Expressway patrol management system and method and electronic equipment

Abstract

The invention relates to the technical field of intelligent traffic, in particular to an expressway patrol management system and method and electronic equipment. The system comprises a data acquisition module, a road condition identification module, an illegal driving identification module and a communication module. The data acquisition module is used for acquiring pavement image data of the expressway and driving data and snapshot image data of vehicles on the expressway; the pavement condition identification module is used for identifying the pavement abnormal condition of the expressway according to the pavement image data and the driving data; the illegal driving recognition module is used for recognizing illegal driving behaviors according to the driving data and the snapshot image data; and the communication module is used for sending an alarm signal when the road surface abnormal condition or the illegal driving behavior is recognized. According to the invention, timely early warning can be carried out on traffic potential safety hazards of the expressway, and the driving safety is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a highway patrol management system, method and electronic equipment. Background Art

[0002] In the current road traffic safety monitoring system, the traditional road patrol method mainly relies on manual road inspection and traffic order maintenance. However, this method has many limitations. First, manual patrol is inefficient, especially on roads with large traffic volume and high speed such as highways, it is difficult for manual patrol to achieve comprehensive and timely monitoring. Secondly, manual patrol is less safe, and patrol personnel need to frequently shuttle in the traffic, facing extremely high safety risks, especially in bad weather or at night, this risk is more significant.

[0003] With the rapid development of Internet of Things technology and big data analysis, the development and application of intelligent monitoring systems have gradually become an important way to solve the above problems. However, most of the existing intelligent monitoring systems rely on fixed cameras and manual video analysis. Fixed cameras collect images and process and identify images through manual video analysis. However, this patrol method has low data analysis efficiency and is difficult to provide timely warnings of safety hazards on the road. Summary of the invention

[0004] In order to solve the problem that the prior art cannot provide timely warnings for traffic safety hazards on highways, the present application provides a highway inspection management system, method and electronic equipment.

[0005] In the first aspect, the present application provides a highway patrol management system, which adopts the following technical solutions: A highway patrol management system, comprising: a data acquisition module, a road condition recognition module, a driving violation recognition module and a communication module; The data acquisition module is used to obtain road surface image data of the expressway and driving data and captured image data of vehicles on the expressway; The road surface condition recognition module is used to recognize abnormal road surface conditions of the expressway according to the road surface image data and the driving data; The illegal driving identification module is used to identify illegal driving behavior based on the driving data and the captured image data; The communication module is used to send an alarm signal when the abnormal road condition or the illegal driving behavior is identified.

[0006] By adopting the above technical solutions, the data acquisition module can efficiently obtain road surface images, vehicle driving data and captured images. The road condition recognition module can accurately identify abnormal road conditions such as road damage or road obstacles on the highway, while the illegal driving recognition module can accurately capture illegal driving behaviors such as speeding, following vehicles, and illegal lane changes. Once these abnormalities or illegal behaviors are identified, the communication module will quickly send an alarm signal, reporting the road abnormality to the management personnel for timely maintenance, and issuing warnings to the illegal vehicles to correct driving behaviors. The system has significantly improved the safety management level of highways, effectively prevented the occurrence of traffic accidents, and enhanced the timeliness and pertinence of road maintenance, providing solid technical support for the safety and smooth flow of highways.

[0007] In a preferred example, the present application may be further configured as follows: the road condition identification module includes a model identification unit and a parameter identification unit; the driving data includes vehicle vibration data; The model recognition unit is used to input the road surface image data into a preset recognition model and receive road surface abnormality information output by the preset recognition model, wherein the road surface abnormality information includes abnormal road surface area and abnormality type, wherein the abnormality type includes: road surface damage and road surface obstacle; The parameter identification unit is used to determine the bumpy road section that the target vehicle has traveled on based on the vehicle vibration data of the target vehicle, and determine the bumpy road section where the number of vehicles that have traveled on it reaches a preset number of vehicles as having abnormal road conditions; the target vehicle is any vehicle traveling on the highway.

[0008] By adopting the above technical solution, the model recognition unit uses advanced preset recognition models to conduct in-depth analysis of road image data, accurately identify abnormal conditions such as road damage and obstacles, and accurately mark abnormal areas and types, providing managers with intuitive and detailed road condition information. The parameter recognition unit uses vehicle vibration data to count the vibration conditions of multiple vehicles on the same section of road, and intelligently identifies bumpy sections. When these sections are verified as bumpy by multiple vehicles (reaching the preset number of vehicles), it is determined that there is an abnormal road condition. This dual recognition mechanism greatly improves the recognition accuracy and reliability of abnormal road conditions.

[0009] In a preferred example, the present application may be further configured as follows: the vehicle vibration data includes vibration frequency and amplitude; When the parameter identification unit determines the bumpy road section traveled by the target vehicle based on the vehicle vibration data of the target vehicle, it is specifically used to: Based on the vibration frequency in the vehicle vibration data of the target vehicle, determining a first road section where the vibration frequency of the target vehicle exceeds a preset vibration frequency; Based on the amplitude in the vehicle vibration data of the target vehicle, determining a second road section where the amplitude exceeds a preset amplitude, which the target vehicle has traveled; Determine the overlapping section of the first section and the second section, and judge whether the time taken by the target vehicle to travel through the overlapping section exceeds a preset effective bumpy time. If the time taken exceeds the preset effective bumpy time, determine the overlapping section as the bumpy section traveled by the target vehicle.

[0010] By adopting the above technical solution, the parameter identification unit can accurately identify the first section where the vibration frequency exceeds the preset range and the second section where the amplitude exceeds the preset range encountered during the vehicle's driving process by analyzing the vibration frequency and amplitude data of the target vehicle. By determining the overlapping part of the two sections and judging whether the time the vehicle travels on the overlapping section exceeds the preset effective bump duration, the system can accurately screen out the sections where there are real bump problems and mark them as bumpy sections. This method based on comprehensive judgment of multiple parameters not only improves the accuracy of road surface abnormality identification, but also avoids false alarms caused by single parameter abnormalities.

[0011] In a preferred example, the present application may be further configured as follows: the illegal driving identification module includes an illegal behavior identification unit and an illegal status identification unit; the illegal driving behavior includes illegal behavior and illegal status; The violation identification unit is used to determine the driving trajectory of each vehicle according to the driving data, and identify whether the target vehicle has any violation based on the driving trajectory of each vehicle; The violation status identification unit is used to compare the driving trajectory of the target vehicle with the lane it is in to determine whether the target vehicle has deviated from the lane; perform image recognition on the captured image data of the target vehicle to determine whether the driver of the target vehicle has inattentive behavior; if the target vehicle has the lane deviation behavior or the inattentive behavior, it is determined that the target vehicle is in a violation state.

[0012] By adopting the above technical solution, the violation identification unit tracks the driving trajectory of each vehicle and identifies whether there are any violations such as speeding and illegal lane changes. The violation status identification unit detects whether the vehicle deviates from the lane by comparing the driving trajectory with the lane information, and uses image recognition technology to analyze the captured image data to determine whether the driver has dangerous behaviors such as distraction and inattention. This comprehensive identification mechanism not only improves the detection accuracy of illegal driving behaviors, but also effectively covers a variety of potential safety hazards, providing more comprehensive and detailed protection for highway traffic safety.

[0013] In a preferred example, the present application can be further configured as follows: the illegal behaviors include: speed violation, illegal overtaking and illegal following; The violation identification unit, when performing the step of identifying whether the target vehicle has a violation based on the driving trajectory of each vehicle, is specifically used to: Determine the real-time speed of the target vehicle based on the driving trajectory of the target vehicle, compare the real-time speed with the standard speed interval, and if the real-time speed exceeds the standard speed interval, determine that the target vehicle has a speed violation; Based on the driving trajectories of the target vehicle and the vehicle in front of the target vehicle, the relative position of the target vehicle and the vehicle in front is determined in real time, and based on the relative position and the driving trajectories of the target vehicle and the vehicle in front, whether the target vehicle has overtaken is determined, and the number of overtaking behaviors of the target vehicle is recorded. If the number of overtaking behaviors exceeds a preset number of overtaking behaviors, the target vehicle is determined to have overtaken illegally; Determine whether the vehicle in front has emergency braking behavior based on the driving trajectory of the vehicle in front; if the vehicle in front has not had emergency braking behavior, determine whether the distance between the target vehicle and the vehicle in front exceeds a preset safety distance; if the distance does not exceed the preset safety distance and the duration exceeds the preset following time, determine that the target vehicle has violated the following regulations.

[0014] By adopting the above technical solution, the violation identification unit accurately calculates the real-time speed of the target vehicle based on the driving trajectory of each vehicle, and compares it with the standard speed range to accurately identify speed violations. The unit can also monitor the relative position and driving trajectory of the target vehicle and the vehicle in front of it in real time, intelligently determine whether there is any illegal overtaking behavior, and count the number of overtaking times. Once the preset number is exceeded, it is determined to be a violation. In addition, it can also intelligently identify whether there is any illegal following behavior based on the driving status of the vehicle in front and the distance between the target vehicle and the vehicle in front, especially when the vehicle in front does not brake urgently and the distance between the two vehicles is too close and the duration is too long. This comprehensive and detailed illegal driving behavior identification mechanism not only significantly improves the safety management level of highways, but also effectively prevents traffic accidents caused by illegal driving, providing a safer driving environment for drivers and other road users.

[0015] In a preferred example, the present application may be further configured as follows: the communication module includes a road surface abnormality warning unit and an illegal driving behavior warning unit; The road surface abnormality alarm unit is used to generate a first alarm signal including the type and location of the road surface abnormality when the road surface condition recognition module recognizes the road surface abnormality, and send the first alarm signal to the terminal device of the manager of the expressway; The illegal driving behavior alarm unit is used to determine the illegal vehicle with the illegal driving behavior when the illegal driving identification module identifies the illegal driving behavior, generate a second alarm signal containing the type of the illegal driving behavior, and send the second alarm signal to the vehicle-mounted terminal of the illegal vehicle.

[0016] By adopting the above technical solution, when the road condition recognition module detects a road abnormality, the road abnormality alarm unit can quickly generate a first alarm signal containing the abnormality type and location, and immediately convey this key information to the highway manager, ensuring that the manager can grasp the road condition at the first time and take maintenance measures in time; at the same time, once the illegal driving recognition module identifies the illegal driving behavior, the illegal driving behavior alarm unit will immediately lock the illegal vehicle, generate a second alarm signal containing the violation type, and send it directly to the on-board terminal of the illegal vehicle to warn the driver to correct the driving behavior immediately; this instant and accurate alarm mechanism not only greatly improves the safety management level of the highway, but also effectively promotes the driver's awareness of compliance driving by directly sending warning information to violators.

[0017] In a preferred example, the present application may be further configured as follows: the system further includes an early warning module; The early warning module is used to record the historical illegal driving behavior of each vehicle, and when monitoring any vehicle with the historical illegal driving behavior driving on the highway, send a warning signal to the on-board terminal of the vehicle with the historical illegal driving behavior to prompt the driver to avoid the historical illegal driving behavior.

[0018] By adopting the above technical solution, the early warning module records the historical illegal driving behaviors of each vehicle. When these vehicles enter the highway again, the early warning module will immediately send a warning signal to their on-board terminals to remind the drivers to pay attention and avoid repeating previous illegal driving behaviors. This personalized early warning mechanism not only helps to improve the driver's awareness of safe driving, but also effectively reduces potential traffic accidents caused by repeated violations.

[0019] In the second aspect, the present application provides a highway patrol management method, which adopts the following technical solution: A highway inspection management method, comprising: Acquire road surface image data of the expressway and driving data and captured image data of vehicles on the expressway; Identify abnormal road conditions of the highway according to the road image data and the driving data; Identify illegal driving behavior based on the driving data and the captured image data; When the abnormal road condition or the illegal driving behavior is identified, a warning signal is sent.

[0020] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: one or more processors; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the highway patrol management method as described in any one of the first aspects.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium stores a computer program, which, when executed in a computer, causes the computer to execute the highway patrol management method as described in any one of the first aspects.

[0022] In a fifth aspect, the present application provides a computer program product, which adopts the following technical solution: A computer program product includes a computer program. When the computer program is executed by a processor, it implements the highway patrol management method as described in any one of the first aspects.

[0023] In summary, this application includes the following beneficial technical effects: In this application, the data acquisition module efficiently acquires road surface images, vehicle driving data and captured images. The road condition recognition module can accurately identify abnormal road conditions such as road damage or road obstacles on the highway, while the illegal driving recognition module can accurately capture illegal driving behaviors such as speeding, following vehicles, and illegal lane changes. Once these abnormalities or illegal behaviors are identified, the communication module will quickly send an alarm signal, reporting the road abnormality to the management personnel for timely maintenance, and issuing warnings to the illegal vehicles to correct driving behavior. The system significantly improves the safety management level of highways, effectively prevents the occurrence of traffic accidents, and at the same time enhances the timeliness and pertinence of road maintenance, providing solid technical support for the safety and smooth flow of highways. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a structural diagram of a highway patrol management system provided by an embodiment of the present application; Figure 2 It is a flowchart of a highway patrol management method provided by an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following is combined with Figure 1 -Attached Figure 3 This application is described in further detail.

[0026] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0029] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0030] The present application embodiment provides a highway patrol management method, such as Figure 1 As shown, the system includes: a data acquisition module, a road condition recognition module, a driving violation recognition module and a communication module.

[0031] The data acquisition module is used to obtain the road surface image data of the expressway and the driving data and captured image data of the vehicles on the expressway.

[0032] Specifically, the road surface image data of the highway can be collected by a drone or mobile inspection equipment equipped with a camera device, and the road surface image data includes the highway road surface and roadside railings. Vehicles are equipped with on-board sensors, including but not limited to: GPS positioning sensors, speed sensors, acceleration sensors, vibration sensors, etc. The sensors collect vehicle driving data in real time and transmit it to the highway inspection management system in real time through the on-board communication system. Camera devices are fixed along the highway to capture moving vehicles and obtain captured image data.

[0033] The road surface condition recognition module is used to recognize abnormal road surface conditions on the expressway based on road surface image data and driving data.

[0034] Specifically, the road condition recognition module inputs the road image data into the built-in preset recognition model, and the preset recognition model outputs the road abnormality information, which includes the abnormal road area and the abnormal condition type. The vehicle driving data includes the vibration data collected by the vibration sensor. The road condition recognition module identifies the bumpy sections on the highway based on the vibration data of each vehicle, and identifies the determined bumpy sections as having abnormal road conditions.

[0035] The illegal driving identification module is used to identify illegal driving behaviors based on driving data and captured image data.

[0036] Specifically, the illegal driving identification module determines whether the vehicle has any illegal behaviors, such as speed violations, illegal overtaking, and illegal following, based on the vehicle's driving data. On the other hand, the illegal driving identification module determines whether the driver of the vehicle has deviated from the lane or has not focused based on the vehicle's driving data and captured image data. If any of these exists, the driver can be determined to be distracted driving, and a warning can be issued to the driver. Among them, inattentive behaviors include but are not limited to: taking both hands off the steering wheel and making phone calls.

[0037] The communication module is used to send an alarm signal when abnormal road conditions or illegal driving behaviors are identified.

[0038] Specifically, when the road condition recognition module recognizes an abnormal road condition, or the illegal driving recognition module recognizes illegal driving behavior, the communication module sends an alarm signal to the corresponding personnel to avoid vehicle accidents.

[0039] In this embodiment, the data acquisition module efficiently acquires road surface images, vehicle driving data and captured images. The road condition recognition module can accurately identify abnormal road conditions such as road damage or road obstacles on the highway, and the illegal driving recognition module can accurately capture illegal driving behaviors such as speeding, following vehicles, and illegal lane changes. Once these abnormalities or illegal behaviors are identified, the communication module will quickly send an alarm signal, reporting the road abnormality to the management personnel for timely maintenance, and issuing a warning to the illegal vehicle to correct the driving behavior. The system significantly improves the safety management level of highways, effectively prevents the occurrence of traffic accidents, and at the same time enhances the timeliness and pertinence of road maintenance, providing solid technical support for the safety and smooth flow of highways.

[0040] In a possible implementation of the embodiment of the present application, the road condition identification module includes a model identification unit and a parameter identification unit; the driving data includes vehicle vibration data; A model recognition unit, used to input the road surface image data into a preset recognition model, and receive road surface abnormality information output by the preset recognition model, wherein the road surface abnormality information includes abnormal road surface area and abnormality type, wherein the abnormality type includes: road surface damage and road surface obstacle; A parameter identification unit is used to determine the bumpy road section traveled by the target vehicle based on the vehicle vibration data of the target vehicle, and to determine the bumpy road section where the number of vehicles traveling reaches a preset number of vehicles as having abnormal road conditions; the target vehicle is any vehicle traveling on the highway.

[0041] In this embodiment, the training process of the preset recognition model includes: collecting a large amount of road surface image data from multiple sources (such as surveillance cameras, drone photography, vehicle-mounted cameras, etc.), annotating the collected road surface image data, marking normal road surfaces and abnormal road surfaces, and the annotation of abnormal road surfaces also includes abnormal road surface areas and abnormal condition types. The abnormal road surface area represents the area where abnormal conditions occur, and the abnormal condition types include but are not limited to: damaged road surface, missing road guardrails, and road surface obstacles. This annotation process can be achieved through image annotation tools, and the image annotation tools can use LabelImg, VIA, etc. According to experience, a suitable model architecture is manually selected, such as a convolutional neural network, a residual network, etc., and the annotated road surface image data is used as a training set to train the model. The trained model can identify abnormal road surface conditions on highways and output abnormal road surface condition information.

[0042] The vibration data of the target vehicle is the vibration data with time series generated when the target vehicle is driving on the highway. The vehicle vibration data includes vibration frequency and amplitude. Then, the bumpy road section that the target vehicle has driven on is determined. The bumpy road section can be a continuous road section or multiple discontinuous road sections. The vehicles driving on the highway are dynamic. For a certain road section on the highway, if the road section is determined to be a bumpy road section after a certain vehicle has driven on it, it is counted once. When the number of times that the road section is determined to be a bumpy road section after the counted vehicles have driven on it reaches a preset number of vehicles, the road section is determined to have abnormal road conditions, and the location information of the road section is recorded. The location information includes the specific locations of the two ends of the road section (which can be expressed in longitude and latitude).

[0043] In this embodiment, the model recognition unit uses an advanced preset recognition model to perform in-depth analysis of road surface image data, accurately identifies abnormal conditions such as road damage and obstacles, and accurately marks abnormal areas and types, providing management personnel with intuitive and detailed road condition information. The parameter recognition unit uses vehicle vibration data to intelligently identify bumpy sections by counting the vibration conditions of multiple vehicles on the same section of road. When these sections are verified to be bumpy by multiple vehicles (reaching the preset number of vehicles), it is determined that an abnormal road condition exists. This dual recognition mechanism greatly improves the recognition accuracy and reliability of abnormal road conditions.

[0044] In a possible implementation of the embodiment of the present application, the vehicle vibration data includes vibration frequency and amplitude; When the parameter identification unit determines the bumpy road section traveled by the target vehicle based on the vehicle vibration data of the target vehicle, it is specifically used to: Based on the vibration frequency in the vehicle vibration data of the target vehicle, determining a first road section traveled by the target vehicle, on which the vibration frequency exceeds a preset vibration frequency; Based on the amplitude in the vehicle vibration data of the target vehicle, determining a second road section traveled by the target vehicle and having an amplitude exceeding a preset amplitude; Determine the overlapping section of the first section and the second section, and judge whether the time taken by the target vehicle to travel through the overlapping section exceeds the preset effective bumping time. If the time taken exceeds the preset effective bumping time, the overlapping section is determined as the bumpy section traveled by the target vehicle.

[0045] In this embodiment, the vehicle vibration data of the target vehicle is synchronized with the driving trajectory of the target vehicle (such as GPS location information) to obtain the vehicle vibration data of the target vehicle at various locations on the highway. According to the highway road condition standard or human experience, the preset vibration frequency and preset amplitude are pre-set to determine whether the vehicle is driving on a bumpy road section.

[0046] Furthermore, based on the vehicle vibration data of the target vehicle, a first road section where the vibration frequency of the target vehicle exceeds a preset vibration frequency is found, and a second road section where the amplitude of the target vehicle exceeds a preset amplitude is found. The first road section and the second road section can both be a continuous road section or multiple discontinuous road sections.

[0047] Determine the overlapping sections of the first section and the second section. If the number of overlapping sections is 1, determine whether the time taken by the target vehicle to travel through the overlapping section exceeds the preset effective bumping time (a section where continuous vibration exceeds a certain time is considered a bumpy section). If so, determine the overlapping section as the bumpy section where the target vehicle has traveled. If the number of overlapping sections of the first section and the second section is multiple, determine whether the time taken by the target vehicle to travel through each overlapping section exceeds the preset effective bumping time. If there is an overlapping section where the time taken exceeds the preset effective bumping time, determine the overlapping section where the time taken exceeds the preset effective bumping time as the bumpy section where the target vehicle has traveled. The preset effective bumping time can be set manually based on the experience of the management personnel.

[0048] The parameter identification unit in this embodiment can accurately identify the first section where the vibration frequency exceeds the preset range and the second section where the amplitude exceeds the preset range encountered during the vehicle's driving process by analyzing the vibration frequency and amplitude data of the target vehicle. By determining the overlapping part of the two sections and judging whether the time the vehicle travels on the overlapping section exceeds the preset effective bump duration, the system can accurately screen out the sections where there are real bump problems and mark them as bumpy sections. This method based on comprehensive judgment of multiple parameters not only improves the accuracy of road surface abnormality identification, but also avoids false alarms caused by single parameter abnormalities.

[0049] In a possible implementation of the embodiment of the present application, the illegal driving identification module includes an illegal behavior identification unit and an illegal status identification unit; the illegal driving behavior includes illegal behavior and illegal status; A violation identification unit, used to determine the driving trajectory of each vehicle according to the driving data, and to identify whether the target vehicle has any violation based on the driving trajectory of each vehicle; The violation status recognition unit is used to compare the target vehicle's driving trajectory with the lane it is in to determine whether the target vehicle has deviated from the lane; perform image recognition on the captured image data of the target vehicle to determine whether the driver of the target vehicle has inattentive behavior; if the target vehicle has deviated from the lane or has inattentive behavior, it is determined that the target vehicle is in a violation state.

[0050] In this embodiment, the vehicle driving data includes the location information collected by the GPS positioning sensor. Based on the location information, a series of continuous trajectory points corresponding to each vehicle are generated. Each trajectory point contains the location information and a timestamp. The trajectory points generated for each vehicle are smoothed to obtain the driving trajectory of each vehicle. The violation identification unit is used to identify whether the vehicle has a violation. The violation types include but are not limited to: speed violation, illegal overtaking and illegal following. When the target vehicle is identified to have a violation, the violation type, time, location and other information of the violation are recorded.

[0051] The violation state identification unit is used to identify whether the driver is in a distracted violation state during driving. Specifically, high-precision map data containing lane information of the highway is imported, and the lane of the target vehicle is identified based on the location information and map data of the target vehicle. The lane boundary is detected based on the high-precision map data, and it is determined whether the target vehicle exceeds the lane boundary based on the driving trajectory of the target vehicle. If the target vehicle exceeds the lane boundary, it is determined that the target vehicle has deviated from the lane. The process of determining whether the target vehicle exceeds the lane boundary includes: statistically analyzing the size information of multiple cars with the highest sales on the market, the size information includes vehicle width information, calculating the average width of multiple cars, and taking half of the average width as the distance threshold. When the distance between the location information of the target vehicle and the lane boundary is less than the distance threshold, it is determined that the target vehicle exceeds the lane boundary.

[0052] Furthermore, the captured image data of the target image is subjected to image recognition to identify the inattentive behavior. This process can be achieved through a preset classification model. The real-time captured image data of the target vehicle is input into the preset classification model. The preset classification model outputs the judgment result of whether the driver has inattentive behavior. If inattentive behavior exists, the preset classification model also outputs the behavior type of inattentive behavior. The behavior types of inattentive behavior include but are not limited to: answering and making phone calls, playing with mobile phones, turning the head, etc. If the recognition result is that the target vehicle is in a violation state, the violation state type is recorded, whether there is lane deviation behavior or inattentive behavior. If inattentive behavior exists, the behavior type of inattentive behavior is also recorded.

[0053] The training process of the preset classification model includes: collecting a large number of images containing different behaviors of drivers from vehicle-mounted cameras or traffic monitoring cameras, using image annotation tools to annotate the driver's behavior in the images, and annotating the content including the presence or absence of inattention behavior. For images with inattention behavior, annotate the behavior type of inattention behavior. Use the annotated image data to train the classification model (the model architecture can be selected from YOLO, EfficientDet, FasterR-CNN), and obtain the trained preset classification model.

[0054] In this embodiment, the violation identification unit tracks the driving trajectory of each vehicle and identifies whether there are any violations such as speeding, illegal lane changes, etc. The violation status identification unit detects whether the vehicle deviates from the lane by comparing the driving trajectory with the lane information, and uses image recognition technology to analyze the captured image data to determine whether the driver has dangerous behaviors such as distraction and inattention. This comprehensive identification mechanism not only improves the detection accuracy of illegal driving behaviors, but also effectively covers a variety of potential safety hazards, providing more comprehensive and detailed protection for highway traffic safety.

[0055] In a possible implementation of the embodiment of the present application, the violation behaviors include: speed violation, illegal overtaking and illegal following; The violation identification unit, when performing the identification of whether the target vehicle has violated the regulations based on the driving trajectory of each vehicle, is specifically used to: Determine the real-time speed of the target vehicle based on the target vehicle's driving trajectory, compare the real-time speed with the standard speed range, and if the real-time speed exceeds the standard speed range, determine that the target vehicle has violated the speed regulations; Based on the driving trajectories of the target vehicle and the vehicle in front of the target vehicle, the relative position of the target vehicle and the vehicle in front is determined in real time. Based on the relative position and the driving trajectories of the target vehicle and the vehicle in front, it is determined whether the target vehicle has overtaken, and the number of overtaking behaviors of the target vehicle is recorded. If the number of overtaking behaviors exceeds the preset number of overtaking behaviors, it is determined that the target vehicle has overtaken illegally. Determine whether the vehicle in front has emergency braking behavior based on its driving trajectory. If the vehicle in front has not emergency braking behavior, determine whether the distance between the target vehicle and the vehicle in front exceeds the preset safety distance. If the distance does not exceed the preset safety distance and the duration exceeds the preset following time, determine that the target vehicle is following the vehicle illegally.

[0056] In this embodiment, the real-time speed of the target vehicle can be determined according to the driving trajectory of the target vehicle, and the real-time speed of the target vehicle collected by the speed sensor configured for the target vehicle can also be obtained. The standard speed interval is set according to traffic regulations. If the real-time speed does not exceed the standard speed interval, it is determined that the target vehicle has no violation behavior.

[0057] For illegal overtaking, the driving trajectories of each vehicle on the highway are synchronized to ensure that the data of all vehicles are compared within the same time frame. Two overtaking conditions are set in advance. If the target vehicle and the vehicle ahead both meet the two overtaking conditions, the target vehicle is judged to have overtaken.

[0058] Overtaking condition one: Identify the driving trajectory of the target vehicle and the vehicle in front of the target vehicle, and determine whether the target vehicle and the vehicle in front have changed lanes. If the target vehicle changes lanes briefly and returns to the initial lane, it is determined that overtaking condition one is met. Overtaking condition two: Determine the relative position of the target vehicle and the vehicle in front based on the driving trajectory. The relative position is expressed by the position difference between the vehicle in front and the target vehicle in the driving direction, that is, the longitudinal distance. If the position difference is a positive number, it means that the vehicle in front is in front of the target vehicle in the driving direction. If the position difference changes from a positive number to a negative number, it means that the vehicle in front is behind the target vehicle in the driving direction. It is determined that the relative position of the target vehicle and the vehicle in front has changed, and overtaking condition two is met.

[0059] Among them, a time window (such as 1 hour) can be set, and the number of overtaking behaviors of the target vehicle can be counted within the time window. If the number of overtaking behaviors of the target vehicle within the time window exceeds the preset number of overtaking behaviors, the target vehicle is judged to have overtaken illegally.

[0060] For the illegal behavior of following a vehicle in violation of regulations, the acceleration of the vehicle in front can be calculated based on the driving trajectory of the vehicle in front, and the acceleration data collected by the acceleration sensor installed on the vehicle in front can also be obtained. If the acceleration of the vehicle in front is negative and the absolute value exceeds the preset acceleration threshold, it is determined that the vehicle in front has emergency braking behavior, otherwise, it is determined that the vehicle in front has no emergency braking behavior. Among them, the preset acceleration threshold and the preset safety distance can be manually preset according to traffic regulations and actual experience.

[0061] When the vehicle ahead does not have emergency braking behavior, the distance between the target vehicle and the vehicle ahead does not exceed the preset safety distance, indicating that the target vehicle and the vehicle ahead are too close, and the duration of the close distance exceeds the preset following time, indicating that the target vehicle did not increase the distance from the vehicle ahead in time and was too close to the vehicle ahead for a long time, and the target vehicle is judged to be following the vehicle illegally. If the distance between the target vehicle and the vehicle ahead exceeds the preset safety distance, or the duration of the distance not exceeding the preset safety distance does not exceed the preset following time, it is judged that the target vehicle does not follow the vehicle illegally.

[0062] When the vehicle ahead performs emergency braking, it indicates that a sudden situation may occur and the target vehicle is not judged to have violated the following regulations.

[0063] In this embodiment, the violation identification unit accurately calculates the real-time speed of the target vehicle based on the driving trajectory of each vehicle, and compares it with the standard speed range to accurately identify speed violations. The unit can also monitor the relative position and driving trajectory of the target vehicle and the vehicle in front of it in real time, intelligently determine whether there is any illegal overtaking behavior, and count the number of overtaking times. Once the preset number is exceeded, it is determined to be a violation. In addition, it can also intelligently identify whether there is any illegal following behavior based on the driving status of the vehicle in front and the distance between the target vehicle and the vehicle in front, especially when the distance between the two vehicles is too close and the duration is too long without emergency braking. This comprehensive and detailed illegal driving behavior identification mechanism not only significantly improves the safety management level of highways, but also effectively prevents traffic accidents caused by illegal driving, providing a safer driving environment for drivers and other road users.

[0064] In a possible implementation of the embodiment of the present application, the communication module includes a road surface abnormality warning unit and an illegal driving behavior warning unit; A road surface abnormality alarm unit, for generating a first alarm signal including the type and location of the road surface abnormality when the road surface condition recognition module recognizes the road surface abnormality, and sending the first alarm signal to a terminal device of a manager of the expressway; The illegal driving behavior alarm unit is used to determine the illegal vehicle with illegal driving behavior when the illegal driving identification module identifies the illegal driving behavior, generate a second alarm signal containing the type of illegal driving behavior, and send the second alarm signal to the vehicle-mounted terminal of the illegal vehicle.

[0065] In this embodiment, the first warning signal is sent to the terminal device of the manager so that the manager can take timely measures to deal with the abnormal road conditions. The second warning signal is sent to the vehicle terminal of the illegal vehicle to remind the driver to correct the illegal driving behavior and improve driving safety.

[0066] In this embodiment, when the road condition recognition module detects a road abnormality, the road abnormality alarm unit can quickly generate a first alarm signal containing the abnormality type and location, and immediately convey this key information to the highway manager, ensuring that the manager can grasp the road condition at the first time and take maintenance measures in time; at the same time, once the illegal driving recognition module identifies the illegal driving behavior, the illegal driving behavior alarm unit will immediately lock the illegal vehicle, generate a second alarm signal containing the violation type, and send it directly to the on-board terminal of the illegal vehicle to warn the driver to correct the driving behavior immediately; this instant and accurate alarm mechanism not only greatly improves the safety management level of the highway, but also effectively promotes the driver's awareness of compliance driving by directly sending warning information to violators.

[0067] In a possible implementation of the embodiment of the present application, the system further includes an early warning module; The early warning module is used to record the historical illegal driving behavior of each vehicle, and when any vehicle with a history of illegal driving behavior is detected driving on the highway, a warning signal is sent to the on-board terminal of the vehicle with a history of illegal driving behavior to remind the driver to avoid the historical illegal driving behavior.

[0068] In this embodiment, historical driving violations include violations on the highway during historical time periods. When new violations occur, the system promptly updates the historical driving violations stored in the database. At the same time, for vehicles that have corrected the violations (for example, they have traveled a preset distance and have not committed a certain type of violation again), the historical records of this type can be deleted from the records of historical driving violations as a sign of encouragement.

[0069] When the recognition device at the entrance of the expressway identifies a vehicle driving on the expressway, the historical illegal driving behavior corresponding to the vehicle is queried, and the illegal driving behavior of each type can be sorted according to the historical number of violations, and a warning signal containing the sorting results is generated to prompt the driver to avoid illegal driving behavior during this driving.

[0070] In this embodiment, the early warning module records the historical illegal driving behaviors of each vehicle. When these vehicles enter the highway again, the early warning module will immediately send a warning signal to their on-board terminals to remind the drivers to pay attention and avoid repeating previous illegal driving behaviors. This personalized early warning mechanism not only helps to improve the driver's awareness of safe driving, but also effectively reduces potential traffic accidents caused by repeated violations.

[0071] The present application embodiment provides a highway patrol management method, such as Figure 2 As shown, the method provided in the embodiment of the present application is executed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application. The method includes steps S201 to S204, wherein: S201, obtaining road surface image data of the expressway and driving data and captured image data of vehicles on the expressway.

[0072] S202: Identify abnormal road conditions on the expressway based on road image data and driving data.

[0073] S203: Identify illegal driving behaviors based on the driving data and captured image data.

[0074] S204: When an abnormal road condition or illegal driving behavior is identified, a warning signal is sent.

[0075] An electronic device is provided in an embodiment of the present application, such as Figure 3 As shown, Figure 3 The electronic device 300 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation on the embodiments of the present application.

[0076] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0077] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but it does not mean that there is only one bus or only one type of bus.

[0078] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0079] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above-mentioned highway patrol management method embodiment.

[0080] Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0081] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the contents shown in the aforementioned highway patrol management method embodiment.

[0082] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0083] An embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the contents shown in the aforementioned highway patrol management method embodiment are implemented.

[0084] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A highway inspection management system, characterized in that: include: Data acquisition module, road condition recognition module, illegal driving recognition module and communication module; The data acquisition module is used to obtain road surface image data of the expressway and driving data and captured image data of vehicles on the expressway; The road surface condition recognition module is used to recognize abnormal road surface conditions of the expressway according to the road surface image data and the driving data; The illegal driving identification module is used to identify illegal driving behavior based on the driving data and the captured image data; The communication module is used to send an alarm signal when the abnormal road condition or the illegal driving behavior is identified.

2. The highway inspection management system according to claim 1 is characterized in that: The road condition identification module includes a model identification unit and a parameter identification unit; the driving data includes vehicle vibration data; The model recognition unit is used to input the road surface image data into a preset recognition model and receive road surface abnormality information output by the preset recognition model, wherein the road surface abnormality information includes abnormal road surface area and abnormality type, wherein the abnormality type includes: road surface damage and road surface obstacle; The parameter identification unit is used to determine the bumpy road section that the target vehicle has traveled on based on the vehicle vibration data of the target vehicle, and determine the bumpy road section where the number of vehicles that have traveled on it reaches a preset number of vehicles as having abnormal road conditions; the target vehicle is any vehicle traveling on the highway.

3. The highway inspection management system according to claim 2 is characterized in that: The vehicle vibration data includes vibration frequency and amplitude; When the parameter identification unit determines the bumpy road section traveled by the target vehicle based on the vehicle vibration data of the target vehicle, it is specifically used to: Based on the vibration frequency in the vehicle vibration data of the target vehicle, determining a first road section where the vibration frequency of the target vehicle exceeds a preset vibration frequency; Based on the amplitude in the vehicle vibration data of the target vehicle, determining a second road section where the amplitude exceeds a preset amplitude, which the target vehicle has traveled; Determine the overlapping section of the first section and the second section, and judge whether the time taken by the target vehicle to travel through the overlapping section exceeds a preset effective bumpy time. If the time taken exceeds the preset effective bumpy time, determine the overlapping section as the bumpy section traveled by the target vehicle.

4. The highway inspection management system according to claim 1, characterized in that: The illegal driving identification module includes an illegal behavior identification unit and an illegal status identification unit; the illegal driving behavior includes illegal behavior and illegal status; The violation identification unit is used to determine the driving trajectory of each vehicle according to the driving data, and identify whether the target vehicle has any violation based on the driving trajectory of each vehicle; The violation state identification unit is used to compare the driving trajectory of the target vehicle with the lane it is in to determine whether the target vehicle has deviated from the lane; Performing image recognition on the captured image data of the target vehicle to determine whether the driver of the target vehicle has inattentive behavior; If the target vehicle has the lane departure behavior or the inattentive behavior, it is determined that the target vehicle is in a violation state.

5. The highway inspection management system according to claim 4 is characterized in that: The violations include: speed violations, illegal overtaking and illegal following; The violation identification unit, when performing the step of identifying whether the target vehicle has a violation based on the driving trajectory of each vehicle, is specifically used to: Determine the real-time speed of the target vehicle based on the driving trajectory of the target vehicle, compare the real-time speed with the standard speed interval, and if the real-time speed exceeds the standard speed interval, determine that the target vehicle has a speed violation; Based on the driving trajectories of the target vehicle and the vehicle in front of the target vehicle, the relative position of the target vehicle and the vehicle in front is determined in real time, and based on the relative position and the driving trajectories of the target vehicle and the vehicle in front, whether the target vehicle has overtaken is determined, and the number of overtaking behaviors of the target vehicle is recorded. If the number of overtaking behaviors exceeds a preset number of overtaking behaviors, the target vehicle is determined to have overtaken illegally; Determine whether the vehicle in front has emergency braking behavior based on the driving trajectory of the vehicle in front; if the vehicle in front has not had emergency braking behavior, determine whether the distance between the target vehicle and the vehicle in front exceeds a preset safety distance; if the distance does not exceed the preset safety distance and the duration exceeds the preset following time, determine that the target vehicle has violated the following regulations.

6. The highway inspection management system according to claim 1, characterized in that: The communication module includes a road surface abnormality warning unit and an illegal driving behavior warning unit; The road surface abnormality alarm unit is used to generate a first alarm signal including the type and location of the road surface abnormality when the road surface condition recognition module recognizes the road surface abnormality, and send the first alarm signal to the terminal device of the manager of the expressway; The illegal driving behavior alarm unit is used to determine the illegal vehicle with the illegal driving behavior when the illegal driving identification module identifies the illegal driving behavior, generate a second alarm signal containing the type of the illegal driving behavior, and send the second alarm signal to the vehicle-mounted terminal of the illegal vehicle.

7. The highway inspection management system according to claim 1, characterized in that: The system also includes an early warning module; The early warning module is used to record the historical illegal driving behavior of each vehicle, and when monitoring any vehicle with the historical illegal driving behavior driving on the highway, send a warning signal to the on-board terminal of the vehicle with the historical illegal driving behavior to prompt the driver to avoid the historical illegal driving behavior.

8. A highway inspection management method, characterized in that: include: Acquire road surface image data of the expressway and driving data and captured image data of vehicles on the expressway; Identify abnormal road conditions of the highway according to the road image data and the driving data; Identify illegal driving behavior based on the driving data and the captured image data; When the abnormal road condition or the illegal driving behavior is identified, a warning signal is sent.

9. An electronic device, characterized in that: include: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the highway patrol management method described in claim 8.

Images