Method, system and equipment for full-route inspection, monitoring and early warning of highway maintenance management

Abstract

The present invention discloses a method, system and equipment for patrolling, monitoring and early warning of the entire route of highway maintenance and management, and belongs to the field of video monitoring technology. The equipment includes an intelligent terminal device capable of patrolling and moving and a monitoring sensing device at a fixed point. The intelligent terminal device is arranged on guardrails on both sides of the highway, and has functions such as patrolling, data collection, transmission, monitoring, light warning, broadcasting, and voice intercom. The monitoring sensing device is arranged in tunnels, bridges, high slopes, sharp bends, sections with poor visibility, and sections susceptible to rain and snow weather, and can realize the monitoring of images, environmental changes, topography, and real-time data of road conditions. The present invention can realize all-round monitoring and management of the entire highway through the movable intelligent terminal and the collection terminals at various fixed points, and timely collect, organize, and transmit management data such as real-time changes in road conditions, weather and environmental impacts, and driving safety status to a monitoring command and management platform.

Description

Technical Field

[0001] The present invention belongs to the technical field of video monitoring, and in particular relates to a full-route patrol monitoring and early warning method, system and equipment for highway maintenance management. Background Art

[0002] Currently, surveillance cameras are installed on urban roads to obtain road traffic conditions. However, highways have the characteristics of high speed and long sections. In order to better monitor the driving sections, the following monitoring schemes are proposed in the prior art. For example, Chinese patent document CN113206979A discloses a highway patrol robot, which is equipped with a surveillance camera. After the robot is installed on the guardrail of the highway, the robot is controlled to walk inside the guardrail through a driving wheel assembly, thereby realizing dynamic movement of the monitoring location, so that the monitoring points in the highway can be changed as needed; for example, Chinese patent document CN117440130A discloses a highway inspection system, which obtains monitoring videos through inspection robots, drones or optional vehicles, transmits them through a 5G network, and displays them in real time on receiving devices such as monitoring large screens, thereby greatly improving the inspection efforts.

[0003] However, the patrol equipment of the above two solutions only has video surveillance function. Therefore, when judging whether traffic congestion occurs, only the monitoring video images sent back by the patrol equipment are processed manually. After determining that there is traffic congestion, it is also necessary to manually remotely control the movement of the patrol robot to determine the point and cause of the congestion. Based on the above description, the patrol robots in the prior art still have the problem of being inconvenient to use. Summary of the invention

[0004] To solve the above problems, the present invention provides a method, system and equipment for full-route inspection, monitoring and early warning of highway maintenance and management, so as to solve the problem that the inspection robots in the prior art are still inconvenient to use.

[0005] In order to achieve the above-mentioned invention object, the present invention proposes a full-route inspection, monitoring and early warning method for highway maintenance management, comprising:

[0006] Fixed-point collection terminals and early warning devices are arranged at intervals on the highway. The collection terminals and early warning devices adjust their positions to obtain the best monitoring images.

[0007] When the early warning device is in the first state, a road video is recorded and uploaded at a first frame rate, and the driving speed of passing vehicles is detected at the same time. When it is detected that the driving speed of multiple consecutive vehicles is less than a first threshold, the early warning device switches itself to the second state and defines itself as the first device. The first device records the road video at a second frame rate in the second state, and the first frame rate is less than the second frame rate.

[0008] defining a warning device that is located in a first direction and adjacent to the first device as a second device, wherein the first direction is a driving direction of the highway;

[0009] If the second device is in the first state, obtaining a distance between the first device and the second device, and if the distance is less than a second threshold, controlling the first device to move toward the first direction, otherwise controlling the first device and the second device to move toward each other;

[0010] Locating a target position based on the first device or the second device, where the target position is a congestion starting point;

[0011] The optimal surveillance picture of the target location is analyzed to identify the cause of congestion.

[0012] Further, obtaining the driving speed of the vehicle includes the following steps:

[0013] Obtaining the brightness in the optimal monitoring picture, and if the brightness is greater than or equal to a third threshold, using a vehicle tracking algorithm to obtain the vehicle position and vehicle information in the optimal monitoring picture, splitting the optimal monitoring picture into a plurality of first sub-areas, setting a positioning area in the first sub-areas, and positioning the first time point at which the vehicle appears in the positioning area;

[0014] If the brightness is less than the third threshold, multiple gray levels are set, each gray level corresponds to a gray range, the gray level is allocated to the first sub-region according to the gray average value of the first sub-region, and the first sub-region in which the gray levels in the optimal monitoring image change in a predetermined order is defined as a focus area. If the focus area appears in sequence from right to left in the optimal monitoring image, the focus area is analyzed to obtain the vehicle information contained therein, and the first time point at which the vehicle appears in the positioning area is obtained;

[0015] The driving speed and the second time point when the vehicle passes the warning device are acquired based on the radar, and the first time point is matched with the second time point to match the vehicle information with the driving speed.

[0016] Further, identifying the cause of congestion includes the following steps:

[0017] The optimal monitoring picture is projected onto a plane to obtain a road plane map, the early warning device captures the movement trajectory of the vehicle based on preset feature points and maps it into the road plane map, establishes a coordinate system in the road plane map, and cuts the road plane map into a plurality of second sub-areas;

[0018] Generate the direction vector of the nth second sub-region based on the first formula , the first formula is: ,in, is the coordinate of the preset feature point when it appears for the last time in the nth second sub-region, is the coordinates of the preset feature point when it first appears in the nth second sub-region;

[0019] A reference vector is set along the first direction in the road plan, and an angle between the direction vector and the reference vector is calculated; if the angle is greater than a preset value, the direction vector is marked as an avoidance vector; a ratio of the number of avoidance vectors and the number of direction vectors for the same vehicle is calculated, and the ratio is used as an offset; if the offset is greater than a fourth threshold, it is defined as an avoidance vehicle; if the number of avoidance vehicles is greater than a fifth threshold, it is considered that the cause of congestion is obstacle congestion, otherwise it is phantom congestion.

[0020] Furthermore, if the grayscale level of the first sub-area increases and no longer changes, it is defined as an illegal parking area, the illegal parking area is analyzed to obtain the vehicle information contained therein, the vehicle information is uploaded and an illegal parking warning is issued.

[0021] Furthermore, obtaining the optimal monitoring picture includes the following steps:

[0022] Obtain a target image that does not contain a vehicle, set a pixel range, define pixel points in the target image whose pixel values ​​are within the pixel range as road pixel points, identify a road contour based on the road pixel points, and define all pixel points within the road contour as road pixel points;

[0023] The target image is split into a plurality of third sub-areas in the horizontal direction, and the evaluation value of the current shooting angle of view is calculated based on the second formula, where the second formula is: , where M is the number of the third sub-regions, and are respectively the total number of pixel points in the mth third sub-area and the number of pixel points on the road, rotating the warning device in a horizontal plane to obtain a new shooting angle of view, continuing to obtain the evaluation value of the new shooting angle of view by using the second formula, repeating this step until the number of rotations reaches a predetermined number, rotating the warning device to a shooting angle corresponding to the highest evaluation value, which is defined as a first shooting angle of view;

[0024] The early warning device rotates in a vertical plane at the first shooting angle, and obtains the evaluation values ​​of multiple shooting angles based on the second formula, and defines the shooting angle with the evaluation value closest to the critical value as the optimal monitoring picture.

[0025] Furthermore, in the first state, the early warning device records the road video at the second frame rate at a first time interval, which is defined as a sample video, and obtains the target image from the sample video.

[0026] Furthermore, uploading the road video includes the following steps:

[0027] Each warning device is numbered along the first direction. Whenever the road video recorded in the warning device reaches the first duration, the road video is uploaded. When the warning device numbered k completes uploading, the transmission duration is recorded. The warning device numbered k+1 starts uploading the road video after a delay time. The delay time Calculated based on the third formula, the third formula is: ,in, is the first duration of the early warning device numbered k, and is the delay time of the early warning device numbered k.

[0028] The present invention also provides a full-route inspection, monitoring and early warning system for highway maintenance management, which is used to implement the above-mentioned full-route inspection, monitoring and early warning method for highway maintenance management, and the system includes:

[0029] An adjustment module, which is used to adjust the positions of the early warning equipment and the acquisition terminal to obtain the best monitoring picture;

[0030] The detection module records and uploads a road video at a first frame rate when the early warning device is in a first state, and detects the driving speed of passing vehicles at the same time. When it is detected that the driving speed of multiple consecutive vehicles is less than a first threshold, the early warning device switches itself to a second state and defines itself as a first device. The first device records the road video at a second frame rate in the second state, and the first frame rate is less than the second frame rate.

[0031] a moving module, defining an early warning device adjacent to the first device in a first direction as a second device, wherein the first direction is a driving direction of the expressway, and if the second device is in the first state, obtaining a distance between the first device and the second device, and if the distance is less than a second threshold, controlling the first device to move toward the first direction, otherwise controlling the first device and the second device to move toward each other;

[0032] The analysis module locates a target position based on the first device or the second device, the target position being a starting point of congestion, and analyzes the optimal monitoring picture of the target position to identify the cause of congestion.

[0033] The present invention also provides a device for patrolling, monitoring and warning of the entire route of highway maintenance management. The system is used to implement the above-mentioned method for patrolling, monitoring and warning of the entire route of highway maintenance management. The device includes:

[0034] A power system, the power system comprising a wind blade wheel, a travel drive wheel integrated device, a battery and a solar panel, the solar panel and the wind blade wheel charge the battery through solar energy and wind energy, the battery is used to supply power to the travel drive wheel integrated device, and the early warning device moves through the travel drive wheel integrated device;

[0035] Radar system, used to collect vehicle speed;

[0036] A recording system, including a high-definition infrared camera and a recorder, wherein the high-definition infrared camera is used to record and generate road videos, and the recorder is used to record sounds in the road videos;

[0037] A sensing system, including a temperature sensor and a vibration actuator, wherein the temperature sensor is used to detect the road environment temperature and the vibration sensor is used to detect the road vibration;

[0038] A transmission system, including a 5G transmission chip and a wireless network transmission chip, through which the road video is uploaded;

[0039] Lighting system, including yellow warning lights, red warning lights and outline light strips;

[0040] A voice system, including a speaker, through which voice and warnings are played externally;

[0041] Remote control system, used for remote control of early warning equipment.

[0042] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0043] The present invention sets intelligent terminal equipment that can patrol on the guardrails on both sides (middle) of the highway. The intelligent terminal equipment has the functions of data collection, transmission, monitoring, light warning, broadcasting, voice intercom, etc. It also comes with solar panels, small wind turbines, walking drive and other systems. In key sections such as tunnels, bridges, high slopes, sharp bends, poor sight distance, and susceptible to rain and snow weather, fixed-point image, environmental change, topography, real-time road conditions and other data monitoring sensing equipment collection terminals are set. Through the movable collection equipment and the collection terminals at each fixed point, the present invention can realize all-round monitoring and management of the entire highway, and timely collect, organize and transmit the management data involved in real-time changes in road conditions, weather and environmental impacts, driving safety status, deformation and change usage status of tunnels, bridges, high slopes, sharp bends, etc. to the monitoring command management platform. The functional departments of highway maintenance and safety management at all levels can view various online monitoring data and images of highway management in real time through the network.

[0044] The present invention first arranges a plurality of early warning devices at intervals on the highway, and installs speed measuring devices in the early warning devices, so as to realize vehicle speed measurement at a plurality of locations. On this basis, if the early warning device detects that the driving speed of a plurality of vehicles is too low, it is considered that congestion has occurred. In order to quickly obtain a picture of the location where the congestion occurs, the present invention also determines whether to move only the first device to determine the location of the congestion, or to move the first device and the second device at the same time to determine the location of the congestion according to actual conditions. In this way, the early warning device can also automatically move toward the direction of the congestion location and determine the cause of the congestion without manual operation, thereby greatly improving the convenience of using the early warning device. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of the steps of a patrol warning method for a highway according to the present invention;

[0046] Figure 2 A schematic diagram showing the principle of dividing the first sub-area according to the present invention;

[0047] Figure 3 A schematic diagram showing the principle of dividing the second sub-area according to the present invention;

[0048] Figure 4 It is a schematic diagram of the principle of dividing the third sub-area of ​​the present invention;

[0049] Figure 5 The present invention is a schematic structural diagram of a patrol warning system for expressways. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0052] First, the overall system composition and advantages of the present invention are introduced. The present invention sets intelligent terminal devices that can patrol on the guardrails on both sides (middle) of the highway. The intelligent terminal devices have functions such as data collection, transmission, monitoring, light warning, broadcasting, voice intercom, etc. And it comes with solar panels, small wind turbines, walking drive and other systems. In key sections such as tunnels, bridges, high slopes, sharp bends, poor visibility, and susceptible to rain and snow weather, fixed-point image, environmental changes, topography, real-time road conditions and other data monitoring sensing equipment collection terminals are set. Through the movable collection equipment and the collection terminals at each fixed point, the present invention can realize all-round monitoring and management of the entire highway, and timely collect, organize and transmit the management data involved in real-time changes in road conditions, weather and environmental impacts, driving safety status, deformation changes and use status of tunnels, bridges, high slopes, sharp bends, etc. to the monitoring command management platform. The functional departments of highway maintenance and safety management at all levels can view various online monitoring data and images of highway management in real time through the network.

[0053] The highway safety patrol warning system of the present invention also has the following functions:

[0054] 1. Automated monitoring: realize automatic collection, transmission, storage, processing and analysis of monitoring data in all directions and comprehensive early warning;

[0055] 2. Remote online viewing: Highway maintenance and safety management departments at all levels can view various online monitoring data and images of highway management in real time through the Internet;

[0056] 3. Online data analysis: It has the functions of basic data management, timely release of various monitoring data and contents, graphic report production, data analysis, comprehensive early warning, etc.;

[0057] 4. Dangerous alarm for exceeding the standard (limit): Through the real-time online analysis of the monitoring parameters by the software, once the monitoring parameters exceed the standard (limit) or reach the dangerous value, the system can make sound and light alarms and on-site broadcasts, and the data will be reported to the management department by SMS, voice, email, etc.

[0058] Based on the above description, the present invention has the following objectives and effects:

[0059] 1. When illegal parking, scattering, rolling objects on the slope, rain, snow and ice on the road occur, which may affect the safety of pedestrians and vehicles, the system can collect and alarm in time;

[0060] 2. When highway tunnels, bridges, slopes, and road surfaces are damaged, the system can promptly detect and collect data and issue an alarm at the same time, and the on-site terminal will issue an audible and visual warning and broadcast;

[0061] 3. When there is traffic congestion, abnormal driving, or traffic accidents on the road, data can be collected and alarms can be issued in a timely manner, and the on-site terminal can broadcast and command management.

[0062] The following is a detailed description of the method of this system. Figure 1 As shown, a full-route inspection, monitoring and early warning method for highway maintenance management includes:

[0063] Step S1: The fixed-point collection terminals and the early warning devices are arranged at intervals on the highway. The collection terminals and the early warning devices adjust their positions to obtain the best monitoring image.

[0064] Reference Figure 2 , A is the best monitoring picture. In this embodiment, an early warning device is deployed every 10 km on the highway. After the early warning device is installed, the camera carried by it can be rotated to obtain the best monitoring picture. The specific process of obtaining the best monitoring picture will be described later. In addition, after the early warning device is deployed, the early warning device automatically moves on the guardrail or moves based on a remote control command, so the distance between adjacent early warning devices is not fixed.

[0065] In particular, in this embodiment, fixed-point collection terminals are set up in key sections such as tunnels, bridges, high slopes, sharp bends, poor sight distance, and sections susceptible to rain and snow weather. Cameras are set up in the collection terminals, and images of fixed points and environmental changes, such as rain and snow, topography, road conditions, etc., can be obtained through the cameras. By uploading these data to remote monitoring equipment in real time, the maintenance and safety management functional departments at all levels of highways can monitor the road conditions in real time. For example, in the case of damage to highway tunnels, bridges, slopes, and road surfaces, the system will promptly discover and collect data and alarm at the same time, and the on-site terminal will sound and light alarms and broadcast.

[0066] Step S2: When the early warning device is in the first state, the road video is recorded and uploaded at a first frame rate, and the driving speed of passing vehicles is detected at the same time. When it is detected that the driving speed of multiple consecutive vehicles is less than the first threshold, the early warning device switches itself to the second state and defines itself as the first device. The first device records the road video at a second frame rate in the second state, and the first frame rate is less than the second frame rate.

[0067] The early warning device in this embodiment has a first state and a second state. The first state is the default state of the early warning device. In the first state, the early warning device records and uploads the road video at a lower first frame rate. In this way, the space occupied by the road video can be reduced. The early warning device is equipped with a speed radar for monitoring the speed of the vehicle. When the vehicle passes the early warning device, the early warning device obtains the vehicle's driving speed through the speed radar. When the driving speed of multiple consecutive vehicles is less than the first threshold, such as 10km / h, it is considered that congestion has occurred; at this time, in order to record a more complete monitoring picture, the early warning device starts recording the road video at the second frame rate.

[0068] Step S3: define the warning device adjacent to the first device in the first direction as the second device, where the first direction is the driving direction of the highway.

[0069] Step S4: If the second device is in the first state, obtain the distance between the first device and the second device. If the distance is less than a second threshold, control the first device to move in the first direction; otherwise, control the first device and the second device to move toward each other.

[0070] Since the expressway is a one-way road, when the early warning device detects that congestion has occurred, the location causing the congestion must be behind the early warning device. Therefore, a second device in the first direction and adjacent to the first device is obtained here. If the second device is in the first state, it proves that there is no congestion at the second device. At this time, the distance between the first device and the second device is obtained. If the distance is less than the second threshold, such as 1km, it proves that the congestion length is short, and the first device is only moved in the first direction to determine the cause of the congestion. If the distance is greater than the second threshold, it indicates that the congestion length is long. In order to quickly determine the location of the congestion, the system simultaneously controls the first device and the second device to move in a direction close to each other, that is, the first device moves in the first direction, and the second device moves in the second direction, and the second direction is the opposite direction of the first direction.

[0071] In particular, if the second device is also in the second state, continue to obtain the early warning device adjacent to the second device in the first direction, and repeat the process until the early warning device in the first state is obtained, and the early warning device with the rearmost position is defined as the first device, and the early warning device in the first state is defined as the second device.

[0072] Step S5: Locate the target location based on the first device or the second device, where the target location is the starting point of the congestion.

[0073] Step S6: Analyze the best monitoring picture of the target location to identify the cause of congestion.

[0074] When the first device moves to a location where the vehicle speed is greater than or equal to the first threshold, or when the second device moves to a location where the vehicle speed is less than the first threshold, it is considered that the location where the congestion occurs, that is, the target location, has been reached. Finally, the system determines the cause of the congestion by obtaining the best monitoring picture of the target location. This embodiment divides the causes of congestion into obstacle congestion and phantom congestion. Obstacle congestion refers to the appearance of obstacles on the road that hinder normal driving, thus causing congestion. Phantom congestion refers to congestion caused by the absence of obstacles but due to large traffic volume and vehicles at the head of the traffic flow changing lanes, driving slowly or braking.

[0075] The present invention first arranges a plurality of early warning devices at intervals on the highway, and installs speed measuring devices in the early warning devices, so as to realize vehicle speed measurement at a plurality of locations. On this basis, if the early warning device detects that the driving speed of a plurality of vehicles is too low, it is considered that congestion has occurred. In order to quickly obtain a picture of the location where the congestion occurs, the present invention also determines whether to move only the first device to determine the location of the congestion, or to move the first device and the second device at the same time to determine the location of the congestion according to actual conditions. In this way, the early warning device can also automatically move toward the direction of the congestion location and determine the cause of the congestion without manual operation, thereby greatly improving the convenience of using the early warning device.

[0076] The early warning device of the present invention records video at a smaller first frame rate under normal conditions, which can reduce the size occupied by the recorded video and save storage resources. When congestion is detected, the video is recorded at a higher second frame rate, thereby obtaining a more detailed congestion scene video, which is convenient for subsequent review and analysis.

[0077] It is particularly noteworthy that the patrol robot in the present invention can automatically move to the congested location according to the vehicle speed and automatically determine the cause of the congestion, thereby solving the problem of the inconvenience of using the patrol robot.

[0078] In this embodiment, obtaining the driving speed of the vehicle includes the following steps:

[0079] The brightness in the best monitoring picture is obtained. If the brightness is greater than or equal to the third threshold, a vehicle tracking algorithm is used to obtain the vehicle position and vehicle information in the best monitoring picture, and the best monitoring picture is divided into multiple first sub-areas. A positioning area is set in the first sub-area to locate the first time point when the vehicle appears in the positioning area.

[0080] Here, the monitoring screen is first converted into a grayscale image, and the pixel value of each pixel in the grayscale image is between 0-255. Then, the average grayscale value of the entire optimal monitoring screen is obtained as the brightness. If the brightness is greater than the third threshold, it indicates that it is currently daytime, because the grayscale value in the screen will be larger during the day. In the case of daytime, due to the good light, a vehicle tracking algorithm such as yolov8 can be used to track the vehicle and obtain the vehicle license plate number at the same time; then, this embodiment divides the optimal monitoring screen into the first sub-area A1-A9, referring to Figure 2 As shown in B, Figure 2 The vehicle in the image moves from right to left, so the best position is set as the first sub-area A7, and the first time point when the vehicle appears in the first sub-area A7 is obtained.

[0081] If the brightness is less than the third threshold, multiple grayscale levels are set, each grayscale level corresponds to a grayscale range, and the grayscale level is allocated to the first sub-area according to the grayscale average value of the first sub-area. The first sub-area in which the grayscale level changes in a predetermined order in the best monitoring picture is defined as the focus area. If the focus areas appear from right to left in the best monitoring picture, the focus areas are analyzed to obtain the vehicle information contained therein, and the first time point when the vehicle appears in the positioning area is obtained.

[0082] If the brightness is too low, it is considered that it is night. Due to the presence of car lights and the fact that the device is installed on the railing, it will be illuminated by the car lights, and there will be more noise in the night monitoring screen, which may make the vehicle tracking algorithm unable to locate the vehicle or lose the vehicle being tracked. In order to accurately obtain vehicle information, this embodiment first sets 4 gray levels. For example, the gray range corresponding to the gray level is 0-65. When the average gray value of the first sub-area A1 is 52, it will be classified as gray level 1. Figure 2 It can be seen that the vehicle will first appear in the first sub-area A6, so the average grayscale value of the first sub-area A6 will change from small to large and then from large to small, and the grayscale level will also experience the same process, and the corresponding predetermined order is from small to large and then from large to small.

[0083] According to the above analysis, when a vehicle is driving normally on the road and passes through the best monitoring screen, the corresponding focus areas are the first sub-areas A4, A5 and A6. By cutting out these first sub-areas in the monitoring screen for identification, the vehicle information can be obtained. The identification method can be a differential algorithm, a CNN neural network, etc.; due to the existence of multiple focus areas, the license plate cannot be identified in the first sub-area A4, but it can also be identified in the first sub-area A5; in particular, when the vehicle passes through the best monitoring screen via the emergency lane, the corresponding focus areas are the first sub-areas A5, A6, A7, A8 and A9. Therefore, by obtaining the appearance names and order of the focus areas in the present invention, it is also possible to determine whether the vehicle is traveling in the emergency lane;

[0084] After capturing the vehicle information, the first time point when the vehicle passes the warning device can be determined by tracking the change in the brightness level of the first sub-area. For example, if the vehicle information is captured in the first sub-area A4, and the gray level of the first sub-area A4 decreases, and then the gray level of the second sub-area A7 increases, even if the vehicle information is not captured in the first sub-area A7, it is considered to be the same vehicle as the first sub-area A4.

[0085] Based on the radar acquisition of the driving speed and the second time point when the vehicle passes the warning device, the first time point is matched with the second time point to match the vehicle information and the driving speed.

[0086] If the first time point is 18:00, the vehicle information is license plate number E, and the early warning device detects that the vehicle speed is 115km / h at 18:00, then the driving speed of license plate number E is 115km / h.

[0087] In this embodiment, identifying the cause of congestion includes the following steps:

[0088] The best monitoring image is projected onto a plane to obtain a road plan. The early warning device captures the vehicle's moving trajectory based on preset feature points and maps it into the road plan. A coordinate system is established in the road plan, and the road plan is cut into multiple second sub-areas.

[0089] This embodiment uses the following method for projection: first, specify the length of the road virtual line and the distance between the two virtual lines. Then, the length and width of the road in the monitoring image can be inferred based on the length and distance, thus realizing the generation of the road plan. The vehicle feature point is also a feature point for setting. The vehicle feature point of this embodiment can be the headlight, body color and license plate light. By detecting the moving trajectory of the vehicle feature point in the best image and mapping it to the road plan, the following can be obtained: Figure 3 The moving track L is shown in A. In this embodiment, the road plan is further divided into a plurality of grids, namely, second sub-areas.

[0090] Generate the direction vector of the nth second sub-region based on the first formula , the first formula is: ,in, is the coordinate of the preset feature point when it appears for the last time in the nth second sub-region, is the coordinate of the preset feature point when it first appears in the nth second sub-region.

[0091] Specifically, since the video is composed of multiple frames of static images, the image frame time when the vehicle feature point first appears in the nth second sub-region is obtained, thereby obtaining the corresponding coordinate p1, and the image frame time when the vehicle feature point last appears in the nth second sub-region is obtained, thereby obtaining the corresponding coordinate p2. The direction vector of the vehicle in the nth second sub-region can be obtained through p1-p2. The direction vector finally obtained is as follows Figure 3 The individual grids in the diagram are indicated by arrows.

[0092] A reference vector is set along a first direction in a road plan, and the angle between the direction vector and the reference vector is calculated. If the angle is greater than a preset value, the direction vector is marked as an avoidance vector. The ratio of the number of avoidance vectors and the number of direction vectors for the same vehicle is calculated, and the ratio is used as an offset. If the offset is greater than a fourth threshold, it is defined as an avoidance vehicle. If the number of avoidance vehicles is greater than a fifth threshold, it is considered that the cause of the congestion is obstacle congestion, otherwise it is phantom congestion.

[0093] The reference vector is set as Figure 3 As shown in F, the angle between each direction vector and the reference vector is then calculated. Calculating the angle between vectors is basic knowledge in this field and will not be introduced here. After obtaining the angle between each direction vector, the direction vector with an angle greater than 45 degrees is defined as the avoidance vector. Finally, the number of second sub-areas occupied by the avoidance vector, such as 6, and the ratio of the number of all second sub-areas including the direction vector, such as 13, are obtained. Then the ratio is 6 / 13≈0.46, and 0.46 is set as the offset of the vehicle. The fourth threshold is set to 0.3, and since the offset is greater than the fourth threshold, the vehicle is set as an avoidance vehicle. If multiple vehicles, such as 50 vehicles, are judged as avoidance vehicles here, it is considered that there is an obstacle here, and the cause of the congestion is considered to be obstacle congestion.

[0094] In this embodiment, if the gray level of the first sub-area increases and does not change any more, it is defined as an illegal parking area, the illegal parking area is analyzed to obtain the vehicle information contained therein, the vehicle information is uploaded and an illegal parking warning is issued.

[0095] When this happens, it indicates that a vehicle has been staying in a first sub-area. Therefore, the system determines that the vehicle is an illegally parked vehicle. By analyzing the illegally parked area, the system obtains vehicle information, such as the license plate number, and sends out an audible and visual alarm to remind the vehicle to leave as soon as possible.

[0096] In this embodiment, obtaining the best monitoring picture includes the following steps:

[0097] A target image excluding vehicles is obtained, a pixel range is set, pixel points whose pixel values ​​in the target image are within the pixel range are defined as road pixel points, a road contour is identified based on the road pixel points, and all pixel points within the road contour are defined as road pixel points.

[0098] The color of the highway is fixed, so we can get which pixels in the video are road pixels based on the pixel values. In addition, since there are lane lines in the lanes, the lane lines are white and located within the road surface. Therefore, after obtaining the road surface pixels, we get the road contour based on the road surface pixels, and locate all the pixels within the road contour as road pixels. In this way, the pixels that make up the lane lines are also divided into road pixels.

[0099] The target image is split into multiple third sub-areas along the horizontal direction, and the evaluation value of the current shooting angle is calculated based on the second formula , the second formula is: , where M is the number of the third sub-regions, and are the total number of pixel points and the number of road pixel points in the mth third sub-region, respectively; the warning device is rotated in the horizontal plane to obtain a new shooting angle; the evaluation value of the new shooting angle is obtained by the second formula; this step is repeated until the number of rotations reaches a predetermined number; the warning device is rotated to a shooting angle corresponding to the highest evaluation value, which is defined as the first shooting angle;

[0100] like Figure 4 As shown, the target image is split into the third sub-area B1, B2 and B3, first the number of road pixels in the third sub-area B1 is obtained, and then the total number of pixels included in the third sub-area B1 is obtained. The ratio of the road area in the third sub-area B1 can be determined by the ratio of the two. The ratio of the road area in the third sub-area B2 and B3 is calculated in this way, and finally the average of the three is obtained to obtain the evaluation value. It can be inferred that the larger the evaluation value, the more road images are included in the picture, which means that the shooting angle is more appropriate. Therefore, the best angle of view in the horizontal plane can be obtained by rotating the warning device.

[0101] The early warning device rotates in a vertical plane at a first shooting angle, and obtains evaluation values ​​of multiple shooting angles based on a second formula, and defines the shooting angle with the evaluation value closest to the critical value as the best monitoring picture.

[0102] After determining the best horizontal shooting angle, the warning device is rotated up and down in the vertical plane, and the evaluation value of the shooting picture obtained after each rotation is calculated, and the shooting angle with the largest evaluation value is set as the best monitoring picture. In particular, the angle with the evaluation value closest to the critical value is set as the best picture. The reason is that when rotating up and down, when the angle is facing the ground, the road occupancy area will reach 100%, but this angle cannot capture vehicles coming from a distance. Therefore, a critical value is set, such as 1 / 3, that is, when rotating vertically, when the road occupancy ratio reaches 2 / 3, it is the best.

[0103] In this embodiment, in the first state, the warning device records a road video at a second frame rate at a first time interval, which is defined as a sample video, and obtains a target image from the sample video.

[0104] The road video is recorded at a high frame rate at regular intervals. When the number of vehicles is large, it is easier to obtain a target image that does not contain any vehicles from the high frame rate road video.

[0105] In this embodiment, uploading the road video includes the following steps:

[0106] Each warning device is numbered along the first direction. Whenever the road video recorded in the warning device reaches the first duration, the road video is uploaded. When the warning device numbered k completes uploading, the transmission duration is recorded. The warning device numbered k+1 starts uploading the road video after the delay time. The delay time Calculated based on the third formula, the third formula is: ,in, It is the first duration of the early warning device numbered k. is the delay time of the early warning device numbered k.

[0107] Here the early warning devices are numbered 1.2.3..., and the delay time for the first transmission is set to 0s. When the early warning device numbered 1 records the road video and completes the upload, the recorded transmission time is 20s. The early warning device numbered 2 starts uploading its own recorded road video after 20*2+0=40s. The delay time is 40s, and the recorded transmission time is 15s. The early warning device numbered 3 starts uploading its own recorded road video after 15*2+40=70s. In this way, each early warning device can upload videos in a dispersed manner, which can reduce the receiving pressure on the server.

[0108] like Figure 5As shown, the present invention also provides a full-route inspection, monitoring and early warning system for highway maintenance management, which is used to implement the above-mentioned full-route inspection, monitoring and early warning method for highway maintenance management, and the system includes:

[0109] An adjustment module is used to adjust the position of the early warning device and the acquisition terminal to obtain the best monitoring picture;

[0110] The detection module records and uploads the road video at a first frame rate when the early warning device is in the first state, and detects the driving speed of the passing vehicles at the same time. When it is detected that the driving speed of multiple consecutive vehicles is less than the first threshold, the early warning device switches itself to the second state and defines itself as the first device. The first device records the road video at a second frame rate in the second state, and the first frame rate is less than the second frame rate.

[0111] A moving module, defining a warning device that is adjacent to the first device in a first direction as a second device, where the first direction is the driving direction of the highway, and if the second device is in a first state, obtaining a distance between the first device and the second device, and if the distance is less than a second threshold, controlling the first device to move toward the first direction, otherwise controlling the first device and the second device to move toward each other;

[0112] The analysis module locates the target location based on the first device or the second device, and the target location is the starting point of the congestion. The best monitoring picture of the target location is analyzed to identify the cause of the congestion. The analysis module also has basic data management functions, monitoring data and content publishing functions, image report production functions, data analysis functions and comprehensive early warning functions. When the analysis module finds that the data parameter meter reading or reaches a dangerous value, it can remotely control the early warning device to sound and light alarm and broadcast warning words, and report to the management department and management personnel by SMS, voice, email, etc.

[0113] Therefore, through the above-mentioned highway safety early warning system, it is possible to realize the automatic collection, transmission, storage, processing, analysis and early warning of various environmental data, driving data and path data along the entire route and in all directions of the highway. Based on this technical foundation, the maintenance and safety management functional departments at all levels of highways can view various online monitoring data and images of highway management in real time through the network, so that relevant personnel on the monitoring command and management platform can remotely understand the detailed situation of the highway even if they are not on site.

[0114] The present invention also provides a full-route inspection, monitoring and early warning device for highway maintenance management. The system is used to implement the above-mentioned full-route inspection, monitoring and early warning method for highway maintenance management. The device includes:

[0115] A power system, which includes a wind blade wheel, a travel drive wheel integrated device, a battery and a solar panel. The solar panel and the wind blade wheel charge the battery through solar energy and wind energy. The battery is used to supply power to the travel drive wheel integrated device. The early warning device moves through the travel drive wheel integrated device.

[0116] Radar system, used to collect vehicle speed;

[0117] A recording system, including a high-definition infrared camera and a recorder, wherein the high-definition infrared camera is used to record and generate road videos, and the recorder is used to record sounds in the road videos;

[0118] A sensing system, including a temperature sensor and a vibration actuator, wherein the temperature sensor is used to detect the road environment temperature and the vibration sensor is used to detect the road vibration;

[0119] The transmission system includes a 5G transmission chip and a wireless network transmission chip, through which the road video is uploaded;

[0120] Lighting system, including yellow warning lights, red warning lights and outline light strips;

[0121] A voice system, including a speaker, through which voice and warnings are played externally;

[0122] Remote control system, used for remote control of early warning equipment.

[0123] The early warning device of the present invention is provided with solar panels and small wind turbines in the hardware part, which can supplement electric energy during use. Through the travel drive system, the early warning device can be moved along the guardrails on both sides or in the middle of the road; through the provision of various sensors and network transmission modules, the early warning device can realize data collection, monitoring and transmission during the patrol process, and through the provision of microphones and speakers, the early warning device can realize remote intercom between the monitoring command management platform. For example, when illegal parking, scattering, rolling objects on the slope, rain, snow and ice on the road may affect the driving safety of pedestrians, the system collects and alarms, or when vehicles on the road are congested, driving abnormally, or traffic accidents occur, data collection and alarms are carried out in time, and remote management personnel can broadcast on the spot and conduct on-site command management; in summary, the present invention can realize all-round monitoring and management of the entire highway through movable early warning equipment and collection terminals at various fixed points, and timely collect, organize and transmit management data such as real-time changes in road conditions, weather and environmental impacts, driving safety status, deformation and change usage status of tunnels, bridges, high slopes, sharp bends, etc. to the monitoring command management platform.

[0124] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0125] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0127] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for full-route inspection, monitoring and early warning of highway maintenance management, characterized in that: include: Fixed-point collection terminals and early warning devices are arranged at intervals on the highway. The collection terminals and early warning devices adjust their positions to obtain the best monitoring images. When the early warning device is in the first state, a road video is recorded and uploaded at a first frame rate, and the driving speed of passing vehicles is detected at the same time. When it is detected that the driving speed of multiple consecutive vehicles is less than a first threshold, the early warning device switches itself to the second state and defines itself as the first device. The first device records the road video at a second frame rate in the second state, and the first frame rate is less than the second frame rate. defining a warning device that is located in a first direction and adjacent to the first device as a second device, wherein the first direction is a driving direction of the highway; If the second device is in the first state, obtaining a distance between the first device and the second device, and if the distance is less than a second threshold, controlling the first device to move toward the first direction, otherwise controlling the first device and the second device to move toward each other; Locating a target position based on the first device or the second device, where the target position is a congestion starting point; The optimal surveillance picture of the target location is analyzed to identify the cause of congestion.

2. The method according to claim 1, characterized in that Obtaining the driving speed of the vehicle comprises the following steps: Obtaining the brightness in the optimal monitoring picture, and if the brightness is greater than or equal to a third threshold, using a vehicle tracking algorithm to obtain the vehicle position and vehicle information in the optimal monitoring picture, splitting the optimal monitoring picture into a plurality of first sub-areas, setting a positioning area in the first sub-areas, and positioning the first time point at which the vehicle appears in the positioning area; If the brightness is less than the third threshold, multiple gray levels are set, each gray level corresponds to a gray range, the gray level is allocated to the first sub-region according to the gray average value of the first sub-region, the first sub-region in which the gray levels in the optimal monitoring image change in a predetermined order is defined as a focus area, and if the focus area appears in sequence from right to left in the optimal monitoring image, the focus area is analyzed to obtain the vehicle information contained therein, and the first time point at which the vehicle appears in the focus area is obtained; The driving speed and the second time point when the vehicle passes the warning device are acquired based on the radar, and the first time point is matched with the second time point to match the vehicle information with the driving speed.

3. The method according to claim 2, characterized in that Identifying the cause of congestion involves the following steps: The optimal monitoring picture is projected onto a plane to obtain a road plane map, the early warning device captures the movement trajectory of the vehicle based on preset feature points and maps it into the road plane map, establishes a coordinate system in the road plane map, and cuts the road plane map into a plurality of second sub-areas; Generate the direction vector of the nth second sub-region based on the first formula , the first formula is: ,in, is the coordinate of the preset feature point when it appears for the last time in the nth second sub-region, is the coordinates of the preset feature point when it first appears in the nth second sub-region; A reference vector is set along the first direction in the road plan, and an angle between the direction vector and the reference vector is calculated; if the angle is greater than a preset value, the direction vector is marked as an avoidance vector; a ratio of the number of avoidance vectors and the number of direction vectors for the same vehicle is calculated, and the ratio is used as an offset; if the offset is greater than a fourth threshold, it is defined as an avoidance vehicle; if the number of avoidance vehicles is greater than a fifth threshold, it is considered that the cause of congestion is obstacle congestion, otherwise it is phantom congestion.

4. The method according to claim 2, characterized in that: If the gray level of the first sub-area increases and does not change any more, it is defined as an illegal parking area, the illegal parking area is analyzed to obtain the vehicle information contained therein, the vehicle information is uploaded and an illegal parking warning is issued.

5. The method according to claim 1, characterized in that Obtaining the optimal monitoring picture includes the following steps: Obtain a target image that does not contain a vehicle, set a pixel range, define pixel points in the target image whose pixel values ​​are within the pixel range as road pixel points, identify a road contour based on the road pixel points, and define all pixel points within the road contour as road pixel points; The target image is split into a plurality of third sub-areas in the horizontal direction, and the evaluation value of the current shooting angle is calculated based on the second formula , the second formula is: , where M is the number of the third sub-regions, and are respectively the total number of pixel points in the mth third sub-area and the number of pixel points on the road, rotating the warning device in a horizontal plane to obtain a new shooting angle of view, continuing to obtain the evaluation value of the new shooting angle of view by using the second formula, repeating this step until the number of rotations reaches a predetermined number, rotating the warning device to a shooting angle corresponding to the highest evaluation value, which is defined as a first shooting angle of view; The early warning device rotates in a vertical plane at the first shooting angle, and obtains the evaluation values ​​of multiple shooting angles based on the second formula, and defines the shooting angle with the evaluation value closest to the critical value as the optimal monitoring picture.

6. The method according to claim 5, characterized in that In the first state, the early warning device records the road video at the second frame rate at a first time interval, which is defined as a sample video, and obtains the target image from the sample video.

7. The method according to claim 1, characterized in that Uploading the road video includes the following steps: Each warning device is numbered along the first direction. Whenever the road video recorded in the warning device reaches the first duration, the road video is uploaded. When the warning device numbered k completes uploading, the transmission duration is recorded. The warning device numbered k+1 starts uploading the road video after a delay time. The delay time Calculated based on the third formula, the third formula is: ,in, is the transmission duration of the early warning device numbered k, is the delay time of the early warning device numbered k.

8. A full-route inspection, monitoring and early warning system for highway maintenance management, used to implement the method according to any one of claims 1 to 7, characterized in that: include: An adjustment module, which is used to adjust the positions of the early warning equipment and the acquisition terminal to obtain the best monitoring picture; The detection module records and uploads a road video at a first frame rate when the early warning device is in a first state, and detects the driving speed of passing vehicles at the same time. When it is detected that the driving speed of multiple consecutive vehicles is less than a first threshold, the early warning device switches itself to a second state and defines itself as a first device. The first device records the road video at a second frame rate in the second state, and the first frame rate is less than the second frame rate. a moving module, defining an early warning device adjacent to the first device in a first direction as a second device, wherein the first direction is a driving direction of the expressway, and if the second device is in the first state, obtaining a distance between the first device and the second device, and if the distance is less than a second threshold, controlling the first device to move toward the first direction, otherwise controlling the first device and the second device to move toward each other; The analysis module locates a target position based on the first device or the second device, the target position being a starting point of congestion, and analyzes the optimal monitoring picture of the target position to identify the cause of congestion.

9. A full-route inspection, monitoring and early warning device for highway maintenance management, used to implement the method according to any one of claims 1 to 7, characterized in that: include: A power system, the power system comprising a wind blade wheel, a travel drive wheel integrated device, a battery and a solar panel, the solar panel and the wind blade wheel charge the battery through solar energy and wind energy, the battery is used to supply power to the travel drive wheel integrated device, and the early warning device moves through the travel drive wheel integrated device; Radar system, used to collect vehicle speed; A recording system, including a high-definition infrared camera and a recorder, wherein the high-definition infrared camera is used to record and generate road videos, and the recorder is used to record sounds in the road videos; A sensing system, including a temperature sensor and a vibration actuator, wherein the temperature sensor is used to detect the road environment temperature and the vibration sensor is used to detect the road vibration; A transmission system, including a 5G transmission chip and a wireless network transmission chip, through which the road video is uploaded; Lighting system, including yellow warning lights, red warning lights and outline light strips; A voice system, including a speaker, through which voice and warnings are played externally; Remote control system, used for remote control of early warning equipment.

Images