A Method for Monitoring High-Speed Tunnel Traffic Conditions Based on Tunnel Patrol Robots

Abstract

The present invention discloses a method for monitoring the traffic status of a high-speed tunnel based on a tunnel patrol robot. The tunnel patrol robot acquires continuous-frame patrol images in the tunnel according to a preset acquisition task, and at the same time, after patrolling in the tunnel at a fixed camera angle according to the preset patrol task, relationship M is obtained; the continuous-frame patrol images are input into a multi-task head neural network model for detection, and vehicle target detection result information and lane line detection result information are output; the vehicle target detection result information is tracked based on the continuous-frame patrol images to obtain target vehicle information; ID information is assigned to the target vehicle information and the target vehicle information is framed by a detection box; according to relationship M, the target vehicle information, and the lane line detection result information, the driving state of the vehicle corresponding to the target vehicle information is analyzed, and an alarm is given to the vehicle with an abnormal driving state; full-tunnel coverage is achieved to realize comprehensive monitoring.

Description

Technical Field

[0001] The present invention relates to the field of traffic monitoring, and particularly to a method for monitoring the traffic status of a high-speed tunnel based on a tunnel patrol robot. Background Art

[0002] In recent years, with the rapid economic development of our country, the volume of urban roads has been unable to meet the increasing vehicle travel. As a supplement to urban traffic, tunnels play an important role. While tunnel projects bring traffic convenience, there are also some safety problems. Especially, the road in the tunnel is closed and there are no intersections, which leads to easy speed increase of vehicles, and then vehicle safety problems such as collisions and hitting the edge occur. Especially, the rescue conditions in the tunnel are poor, and secondary additional accidents such as rear-end collisions are likely to occur during the rescue stage. Therefore, when a traffic accident occurs in the tunnel, reporting the accident signal to the traffic management and rescue departments in the first time can win time for rescue; more importantly, it can give a warning to the vehicles driving into the tunnel subsequently to avoid secondary accidents.

[0003] At present, traffic monitoring in the tunnel adopts the method of deploying monitoring cameras. However, the monitoring cameras have the disadvantage of being immovable, and their deployment density is generally limited, and a large number of sections between adjacent monitors cannot be covered. While using a tunnel patrol robot for traffic monitoring in the tunnel can expand the monitoring coverage to the entire tunnel through the movement of the patrol robot, which has a more comprehensive coverage than the monitoring camera scheme. Therefore, the tunnel patrol robot is an efficient and convenient traffic safety detection scheme in the tunnel.

[0004] Existing other technical solutions mainly aim at the situation of fixed sensors in the tunnel, such as cameras, vibration, acceleration, smoke, sound and other sensors at fixed positions in the tunnel. Since the sensors of the tunnel patrol robot are not at fixed positions but will move along with the tunnel patrol robot, the existing technical solutions are not applicable;

[0005] Therefore, for those skilled in the art, it is particularly difficult to efficiently and accurately identify vehicle safety events such as vehicle collisions and vehicle congestion for the monitoring of moving tunnel images. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for monitoring the traffic status of a high-speed tunnel based on a tunnel patrol robot, which solves the above-mentioned technical problems pointed out in the prior art.

[0007] The present invention provides a method for monitoring the traffic status of a high-speed tunnel based on a tunnel patrol robot, including the following operating steps:

[0008] S10: The tunnel patrol robot acquires continuous frame patrol images in the tunnel according to a preset acquisition task;

[0009] S20: The tunnel patrol robot patrols in the tunnel at a fixed camera angle according to the preset patrol task: obtaining the displacement distance S' of the tunnel patrol robot in real time, and simultaneously analyzing and obtaining the displacement distance D' of each pixel in the consecutive frame patrol images; analyzing and obtaining the relationship M based on the displacement distance S' of the tunnel patrol robot and the pixel displacement distance D'; obtaining the running state of the objects in the consecutive frame patrol images through the relationship M; the running state includes stationary, moving, and the direction of movement; the relationship is the relationship between the pixel displacement distance of each stationary object in two adjacent frame patrol images and the displacement distance of the corresponding stationary object in the real scene.

[0010] S30: Input the consecutive frame patrol images into a multi-task head neural network model for detection, and output the vehicle target detection result information and the lane line detection result information.

[0011] S40: Track the vehicle target detection result information based on the consecutive frame patrol images to obtain the target vehicle information; assign ID information to the target vehicle information, and at the same time perform bounding box selection on the target vehicle information.

[0012] S50: Calculate the displacement distance error α corresponding to the target vehicle information according to the relationship M, and then use the displacement distance error α, the target vehicle information, and the lane line detection result information to determine whether the target vehicle information meets the conditions of the abnormal vehicle driving state; if so, issue an alarm.

[0013] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0014] Analyzing the above-mentioned method for monitoring the high-speed tunnel traffic state based on a tunnel patrol robot provided by the present invention, it can be seen that in specific applications, first, the tunnel patrol robot collects consecutive frame patrol images in the tunnel according to the preset acquisition task; the tunnel patrol robot realizes full-tunnel coverage to achieve comprehensive monitoring; at the same time, according to the obtained displacement distance S' of the tunnel patrol robot and the analysis of the displacement distance D' of each pixel in the consecutive frame patrol images, the relationship M is obtained, so as to reflect the running state of the vehicles in the consecutive frame patrol images, comprehensively monitoring the state information of the vehicles in the tunnel, and improving the patrol monitoring ability of the tunnel patrol robot.

[0015] Further, the continuous frame patrol images are input into a multi-task head neural network model for detection to obtain vehicle target detection result information and lane line detection result information; feature extraction is performed on the patrol images, increasing the extraction speed and the extraction accuracy of information features; by tracking the vehicle target detection result information, target vehicle information is obtained; the target vehicle information is boxed by a detection frame; the target vehicle information is extracted more accurately and quickly; finally, according to relationship M, the error α of the displacement distance of the target vehicle is calculated through M, and the driving states of the vehicles corresponding to the respective target vehicle information are judged by using the displacement distance error α, the target vehicle information, and the lane line detection result information; the obtained driving states of the vehicles can supervise the highway tunnel to prevent abnormal driving and cause congestion in the highway tunnel. Description of the Drawings

[0016] Figure 1 It is a simple flowchart of the high-speed tunnel traffic state monitoring method based on a tunnel patrol robot provided by the present invention;

[0017] Figure 2 It is a flowchart of the high-speed tunnel traffic state monitoring method based on a tunnel patrol robot provided by the present invention;

[0018] Figure 3 It is an overall step flowchart of the high-speed tunnel traffic state monitoring method based on a tunnel patrol robot provided by the present invention;

[0019] Figure 4 It is a step flowchart of building a multi-task head neural network model for the high-speed tunnel traffic state monitoring method based on a tunnel patrol robot provided by the present invention;

[0020] Figure 5 It is a step flowchart of tracking and obtaining target vehicle information for the high-speed tunnel traffic state monitoring method based on a tunnel patrol robot provided by the present invention;

[0021] Figure 6 It is a step flowchart of obtaining the driving states of the vehicles corresponding to the respective vehicle target detection result information for the high-speed tunnel traffic state monitoring method based on a tunnel patrol robot provided by the present invention;

[0022] Figure 7 It is a step flowchart of judging whether the vehicle is in an abnormal driving state for the high-speed tunnel traffic state monitoring method based on a tunnel patrol robot provided by the present invention;

[0023] Figure 8 It is a structural diagram of YOLOv8 for the high-speed tunnel traffic state monitoring method based on a tunnel patrol robot provided by the present invention;

[0024] Figure 9 Structural diagram of Ultra Fast Lane Detection for a method of monitoring high - speed tunnel traffic status based on a tunnel patrol robot provided by the present invention;

[0025] Figure 10 Structural diagram of YOLO - traffic for a method of monitoring high - speed tunnel traffic status based on a tunnel patrol robot provided by the present invention;

[0026] Figure 11 Structural diagram of deepsort for a method of monitoring high - speed tunnel traffic status based on a tunnel patrol robot provided by the present invention. Detailed implementation manners

[0027] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0028] The present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0029] As Figure 1 、 Figure 2 or Figure 3 shown, the present invention proposes a method for monitoring the traffic status of a high - speed tunnel based on a tunnel patrol robot, including the following operating steps:

[0030] Step S10: The tunnel patrol robot acquires continuous - frame patrol images in the tunnel according to a preset acquisition task;

[0031] Step S20: The tunnel patrol robot patrols in the tunnel at a fixed camera angle according to a preset patrol task: The displacement distance S' of the tunnel patrol robot obtained in real - time is acquired, and at the same time, the displacement distance D' of each pixel in the continuous - frame patrol images is analyzed; According to the displacement distance S' of the tunnel patrol robot and the pixel displacement distance D', the relationship M is analyzed and obtained (the relationship M is the relationship between the pixel displacement distance of each stationary object in two adjacent patrol images and the displacement distance of the corresponding stationary object in the real - world scene); The running state of the objects in the continuous - frame patrol images is obtained through the relationship M; The running state includes stationary, moving, and the direction of movement;

[0032] The preset patrol task includes a preset moving speed v of the patrol robot and a preset interval time t between two adjacent frames of images to be detected;

[0033] The calculation method of the relationship M is:

[0034] M = S' / D';

[0035] It should be noted that in the embodiment of the present application, the tunnel patrol robot is controlled to fix the camera angle and patrol in the target tunnel at a speed of 1 m / s, photograph the stationary object A, and then take another photograph after an interval of 0.5 s;

[0036] In the two photographed pictures, the displacement of the pixel position of the stationary object A is D, and the displacement of the patrol robot in the real scene is S (here it is 0.5 m), then the relationship M = S / D can be obtained; (The relationship M is equivalent to the patrol robot fixing the camera angle and being stationary, taking pictures of the object A moving at a speed of 1 m / s twice at an interval of 0.5 s, and obtaining the relationship between the pixel displacement distance of the object A in the two photographed images and the displacement distance of the object A in the real scene)

[0037] The patrol robot repeats the above steps according to different speeds and different shooting intervals, records multiple groups of relationships M to assist the patrol robot in perceiving the motion state of objects in the target tunnel scene;

[0038] It should be noted that in areas with relatively dim light, the patrol robot will take pictures at shorter intervals to ensure that the patrol robot can perceive the motion state of objects in this area when the light is dim. In addition, when the tracked vehicle is larger than the threshold (which can be set according to actual situations), it indicates that the current traffic flow is relatively large. Similarly, the patrol robot will also take pictures at shorter intervals and increase the patrol speed to enhance the monitoring ability of the vehicle.

[0039] Step S30: Input the continuous-frame patrol images into a multi-task head neural network model for detection, and output the vehicle target detection result information and the lane line detection result information;

[0040] The vehicle target detection result information includes the vehicle target detection result information and the vehicle target running direction information; the vehicle target detection result information includes the vehicle target and the coordinate information corresponding to the vehicle target;

[0041] It should be noted that in the above embodiment of the present application, by obtaining the output results of the vehicle target detection task head, the number N of the vehicle target detection result information, the coordinate information (x1, y1, x2, y2) corresponding to each vehicle target detection result information, and the orientation information direction of each vehicle target detection result information are obtained, that is, N groups of (x1, y1, x2, y2, direction) information, where N represents the number of vehicles detected in the input image, and each group of information represents the abscissa of the upper left corner, the ordinate of the upper left corner, the abscissa of the lower right corner, the ordinate of the lower right corner, and the vehicle direction of the vehicle in the image;

[0042] Obtain the output result of the lane line detection task header, that is, L×M groups of (x, y), where L represents the number of lane lines detected in the input image, and each lane line has the horizontal and vertical coordinate information of M points. Then, based on the least squares method, fit the straight line expressions of each lane line (i.e., the above-mentioned lane line detection result information) for the points on the extracted lane lines.

[0043] Step S40: Track the vehicle target detection result information based on the continuous frame patrol images to obtain target vehicle information; assign ID information to the target vehicle information, and at the same time perform detection box selection on the target vehicle information.

[0044] Step S50: Calculate the displacement distance error α corresponding to the target vehicle information according to the relationship M, and then use the displacement distance error α, the target vehicle information, and the lane line detection result information to determine whether the target vehicle information meets the conditions of the abnormal vehicle driving state; if so, give an alarm; the conditions of the abnormal vehicle driving state include vehicle stop state, vehicle reverse state, vehicle congestion state, vehicle lane change state, and vehicle abnormal direction change state.

[0045] It should be noted that in the above embodiments of the present application, through step S10, the tunnel patrol robot performs patrol collection in the tunnel according to the preset collection task to obtain continuous frame patrol images, obtains the displacement distance S' of the tunnel patrol robot through the images, and at the same time analyzes and obtains the pixel displacement distance D' of each pixel in the continuous frame patrol images; and uses the above information to calculate the relationship M between the pixel displacement distance of each stationary object in two adjacent frame patrol images and the displacement distance of the corresponding stationary object in the real scene in step S20, and judges the vehicle running state in the tunnel through M to indicate the size of the traffic flow, and also improves the perception ability of the patrol robot, enabling it to better adapt to different working scenarios; further, in step S30, the continuous frame patrol images are detected and recognized through a multi-task head neural network model to obtain vehicle target detection result information and lane line target information. Among them, the multi-task head neural network model extracts features more comprehensively and clearly; and in step S40, the target vehicle information is screened by tracking the vehicle target detection result information, and a detection box selection operation is performed on the target vehicle information, and the target vehicle information is extracted more accurately and quickly; finally, in step S50, through comprehensive research and judgment analysis of the relationship M, the target vehicle information, and the lane line detection result information, judge the driving state corresponding to each target vehicle information, and the obtained vehicle driving state can supervise the highway tunnel to prevent abnormal driving and cause congestion in the highway tunnel.

[0046] Specifically, such as Figure 4As shown, before inputting the continuous frame patrol images into the multi-task head neural network model for detection in step S30, it also includes designing and training the multi-task head neural network model; the operations for designing and training the multi-task head neural network model include the following steps:

[0047] Step S31: Obtain the original video data; extract frames from the original video data to obtain a plurality of original image data;

[0048] It should be noted that in the above embodiments of the present application, a large amount of original video data in the tunnel is obtained through the tunnel patrol robot and frames are extracted to obtain the original image data, and the original image data is used for the next processing to train the multi-task head neural network model;

[0049] Step S32: Label all the original image data to obtain labeled image data;

[0050] It should be noted that in the above embodiments of the present application, the original image data is manually labeled with targets, that is, the vehicle targets, lane line targets, and vehicle orientation targets in the original image data are manually labeled to obtain the image data after labeling processing (i.e., the above-mentioned labeled image data);

[0051] Step S33: Divide all the labeled image data into a training set and a test set;

[0052] Step S34: Construct a multi-task head neural network model; define the vehicle orientation loss function Loss;

[0053] The vehicle orientation loss function Loss is expressed as:

[0054]

[0055] In the formula, is the probability that the model predicts the sample as a positive example, and y is the sample label. If the sample belongs to a positive example, it takes the value of 1, otherwise it takes the value of 0;

[0056] The multi-task head neural network model includes a backbone network and two task heads; among them, the backbone network extracts the feature information in the image; the two task heads respectively output the vehicle detection result and the lane line detection result; the feature information includes the vehicle target detection result information, the lane line target information, and the vehicle orientation information;

[0057] The multi-task head neural network model is constructed based on the Pytorch deep learning framework;

[0058] It should be noted that in the embodiments of the present application, on the basis of YOLOv8, the original detection head is used as the task head for vehicle target detection, such as Figure 8As shown in the figure; replace the backbone network part (Res blocks) of Ultra Fast Lane Detection with the backbone network of YOLOv8, that is, CSPDarknet, and the rest is used as the task head for lane line detection, such as Figure 9 shown in the figure.

[0059] Based on this design, in the object detection head, a new output for predicting the vehicle direction is added (in the image captured by the patrol robot, the front of the vehicle facing the patrol robot is the positive direction, and the rear of the vehicle facing the patrol robot is the negative direction).

[0060] The constructed multi-task head neural network (YOLO-traffic) can perform vehicle object detection, vehicle orientation detection, and lane line detection on the input video frames, such as Figure 10 shown in the figure. Compared with the method of using multiple models to complete these tasks separately, the model designed in this paper has fewer parameters and can be adapted to a variety of edge devices.

[0061] It should be noted that in this paper, a new prediction output of vehicle orientation is added to the object detection head, and the binary cross-entropy loss function is used. Its positive significance lies in that during the model training process, together with other loss functions, it helps the model adjust the weight parameters to obtain ideal model weight parameters, so that the finally obtained model can correctly predict the vehicle orientation;

[0062] Among them, the calculation method of the loss of the remaining prediction outputs is the same as the loss function used by the model before improvement.

[0063] Step S35: Based on the loss function Loss, use the training set and the test set to train the multi-task head neural network model to obtain a trained multi-task head neural network model;

[0064] It should be noted that the above embodiments of the present application use the training set and the test set obtained in step S33 to train the multi-task model, and use the momentum gradient descent method (Adam algorithm) to continuously adjust and optimize the weight parameters of the neural network, gradually improving the accuracy of the object detection result and the lane line detection result. The training is divided into 3 stages, namely the backbone network training stage, the vehicle object detection task head training stage, and the lane line detection task head training stage.

[0065] In the backbone network training stage, the backbone network, all task heads (vehicle object detection task head and lane line detection task head), and all data sets (vehicle object detection data set and lane line detection data set) are included in the training to obtain a backbone network that is applicable to both the foreign object detection task and the lane line detection task;

[0066] In the training phase of the vehicle target detection task head, freeze the weight parameters of the backbone network and the lane line detection task head, and only update the weight parameters of the vehicle target detection task head; and only use the target detection dataset to train the foreign object target detection task head;

[0067] In the training phase of the lane line detection task head, freeze the weight parameters of the backbone network and the vehicle target detection task head, and only update the weight parameters of the lane line detection task head; and only use the lane line detection dataset to train the lane line detection task head;

[0068] Specifically, as Figure 5 shown, in step S40, based on the consecutive frame patrol images, track the vehicle target detection result information to obtain target vehicle information, including the following operation steps:

[0069] Step S41: Obtain consecutive frame patrol images; create track objects (track objects or the first detection boxes) corresponding to each of the vehicle target detection result information (i.e., the first vehicle target detection result information) in the first frame of the consecutive frame patrol images; and perform track prediction on each of the vehicle target detection result information according to the running state to obtain prediction boxes corresponding to each of the vehicle target detection result information; the above prediction boxes refer to the prediction boxes corresponding to the current vehicle target detection result information in the next frame obtained by performing Kalman filtering prediction on the first detection box after obtaining the first detection box;

[0070] It should be noted that the multi-object tracking algorithm deepsort is used in the embodiment of the present application to track the detection box results of the vehicle targets (i.e., the above vehicle target detection result information) obtained in the above step S30, create corresponding Tracks (track objects) for the results detected in the first frame of the image to be detected; initialize the motion variables of the Kalman filter, and predict the corresponding prediction boxes through the Kalman filter based on the running state of the objects in the consecutive frame patrol images obtained in the above step S20; because the prediction track boxes corresponding to the prediction tracks are obtained at this time, the track objects Tracks must be in the unconfirmed state at this time.

[0071] The above prediction boxes refer to predicting the position of the next frame of the vehicle target detection box (i.e., the above track object) using the Kalman filter, that is, the prediction box.

[0072] Step S42: Determine the vehicle target detection result information of each second frame to-be-detected image in the continuous-frame patrol images as the second vehicle target detection result information; set a second detection box for the second vehicle target detection result information, and perform IOU matching between each second detection box and the prediction box to obtain an IOU matching result, i.e., iou; calculate and obtain a first cost matrix (cost matrix) CM according to the iou matching result.

[0073] The calculation method of the first cost matrix is as follows:

[0074] CM = 1 - iou;

[0075] Step S43: Based on each first cost matrix CM, perform matching between each prediction box and each second detection box through the Hungarian algorithm to obtain a matching result; the matching result includes a prediction box mismatch result, a detection box mismatch result, and a matching success result; and update the variable of the trajectory object corresponding to the matching success result with the detection box corresponding to the matching success result to obtain a first confirmed state trajectory object.

[0076] Among them, the prediction box mismatch result means that the above prediction box cannot match any second detection box (if it still cannot be successfully matched after 30 detections, the trajectory object corresponding to the prediction box needs to be deleted); the above detection box mismatch result means that the above second detection box fails to match any prediction box (then a new trajectory object needs to be created for this second detection box for recognition); the above matching success result means that the above second detection box is successfully matched with any one of the above prediction boxes, which means that the trajectory object corresponding to the successful match between the above second detection box and the above prediction box is successfully matched, that is, the above second detection box is the second frame recognition result of one of the above all trajectory objects.

[0077] The embodiments of the present application take all the first cost matrices obtained in the above step S42 as the input of the Hungarian algorithm to obtain the result of linear matching. At this time, there are three types of results. The first is that the Tracks are mismatched (Unmatched Tracks), and the mismatched Tracks are directly deleted (because this Tracks is in an uncertain state. If it is in a definite state, it can be deleted only after reaching a certain number of consecutive times (the default is 30 times)). The second is that the Detections are mismatched (Unmatched Detections), and such Detections are initialized as a new Tracks (newTracks). Each detected target should be recorded with a track of its state. If a new target appears in the second frame compared with the first frame, a new track should be created for the new target to record its state. The third is that the detection box and the prediction box are successfully paired, and the corresponding Detections are used to update the corresponding Tracks variable through the Kalman filter.

[0078] Step S44: Repeat the above steps S42 - S43 according to the order of the consecutive frame patrol images to determine whether a first confirmed state track object is obtained; if so, perform the next operation; if not, repeat the detection until all consecutive frame patrol images are detected to obtain a plurality of unconfirmed state track objects;

[0079] It should be noted that if the prediction box of a track is successfully matched with the detection box for multiple consecutive times, it can change from the unconfirmed state to the confirmed state (corresponding to 3 consecutive hits in the deepsort structure flow chart, that is, 3 consecutive hits, and the specific number can be set according to the actual situation), as Figure 11 shown;

[0080] The embodiments of the present application repeatedly loop through the above steps S42 - S43 in the order of the images to be detected until confirmed state (confirmed) Tracks appear, and then enter the subsequent steps, otherwise loop until the end of the video frame.

[0081] Step S45: Predict the confirmed state prediction box for the first confirmed state track object, and predict the unconfirmed state prediction box for the unconfirmed state track object; cascade and match the confirmed state prediction box with the detection boxes obtained in each frame to obtain a cascade matching result;

[0082] The cascade matching result includes a successful cascade matching result of the track object, a cascade mismatch result of the detection box, and a cascade mismatch result of the prediction box;

[0083] It should be noted that in the above embodiments of the present application, the boxes corresponding to the Tracks in the confirmed state and the Tracks in the unconfirmed state are predicted through Kalman filtering. The boxes of the Tracks in the confirmed state are cascade-matched with the Detections (before a certain frame is input, all Tracks are in an uncertain state. When this frame is input, the confirmed Tracks appear, and then the detection boxes Detections of the current frame will be cascade-matched with these confirmed Tracks). (Previously, every time a Track was matched, the appearance features and motion information of the Detections were saved. By default, the first 100 frames were saved, and the appearance features and motion information were used for cascade matching with the Detections).

[0084] Step S46: Update the variables of the track object corresponding to the successful result of the cascade matching of the detection box, and obtain the second confirmed state track object;

[0085] Perform IOU matching on the detection box corresponding to the result of the cascade mismatch of the detection box, the prediction box corresponding to the result of the cascade mismatch of the prediction box, the unconfirmed state track object, and the track object corresponding to the mismatch result of the prediction box to obtain the IOU matching result, that is, iou'; calculate the second cost matrix CM' according to the IOU matching result iou';

[0086] The calculation method of the second cost matrix CM' is as follows:

[0087] CM' = 1 - iou';

[0088] It should be noted that there are three possible results after cascade matching. First, the Tracks are matched, and the corresponding Track variables of such Tracks are updated through Kalman filtering. Second and third, the Detections and Tracks are mismatched. At this time, the previous unconfirmed state Tracks and the mismatched Tracks are matched with the Unmatched Detections one by one for IOU matching, and then the cost matrix (cost matrix, whose calculation method is 1 - IOU) is calculated according to the result of the IOU matching;

[0089] In the above present application, after the rough matching and screening in step S43, the initial screening result (that is, the above-mentioned confirmed state track object) is obtained, and after the screening is completed, cascade matching is performed for re-matching and screening, so as to obtain a more accurate matching result.

[0090] Step S47: Based on each of the second cost matrices CM', use the Hungarian algorithm to match each prediction box with each of the second detection boxes to obtain a final matching result; the final matching result includes the final mismatch result of the prediction box, the final mismatch result of the detection box, and the final successful matching result; and update the detection box corresponding to the final successful matching result, and at the same time update the variables of the trajectory object corresponding to the final successful matching result to obtain the target vehicle information;

[0091] It should be noted that taking all the cost matrices obtained in the above step S46 as the input of the Hungarian algorithm to obtain a linear matching result, there are three types of results at this time. The first is that Tracks are mismatched (UnmatchedTracks), and the mismatched Tracks (since this Tracks is in an uncertain state, if it is in a definite state, it needs to reach a certain number of times continuously (default 30 times) before it can be deleted) are directly deleted; the second is that Detections are mismatched (UnmatchedDetections), and such Detections are initialized as a new Tracks (new Tracks); the third is that the detection box and the predicted box are successfully paired, indicating successful tracking. Update the corresponding Tracks variable of the Detections through Kalman filtering, that is, obtain the target vehicle information.

[0092] Step S48: Assign ID information to the target vehicle information to obtain the target vehicle information ID; and track the detection box corresponding to the target vehicle information;

[0093] It should be noted that record the ID information (deepsort will automatically assign IDs to the tracked targets) and detection box information corresponding to the tracked vehicle targets.

[0094] It should be noted that in the above embodiments of the present application, first, by acquiring consecutive frame patrol images, tracking the vehicle target detection result information of the first frame of the consecutive frame patrol images, a prediction box corresponding to the vehicle target detection result information is obtained; further, a first cost matrix is obtained by performing matching calculation with the prediction box using the second frame of the consecutive frame patrol images; and a confirmed state trajectory object is obtained by matching the first cost matrix with the detection box of the second frame; the obtained confirmed state trajectory object is predicted to obtain a confirmed state prediction detection box, and successful cascade matching is performed with the detection boxes of each frame, and variables of the trajectory object are performed on the successfully matched cascade to obtain a second confirmed state trajectory object, and a second cost matrix is calculated by calculating the cascade mismatch result and the unconfirmed state trajectory object; the second cost matrix is matched with the second detection box, and the successfully matched one is updated to obtain the target vehicle information, and finally, the detection box of the target vehicle information is tracked; thereby obtaining the vehicle driving state and whether it affects the normal operation of the highway tunnel; at the same time, the improved multi-task deep learning model realizes real-time vehicle target detection and lane line detection, and uses a tracking algorithm to track the vehicle in real time. The improved multi-task model and tracking algorithm have very good real-time performance and can reach more than 30 FPS on the NVIDIA Jetson Xavier development board.

[0095] Specifically, as Figure 6 shown, before step S50 of obtaining the vehicle driving state corresponding to each of the target vehicle information, it further includes pre-calculating and obtaining the displacement distance error α corresponding to the target vehicle information; the obtaining of the displacement distance error α corresponding to the target vehicle information includes the following operation steps:

[0096] Step S51: Calculate and obtain the displacement distance S of the current tunnel patrol robot according to the preset moving speed v of the tunnel patrol robot and the time interval t between two adjacent frames of images to be detected.

[0097] The calculation method of the displacement distance S of the tunnel patrol robot is:

[0098] S = v × t;

[0099] In the formula, S is the displacement distance of the tunnel patrol robot; v is the preset moving speed of the tunnel patrol robot; t is the time interval between two adjacent frames of images to be detected;

[0100] Step S52: Calculate and obtain the pixel displacement distance D of the stationary object in two adjacent frames of images to be detected according to the relationship M and the displacement distance S of the tunnel patrol robot.

[0101] The calculation method of the pixel displacement distance D of the stationary object in two adjacent frames of images to be detected is:

[0102] D = S / M;

[0103] Wherein, D is the pixel displacement distance of the stationary object in two adjacent frames of images to be detected; S is the displacement distance of the tunnel patrol robot; M is the relationship (the relationship M between the pixel displacement distance of each stationary object in two adjacent frames of images to be detected and the displacement distance of the corresponding stationary object in the real scene).

[0104] Step S53: Traverse all the target vehicle information; obtain the center point coordinates of the detection frame corresponding to the target vehicle information; extract the displacement distance d of the center point coordinates of the detection frame in two adjacent frames of images to be detected; calculate and obtain the displacement distance error α corresponding to the target vehicle information according to the displacement distance d of the center point coordinates of the detection frame and the pixel displacement distance D of the stationary object in two adjacent frames of images to be detected.

[0105] The calculation method of the displacement distance error α corresponding to the target vehicle information is as follows:

[0106] α = D - d;

[0107] Wherein, α is the displacement distance error corresponding to the target vehicle information; D is the pixel displacement distance of the stationary object in two adjacent frames of images to be detected; d is the displacement distance of the center point coordinates of the detection frame in two adjacent frames of images to be detected.

[0108] It should be noted that the displacement distance of the tunnel patrol robot is calculated through the moving speed of the tunnel patrol robot and the preset time interval between two adjacent frames of images to be detected. Subsequently, the pixel displacement distance of the stationary object in two adjacent frames of images to be detected is calculated through the displacement distance of the tunnel patrol robot and the relationship M. Finally, the displacement distance error corresponding to the target vehicle information is calculated in combination with the displacement distance of the center point coordinates of the detection frame in two adjacent frames of images to be detected; in this way, the tunnel patrol robot can take pictures at a shorter interval time and increase the patrol speed to enhance the monitoring ability of the vehicle.

[0109] Specifically, as Figure 7 shown, in step S50, determining whether the vehicle driving state belongs to an abnormal vehicle driving state includes the following operation steps:

[0110] Step S501: Preset the vehicle stop judgment hyperparameter k and the maximum threshold T of the vehicle stop judgment times; determine whether the absolute value of the displacement distance error is less than the vehicle stop judgment hyperparameter k; if so, set the vehicle stop times counter; increment by 1 the vehicle stop times of the vehicle stop times counter corresponding to the ID of the target vehicle information; further determine whether the vehicle stop times of the vehicle stop times counter is greater than the maximum threshold T of the vehicle stop judgment times; if so, determine that the current target vehicle information is in a stopped state, set the vehicle congestion counter, and increment by 1 the vehicle congestion times of the vehicle congestion counter; the vehicle stop times of the vehicle stop times counter is initially 0; the vehicle congestion times of the vehicle congestion counter is initially 0;

[0111] It should be noted that for vehicle stop, if |α| < 2 (2 is the set hyperparameter and can be modified according to specific scenarios), it is determined that the vehicle stops within the time interval t, and 1 stop flag is recorded for it according to the vehicle ID. If the number of stop flags of a certain vehicle is greater than the threshold T (T is a hyperparameter and can be modified according to the actual situation, and the default value of T is 7), it is determined that the vehicle is in a stopped state.

[0112] Step S502: Preset the vehicle reverse judgment hyperparameter u and the maximum threshold R of the vehicle reverse judgment; traverse each target vehicle information; determine whether the absolute value of the displacement distance error α is greater than the vehicle reverse judgment hyperparameter u; if so, set the vehicle reverse times counter; increment by 1 the vehicle reverse times of the vehicle reverse times counter and increment by 1 the vehicle congestion times of the vehicle congestion counter; determine whether the vehicle reverse times of the vehicle reverse times counter is greater than the maximum threshold R of the vehicle reverse judgment; if so, determine that the target vehicle information is in a reverse state; the vehicle reverse times of the vehicle reverse times counter is initially 0;

[0113] It should be noted that for vehicle reverse judgment: when the tunnel patrol robot travels along the tunnel exit direction, if α > 3, it is determined that the vehicle reverses within the time interval t (when the tunnel patrol robot travels along the tunnel entrance direction, if α < -3, it is determined that the vehicle reverses within the time interval t), and 1 reverse flag is recorded for it according to the vehicle ID. If the number of reverse flags of a certain vehicle is greater than the threshold R (R is a hyperparameter and can be modified according to the actual situation, and the default value of R is 5), it is determined that the vehicle is in a reverse state;

[0114] In the embodiments of the present application, when the tunnel patrol robot travels in the direction of the tunnel exit or the tunnel entrance, it is compared with the preset vehicle reverse judgment hyperparameter. The difference is that when the tunnel patrol robot travels in the direction of the tunnel exit, it is necessary to judge whether the displacement distance error is greater than the vehicle reverse judgment hyperparameter, and when the tunnel patrol robot travels in the direction of the tunnel entrance, it is necessary to judge whether the displacement distance error is less than the negative vehicle reverse judgment hyperparameter. Therefore, in the embodiments of the present application, whether the patrol robot travels in the direction of the tunnel exit or the tunnel entrance, it is judged whether the absolute value of the displacement distance error is greater than the vehicle reverse judgment hyperparameter.

[0115] Step S503: Count the number of the obtained target vehicle information; judge whether the number of vehicle congestion times of the vehicle congestion counter is greater than or equal to 3. If so, further judge whether the number of the target vehicle information is greater than or equal to 7. If so, judge that the target vehicle information is in a congested state;

[0116] It should be noted that for the judgment of the vehicle congestion state: if it is detected that 3 vehicles are in a reverse or stopped state and 7 vehicles are tracked, it is determined that the vehicle is congested.

[0117] Step S504: Traverse each piece of the target vehicle information, and judge whether the detection frame corresponding to the target vehicle information passes through the lane line target information. If so, set a vehicle lane change judgment counter and increment the number of vehicle lane change times of the vehicle lane change judgment counter by 1; judge whether the number of vehicle lane change times of the vehicle lane change judgment counter is greater than the preset maximum threshold C for vehicle lane change judgment. If so, judge that the target vehicle information is in a lane change state; the initial number of vehicle lane change judgment times of the vehicle lane change judgment counter is 0;

[0118] It should be noted that for vehicle lane change. According to the lane line information and the tracked vehicle coordinate information, if the vehicle target frame passes through the internal lane line (the lane line between the lane lines on both sides of the tunnel), a lane change flag is recorded for it according to the vehicle ID. If the number of lane change flags of a certain vehicle is greater than the threshold C (C is a hyperparameter and can be modified according to actual situations, and the default value of C is 5), it is determined that the vehicle has a lane change behavior.

[0119] Step S505: Preset a vehicle direction change judgment time period t'; Traverse each piece of the target vehicle information, and determine whether the vehicle orientation information corresponding to the target vehicle information has changed. If so, set a vehicle direction change abnormal judgment counter, and increment the vehicle direction change abnormal count of the vehicle direction change abnormal judgment counter by 1; Determine whether the vehicle direction change abnormal count of the vehicle direction change abnormal judgment counter is greater than a preset maximum threshold E for vehicle direction change judgment; If so, determine that the target vehicle information is in a vehicle direction change abnormal state; The vehicle direction change abnormal count of the vehicle direction change abnormal judgment counter is initially 0;

[0120] It should be noted that for vehicle direction change abnormalities. The direction of the tracked vehicle changes within a time interval t, and a direction change flag is recorded once for it according to the vehicle ID. If the number of direction change flags of a certain vehicle is greater than the threshold E (E is a hyperparameter that can be modified according to actual situations, and the default value of E is 3), then it is determined that the vehicle has an abnormal direction change.

[0121] In the above embodiments of the present application, S501: By presetting a vehicle stop judgment hyperparameter k and a maximum threshold T for the number of vehicle stop judgments, if the number of vehicle stops is greater than the maximum threshold, then the vehicle is congested; S502: By presetting a vehicle reverse judgment hyperparameter u and a maximum threshold R for vehicle reverse judgment, if the number of vehicle reversals is greater than the maximum threshold, S503: Finally, calculate the number of pieces of target vehicle information. If it is greater than a predetermined number, it will cause vehicle congestion; S504: Monitor whether the driving state of the vehicle has a lane change through the target vehicle information. If the number of vehicle lane changes is greater than the maximum threshold, the target vehicle is in a lane change state, which may thus cause road congestion and the probability of traffic accidents; S505 can also preset a vehicle direction change judgment time period to determine whether the vehicle orientation information corresponding to the target vehicle information has changed, and judge the vehicle direction change abnormal state, thereby improving the tracking ability of the tunnel robot and the detection of vehicles, taking preventive measures in advance to avoid collisions in the tunnel, which is also impossible for other monitoring methods based on fixed points.

[0122] In summary, a method for monitoring the traffic status of a high-speed tunnel based on a tunnel patrol robot proposed in an embodiment of the present invention, through step S10, the tunnel patrol robot patrols and collects continuous frame patrol images in the tunnel according to a preset collection task, and the tunnel patrol robot realizes full-tunnel coverage to achieve comprehensive monitoring; in step S20, the displacement distance S' of the tunnel patrol robot is obtained from the image, and at the same time, the displacement distance D' of each pixel in the continuous frame patrol images is analyzed and obtained; and the relationship M between the pixel displacement distance of each stationary object in two adjacent frame patrol images and the displacement distance of the corresponding stationary object in the real scene is calculated using the above information, and the running state of the vehicles in the tunnel is judged through M to indicate the magnitude of the traffic flow, and the perception ability of the tunnel patrol robot is also improved, enabling it to better adapt to different working scenarios;

[0123] Further, in step S30, the original data is framed and labeled with information, so as to calculate the loss function of the vehicle orientation, and finally the multi-task head network model is obtained through the training of the data set, which improves the speed of patrol image feature extraction and the clarity of image information; and in step S40, through the first frame of the vehicle target detection result information, a trajectory object corresponding to the first vehicle target detection result information is created, and then the prediction box corresponding to the target vehicle information is screened; the first cost matrix is calculated for the second frame of the patrol image; the cascade matching result is obtained through the matching of the first cost matrix, the second confirmed state trajectory object is updated for the cascade matching result, the second cost matrix is calculated, and the final target vehicle information is obtained through the matching of the second cost matrix; the target vehicle information is extracted more accurately and quickly;

[0124] Further, in step S50, the pixel displacement distance of the stationary object in two adjacent frames to be detected is calculated by using the preset moving speed of the tunnel patrol robot, the time interval between two adjacent frames to be detected, and the pixel displacement distance of the stationary object in two adjacent frames to be detected, and then the center point coordinates of the detection box corresponding to the target vehicle information are obtained, and the displacement distance error corresponding to the target vehicle information is calculated, which improves the accuracy of target detection and optimizes the reliability of the algorithm at the same time; and the traffic congestion situation of the vehicle is judged by using the vehicle stop judgment hyperparameter k, the maximum threshold T of the vehicle stop judgment times, the preset vehicle reverse judgment hyperparameter u, and the maximum threshold R of the vehicle reverse judgment; it is judged whether the detection box corresponding to the target vehicle information passes through the lane line target information and the preset vehicle direction change judgment time period t', and the driving states of each target vehicle information are judged, and the obtained vehicle driving states can supervise the highway tunnel to prevent abnormal driving and cause traffic congestion in the highway tunnel.

Claims

1. A high-speed tunnel traffic status monitoring method based on a tunnel patrol robot, characterized in that: The steps are as follows: S10: The tunnel patrol robot collects and acquires continuous-frame patrol images in the tunnel according to the preset collection task; S20: The tunnel patrol robot patrols in the tunnel with a fixed camera angle according to a preset patrol task: the displacement distance S' of the tunnel patrol robot is obtained in real time, and the displacement distance D' of each pixel in the continuous frame patrol image is obtained by analysis; the relationship M is obtained according to the displacement distance S' of the tunnel patrol robot and the pixel displacement distance D'; the running state of the object in the continuous frame patrol image is obtained through the relationship M; the running state includes stationary, moving, and the direction of movement; The relationship M is the relationship between the pixel displacement distance of each stationary object in two adjacent patrol images and the displacement distance of the corresponding stationary object in the real scene; The preset patrol task includes a preset patrol robot moving speed v and a preset interval time t between two adjacent frames of images to be detected; The relationship M is calculated as follows: M = S' / D'; S30: inputting the continuous frame patrol image into the multi-task head neural network model for detection, and outputting vehicle target detection result information and lane line detection result information; S40: Tracking the vehicle target detection result information based on the continuous frame patrol image to obtain target vehicle information; Assigning ID information to the target vehicle information, and performing a detection frame selection on the target vehicle information; S50: Calculate the displacement distance error α corresponding to the target vehicle information. The calculation method of the displacement distance error α corresponding to the target vehicle information is: α=Dd; Wherein, α is the displacement distance error corresponding to the target vehicle information; D is the pixel displacement distance of the stationary object in two adjacent frames of the image to be detected; d is the displacement distance of the center point coordinates of the detection frame in two adjacent frames of the image to be detected; Then, the displacement distance error α, the target vehicle information and the lane line detection result information are used to determine whether the target vehicle information meets the condition of the abnormal driving state of the vehicle; if so, an alarm is issued.

2. The high-speed tunnel traffic status monitoring method based on the tunnel patrol robot according to claim 1 is characterized in that: The relationship M is the relationship between the pixel displacement distance of each stationary object in two adjacent patrol image frames and the displacement distance of the corresponding stationary object in the real scene.

3. The high-speed tunnel traffic status monitoring method based on the tunnel patrol robot according to claim 2 is characterized in that: The conditions of the abnormal driving state of the vehicle include the abnormal driving state of the vehicle including the vehicle stopping state, the vehicle going in the wrong direction state, the vehicle being in a congested state, the vehicle changing lanes state, and the vehicle changing direction abnormally state.

4. The high-speed tunnel traffic status monitoring method based on the tunnel patrol robot according to claim 3 is characterized in that: The preset patrol task includes a preset patrol robot moving speed v and a preset interval time t between two adjacent frames of images to be detected; The relationship M is calculated as follows: M=S' / D'.

5. The high-speed tunnel traffic status monitoring method based on the tunnel patrol robot according to claim 4 is characterized in that: Before inputting the continuous frame patrol images into the multi-task head neural network model for detection, the multi-task head neural network model is also designed and trained; the design and training of the multi-task head neural network model includes the following steps: S31: Acquire original video data; extract frames from the original video data to obtain a plurality of original image data; S32: annotating all the original image data to obtain annotated image data; S33: Divide all the labeled image data into a training set and a test set; S34: Build a multi-task neural network model; define the vehicle orientation loss function Loss; The vehicle orientation loss function Loss is expressed as: In the formula, is the probability that the model predicts that the sample is a positive example, y is the sample label, if the sample is a positive example, the value is 1, otherwise the value is 0; S35: Based on the loss function Loss, the multi-task head neural network model is trained using the training set and the test set to obtain a trained multi-task head neural network model.

6. The high-speed tunnel traffic status monitoring method based on the tunnel patrol robot according to claim 5 is characterized in that: The multi-task head neural network model includes a backbone network and two task heads; wherein the backbone network extracts feature information from the image; the two task heads respectively output vehicle detection results and lane line detection results; the feature information includes vehicle target detection result information, lane line target information, and vehicle orientation information.

7. The high-speed tunnel traffic status monitoring method based on the tunnel patrol robot according to claim 6 is characterized in that: The step of tracking the vehicle target detection result information based on the continuous frame patrol image to obtain the target vehicle information includes the following steps: S41: Acquire a continuous-frame patrol image; create a trajectory object corresponding to each first vehicle target detection result information for the vehicle target detection result information in the first frame of the continuous-frame patrol image; and perform trajectory prediction on each of the vehicle target detection result information according to the running state to obtain a prediction box corresponding to each of the vehicle target detection result information; S42: determining each of the vehicle target detection result information of the second frame to be detected image in the continuous frame patrol image as the second vehicle target detection result information; A second detection frame is set for the second vehicle target detection result information, and each second detection frame is matched with the prediction frame to obtain an IOU matching result, i.e., an iou; and a first cost matrix CM is obtained according to the iou calculation; The first cost matrix is ​​calculated as follows: CM = 1-iou; S43: Matching each prediction box with each second detection box by using the Hungarian algorithm based on each of the first cost matrices CM to obtain a matching result; the matching result includes a prediction box mismatch result, a detection box mismatch result, and a matching success result; and updating the detection frame corresponding to the successful matching result, and at the same time updating the variable of the trajectory object corresponding to the successful matching result, to obtain a first confirmed state trajectory object; S44: Repeat the above steps S42 to S43 according to the order of the continuous frame patrol images to determine whether the first confirmed state trajectory object is obtained; if so, execute the next step; if not, repeat the detection until all continuous frame patrol images are detected to obtain multiple unconfirmed state trajectory objects; S45: predicting the first confirmed state trajectory object to obtain a confirmed state prediction frame, and predicting the unconfirmed state trajectory object to obtain an unconfirmed state prediction frame; performing cascade matching on the confirmed state prediction frame and the detection frame obtained in each frame to obtain a cascade matching result; The cascade matching results include trajectory object cascade matching success results, detection frame cascade mismatch results, and prediction frame cascade mismatch results; S46: updating the variable of the trajectory object corresponding to the successful cascade matching result with the detection frame corresponding to the successful cascade matching result of the trajectory object, to obtain a second confirmed state trajectory object; Perform IOU matching on the detection frame corresponding to the detection frame cascade mismatch result, the prediction frame corresponding to the prediction frame cascade mismatch result, the uncertain dynamic trajectory object, and the trajectory object corresponding to the prediction frame mismatch result to obtain an IOU matching result, i.e., iou'; calculate and obtain a second cost matrix CM' according to iou'; The second cost matrix CM' is calculated as follows: CM' = 1 - iou'; S47: Matching each prediction box with each second detection box by using the Hungarian algorithm based on each second cost matrix CM' to obtain a final matching result; the final matching result includes a final mismatch result of the prediction box, a final mismatch result of the detection box, and a final successful matching result; The detection frame corresponding to the final successful matching result is updated, and the variable of the track object corresponding to the final successful matching result is updated to obtain the target vehicle information; S48: assigning ID information to the target vehicle information to obtain the target vehicle information ID; And track the detection frame corresponding to the target vehicle information.

8. The high-speed tunnel traffic status monitoring method based on the tunnel patrol robot according to claim 7 is characterized in that: Before obtaining the vehicle driving status corresponding to each of the target vehicle information, the method further includes precalculating and obtaining the displacement distance error α corresponding to the target vehicle information; the method of obtaining the displacement distance error α corresponding to the target vehicle information includes the following steps: S51: Calculate and obtain the current displacement distance S of the tunnel patrol robot according to the preset moving speed v of the tunnel patrol robot and the preset time interval t between two adjacent frames of images to be detected; The calculation method of the displacement distance S of the tunnel patrol robot is: S = v × t; Where S is the displacement distance of the tunnel patrol robot; v is the preset moving speed of the tunnel patrol robot; t is the time interval between two adjacent frames of images to be detected; S52: Calculate and obtain the pixel displacement distance D of the stationary object in two adjacent frames of the image to be detected according to the relationship M and the displacement distance S of the tunnel patrol robot; The calculation method of the pixel displacement distance D of the stationary object in the two adjacent frames of the image to be detected is: D = S / M; Where D is the pixel displacement distance of the stationary object in two adjacent frames of the image to be detected; S is the displacement distance of the tunnel patrol robot; M is the relationship; S53: traverse all the target vehicle information; obtain the center point coordinates of the detection frame corresponding to the target vehicle information; extract the displacement distance d of the center point coordinates of the detection frame in two adjacent frames of the image to be detected; calculate and obtain the displacement distance error α corresponding to the target vehicle information according to the displacement distance d of the center point coordinates of the detection frame and the pixel displacement distance D of the stationary object in the two adjacent frames of the image to be detected; The calculation method of the displacement distance error α corresponding to the target vehicle information is: α=Dd; Wherein, α is the displacement distance error corresponding to the target vehicle information; D is the pixel displacement distance of the stationary object in two adjacent frames of the image to be detected; d is the displacement distance of the center point coordinates of the detection frame in two adjacent frames of the image to be detected.

9. The high-speed tunnel traffic status monitoring method based on the tunnel patrol robot according to claim 8 is characterized in that: Using the displacement distance error α, the target vehicle information, and the lane line detection result information to determine whether the target vehicle information meets the condition of the abnormal vehicle driving state includes the following steps: S501: Preset a vehicle stop judgment hyperparameter k and a maximum threshold value T for vehicle stop judgment times; determine whether the absolute value of the displacement distance error α is less than the vehicle stop judgment hyperparameter k; if so, set a vehicle stop times counter; add 1 to the vehicle stop times counter corresponding to the ID of the target vehicle information; further determine whether the vehicle stop times counter is greater than the maximum threshold value T for vehicle stop judgment times; if so, determine that the current target vehicle information is in a stopped state, set a vehicle congestion counter, and add 1 to the vehicle congestion times counter; S502: Preset a vehicle reverse traffic judgment hyperparameter u and a vehicle reverse traffic judgment maximum threshold R; traverse each of the target vehicle information; determine whether the absolute value of the displacement distance error α is greater than the vehicle reverse traffic judgment hyperparameter u; if so, set a vehicle reverse traffic count counter; add 1 to the vehicle reverse traffic count of the vehicle reverse traffic count counter, and add 1 to the vehicle congestion count of the vehicle congestion counter; determine whether the vehicle reverse traffic count of the vehicle reverse traffic count counter is greater than the vehicle reverse traffic judgment maximum threshold R; if so, determine that the target vehicle information is in a reverse traffic state; S503: Count and obtain the number of target vehicle information; determine whether the number of vehicle congestion counted by the vehicle congestion counter is greater than or equal to 3, and if so, further determine whether the number of target vehicle information is greater than or equal to 7, and if so, determine that the target vehicle information is in a congested state; S504: traverse each of the target vehicle information, determine whether the detection box corresponding to the target vehicle information crosses the lane line target information, if so, set a vehicle lane change determination counter, and add 1 to the number of vehicle lane changes in the vehicle lane change determination counter; determine whether the number of vehicle lane changes in the vehicle lane change determination counter is greater than a preset vehicle lane change determination maximum threshold C; if so, determine that the target vehicle information is in a lane change state; S505: Preset a vehicle change direction judgment time period t'; traverse each of the target vehicle information to determine whether the vehicle orientation information corresponding to the target vehicle information has changed. If so, set a vehicle change direction abnormality judgment counter, and add 1 to the number of vehicle change direction abnormalities in the vehicle change direction abnormality judgment counter; determine whether the number of vehicle change direction abnormalities in the vehicle change direction abnormality judgment counter is greater than a preset vehicle change direction judgment maximum threshold E; if so, determine that the target vehicle information is in a vehicle change direction abnormality state.

10. The high-speed tunnel traffic status monitoring method based on a tunnel patrol robot according to claim 9 is characterized in that: The number of vehicle stops of the vehicle stop count counter is initially 0; the number of vehicle congestion counts of the vehicle congestion counter is initially 0; the number of vehicle reverse movement counts of the vehicle reverse movement count counter is initially 0; the number of vehicle direction change abnormality judgment counter is initially 0; the number of vehicle lane change judgment counts of the vehicle lane change judgment counter is initially 0.

Images