A method, apparatus, terminal equipment, and storage medium for target detection in tunnels.

Abstract

This application relates to the field of target detection technology, and provides a method, apparatus, terminal device, and storage medium for target detection in tunnels. The method includes: performing target detection on image data using at least two sub-detection models in a target detection model, obtaining the target detection result of each sub-detection model; and determining abnormal targets appearing in the image data based on the target detection results of each sub-detection model. This application uses at least two sub-detection models to detect image data, and determines abnormal targets in the image data based on the detection results of each sub-detection model; using multiple sub-detection models can make the detection of abnormal targets more accurate.

Description

Technical Field

[0001] This application belongs to the field of target detection technology, and in particular relates to a target detection method, device, terminal equipment and storage medium in tunnels. Background Technology

[0002] Tunnels are engineering structures buried underground. Since the scene inside a tunnel cannot be accurately obtained from the outside, data acquisition equipment is installed inside the tunnel to facilitate data collection and analysis of the tunnel's internal conditions. Currently, the lack of accurate methods for detecting abnormal targets within tunnels, such as pedestrians and animals, makes detection difficult and hinders safety monitoring within the tunnel. Summary of the Invention

[0003] This application provides a method, apparatus, terminal device, and storage medium for target detection in tunnels, which can accurately detect abnormal targets in tunnels.

[0004] In a first aspect, embodiments of this application provide a target detection method in a tunnel, including:

[0005] Acquire image data of the tunnel interior;

[0006] The image data is subjected to target detection using at least two of the sub-detection models in the target detection model, and the target detection result of each sub-detection model is obtained. The at least two sub-detection models are different sub-detection models.

[0007] Based on the target detection results of each of the sub-detection models, abnormal targets appearing in the image data are identified, wherein the abnormal targets are non-motorized vehicles.

[0008] In one possible implementation of the first aspect, the target detection model includes the DINO model and the CO-DETR model.

[0009] In one possible implementation of the first aspect, determining the anomalous targets appearing in the image data based on the target detection results of each of the sub-detection models includes:

[0010] If an abnormal target is found in the target detection result of at least one of the sub-detection models, then the abnormal target detected by at least one of the sub-detection models is determined to be an abnormal target appearing in the image data.

[0011] In one possible implementation of the first aspect, after determining the anomalous targets appearing in the image data based on the target detection results of each of the sub-detection models, the method further includes:

[0012] Obtain the first time of appearance of each of the anomalous targets in the image data;

[0013] By extracting image data corresponding to each of the first time points from the image data, the anomaly detection results of the abnormal targets in the tunnel are obtained.

[0014] In one possible implementation of the first aspect, the target detection result includes the detected vehicle, the position of the vehicle in the image data, and the speed of the vehicle;

[0015] After performing target detection on the image data using at least two of the sub-detection models in the target detection model, and obtaining the target detection result of each sub-detection model, the method further includes:

[0016] Based on the vehicle's position in the image data, the vehicle's movement trajectory is constructed;

[0017] Based on the vehicle's movement trajectory, abnormal events in the tunnel are determined, wherein the abnormal events include at least one of abnormal speed, lane change, and driving abnormality.

[0018] In one possible implementation of the first aspect, determining the abnormal event in the tunnel based on the vehicle's movement trajectory includes:

[0019] Based on the vehicle's movement trajectory, determine the lane number of the lane where the vehicle is located at each time.

[0020] Based on the lane number corresponding to each time moment, the movement time of the vehicle in each lane is calculated.

[0021] If the vehicle changes lanes from the first lane to the second lane at a given moment, the duration of its movement in the first lane exceeds a first preset duration, and the duration of its movement in the second lane exceeds a second preset duration, then it is determined that the vehicle has changed lanes.

[0022] In one possible implementation of the first aspect, determining the abnormal event in the tunnel based on the vehicle's movement trajectory includes:

[0023] Based on the movement trajectories of each vehicle, the total number of vehicles in the tunnel is determined;

[0024] If the total number is less than the preset number, determine whether the vehicle has an abnormal speed based on the detected vehicle speed;

[0025] If the total number is greater than or equal to the preset number, calculate a standard score of the vehicle's speed based on the detected vehicle speed;

[0026] Based on the standard score of the vehicle's speed, determine whether the vehicle has a speed anomaly.

[0027] In one possible implementation of the first aspect, determining the abnormal event in the tunnel based on the vehicle's movement trajectory includes:

[0028] Based on the vehicle's movement trajectory, calculate the lateral distance between the vehicle and the centerline of its lane at different times;

[0029] Based on the lateral distance, it is determined whether the vehicle exhibits the driving abnormality.

[0030] In one possible implementation of the first aspect, determining whether the vehicle exhibits the driving abnormality based on the lateral distance includes:

[0031] Calculate the standard deviation of the lateral distance based on the lateral distance;

[0032] If the standard deviation is less than the preset value, it is determined that the vehicle does not have the driving abnormality;

[0033] If the standard deviation is greater than or equal to the preset value, it is determined that the vehicle has the driving abnormality.

[0034] Secondly, embodiments of this application provide a target detection device in a tunnel, comprising:

[0035] The data acquisition module is used to acquire image data of the tunnel interior.

[0036] The target detection module is used to perform target detection on the image data using at least two of the sub-detection models in the target detection model, and to obtain the target detection result of each of the sub-detection models, wherein the at least two sub-detection models are different sub-detection models;

[0037] An abnormal target extraction model is used to determine abnormal targets appearing in the image data based on the target detection results of each of the sub-detection models, wherein the abnormal target is a non-motorized vehicle.

[0038] Thirdly, embodiments of this application provide a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the tunnel target detection method described in any one of the first aspects above.

[0039] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the tunnel target detection method described in any one of the first aspects above.

[0040] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the tunnel target detection method described in any one of the first aspects.

[0041] The beneficial effects of the first aspect of this application compared with the prior art are as follows: This application utilizes at least two sub-detection models in the target detection model to perform target detection on the image data respectively, obtaining the target detection result of each sub-detection model; based on the target detection result of each sub-detection model, abnormal targets appearing in the image data are determined. This application uses at least two sub-detection models to detect image data, and determines abnormal targets in the image data based on the detection result of each sub-detection model; using multiple sub-detection models can make the detected abnormal targets more accurate.

[0042] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic flowchart of a target detection method in a tunnel provided in an embodiment of this application;

[0045] Figure 2 This is a flowchart illustrating a target detection method in a tunnel according to another embodiment of this application;

[0046] Figure 3 This is a flowchart illustrating a method for determining abnormal events provided in an embodiment of this application;

[0047] Figure 4 This is a flowchart illustrating a method for determining lane change events according to an embodiment of this application;

[0048] Figure 5 This is an illustrative diagram illustrating an example of a lane change event provided in an embodiment of this application;

[0049] Figure 6This is a flowchart illustrating a method for determining speed anomaly events according to an embodiment of this application;

[0050] Figure 7 This is a flowchart illustrating a method for determining abnormal driving events according to an embodiment of this application;

[0051] Figure 8 This is a schematic diagram of the structure of a target detection device in a tunnel provided in an embodiment of this application;

[0052] Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0053] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0054] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0055] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0056] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0057] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0058] To obtain information about the interior of a tunnel, data acquisition equipment, such as base stations, cameras, or radar, is often installed inside. This equipment collects tunnel data, and the scene inside the tunnel is then analyzed.

[0059] Because tunnels are typically dimly lit, creating a significant contrast with the bright outside environment, drivers often struggle to clearly spot pedestrians or animals within the tunnel. Furthermore, tunnels are narrower than regular roads, with a limited number of lanes, resulting in very little lateral space for maneuvering. If a pedestrian or animal is spotted ahead, drivers often find it difficult to quickly and safely change lanes to avoid it, frequently resorting to emergency braking. However, this can lead to rear-end collisions or other secondary accidents if following too closely.

[0060] Therefore, the presence of non-motorized vehicles in tunnels significantly impacts traffic safety. Currently, issues such as large vehicles obstructing traffic and lighting problems lead to inaccurate detection of abnormal targets, affecting tunnel safety management.

[0061] To ensure driving safety inside tunnels, accurately detecting abnormal targets within tunnels is a problem that needs to be solved.

[0062] Based on this, this application proposes a target detection method in tunnels. This application uses a trained target detection model to detect image data inside the tunnel, thereby obtaining image data of abnormal targets existing in the tunnel.

[0063] Specifically, such as Figure 1 As shown, object detection models can include dual-model and data mining models.

[0064] The tunnel image data is input into the dual-model, which includes two different sub-detection models. The two sub-detection models are used to perform target detection on the image data and identify each target object in the image data. Each sub-detection model will output the target detection result after performing target detection on the image data.

[0065] The dual-model approach inputs the target detection results output by each sub-detection model into the data mining model.

[0066] The data mining model analyzes the detection results of each target, identifies the abnormal targets detected by each sub-detection model, and finally outputs the abnormal targets detected by each sub-detection model.

[0067] The abnormal targets detected by each sub-detection model are analyzed.

[0068] If the detection results of two sub-detection models are consistent, and the detected target is an anomalous target, then the anomalous target is considered an anomalous target in the image data. For example, if both sub-detection models detect the presence of anomalous target A at the same time, then anomalous target A is considered an anomalous target in the image data.

[0069] If the detection results of the two sub-detection models are inconsistent, for example, one sub-detection model detects the existence of an abnormal target B at a certain time, while the other sub-detection model does not detect the abnormal target B at the same time, then the abnormal target B will also be regarded as an abnormal target in the image data.

[0070] Finally, the image data corresponding to each abnormal target in the image data is extracted to obtain the anomaly detection results of abnormal targets in the tunnel.

[0071] After detecting anomalous targets using an object detection model, the anomaly detection results can be compared with the image data to determine if there are any anomalous targets that the model failed to detect. If there are anomalous targets in the image data that were not detected by the model, the image data corresponding to the undetected anomalous targets is extracted and added to the anomaly detection results. The anomaly detection results can be used to train the object detection model, thereby improving its performance.

[0072] This application uses two different sub-detection models to detect abnormal targets in image data, which can improve the accuracy of abnormal target recognition and reduce the probability of false detection.

[0073] The following combination Figure 1 The target detection method in the tunnel according to the embodiments of this application will be described in detail.

[0074] Figure 2 A schematic flowchart of the target detection method in tunnels provided in this application is shown, with reference to... Figure 2 The method is described in detail below:

[0075] S101, acquire image data of the tunnel interior.

[0076] In this embodiment, the image data can be video data or point cloud data, etc.

[0077] S102, target detection is performed on the image data using at least two of the sub-detection models in the target detection model, and the target detection result of each sub-detection model is obtained. The at least two sub-detection models are different sub-detection models.

[0078] In this embodiment, the target detection model has at least two sub-detection models. For example, the number of sub-detection models can be 2 or 3.

[0079] When there are two sub-detection models, the object detection models include the DINO model and the CO-DETR model. DINO (Distillation with No Labels) is an unsupervised learning training method that mainly utilizes the VisionTransformer (ViT) architecture for training. It does not require manually labeled data and learns the semantic features of images in a self-supervised manner. The CO-DETR model is an improved DETR (Detection Transformer) model designed to improve the efficiency and performance of object detection. CO-DETR optimizes the model training process by co-mixing training assignments and customizing positive queries, reducing matching instability and demonstrating superior performance on multiple datasets.

[0080] Specifically, image data is input into at least two sub-detection models. These models perform target recognition on the image data, detecting objects appearing in the image data and obtaining target detection results. Target objects can include motor vehicles, non-motor vehicles, pedestrians, animals, etc. The target detection results can include the target object's category, the time of its appearance, its position in the image data, its movement speed, and the accuracy of the detected target. The category of the target object can be determined based on its shape.

[0081] S103, based on the target detection results of each of the sub-detection models, determine the abnormal targets appearing in the image data, wherein the abnormal targets are non-motorized vehicles.

[0082] In one embodiment, the occurrence time of each target object detected by each sub-detection model is extracted. If all sub-detection models (or a preset number of sub-detection models) detect a target object at the same time, and the types of target objects detected by all sub-detection models are the same, the target object is extracted and recorded as the first target object. If the first target object is an anomalous target object, it is determined as an anomalous target appearing in the image data. Anomalous targets can be people, bicycles, motorcycles, animals, etc.

[0083] As an example, taking object detection using two sub-detection models, if the first sub-detection model detects an object, specifically a bicycle, at 5:10 AM, and the second sub-detection model also detects a bicycle at 5:10 AM, then the bicycle appearing at 5:10 AM is considered an abnormal target in the image data.

[0084] In another embodiment, targets whose accuracy in detecting each sub-detection model is greater than a preset value are selected to obtain second targets, and abnormal targets in each second target are identified as abnormal targets appearing in the image data.

[0085] In another embodiment, if an abnormal target is found in the target detection result of at least one of the sub-detection models, then the abnormal target detected by at least one of the sub-detection models is determined to be an abnormal target appearing in the image data.

[0086] In this embodiment, if any sub-detection model detects an abnormal target, then the abnormal target is identified as an abnormal target appearing in the image data.

[0087] As an example, taking object detection using two sub-detection models, if the first sub-detection model detects an object, specifically a bicycle, at 5:10 AM, and the second sub-detection model does not detect the bicycle at 5:10 AM, then the bicycle detected by the first sub-detection model at 5:10 AM is considered an anomalous target in the image data.

[0088] In this application, at least two sub-detection models are used to perform target detection on image data, obtaining the target detection result of each sub-detection model; based on the target detection result of each sub-detection model, abnormal targets appearing in the image data are identified. This application uses at least two sub-detection models to detect image data, and determines abnormal targets in the image data based on the detection result of each sub-detection model; using multiple sub-detection models can make the detection of abnormal targets more accurate.

[0089] In one possible implementation, after step S103, the above method may further include:

[0090] Image data of each of the abnormal targets are extracted from the image data to obtain the anomaly detection results of the abnormal targets in the tunnel.

[0091] In this embodiment, based on the time when the abnormal target appears, image data containing the abnormal target is extracted from the image data, and the image data containing the abnormal target constitutes the anomaly detection result.

[0092] Specifically, the first time when each of the abnormal targets appears in the image data is obtained; the image data corresponding to each first time is extracted from the image data to obtain the abnormal detection result of the abnormal targets in the tunnel.

[0093] In this embodiment, after obtaining the anomaly detection results, the anomaly detection results can be saved, the abnormal events in the anomaly detection results can be sliced ​​and labeled, and the anomaly detection results can be used to optimize the target detection model to make the target detection model more accurate.

[0094] In this embodiment, tunnel safety management can be carried out based on the anomaly detection results, reminding pedestrians, bicycles, motorcycles, etc., who have entered the tunnel to leave the tunnel as soon as possible.

[0095] In one possible implementation, after obtaining the target detection result in step S102, the information of the detected vehicles in the target detection result is extracted. The vehicle information includes the vehicle's position and speed, etc., and abnormal events of vehicles in the tunnel are determined based on the vehicle information.

[0096] like Figure 3 As shown, specifically, after step S102, the above method may further include:

[0097] S201, Based on the vehicle's position in the image data, construct the vehicle's movement trajectory.

[0098] In this embodiment, when the sub-detection model performs target detection, the same identification code is used for the same target. Based on the identification codes of each vehicle, the vehicle is tracked to obtain the position of each vehicle at different times. Based on the positions of the same vehicle at different times, the vehicle's movement trajectory is generated.

[0099] In this embodiment, the target detection results of each sub-detection model are fused to obtain a fused image of the image data. Target tracking is performed on each vehicle in the fused image to obtain the position of each vehicle at different times. The vehicle's movement trajectory is generated based on the position of each vehicle at different times.

[0100] S202, based on the vehicle's movement trajectory, determine the abnormal events in the tunnel, wherein the abnormal events include at least one of abnormal speed, lane change, and driving abnormality.

[0101] In this embodiment, abnormal driving can be a prolonged deviation from the lane line, etc.

[0102] The driving trajectory is input into the trajectory detection model to obtain abnormal events in the tunnel.

[0103] In this application, the vehicle's trajectory is determined based on its position in the image data, and abnormal events of the vehicle in the tunnel are determined based on the trajectory. This can not only ensure tunnel safety, but also provide data support for subsequent penalties against the vehicle.

[0104] like Figure 4 As shown, in one possible implementation, the process of detecting vehicle lane changes in step S202 may include:

[0105] S301, Based on the vehicle's movement trajectory, determine the lane number of the lane where the vehicle is located at each time.

[0106] In this embodiment, each lane in the image data is numbered, and the lane number of the vehicle at each time point is extracted.

[0107] S302, based on the lane number corresponding to each time moment, calculate the movement time of the vehicle in each lane.

[0108] S303, if the vehicle changes from the first lane to the second lane at a certain moment, the duration of movement in the first lane is greater than a first preset duration, and the duration of movement in the second lane is greater than a second preset duration, then it is determined that the vehicle has changed lanes.

[0109] In this embodiment, the first preset duration and the second preset duration can be set as needed. The first preset duration can be the same or different. For example, both the first preset duration and the second preset duration can be set to 1 minute.

[0110] For example, such as Figure 5 As shown, in the first scenario: the vehicle travels in lane 1 during the first time period, in lane 2 during the second time period, and in lane 1 during the third time period. The first and third time periods are longer than the first preset duration, while the second time period is shorter than the second preset duration. Although the vehicle changes lanes from lane 1 to lane 2 and then back to lane 1 during its journey, the time spent in lane 2 is relatively short. To avoid false detections, the vehicle is determined to have not changed lanes.

[0111] The second scenario: The vehicle travels in lane 1 during the first time period, in lane 2 during the second time period, and in lane 1 during the third time period. The first and third time periods are longer than the first preset duration, and the second time period is longer than the second preset duration. Therefore, the vehicle changes lanes from lane 1 to lane 2 and then back from lane 2 to lane 1, thus confirming that the vehicle has changed lanes.

[0112] The third scenario: The vehicle travels in lane 1 during the first time period, in lane 2 during the second time period, in lane 1 during the third time period, and in lane 2 during the fourth time period; the first time period is longer than the first preset duration, the second time period is longer than the second preset duration, the third time period is shorter than the first preset duration, and the fourth time period is shorter than the second preset duration; therefore, the vehicle changes lanes from lane 1 to lane 2, thus confirming that the vehicle has changed lanes.

[0113] In another embodiment, if the vehicle changes lanes from the first lane to the second lane at a certain moment, the duration of movement in the first lane is less than or equal to a first preset duration, and / or the duration of movement in the second lane is less than or equal to a second preset duration, then it is determined that the vehicle has changed lanes.

[0114] In this application, determining whether a vehicle has changed lanes based on the duration of its travel in each lane can effectively avoid detection errors caused by false detections, making lane change detection more accurate.

[0115] like Figure 6 As shown, in one possible implementation, the process of detecting abnormal vehicle speed in step S202 may include:

[0116] S401, Based on the movement trajectory of each of the vehicles, determine the total number of vehicles in the tunnel.

[0117] In this embodiment, the total number of vehicles in the tunnel is determined based on the vehicle's identification code.

[0118] S402, if the total number is less than the preset number, determine whether the vehicle has a speed abnormality based on the detected speed of the vehicle.

[0119] In this embodiment, the preset number can be set as needed, for example, the preset number can be set to 10 or 15, etc.

[0120] In this embodiment, if there are relatively few vehicles in the tunnel, the speed detection of vehicles is relatively accurate. Therefore, based on the speed of each vehicle, it can be determined whether there is a speed abnormality. Speed ​​abnormality includes speeds below the minimum specified speed and speeds above the maximum specified speed. In other words, speed abnormality indicates that the speed of the vehicle is not within the specified range.

[0121] Specifically, if the vehicle's speed is less than the minimum speed limit (e.g., 40 km / h), it is determined that the vehicle is driving too slowly, and the vehicle has an abnormal speed. If the vehicle's speed is greater than the maximum speed limit (e.g., 80 km / h), it is determined that the vehicle is speeding, and the vehicle has an abnormal speed.

[0122] S403, if the total number is greater than or equal to the preset number, calculate the standard score of the vehicle's speed based on the detected vehicle speed.

[0123] In this embodiment, if there are many vehicles in the tunnel, the vehicle speed detection may be inaccurate. To avoid false detections caused by inaccurate vehicle speed detection, when there are many vehicles in the tunnel, the standard score of the vehicle speed is used to determine whether there is a speed abnormality.

[0124] In this embodiment, the standard score is also called the z-score.

[0125] The standard score is calculated as follows:

[0126]

[0127] Where Z is the standard score, x is the detected vehicle speed, μ is the average vehicle speed, and σ is the standard deviation of vehicle speed.

[0128] S404, determine whether the vehicle has a speed abnormality based on the standard score of the vehicle's speed.

[0129] Specifically, if a vehicle's standard speed score is outside the preset score range, the vehicle is determined to have a speed anomaly; if the vehicle's standard speed score is within the preset score range, the vehicle is determined not to have a speed anomaly. The preset score range can be set as needed; for example, the preset score range can be set to [-3, 3].

[0130] For example, if the standard score of vehicle A's speed is 4, and vehicle A's standard score is not within the preset score range [-3, 3], then vehicle A is determined to have an abnormal speed.

[0131] In this application, speed anomalies are determined in different scenarios. When there are few vehicles in the tunnel, the presence of a speed anomaly is determined directly based on the detected speed. When there are many vehicles in the tunnel, the presence of a speed anomaly is determined based on a standard score of the vehicle's speed. In the case of many vehicles in the tunnel, to avoid errors in judgment due to false detections, the presence of a speed anomaly is not determined directly based on speed. Instead, it is determined based on a standard score of speed, making the determination of speed anomalies more accurate.

[0132] like Figure 7 As shown, in one possible implementation, the process of detecting driving abnormalities in step S202 may include:

[0133] S501, Based on the vehicle's movement trajectory, calculate the lateral distance between the vehicle and the centerline of the lane at different times.

[0134] In this embodiment, a first preset number of calibration points are selected on the vehicle's movement trajectory, and the lateral distance between each calibration point and the centerline of the lane is determined.

[0135] S502, based on the lateral distance, determine whether the vehicle has the driving abnormality.

[0136] In this embodiment, the driving abnormality may be caused by factors such as user fatigue or physical discomfort.

[0137] In one embodiment, a method for determining whether a vehicle exhibits abnormal driving behavior includes:

[0138] Extract horizontal distances greater than a preset distance from the first preset number of horizontal distances; calculate the total number of horizontal distances greater than the preset distance.

[0139] If the total number of lateral distances greater than the preset distance is greater than the second preset number, then the vehicle is determined to have a driving abnormality. If the total number of lateral distances greater than the preset distance is less than or equal to the second preset number, then the vehicle is determined not to have a driving abnormality.

[0140] In another embodiment, the method for determining whether a vehicle exhibits abnormal driving behavior includes:

[0141] S5021, Calculate the standard deviation of the lateral distance based on the lateral distance.

[0142] In this embodiment, standard deviation is used to measure the dispersion or width of a set of data. The larger the standard deviation, the greater the dispersion of the data points relative to the mean, and the greater the fluctuation of the data; conversely, the smaller the standard deviation, the more concentrated the data is around the mean, and the smaller the fluctuation.

[0143] The standard deviation is the square root of the variance. First, calculate the variance of the lateral distance, and then calculate the standard deviation of the lateral distance based on the variance.

[0144] Specifically, the variance of the lateral distance is calculated as follows: Where, σ 2 Let x be the variance, N be the total number of lateral distances, and x be the variance. i Let be the i-th horizontal distance, and μ be the mean of the horizontal distances.

[0145] S5022, If the standard deviation is less than a preset value, it is determined that the vehicle does not have the driving abnormality.

[0146] S5023, if the standard deviation is greater than or equal to the preset value, it is determined that the vehicle has the driving abnormality.

[0147] In this embodiment, if the standard deviation is less than the preset value, it indicates that the vehicle's trajectory fluctuation is relatively small, and it is determined that the vehicle does not have any driving abnormalities.

[0148] If the standard deviation is greater than or equal to the preset value, it indicates that the vehicle's trajectory fluctuates significantly and that the vehicle is experiencing abnormal driving behavior.

[0149] In this application, the vehicle's movement trajectory is used to determine whether there is any abnormal driving behavior, so that the user can be alerted as soon as possible to ensure the user's driving safety.

[0150] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0151] Corresponding to the target detection method in the tunnel described in the above embodiments, Figure 8 The diagram shows a structural block diagram of a target detection device in a tunnel provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0152] Reference Figure 8 The device 600 may include: a data acquisition module 610, a target detection module 620, and an abnormal target extraction model 630.

[0153] The data acquisition module 610 is used to acquire image data inside the tunnel.

[0154] The target detection module 620 is used to perform target detection on the image data using at least two of the sub-detection models in the target detection model, and to obtain the target detection result of each of the sub-detection models, wherein the at least two sub-detection models are different sub-detection models;

[0155] An abnormal target extraction model 630 is used to determine abnormal targets appearing in the image data based on the target detection results of each of the sub-detection models, wherein the abnormal target is a non-motorized vehicle.

[0156] In one possible implementation, the anomaly target extraction model 630 can specifically be used for:

[0157] If an abnormal target is found in the target detection result of at least one of the sub-detection models, then the abnormal target detected by at least one of the sub-detection models is determined to be an abnormal target appearing in the image data.

[0158] In one possible implementation, the abnormal target extraction model 630 is also connected to:

[0159] A time determination module is used to obtain the first time when each of the abnormal targets appears in the image data;

[0160] The result output module is used to extract image data corresponding to each first time point from the image data to obtain the anomaly detection result of the abnormal target in the tunnel.

[0161] In one possible implementation, the target detection result includes the detected vehicle, the vehicle's position in the image data, and the vehicle's speed;

[0162] The following are also connected to the target detection module 620:

[0163] The trajectory drawing module is used to construct the vehicle's movement trajectory based on the vehicle's position in the image data;

[0164] An abnormal event determination module is used to determine abnormal events in the tunnel based on the vehicle's movement trajectory, wherein the abnormal events include at least one of abnormal speed, lane change, and driving abnormality.

[0165] In one possible implementation, the exception event determination module can specifically be used for:

[0166] Based on the vehicle's movement trajectory, determine the lane number of the lane where the vehicle is located at each time.

[0167] Based on the lane number corresponding to each time moment, the movement time of the vehicle in each lane is calculated.

[0168] If the vehicle changes lanes from the first lane to the second lane at a given moment, the duration of its movement in the first lane exceeds a first preset duration, and the duration of its movement in the second lane exceeds a second preset duration, then it is determined that the vehicle has changed lanes.

[0169] In one possible implementation, the exception event determination module can specifically be used for:

[0170] Based on the movement trajectories of each vehicle, the total number of vehicles in the tunnel is determined;

[0171] If the total number is less than the preset number, determine whether the vehicle has an abnormal speed based on the detected vehicle speed;

[0172] If the total number is greater than or equal to the preset number, calculate a standard score of the vehicle's speed based on the detected vehicle speed;

[0173] Based on the standard score of the vehicle's speed, determine whether the vehicle has a speed anomaly.

[0174] In one possible implementation, the exception event determination module can specifically be used for:

[0175] Based on the vehicle's movement trajectory, calculate the lateral distance between the vehicle and the centerline of its lane at different times;

[0176] Based on the lateral distance, it is determined whether the vehicle exhibits the driving abnormality.

[0177] In one possible implementation, the exception event determination module can specifically be used for:

[0178] Calculate the standard deviation of the lateral distance based on the lateral distance;

[0179] If the standard deviation is less than the preset value, it is determined that the vehicle does not have the driving abnormality;

[0180] If the standard deviation is greater than or equal to the preset value, it is determined that the vehicle has the driving abnormality.

[0181] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0182] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0183] This application also provides a terminal device, see [link to relevant documentation] Figure 9 The terminal device 700 may include: at least one processor 710, a memory 720, and a computer program stored in the memory 720 and executable on the at least one processor 710. When the processor 710 executes the computer program, it implements the steps in any of the above-described method embodiments, for example... Figure 2 Steps S101 to S103 in the illustrated embodiment. Alternatively, when the processor 710 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 8 The functions of the data acquisition module 610 to the abnormal target extraction module 630 are shown.

[0184] For example, a computer program may be divided into one or more modules / units, one or more of which are stored in memory 720 and executed by processor 710 to complete this application. The one or more modules / units may be a series of computer program segments capable of performing specific functions, which describe the execution process of the computer program in terminal device 700.

[0185] Those skilled in the art will understand that Figure 9This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0186] The processor 710 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0187] The memory 720 can be an internal storage unit of the terminal device or an external storage device, such as a plug-in hard drive, a smart media card (SMC), a secure digital card (SD), or a flash card. The memory 720 is used to store the computer program and other programs and data required by the terminal device. The memory 720 can also be used to temporarily store data that has been output or will be output.

[0188] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0189] The target detection method in tunnels provided in this application can be applied to terminal devices such as computers, tablets, laptops, netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of terminal device.

[0190] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0191] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0192] In the embodiments provided in this application, it should be understood that the disclosed terminal devices, apparatuses, and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0193] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0194] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0195] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by one or more processors, it can implement the steps of the various method embodiments described above.

[0196] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by one or more processors, it can implement the steps of the various method embodiments described above.

[0197] Similarly, as a computer program product, when the computer program product is run on a terminal device, it enables the terminal device to implement the steps in the above-described method embodiments.

[0198] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0199] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A target detection method in a tunnel, characterized in that, include: Acquire image data of the tunnel interior; The image data is subjected to target detection using at least two of the sub-detection models in the target detection model, and the target detection result of each sub-detection model is obtained. The at least two sub-detection models are different sub-detection models. Based on the target detection results of each of the sub-detection models, abnormal targets appearing in the image data are identified, wherein the abnormal targets are non-motorized vehicles.

2. The target detection method in a tunnel as described in claim 1, characterized in that, The target detection models include the DINO model and the CO-DETR model.

3. The target detection method in a tunnel as described in claim 1, characterized in that, The step of determining abnormal targets in the image data based on the target detection results of each of the sub-detection models includes: If an abnormal target is found in the target detection result of at least one of the sub-detection models, then the abnormal target detected by at least one of the sub-detection models is determined to be an abnormal target appearing in the image data.

4. The target detection method in a tunnel as described in any one of claims 1 to 3, characterized in that, After determining the anomalous targets appearing in the image data based on the target detection results of each of the sub-detection models, the method further includes: Image data of each of the abnormal targets are extracted from the image data to obtain the anomaly detection results of the abnormal targets in the tunnel.

5. The target detection method in a tunnel as described in claim 4, characterized in that, Extracting image data of each anomalous target from the image data to obtain the anomaly detection results of the anomalous targets in the tunnel includes: Obtain the first time of appearance of each of the anomalous targets in the image data; The image data corresponding to each first time point is extracted from the image data to obtain the anomaly detection results of the abnormal targets in the tunnel.

6. The target detection method in a tunnel as described in claim 1, characterized in that, The target detection results include the detected vehicle, the vehicle's position in the image data, and the vehicle's speed; After performing target detection on the image data using at least two of the sub-detection models in the target detection model, and obtaining the target detection result of each sub-detection model, the method further includes: Based on the vehicle's position in the image data, the vehicle's movement trajectory is constructed; Based on the vehicle's movement trajectory, abnormal events in the tunnel are determined, wherein the abnormal events include at least one of abnormal speed, lane change, and driving abnormality.

7. The target detection method in a tunnel as described in claim 6, characterized in that, The step of determining abnormal events in the tunnel based on the vehicle's movement trajectory includes: Based on the vehicle's movement trajectory, determine the lane number of the lane where the vehicle is located at each time. Based on the lane number corresponding to each time moment, the movement time of the vehicle in each lane is calculated. If the vehicle changes from the first lane to the second lane at a certain moment, the duration of its movement in the first lane is greater than a first preset duration, and the duration of its movement in the second lane is greater than a second preset duration, then it is determined that the vehicle has changed lanes.

8. The target detection method in a tunnel as described in claim 6, characterized in that, The step of determining abnormal events in the tunnel based on the vehicle's movement trajectory includes: Based on the movement trajectories of each vehicle, the total number of vehicles in the tunnel is determined; If the total number is less than the preset number, determine whether the vehicle has an abnormal speed based on the detected vehicle speed; If the total number is greater than or equal to the preset number, calculate a standard score of the vehicle's speed based on the detected vehicle speed; Based on the standard score of the vehicle's speed, determine whether the vehicle has a speed anomaly.

9. The target detection method in a tunnel as described in claim 6, characterized in that, The step of determining abnormal events in the tunnel based on the vehicle's movement trajectory includes: Based on the vehicle's movement trajectory, calculate the lateral distance between the vehicle and the centerline of its lane at different times; Based on the lateral distance, it is determined whether the vehicle exhibits the driving abnormality.

10. The target detection method in a tunnel as described in claim 9, characterized in that, The determination of whether the vehicle exhibits the driving abnormality based on the lateral distance includes: Calculate the standard deviation of the lateral distance based on the lateral distance; If the standard deviation is less than the preset value, it is determined that the vehicle does not have the driving abnormality; If the standard deviation is greater than or equal to the preset value, it is determined that the vehicle has the driving abnormality.

11. A target detection device in a tunnel, characterized in that, include: The data acquisition module is used to acquire image data of the tunnel interior. The target detection module is used to perform target detection on the image data using at least two of the sub-detection models in the target detection model, and to obtain the target detection result of each of the sub-detection models, wherein the at least two sub-detection models are different sub-detection models; An abnormal target extraction model is used to determine abnormal targets appearing in the image data based on the target detection results of each of the sub-detection models, wherein the abnormal target is a non-motorized vehicle.

12. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the target detection method in the tunnel as described in any one of claims 1 to 9.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the target detection method in the tunnel as described in any one of claims 1 to 9.

Images