A data resource collection and annotation method based on big data mining

Abstract

The present invention proposes a data resource collection and annotation method based on big data mining, which relates to the technical field of data collection and annotation. The specific implementation steps include: collecting multi-perspective traffic images, traffic flow data and speed data of the target road section in real time, and pre-processing the multi-perspective traffic images, dividing them into traffic image groups in chronological order; secondly, calculating the traffic concentration index in the first-perspective traffic image and identifying the blocked area, mapping the blocked area to the traffic image of another perspective, and performing congestion annotation; then, through the projection of overlapping detection boxes and traffic element point cloud computing, combined with the vehicle sag detection model, the traffic accidents are annotated; finally, the congestion annotation results are subjected to environmental consistency verification and annotation conflict verification to ensure the accuracy and integrity of the data. This method improves the processing accuracy of traffic data, provides effective support for traffic management and decision-making, and optimizes the operating efficiency of the intelligent transportation system.

Description

Technical Field

[0001] The present invention relates to the technical field of data collection and annotation, and in particular to a data resource collection and annotation method based on big data mining. Background Art

[0002] With the acceleration of urbanization and increased population mobility, problems such as traffic congestion and accidents are becoming increasingly prominent, posing a significant challenge to urban traffic management. As a crucial component of modern traffic management, intelligent transportation systems (ITS) leverage advanced technologies such as big data, the Internet of Things (IoT), and artificial intelligence (AI) to enable real-time monitoring and intelligent management of traffic flow, road conditions, and transportation facilities. In these systems, the collection and annotation of data resources play a crucial role. By using sensors, cameras, and other devices to collect and accurately annotate multi-dimensional data such as traffic flow, vehicle speed, and road conditions in real time, they not only enable real-time monitoring of traffic flow but also enable applications in traffic forecasting, accident analysis, and emergency response.

[0003] Currently, existing technologies rely solely on traffic image data to annotate traffic elements (such as vehicles, pedestrians, traffic signs, and traffic lights), ignoring the impact of traffic flow and traffic accidents on road conditions. This single-minded approach to data annotation limits a comprehensive understanding of complex traffic scenarios, resulting in many traffic congestion incidents and traffic accidents remaining unlabeled, making intelligent transportation systems less effective in traffic management and accident prevention.

[0004] To this end, a data resource collection and annotation method based on big data mining is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a data resource collection and annotation method based on big data mining, which is used to collect and annotate traffic data. In order to solve the problems existing in the prior art, the present invention collects multi-perspective traffic images, traffic flow data and speed data of the target road section in real time, and pre-processes the multi-perspective traffic images to divide them into traffic image groups in chronological order; secondly, calculates the traffic concentration index in the first-perspective traffic image and identifies the blocked area, maps the blocked area to the traffic image of another perspective, and performs congestion annotation; then, through the projection of overlapping detection frames and traffic element point cloud computing, combined with the vehicle sag detection model, the traffic accidents are annotated; finally, the congestion annotation results are subjected to environmental consistency verification and annotation conflict verification to ensure the accuracy and integrity of the data. This method improves the processing accuracy of traffic data, provides effective support for traffic management and decision-making, and optimizes the operating efficiency of the intelligent transportation system.

[0006] A data resource collection and annotation method based on big data mining, comprising:

[0007] Collect multi-view traffic images, traffic flow data, and vehicle speed data of a target road section in real time; pre-process the multi-view traffic images and divide them into traffic image groups in chronological order; and obtain vehicle detection frames and pedestrian detection frames in traffic images of different viewpoints in the traffic image group using a target detection algorithm;

[0008] Obtaining a position density index based on the position coordinates and confidence levels of different vehicle detection frames in the first-view traffic image of the traffic image group and determining whether the corresponding vehicles are densely packed; merging all densely packed vehicle detection frames to obtain a composite detection frame; calculating a traffic density index within the composite detection frame to determine whether it is a congestion area; mapping the congestion area to another traffic image from a different view to obtain a vehicle distribution area and determine whether it is a congestion area; and combining the determination results of traffic images from all viewpoints to perform congestion annotation.

[0009] Calculating contact risk indices for two different detection frames in the first-view traffic image and determining whether they overlap; mapping the overlapping detection frames to another traffic image from a different view to obtain a mapped detection frame; projecting the overlapping detection frame and the mapped detection frame into a world coordinate system to form a traffic element point cloud; and determining whether to perform an accident labeling based on the corresponding point cloud overlap ratio and the vehicle dent detection model;

[0010] An environmental consistency check is performed on the congestion labeling result according to the vehicle flow data and the vehicle speed data; and a labeling conflict check is performed in combination with the congestion labeling result and the accident labeling result.

[0011] Preferably, preprocessing the multi-view traffic image includes denoising, image enhancement, and standardization; the multi-view traffic image includes M traffic images from different viewpoints at different times; and the process of dividing the preprocessed multi-view traffic image into the traffic image groups according to the time sequence includes:

[0012] The preprocessed multi-perspective traffic images are sorted in chronological order to obtain M groups of time-series traffic images; the time of the first-perspective traffic image in the first group of time-series traffic images is obtained, and in combination with a predetermined time window, the image with the smallest time difference is matched in another M-1 groups of time-series traffic images; the first-perspective traffic image and the corresponding M-1 matching images are divided into a group to obtain the traffic image groups at different times; the traffic image group contains M perspective traffic images at the same time.

[0013] Preferably, the implementation process of determining whether to perform congestion marking by combining traffic images from different perspectives includes:

[0014] Obtain the first-perspective traffic image and another M-1-perspective traffic images of the traffic image group; obtain different vehicle detection frames in the first-perspective traffic image, determine whether different vehicles are densely packed, and obtain the synthetic detection frame; determine whether the synthetic detection frame is a blocked area based on the position coordinates and the number of vehicles in the synthetic detection frame; if the synthetic detection frame is a blocked area, obtain the vehicle distribution area corresponding to the synthetic detection frame in another M-1 different-perspective traffic images through a key point matching algorithm, and determine whether to perform congestion marking.

[0015] Preferably, the synthetic detection frame in the first-view traffic image is obtained, and whether it is a blocked area is determined:

[0016] Obtain different vehicle detection frames in the first-perspective traffic image and determine whether different vehicles are densely packed; calculate the position density index of the corresponding vehicles based on the confidence and position coordinates of the two different detection frames; if the position density index is greater than a predetermined density threshold, calculate the minimum outer frame of the two different detection frames to obtain the composite detection frame; repeat the density determination operation on the composite detection frame and the vehicle detection frame until both the vehicle detection frame and the composite detection frame do not meet the density condition; obtain the position coordinates and the number of vehicles inside the composite detection frame and calculate the traffic concentration index; if the traffic concentration index is greater than the predetermined density threshold, it indicates that the composite detection frame is a blocked area.

[0017] Preferably, the process of obtaining the vehicle distribution area corresponding to the synthetic detection frame in another M-1 traffic images with different viewing angles and determining whether to perform congestion marking includes:

[0018] Key point matching is performed on the first-perspective traffic image and the traffic image from another perspective to obtain the coordinates of the vehicle distribution area of ​​the congestion area in the traffic image from the other perspective; the number of vehicles corresponding to the vehicle distribution area is obtained through a target detection algorithm; the traffic concentration index is calculated based on the coordinates of the vehicle distribution area and the number of vehicles; whether it is a congestion area is determined based on a predetermined density threshold; the operation of determining the congestion area is repeated for another M-2 traffic images from different perspectives; if the vehicle distribution areas corresponding to all the traffic images from different perspectives are congestion areas, the vehicle distribution area and the synthetic detection frame are marked as congested; otherwise, no marking is performed.

[0019] Preferably, the specific process of determining whether to perform accident marking includes:

[0020] The vehicle detection frame and the pedestrian detection frame in the traffic image of the first perspective are obtained, the same vehicle detection frame and / or pedestrian detection frame in different perspectives are matched by a key point matching algorithm, and are projected into a world coordinate system to obtain a traffic element point cloud; the point cloud overlap ratio is calculated, and if the point cloud overlap ratio is greater than a predetermined overlap threshold, an accident is marked; if the point cloud overlap ratio is less than or equal to the predetermined overlap threshold, a dent detection is performed on the vehicle detection frame by a vehicle dent detection model; if the detection result is a dented vehicle, the vehicle detection frame is marked as an accident.

[0021] Preferably, the specific process of calculating the point cloud overlap ratio and performing accident marking includes:

[0022] Obtain the vehicle detection frame and the pedestrian detection frame in the first-perspective traffic image, and determine whether the two different detection frames overlap; calculate a contact risk index based on the position coordinates of the two detection frames; if the contact risk index is greater than a predetermined risk threshold, obtain the mapped detection frames of the two detection frames in another M-1-perspective traffic images through a key point matching algorithm; project the pixel coordinates of the corresponding detection frame or the mapped detection frame into the world coordinate system according to the built-in parameters of different cameras, and obtain the traffic element point cloud of the pedestrian or vehicle corresponding to the detection frame; calculate the point cloud overlap ratio; if the point cloud overlap ratio is greater than the predetermined overlap threshold, merge the corresponding detection frames and mark them as accidents.

[0023] Preferably, the specific process of performing sag detection on the vehicle detection frame using the vehicle sag detection model includes:

[0024] The vehicle dent detection model is constructed and trained on a public car damage dataset; the vehicle dent detection model includes a feature extraction module, a multi-view feature fusion module and a classifier; if the point cloud overlap ratio is less than or equal to the predetermined overlap threshold, a multi-view vehicle image corresponding to the vehicle detection frame is obtained; the multi-view vehicle image is input into the feature extraction module, and the edge, texture and shape features of the vehicle image are obtained through a multi-layer convolutional neural network to obtain a multi-view feature map; the multi-view feature map is input into the multi-view feature fusion module, and the feature maps of different viewpoints are spatially aligned through a key point matching algorithm, and the multi-view feature maps are spatially fused in combination with an attention mechanism and feature weighting to obtain a multi-view fusion map; the multi-view fusion map is subjected to nonlinear feature mapping by the classifier to obtain the vehicle category of the multi-view vehicle image; if the vehicle category is a dented vehicle, the corresponding vehicle detection frame is marked as an accident.

[0025] Preferably, the specific implementation process of performing the environmental consistency check on the congestion annotation result according to the traffic flow data and the vehicle speed data is as follows:

[0026] According to the congestion annotation results from different perspectives, the congested road section, congestion time and driving direction are determined; in combination with the traffic flow data and the vehicle speed data, the congested traffic flow and the average congested speed of the congested road section in the driving direction within a predetermined time before and after the congestion time are obtained; the monthly average traffic flow and the monthly average speed of the congested road section in the driving direction are calculated; if the congested traffic flow is less than or equal to the monthly average traffic flow, or the average congested speed is greater than the monthly average speed, it indicates that the congestion annotation result is wrong, and the corresponding image is fed back to the management personnel.

[0027] Preferably, the specific implementation process of the annotation conflict check is performed in combination with the congestion annotation result and the accident annotation result:

[0028] If the congestion annotation result and the accident annotation result exist in the same group of traffic images, the corresponding congestion annotation box and accident annotation box in the traffic image of the same perspective are obtained, and it is determined whether they overlap; if the congestion annotation box and the accident annotation box overlap, the annotation overlap rate is calculated; if the annotation overlap rates in the traffic images of different perspectives are all greater than the predetermined annotation overlap threshold, it indicates that there is a conflict between the congestion annotation result and the accident annotation result, and the corresponding image is fed back to the management personnel.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. This invention combines vehicle detection frames in traffic images from different perspectives for comprehensive judgment, enabling precise labeling of traffic congestion areas. By calculating the position density index and traffic concentration index between vehicles, it can accurately identify congestion areas in traffic images from the first perspective. It then obtains the vehicle distribution areas corresponding to the first-perspective congestion areas in traffic images from other perspectives and combines them with the traffic concentration index to further determine congestion. This avoids potential misjudgments from a single perspective and ensures the accuracy and reliability of congestion labeling.

[0031] 2. This invention effectively identifies traffic accidents through precise matching of vehicle and pedestrian detection frames and calculation of point cloud overlap ratios, improving the accuracy and efficiency of accident annotation. Using a key point matching algorithm, the detection frames from different perspectives are spatially projected to generate more comprehensive point cloud data, enhancing accident scene reconstruction capabilities. Combined with a vehicle dent detection model, the system extracts edge, texture, and shape features of the vehicle, enabling intelligent identification of vehicle damage. This extends accident annotation beyond simple overlap detection and further enhances the ability to identify complex traffic accidents.

[0032] 3. The present invention can effectively improve the accuracy of annotation results through environmental consistency verification and annotation conflict verification. After determining the congested road section, time, and driving direction, the calculation is combined with traffic flow and speed data to promptly detect annotation errors and feedback to management personnel, thereby improving the response efficiency of traffic management. Conflict verification combined with congestion and accident annotation results helps to identify and resolve possible annotation contradictions, ensure the integrity and consistency of traffic data, and improve the accuracy of annotation results, thereby effectively reducing traffic congestion and accidents and improving road use efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of a data resource collection and annotation method based on big data mining provided in an embodiment of the present invention;

[0034] Figure 2 A flowchart of congestion marking provided for an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of a synthetic detection frame acquisition process provided in an embodiment of the present invention;

[0036] Figure 4 A flowchart of accident marking provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] With the acceleration of urbanization, traffic congestion and frequent traffic accidents have become major challenges worldwide. Intelligent transportation systems have emerged, aiming to improve the efficiency and safety of traffic management through advanced information and communication technologies. These systems rely on massive amounts of traffic data to monitor traffic flow in real time, predict traffic conditions, and optimize resource allocation. In this context, the collection and annotation of traffic data are particularly important. Accurate traffic data not only provides the foundation for intelligent algorithms, but high-quality annotated data also enhances the accuracy of machine learning models, enabling effective analysis and prediction of traffic conditions. Therefore, building an efficient and accurate traffic data collection and annotation system is crucial for achieving the goals of intelligent transportation.

[0039] This invention proposes a data resource collection and annotation method based on big data mining. This method collects and annotates traffic data to improve the effectiveness of intelligent transportation systems in traffic management and accident prevention. To illustrate the effectiveness of this method, the invention is described in detail with reference to the accompanying drawings and the following implementation details.

[0040] Example 1

[0041] The embodiment of the present application discloses a data resource collection and annotation method based on big data mining to realize the collection and annotation of traffic data at intersection A. Figure 1 The specific steps of the method proposed in the present invention include: S1. real-time collection of multi-perspective traffic images, traffic flow data and vehicle speed data of the target road section; S2. pre-processing the multi-perspective traffic images and dividing them into traffic image groups in chronological order; S3. merging the detection frames of dense vehicles according to the position coordinates and confidence of the vehicle detection frames, and calculating the traffic concentration index to determine whether to perform congestion marking; S4. projecting the overlapping detection frames in the traffic images of different perspectives into the world coordinate system to obtain the traffic element point cloud, and combining the point cloud overlap ratio and the vehicle dent detection model to determine whether to perform accident marking; S5. performing environmental consistency verification and marking conflict verification on the congestion marking results and the accident marking results according to the traffic flow data and the vehicle speed data.

[0042] Furthermore, multi-perspective traffic images, traffic flow data, and vehicle speed data at intersection A are collected in real time; the multi-perspective traffic images are preprocessed and divided into traffic image groups in chronological order; vehicle detection frames and pedestrian detection frames are obtained in traffic images of different perspectives in the traffic image groups using a target detection algorithm, corresponding to steps S1 and S2 above; wherein the preprocessing of the multi-perspective traffic images includes denoising, image enhancement, and normalization; the multi-perspective traffic images include M traffic images of different perspectives at different times; and the process of dividing the preprocessed multi-perspective traffic images into the traffic image groups in chronological order includes:

[0043] The preprocessed multi-perspective traffic images are sorted in chronological order to obtain M groups of time-series traffic images; the time of the first-perspective traffic image in the first group of time-series traffic images is obtained, and in combination with a predetermined time window, the image with the smallest time difference is matched in another M-1 groups of time-series traffic images; the first-perspective traffic image and the corresponding M-1 matching images are divided into a group to obtain the traffic image groups at different times; the traffic image group contains M perspective traffic images at the same time.

[0044] The embodiments of the present application improve the image quality and reduce interference factors through denoising, image enhancement and standardization, thereby ensuring the accuracy of subsequent analysis; sort the multi-perspective traffic images in chronological order, so that the traffic conditions under different perspectives can be effectively compared and analyzed, and the time difference between perspectives can be minimized, thereby improving the similarity and consistency of the traffic image group; by dividing each first-perspective traffic image and the corresponding matching image into a group, a rich multi-perspective data set can be formed, providing data support for subsequent traffic status analysis and promoting the effective implementation of the intelligent transportation system.

[0045] Furthermore, the position coordinates and confidence levels of different vehicle detection frames in the first-view traffic image of the traffic image group are used to determine whether the corresponding vehicles are dense; the detection frames of densely packed vehicles are merged to obtain a composite detection frame, and the traffic concentration index is calculated to determine whether it is a congestion area; the area is mapped to traffic images of different viewpoints to obtain a vehicle distribution area, and it is determined whether it is a congestion area; based on the judgment results, the traffic image group is congested, corresponding to the above-mentioned step S3; refer to Figure 2 The specific implementation process includes:

[0046] Obtain the first-perspective traffic image and another M-1-perspective traffic images of the traffic image group; obtain different vehicle detection frames in the first-perspective traffic image, determine whether different vehicles are densely packed, and obtain the synthetic detection frame; determine whether the synthetic detection frame is a blocked area based on the position coordinates and the number of vehicles in the synthetic detection frame; if the synthetic detection frame is a blocked area, obtain the corresponding vehicle distribution area in the other M-1 different-perspective traffic images of the synthetic detection frame through a key point matching algorithm, and determine whether to perform congestion marking.

[0047] The embodiment of the present application analyzes the vehicle detection frames of traffic images from different perspectives in a traffic image group, and combines the position coordinates and confidence information to effectively determine the density of vehicles and identify potential congestion areas, thereby monitoring and improving traffic flow in real time and improving traffic efficiency.

[0048] Furthermore, the synthetic detection frame in the first-view traffic image is obtained, and whether it is a blocked area is determined:

[0049] Obtain different vehicle detection frames in the first-view traffic image and determine whether different vehicles are densely packed. Calculate the position density index of the corresponding vehicles based on the confidence and position coordinates of the two different detection frames. The calculation formula is expressed as:

[0050]

[0051] α c =1-pc ,α d =p d ;

[0052] Among them, DI c,d Represents the position density index of the detection box c and the detection box d; exp() represents the power function with a natural constant as the base; x c ,y c Respectively represent the horizontal and vertical coordinates of the upper left corner of the detection box c; w c ,h c Respectively represent the width and height of the detection box c; x d ,y d Respectively represent the horizontal and vertical coordinates of the upper left corner of the detection box d; w d ,h d Respectively represent the width and height of the detection box d; α c ,α d represents the adjustment parameter; p c ,p d Represent the confidence of detection frames c and d respectively. If detection frames c and / or d are composite detection frames, their corresponding confidence is the average confidence of all vehicle detection frames in the composite detection frame;

[0053] If the position density index is greater than a predetermined density threshold, the minimum outer bounding box of the two different detection frames is calculated to obtain the composite detection frame; the density determination operation is repeated for the composite detection frame and the vehicle detection frame until both the vehicle detection frame and the composite detection frame do not meet the density condition; the position coordinates and the number of vehicles inside the composite detection frame are obtained, and the traffic concentration index is calculated. The specific calculation formula is expressed as follows:

[0054]

[0055] Among them, VD i N represents the traffic concentration index of the i-th synthetic detection frame; i represents the number of vehicles inside the i-th synthetic detection frame; h i ,w i represents the height and width of the i-th synthetic detection frame;

[0056] If the traffic concentration index is greater than a predetermined density threshold, it indicates that the synthetic detection frame is a blocked area.

[0057] Specifically, see Figure 3Repeating the density determination operation on the synthesized detection frame indicates that the synthesized detection frame is regarded as the vehicle detection frame. By calculating the position density index of the synthesized detection frame and the remaining vehicle detection frames or synthesized detection frames, it is determined whether to merge the synthesized detection frame again to obtain a new synthesized detection frame. If neither the vehicle detection frame nor the synthesized detection frame meets the density condition, it means that the position density index between the remaining detection frames in the first-view traffic image cannot meet a predetermined density threshold, thereby obtaining a final synthesized detection frame.

[0058] In order to further illustrate the role of the traffic concentration index proposed in the present invention in determining whether the synthetic detection frame in the first-perspective traffic image is congested, the traffic concentration indices of different synthetic detection frames in the first-perspective traffic image at intersection A are given as an example for comparison. Refer to Table 1 for a list of the traffic concentration indices of different synthetic detection frames in the first-perspective traffic image at intersection A.

[0059] Table 1. Traffic concentration index of different synthetic detection frames in the first-person view traffic image at intersection A

[0060]

[0061] The data in Table 1 shows that the size of the composite detection frame and the number of vehicles within it are correlated with traffic congestion. For example, for composite detection frames 1 and 3, where the size of the frames is similar, the larger the number of vehicles within them, the more likely the area is congested. In other words, within a similar spatial range, a smaller number of vehicles indicates smoother traffic.

[0062] The embodiments of the present application utilize the aforementioned method to effectively identify congested areas in traffic images, thereby improving traffic management efficiency. By utilizing the density determination between different vehicle detection frames, changes in traffic flow can be accurately captured; by calculating the vehicle location density index, potential congested areas can be identified; and the generation of synthetic detection frames and further density determination ensure accurate identification of congested areas. This method not only improves the accuracy of traffic status monitoring but also provides important data support for the optimization of intelligent transportation systems, thereby improving overall traffic efficiency and safety.

[0063] Furthermore, the process of obtaining the vehicle distribution area corresponding to the synthetic detection frame in another M-1 traffic images with different viewing angles and determining whether to perform congestion marking includes:

[0064] Key point matching is performed on the first-perspective traffic image and the traffic image from another perspective to obtain the coordinates of the vehicle distribution area of ​​the congestion area in the traffic image from the other perspective; the number of vehicles corresponding to the vehicle distribution area is obtained through a target detection algorithm; the traffic concentration index is calculated based on the coordinates of the vehicle distribution area and the number of vehicles; whether it is a congestion area is determined based on a predetermined density threshold; the operation of determining the congestion area is repeated for another M-2 traffic images from different perspectives; if the vehicle distribution areas corresponding to all the traffic images from different perspectives are congestion areas, the vehicle distribution area and the synthetic detection frame are marked as congested; otherwise, no marking is performed.

[0065] In the traffic data annotation of Intersection A, traffic mobility and traffic concentration indices can show significant differences in images from different perspectives. Different perspectives provide a multi-dimensional observation of traffic conditions, which can more comprehensively capture the true nature of traffic congestion. For example, certain vehicle detection frames may appear overlapping in a traffic image from one perspective, but non-overlapping in another. Therefore, by analyzing images from multiple perspectives to ensure the accuracy and reliability of blocked area judgments, we can not only reduce misjudgments caused by a single perspective, but also improve the overall understanding of traffic conditions and ensure the quality of annotations.

[0066] By performing key point matching and target detection on traffic images from multiple perspectives, the present embodiment accurately obtains the coordinates of vehicle distribution areas and the corresponding number of vehicles, thereby effectively calculating the traffic concentration index. This method not only improves the accuracy of identifying congested areas but also comprehensively considers image information from different perspectives to ensure consistency and reliability of judgment. When a vehicle distribution area is confirmed to be spatially congested under all perspectives, the area is marked as congested, providing accurate data support for traffic management and intelligent navigation, significantly improving the efficiency of traffic flow monitoring and management, and helping to alleviate traffic congestion.

[0067] Furthermore, it is determined whether two different detection frames in the first perspective traffic image overlap, and the overlapping detection frames in the traffic images of different perspectives are obtained; the overlapping detection frames are projected into the world coordinate system to obtain the traffic element point cloud; by calculating the overlapping number ratio of the point clouds corresponding to the two different detection frames, combined with the vehicle dent detection model, it is determined whether the accident is marked, corresponding Figure 1 S4 step of Figure 4 The specific process includes:

[0068] The vehicle detection frame and the pedestrian detection frame in the traffic image of the first perspective are obtained, the same vehicle detection frame and / or pedestrian detection frame in different perspectives are matched by a key point matching algorithm, and are projected into a world coordinate system to obtain a traffic element point cloud; the point cloud overlap ratio is calculated, and if the point cloud overlap ratio is greater than a predetermined overlap threshold, an accident is marked; if the point cloud overlap ratio is less than or equal to the predetermined overlap threshold, a dent detection is performed on the vehicle detection frame by a vehicle dent detection model; if the detection result is a dented vehicle, the vehicle detection frame is marked as an accident.

[0069] The embodiment of the present application can effectively improve the accuracy and efficiency of accident annotation by analyzing the overlap of different detection frames in the first-person perspective traffic image and projecting the overlapping frames into the world coordinate system to obtain a traffic element point cloud. This method uses a key point matching algorithm to associate vehicle or pedestrian detection frames from different perspectives to ensure data consistency and integrity; it calculates the point cloud overlap ratio to enable timely accident annotation and ensure timely handling of accidents; and when the overlap ratio is insufficient, the vehicle is further analyzed using a vehicle dent detection model to ensure that every vehicle that may have damage is accurately labeled. This method not only enhances the accuracy of accident identification, but also improves the overall efficiency of traffic safety management.

[0070] Furthermore, the specific process of calculating the point cloud overlap ratio and marking the accident includes:

[0071] Obtain the vehicle detection frame and the pedestrian detection frame in the first-view traffic image, and determine whether the two different detection frames overlap; calculate the contact risk index based on the position coordinates of the two detection frames; the specific calculation formula is:

[0072]

[0073] Among them, CP c,d represents the contact risk index of detection box c and detection box d; x c ,y c Respectively represent the horizontal and vertical coordinates of the upper left corner of the detection box c; w c ,h c Respectively represent the width and height of the detection box c; x d ,y d Respectively represent the horizontal and vertical coordinates of the upper left corner of the detection box d; w d ,h d Respectively represent the width and height of the detection box d; max() represents the maximum value function;

[0074] If the contact risk index is greater than a predetermined risk threshold, the mapping detection frames of the two detection frames in the other M-1 viewpoint traffic images are obtained through a key point matching algorithm. The pixel coordinates of the corresponding detection frame or the mapping detection frame are projected into the world coordinate system according to the built-in parameters of different cameras to obtain the traffic element point cloud of the pedestrian or vehicle corresponding to the detection frame. The specific formula is expressed as follows:

[0075] [X,Y,Z] T =R×[X cam ,Y cam ,Z cam ] T +t c ;

[0076]

[0077] Wherein, [X, Y, Z] represents the traffic element point cloud; [] T Represents the matrix transpose operation; R represents the rotation matrix of the camera; t c Indicates the coordinates of the camera in the world coordinate system; [X cam ,Y cam ,Z cam ] represents the coordinates in the camera coordinate system; D represents the camera depth, that is, the distance between the camera and the vehicle; Represents the camera's built-in parameter matrix, f x ,f y is the focal length of the camera on the horizontal and vertical axes, c x ,c y is the center coordinate of the image; u, v represent the horizontal and vertical coordinates of the key point in the image; [] -1 Represents the matrix inversion operation;

[0078] The point cloud overlap ratio is calculated. If the point cloud overlap ratio is greater than the predetermined overlap threshold, the corresponding detection frames are merged and marked as an accident.

[0079] Specifically, based on the traffic element point clouds of the targets corresponding to the two detection frames, the number of overlapping point clouds is calculated; the edge traffic element point cloud of one of the targets is obtained, and the traffic element point cloud of the other target is compared with the edge traffic element point cloud to screen out the number of point clouds within the edge traffic element point cloud area; combined with the total number of point clouds of the corresponding targets, the point cloud overlapping number ratio is obtained.

[0080] The method in this embodiment of the application effectively identifies key elements in traffic accidents by calculating the point cloud overlap ratio and contact risk index, thereby improving the accuracy of accident detection. By acquiring traffic images and their detection frames from different perspectives and mapping them using a key point matching algorithm, multi-perspective information integration is achieved, enabling a more comprehensive assessment of potential collisions between vehicles and pedestrians. Projecting the pixel coordinates of the detection frames onto the world coordinate system accurately captures the point clouds of traffic elements representing pedestrians and / or vehicles, further analyzing the spatial relationships between different targets and improving the efficiency of accident identification.

[0081] Furthermore, the specific process of performing sag detection on the vehicle detection frame using the vehicle sag detection model includes:

[0082] The vehicle dent detection model is constructed and trained on a public car damage dataset. The vehicle dent detection model includes a feature extraction module, a multi-view feature fusion module, and a classifier. If the point cloud overlap ratio is less than or equal to the predetermined overlap threshold, a multi-view vehicle image corresponding to the vehicle detection frame is obtained. The multi-view vehicle image is input into the feature extraction module, and the edge, texture, and shape features of the vehicle image are obtained through a multi-layer convolutional neural network to obtain a multi-view feature map. The multi-view feature map is input into the multi-view feature fusion module, and the spatial feature alignment of the feature maps of different viewpoints is performed using a key point matching algorithm. The spatial feature fusion of the multi-view feature maps is performed by combining the attention mechanism and feature weighting to obtain a multi-view fusion map. The specific formula is:

[0083]

[0084] Among them, F multi F represents the multi-view fusion graph; j W represents the multi-view feature map of the j-th view; j Represents the weight coefficient of the multi-view feature map of the j-th view; Softmax() represents the softmax activation function; Flatten() represents the sequence expansion operation; () T Indicates transpose operation; d f Represents the matrix Flatten(F j ) dimension;

[0085] The multi-view fusion image is subjected to nonlinear feature mapping by the classifier to obtain the vehicle category of the multi-view vehicle image; if the vehicle category is a dented vehicle, the corresponding vehicle detection frame is marked as an accident.

[0086] The embodiment of the present application improves the accuracy and efficiency of vehicle dent detection through a vehicle dent detection model. The model's training on a public dataset of automobile damage ensures its good adaptability to various damage situations, thereby improving recognition reliability. The multi-layer convolutional neural network of the feature extraction module can accurately capture the edge, texture, and shape features in vehicle images, providing rich information for subsequent feature fusion. The multi-view feature fusion module, combined with the key point matching algorithm and attention mechanism, can effectively align and weight feature maps from different perspectives, ensuring the comprehensiveness and consistency of multi-view information. Through the nonlinear feature mapping of the classifier, the model can not only accurately identify dented vehicles, but also quickly annotate related detection boxes.

[0087] Furthermore, the congestion marking result is subjected to the environmental consistency check according to the traffic flow data and the vehicle speed data; Figure 1 The specific implementation process of step S5 is as follows:

[0088] According to the congestion annotation results from different perspectives, the congested road section, congestion time and driving direction are determined; in combination with the traffic flow data and the vehicle speed data, the congested traffic flow and the average congested speed of the congested road section in the driving direction within a predetermined time before and after the congestion time are obtained; the monthly average traffic flow and the monthly average speed of the congested road section in the driving direction are calculated; if the congested traffic flow is less than or equal to the monthly average traffic flow, or the average congested speed is greater than the monthly average speed, it indicates that the congestion annotation result is wrong, and the corresponding image is fed back to the management personnel.

[0089] The embodiments of this application can effectively improve the accuracy of congestion annotation results by verifying the environmental consistency of traffic flow data and speed data. This process not only accurately identifies congested road sections, congestion time, and travel direction, but also dynamically analyzes actual traffic conditions to ensure that the annotation results are consistent with the actual situation. By calculating the monthly average traffic flow and speed of congested sections, annotation errors can be promptly discovered and corrected, thereby providing accurate information to traffic management personnel, improving road traffic efficiency, and alleviating traffic congestion.

[0090] Furthermore, the congestion marking result and the accident marking result are combined to perform the marking conflict check; Figure 1 The specific implementation process of step S5 is as follows:

[0091] If the congestion annotation result and the accident annotation result exist in the same group of traffic images, obtain the corresponding congestion annotation frame and accident annotation frame in the traffic image of the same viewpoint, and determine whether they overlap; if the congestion annotation frame and the accident annotation frame overlap, calculate the annotation overlap rate. The specific formula is:

[0092]

[0093] Wherein, OR represents the overlap ratio of the annotations; A over Indicates the overlapping area of ​​the congestion labeling box and the accident labeling box. If the two labeling boxes do not overlap, then A over =0;A ca A represents the area of ​​the congestion mark box; ia represents the area of ​​the accident marking box; min() represents the minimum function;

[0094] If the annotation overlap rates in traffic images from different viewing angles are all greater than a predetermined annotation overlap threshold, it indicates that there is a conflict between the congestion annotation result and the accident annotation result, and the corresponding image is fed back to the management personnel.

[0095] By combining congestion and accident annotation results to perform annotation conflict verification, this embodiment of the present application can effectively improve the accuracy and reliability of the traffic management system. When congestion and accident annotations are present in the same set of traffic images, the system automatically identifies and analyzes the overlap between these annotations, calculating the annotation overlap ratio. If the annotation overlap ratio in traffic images from different perspectives exceeds a predetermined threshold, a potential annotation conflict is identified. This process optimizes traffic flow management, reduces false alarms, and improves incident response efficiency, thereby enhancing overall traffic safety and traffic efficiency.

[0096] The embodiment of the present application achieves accurate labeling of traffic congestion areas by making comprehensive judgments based on the vehicle detection frames in traffic images with different perspectives. The specific implementation process mainly includes the following steps: (1) Identifying the congestion area in the multi-perspective traffic image; (2) Identifying the accident area in the multi-perspective traffic image; (3) Promptly discovering labeling errors and feeding them back to management personnel through environmental consistency verification and labeling conflict verification. The present invention proposes a congestion labeling method, an accident labeling method and a labeling result verification for the above three processes respectively; the congestion labeling method realizes a comprehensive detection of traffic flow conditions and improves the accuracy of congestion labeling results; the accident labeling method further enhances the ability to identify complex traffic accidents; the labeling result verification realizes the consistency of traffic labeling data, thereby improving the accuracy of labeling results.

[0097] Example 2

[0098] In Example 1, the method of the present invention realizes the collection and annotation of traffic data at intersection A. In the embodiment of this application, the method proposed by the present invention will be described again to realize the collection and annotation of traffic data at roundabout B. The specific implementation process is as follows:

[0099] Multi-perspective traffic images, traffic flow data, and vehicle speed data of the target road section are collected in real time; the multi-perspective traffic images are preprocessed and divided into traffic image groups in chronological order; and vehicle detection frames and pedestrian detection frames are obtained from traffic images of different perspectives in the traffic image group using a target detection algorithm.

[0100] Furthermore, the position density index is obtained through the position coordinates and confidence levels of different vehicle detection frames in the first-perspective traffic image of the traffic image group, and it is judged whether the corresponding vehicles are dense; the detection frames of all dense vehicles are merged to obtain a synthetic detection frame; the traffic concentration index in the synthetic detection frame is calculated to determine whether it is a congestion area; the congestion area is mapped to another traffic image with a different perspective to obtain a vehicle distribution area, and it is determined whether it is a congestion area; and congestion marking is performed by combining the judgment results of traffic images from all perspectives.

[0101] Furthermore, the contact risk index of two different detection frames in the first-view traffic image is calculated, and whether they overlap is determined; the overlapping detection frames are mapped to another traffic image from a different view to obtain a mapped detection frame; the overlapping detection frames and the mapped detection frames are projected into a world coordinate system to form a traffic element point cloud; and whether to perform accident labeling is determined based on the corresponding point cloud overlap ratio and the vehicle dent detection model. The specific implementation process includes:

[0102] Obtain the vehicle detection frame and the pedestrian detection frame in the first-perspective traffic image, and determine whether the two different detection frames overlap; calculate a contact risk index based on the position coordinates of the two detection frames; if the contact risk index is greater than a predetermined risk threshold, obtain the mapped detection frames of the two detection frames in another M-1-perspective traffic images through a key point matching algorithm; project the pixel coordinates of the corresponding detection frame or the mapped detection frame into the world coordinate system according to the built-in parameters of different cameras, and obtain the traffic element point cloud of the pedestrian or vehicle corresponding to the detection frame; calculate the corresponding point cloud overlap ratio; if the point cloud overlap ratio is greater than the predetermined overlap threshold, merge the corresponding detection frames and mark them as accidents; if the point cloud overlap ratio is less than or equal to the predetermined overlap threshold, detect the multi-perspective vehicle image through a vehicle dent detection model to obtain the vehicle type; if the vehicle category is a dented vehicle, mark the corresponding vehicle detection frame as an accident.

[0103] When annotating traffic data for roundabout B, the point cloud overlap ratio can effectively assess the degree of interaction between two detection frames. When different detection frames overlap, calculating the point cloud overlap ratio can reveal the potential collision risk between traffic participants, which is crucial for timely identifying possible accidents. This is especially true in complex traffic environments, such as roundabouts, where there is frequent interaction between vehicles and pedestrians or other vehicles. The point cloud overlap ratio can help determine whether there is an actual collision risk. In cases where the contact risk index is high, by integrating information captured by different cameras and constructing three-dimensional point cloud data, traffic accident judgments can be made from multiple perspectives, thereby improving the accuracy of accident annotation. At the same time, dent detection can analyze vehicle damage. When a dented vehicle is detected, it indicates that the vehicle may have been involved in a traffic accident. Therefore, the corresponding detection frame is supplemented with annotations, improving the completeness of the accident annotation.

[0104] Furthermore, based on the traffic flow data and the vehicle speed data, the congestion labeling results are checked for environmental consistency; and the congestion labeling results and the accident labeling results are combined to perform labeling conflict verification. The specific implementation process of the labeling conflict verification is as follows:

[0105] If the congestion annotation result and the accident annotation result exist in the same group of traffic images, obtain the corresponding congestion annotation frame and accident annotation frame in the traffic image of the same viewpoint, and determine whether they overlap; if the congestion annotation frame and the accident annotation frame overlap, calculate the annotation overlap rate. The specific formula is:

[0106]

[0107] Wherein, OR represents the overlap ratio of the annotations; A over Indicates the overlapping area of ​​the congestion labeling box and the accident labeling box. If the two labeling boxes do not overlap, then A over =0;A ca A represents the area of ​​the congestion mark box; ia represents the area of ​​the accident marking box; min() represents the minimum function;

[0108] If the annotation overlap rates in traffic images from different viewing angles are all greater than a predetermined annotation overlap threshold, it indicates that there is a conflict between the congestion annotation result and the accident annotation result, and the corresponding image is fed back to the management personnel.

[0109] In order to further illustrate the role of the annotation overlap rate proposed in the present invention in determining the accuracy of the annotation results, the annotation overlap rates of traffic images at different viewing angles at the B roundabout are given as an example for comparison. Table 2 lists the annotation overlap rates of traffic images at different viewing angles at the B roundabout; the unit of area is (pixel 2 ).

[0110] According to the data analysis in Table 2, a certain relationship exists between the annotation overlap rate of traffic images from different viewpoints at Roundabout B and the areas of the congestion and accident annotation boxes. For example, in image TX01, the congestion annotation box has an area of ​​15444 square meters, the accident annotation box has an area of ​​1440 square meters, and the overlap area is 782 square meters. The annotation overlap rate reaches 0.54, which is well above the predetermined threshold of 0.3. This indicates significant overlap between the two annotation boxes, indicating a possible annotation error and requiring further processing. In image TX03, the overlap area is 0, resulting in a 0 annotation overlap rate, indicating no overlap between the annotation boxes, meaning there is no conflict between the two annotations.

[0111] Table 2. Annotation overlap rate of traffic images from different perspectives at roundabout B

[0112]

[0113] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A data resource collection and annotation method based on big data mining, characterized in that: include: Collect multi-view traffic images, traffic volume data, and vehicle speed data of the target road section in real time; pre-process the multi-view traffic images and divide them into traffic image groups in chronological order; Obtain vehicle detection frames and pedestrian detection frames in traffic images of different viewpoints in the traffic image group through the target detection algorithm; The position density index is obtained by using the position coordinates and confidence levels of different vehicle detection frames in the first-view traffic image of the traffic image group, and the corresponding vehicle density is determined. All dense vehicle detection frames are merged to obtain a composite detection frame. If the position density index is greater than a predetermined density threshold, the minimum outer bounding box of the two different detection frames is calculated to obtain a composite detection frame. The density determination operation is repeated for the composite detection frame and the vehicle detection frame until both the vehicle detection frame and the composite detection frame do not meet the density condition. The traffic concentration index within the composite detection frame is calculated to determine whether it is a congestion area. The congestion area is mapped to another traffic image with a different view to obtain the vehicle distribution area and determine whether it is a congestion area. The congestion annotation is performed by combining the judgment results of traffic images from all viewpoints. Calculate the contact risk index of two different detection frames in the first-person traffic image and determine whether they overlap; map the overlapping detection frames to another traffic image from a different perspective to obtain a mapped detection frame; project the overlapping detection frames and the mapped detection frames into the world coordinate system to form a traffic element point cloud; combine the corresponding point cloud overlap ratio and the vehicle dent detection model to determine whether to perform accident labeling; if the point cloud overlap ratio is greater than a predetermined overlap threshold, perform accident labeling; If the point cloud overlap ratio is less than or equal to the predetermined overlap threshold, the vehicle detection frame is subjected to dent detection using the vehicle dent detection model; if the detection result is a dented vehicle, the vehicle detection frame is marked as an accident. Based on the traffic flow data and speed data, the congestion labeling results are checked for environmental consistency; combined with the congestion labeling results and accident labeling results, labeling conflict verification is performed.

2. The data resource collection and annotation method based on big data mining according to claim 1 is characterized in that: Preprocessing the multi-view traffic image includes denoising, image enhancement and standardization; the multi-view traffic image includes M view traffic images at different times; The process of dividing the pre-processed multi-view traffic image into the traffic image groups according to the time sequence includes: Sorting the pre-processed multi-view traffic images in chronological order to obtain M groups of time-series traffic images; Obtain the time of the first-perspective traffic image in the first group of time-series traffic images, and match the image with the smallest time difference in another M-1 groups of time-series traffic images in combination with a predetermined time window; divide the first-perspective traffic image and the corresponding M-1 matching images into a group to obtain the traffic image group at different times; the traffic image group includes the M-perspective traffic images at the same time.

3. The data resource collection and annotation method based on big data mining according to claim 1 is characterized in that: The implementation process of combining traffic images from different perspectives to determine whether to mark congestion includes: Obtain the first-perspective traffic image and another M-1-perspective traffic images of the traffic image group; obtain different vehicle detection frames in the first-perspective traffic image, determine whether different vehicles are densely packed, and obtain the synthetic detection frame; determine whether the synthetic detection frame is a blocked area based on the position coordinates and the number of vehicles in the synthetic detection frame; if the synthetic detection frame is a blocked area, obtain the vehicle distribution area corresponding to the synthetic detection frame in another M-1 different-perspective traffic images through a key point matching algorithm, and determine whether to perform congestion marking.

4. The data resource collection and annotation method based on big data mining according to claim 3 is characterized in that: The process of obtaining the vehicle distribution area corresponding to the synthetic detection frame in the other M-1 traffic images with different viewing angles and determining whether to perform congestion marking includes: Key point matching is performed on the first-perspective traffic image and the traffic image from another perspective to obtain the coordinates of the vehicle distribution area of ​​the congestion area in the traffic image from the other perspective; the number of vehicles corresponding to the vehicle distribution area is obtained through a target detection algorithm; the traffic concentration index is calculated based on the coordinates of the vehicle distribution area and the number of vehicles; whether it is a congestion area is determined based on a predetermined density threshold; the operation of determining the congestion area is repeated for another M-2 traffic images from different perspectives; if the vehicle distribution areas corresponding to all the traffic images from different perspectives are congestion areas, the vehicle distribution area and the synthetic detection frame are marked as congested; otherwise, no marking is performed.

5. The data resource collection and annotation method based on big data mining according to claim 1 is characterized in that: The specific process of calculating the point cloud overlap ratio and marking accidents includes: Obtain the vehicle detection frame and the pedestrian detection frame in the first-perspective traffic image, and determine whether the two different detection frames overlap; calculate the contact risk index based on the position coordinates of the two detection frames; if the contact risk index is greater than a predetermined risk threshold, obtain the mapped detection frames of the two detection frames in another M-1-perspective traffic images through a key point matching algorithm; project the pixel coordinates of the corresponding detection frame or the mapped detection frame into the world coordinate system according to the built-in parameters of different cameras, and obtain the traffic element point cloud of the pedestrian or vehicle corresponding to the detection frame; calculate the corresponding point cloud overlap ratio; if the point cloud overlap ratio is greater than the predetermined overlap threshold, merge the corresponding detection frames and mark them as accidents.

6. The data resource collection and annotation method based on big data mining according to claim 1 is characterized in that: The specific process of performing sag detection on the vehicle detection frame by using the vehicle sag detection model includes: The vehicle dent detection model is constructed and trained on a public car damage dataset; the vehicle dent detection model includes a feature extraction module, a multi-view feature fusion module and a classifier; if the point cloud overlap ratio is less than or equal to the predetermined overlap threshold, a multi-view vehicle image corresponding to the vehicle detection frame is obtained; the multi-view vehicle image is input into the feature extraction module, and the edge, texture and shape features of the vehicle image are obtained through a multi-layer convolutional neural network to obtain a multi-view feature map; the multi-view feature map is input into the multi-view feature fusion module, and the feature maps of different viewpoints are spatially aligned through a key point matching algorithm, and the multi-view feature maps are spatially fused in combination with an attention mechanism and feature weighting to obtain a multi-view fusion map; the multi-view fusion map is subjected to nonlinear feature mapping by the classifier to obtain the vehicle category of the multi-view vehicle image; if the vehicle category is a dented vehicle, the corresponding vehicle detection frame is marked as an accident.

7. The data resource collection and annotation method based on big data mining according to claim 1 is characterized in that: The specific implementation process of performing the environmental consistency check on the congestion marking result according to the traffic flow data and the vehicle speed data is as follows: According to the congestion annotation results from different perspectives, the congested road section, congestion time and driving direction are determined; in combination with the traffic flow data and the vehicle speed data, the congested traffic flow and the average congested speed of the congested road section in the driving direction within a predetermined time before and after the congestion time are obtained; the monthly average traffic flow and the monthly average speed of the congested road section in the driving direction are calculated; if the congested traffic flow is less than or equal to the monthly average traffic flow, or the average congested speed is greater than the monthly average speed, it indicates that the congestion annotation result is wrong, and the corresponding image is fed back to the management personnel.

8. The data resource collection and annotation method based on big data mining according to claim 1 is characterized in that: The specific implementation process of performing the labeling conflict check based on the congestion labeling result and the accident labeling result is as follows: If the congestion annotation result and the accident annotation result exist in the same group of traffic images, the corresponding congestion annotation box and accident annotation box in the traffic image of the same perspective are obtained, and it is determined whether they overlap; if the congestion annotation box and the accident annotation box overlap, the annotation overlap rate is calculated; if the annotation overlap rates in the traffic images of different perspectives are all greater than the predetermined annotation overlap threshold, it indicates that there is a conflict between the congestion annotation result and the accident annotation result, and the corresponding image is fed back to the management personnel.

Images