Traffic incident illegal behavior early warning system based on AI and multi-source data fusion

Abstract

The invention provides a traffic incident illegal behavior early warning system based on AI and multi-source data fusion, relates to the technical field of intelligent traffic management, and aims to realize timestamp alignment, image feature extraction and difference analysis by obtaining videos of multiple cameras and external traffic data, distinguish repeated video information and complementary video information, and improve the early warning efficiency. The similarity threshold is automatically and dynamically adjusted according to data such as traffic flow, weather and vehicle speed; according to the method, space correction between cameras is carried out by utilizing an automatic calibration subunit, multi-partition management is carried out on a target road on the basis of a detection result, and cooperative correction is carried out on an overlapping region of adjacent partitions, so that the robustness of cross-camera tracking and illegal behavior recognition is improved; the system can also compare vehicle tracks in combination with road geometric information, accurately detect behaviors such as overspeed, illegal lane change and retrograde driving, and output corresponding illegal early warning signals.

Description

Technical Field

[0001] The present application relates to the field of intelligent traffic management technology, and more specifically, to a traffic incident violation warning system based on AI and multi-source data fusion. Background Art

[0002] In the field of intelligent traffic management, it is usually necessary to continuously monitor vehicles and traffic events in the target road area in order to timely detect and warn of traffic violations or potential safety risks. In the prior art, a single camera or a combination of multiple cameras is often used for traffic video acquisition. Although a certain degree of illegal capture and traffic flow analysis can be achieved, there are still problems such as inaccurate image similarity judgment, inability to dynamically adapt to different traffic flows and weather environments, and redundant data in overlapping areas between cameras or insufficient blind spot recognition, resulting in high missed detection and false alarm rates. Especially when there is overlap or partial occlusion of multiple cameras in the coverage area, there is a lack of effective difference analysis and adjacent partition collaborative correction mechanism, making it difficult to accurately obtain the complete trajectory of the vehicle and identify potential violations in real time, affecting the overall traffic monitoring efficiency and safety management effect. Therefore, a solution that can adaptively integrate external traffic data in a multi-camera monitoring scenario is needed to further improve the detection accuracy and warning timeliness of traffic event violations. Summary of the invention

[0003] In view of the shortcomings of the prior art, the embodiment of the present application provides a traffic incident violation warning system based on AI and multi-source data fusion, including: An acquisition unit, which acquires video data collected by multiple cameras deployed in a target road area and multi-source traffic data provided by at least one external data source; A detection unit, based on the video data, determines the camera images with overlapping coverage areas and generates a detection result; wherein the detection result includes repeated video information and complementary video information; A fusion unit, which performs fusion analysis on the detection result and the multi-source traffic data to generate traffic violation indication information; The warning unit generates a warning signal based on the traffic violation indication information; wherein the warning signal is used to indicate potential and / or ongoing traffic violation events.

[0004] As an optional implementation manner, generating the detection result includes: Performing time stamp alignment on the video data collected by the multiple cameras to generate multiple image frame sets; For the image frame set, extract image features using a preset model to generate a feature vector for evaluating picture similarity; Based on the feature vector, determining the picture similarity between multiple image frames in the same image frame set; In response to the picture similarity score being greater than or equal to a first threshold, marking the corresponding image frame as repeated video information; For the image frames whose picture similarity scores are greater than or equal to the second threshold and less than the first threshold, performing difference analysis and generating complementary video information; The image frames marked as repeated video information and complementary video information are integrated to generate the detection result, and the detection result is output to the fusion unit.

[0005] As an optional implementation, the detection unit further includes: The automatic calibration subunit obtains the spatial mapping relationship of the multiple cameras based on the feature points of the target road area, and performs spatial correction on the video data before calculating the picture similarity.

[0006] As an optional implementation manner, the detection unit performs a first dynamic adjustment on the first threshold and the second threshold based on the multi-source traffic data; The multi-source traffic data includes: traffic volume, weather conditions and vehicle speed; Among them, the traffic flow is obtained through radar detection data, the weather conditions are obtained through meteorological data, and the vehicle speed is obtained through Internet of Vehicles information.

[0007] As an optional implementation manner, the detection unit is further configured as: Based on the detection result, the target road area is divided into a plurality of camera partitions; wherein the cameras corresponding to the image frames whose picture similarity scores are greater than or equal to a second threshold are regarded as the same partition; In each of the camera partitions, based on the local multi-source traffic data, calculating the partition environment complexity index of the camera partition; Based on the partition environment complexity index, performing a second dynamic adjustment on the first threshold and the second threshold; The first threshold and the second threshold of different camera partitions are independent of each other.

[0008] As an optional implementation manner, the detection unit is further configured as: When overlapping areas are detected between different partitions, the corresponding partitions are marked as adjacent partitions; For the adjacent partitions, the similarity determination and difference analysis results are collaboratively corrected.

[0009] As an optional implementation, the collaborative correction includes: Based on the overlapping degree of coverage areas between the adjacent partitions, confidence weighting processing is performed on the similarity determination and the difference analysis results; In response to detecting that the same target vehicle and / or the same frame image generates a conflict determination in the adjacent partitions, recalculating the similarity score and / or the difference score through cross-validation; In response to completing the recalculation, the repeated video information and the complementary video information are updated and marked, and the updated detection result is output.

[0010] As an optional implementation, the multi-source traffic data further includes: road geometry information; the fusion analysis of the detection result and the multi-source traffic data to generate traffic violation indication information includes: Based on the image frames marked as repeated video information, the consistency of the continuous tracking trajectory of the target vehicle in the overlapping area covered by multiple cameras is confirmed, and the repeated video information is used as the vehicle trajectory benchmark; Based on the image frames marked with complementary video information, the moving position of the target vehicle under different viewing angles or partial occlusion conditions is identified to generate vehicle trajectory data; Aligning and comparing the vehicle trajectory data with the road geometry information to generate a comparison result; Based on the comparison result, determining whether the target vehicle has a preset illegal behavior; In response to the target vehicle having the preset illegal behavior, corresponding vehicle identification information and timestamp are extracted, and indication information for indicating the illegal behavior is generated.

[0011] As an optional implementation manner, the generating complementary video information includes: Performing foreground extraction processing on the image frame to determine a suspicious target area; Matching the suspicious target area with the vehicle position and / or lane information in the multi-source traffic data to generate a difference parameter; In response to the difference parameter being greater than or equal to a preset determination parameter, marking the corresponding image frame as complementary video information; In response to the difference parameter being smaller than the preset determination parameter, the corresponding image frame is updated to repeated video information.

[0012] Compared with the prior art, this application improves the ability to continuously track vehicles across cameras by introducing the distinction and application of repeated video information and complementary video information in a multi-camera networking scenario, reduces missed detections caused by repeated identification or occlusion of the same target, and can obtain a more complete target trajectory through the complementarity of different perspectives. The mechanism of dynamically adjusting the similarity threshold based on multi-source traffic data enables the system to flexibly determine the similarity and difference of the images in complex environments such as a sudden increase in traffic volume, bad weather, or high-speed driving of vehicles, thereby reducing the misjudgment rate caused by environmental factors. On this basis, the environmental complexity index of local partitions is used to independently adjust the threshold, and a collaborative correction method is used in overlapping areas between partitions, so as to achieve a more sophisticated multi-camera fusion monitoring strategy, which not only ensures robustness on busy roads, but also avoids waste of resources and unnecessary false alarms in areas with low traffic volume. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a traffic incident violation warning system based on AI and multi-source data fusion provided in an embodiment of the present application; Figure 2 A schematic diagram of camera partitioning provided in an embodiment of the present application; Figure 3 A flowchart of a method for generating complementary video information provided in an embodiment of the present application.

[0014] Figure numerals: 10, acquisition unit; 20, detection unit; 30, fusion unit; 40, early warning unit. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0016] See also Figure 1 FIG. 1 is a schematic diagram of a traffic incident violation warning system based on AI and multi-source data fusion provided in an embodiment of the present application. The system includes an acquisition unit 10, a detection unit 20, a fusion unit 30, and a warning unit 40, wherein: An acquisition unit 10 acquires video data collected by multiple cameras deployed in a target road area and multi-source traffic data provided by at least one external data source; The detection unit 20 determines the camera images with overlapping coverage areas based on the video data and generates a detection result; wherein the detection result includes repeated video information and complementary video information; A fusion unit 30 performs fusion analysis on the detection result and the multi-source traffic data to generate traffic violation indication information; The warning unit 40 generates a warning signal based on the traffic violation indication information; wherein the warning signal is used to indicate potential and / or ongoing traffic violation events.

[0017] The application as a whole can be deployed in a traffic monitoring center or cloud server to quickly detect and warn potential or ongoing traffic violations in the target road area. Among them, multi-source traffic data may include but is not limited to: radar detection data, meteorological data, vehicle network information, map data, etc.

[0018] When the road area is large or the road conditions are complex, if only a single camera is relied upon (regardless of whether its resolution is high or not), it is still difficult to fully grasp the continuous driving conditions of vehicles at different locations, and it is easy to miss detections and make false detections when the vehicle is partially blocked or visibility is reduced at night. In addition, in multi-camera networking scenarios, if there is a lack of overlapping analysis of the camera coverage area, the same target vehicle is often independently identified multiple times, and the system cannot make a comprehensive judgment across cameras, resulting in insufficient efficiency and accuracy in the identification and evidence collection of illegal behaviors.

[0019] This application emphasizes the use of the detection unit 20 to perform overlapping coverage analysis on the video data collected by multiple cameras, distinguishing between duplicate video information and complementary video information, thereby strengthening the continuous tracking of the same vehicle (reducing repeated calculations and redundant analysis), and using the complementary information provided by different perspectives to compensate for occluded areas and blind spots, thereby improving the accuracy of detection of illegal activities.

[0020] In addition, general video analysis algorithms judge the behavior of vehicles or pedestrians on the road, and often lack effective integration of external environment or traffic factors such as weather, traffic flow, and vehicle speed.

[0021] For example, in heavy rain, heavy snow, or sandstorms, the camera's image clarity and lighting conditions will be seriously affected; when traffic is extremely dense, single video analysis is prone to decreased detection performance or increased misjudgment rate.

[0022] This application can make a more comprehensive dynamic assessment of traffic conditions by integrating external data sources (such as radar detection data, meteorological data, and Internet of Vehicles information) with video detection results. For example, when traffic suddenly surges or the weather deteriorates, the similarity threshold can be increased or decreased, the detection area can be increased or decreased, etc.; and when determining illegal behavior, multi-dimensional data such as vehicle speed and lane deviation can also be combined to reduce missed detections or misjudgments caused by visual interference.

[0023] For the above acquisition unit 10: In specific implementation, multiple high-definition cameras (with infrared or night vision functions) can be installed at each traffic artery or key intersection in the target road area to achieve 24-hour monitoring. The cameras can be connected to the Internet and transmit the monitoring video stream to the data center or cloud in real time through optical fiber, 5G network or wired Ethernet.

[0024] At the external data source level, it can connect to the traffic management department's Internet of Vehicles platform, road radar detection system, meteorological department's weather database, etc., and obtain corresponding multi-source data through API interface or data subscription.

[0025] The acquisition unit 10 can be implemented by a high-performance server or an edge computing device, which is responsible for performing preliminary timestamp or location index association on the above-mentioned video stream and multi-source traffic data to ensure synchronous or approximately synchronous processing in the subsequent detection and analysis link.

[0026] For the above detection unit 20: In a specific implementation, the detection unit 20 can use a computer vision algorithm on a server or cloud to identify possible overlapping areas in different camera images, and determine which cameras' monitoring ranges overlap in space by comparing image content or road geographic location information.

[0027] In order to improve accuracy, the detection unit 20 can combine the calibration information of the camera (such as installation altitude, installation angle, lens focal length) and the map information of the target road area, and perform similarity analysis on the frame images through image recognition algorithms and feature matching algorithms (such as SIFT, SURF, ORB, etc.). When two or more video images are detected to present similar content in the same field of view (such as the same lane, the same vehicle), the part of the image is identified as duplicate video information, so as to be used for subsequent confirmation of vehicle trajectory consistency or location of illegal behavior.

[0028] On the other hand, images that can complement each other's target vehicle information from different perspectives are identified as complementary video information to help the algorithm obtain complete vehicle trajectory information in cases of partial occlusion, insufficient light, etc.

[0029] For the above fusion unit 30: In a specific implementation, the fusion unit 30 can make a comprehensive judgment based on an AI model (such as a deep learning network, an expert system based on fusion rules, etc.) by combining the detection results from different cameras with information such as traffic flow, weather, and vehicle speed provided by an external data source.

[0030] Vehicle cross-camera matching is performed on repeated video information to confirm whether the trajectory of the same vehicle in different images is continuous, and whether there are any illegal behaviors such as abnormal lane changes, speeding or crossing the line.

[0031] Complementary video information is analyzed to identify violations that may be blocked or unrecognized in the field of view of a single camera; at the same time, the traffic volume and vehicle network GPS information in multi-source data are matched with the vehicle identification information in the video to obtain a more comprehensive road traffic status. This multi-data fusion method can significantly improve the accuracy of illegal behavior identification and reduce the misjudgment rate.

[0032] Regarding the above-mentioned early warning unit 40: In a specific implementation, when the fusion unit 30 determines that there may be traffic violations (such as illegal lane change, speeding, running a red light, etc.), the warning unit 40 will generate a corresponding warning signal. The warning signal can be transmitted through the monitoring screen of the traffic command center, mobile terminal APP, SMS alarm, etc.

[0033] In terms of potential violation warning, the system can mark in real time vehicles that have not yet violated the law but have high-risk driving behaviors. For example, in rainy and snowy weather, when the speed is too fast and the traffic volume is heavy, if a vehicle is detected to change lanes frequently or is too close to the vehicle in front, a prompt message can be sent to the backend or roadside display screen.

[0034] In terms of detecting violations that have already occurred, the system can automatically capture or extract relevant video clips of violations, and upload them together with vehicle license plate, timestamp, location and other information to the database of relevant departments for subsequent punishment or intervention.

[0035] It should be noted that the functions of each unit can be performed by different functional modules on a single server or cloud platform, or preliminary video analysis can be performed on the edge side and the results can be uploaded to the cloud for secondary fusion.

[0036] Among them, the acquisition unit 10 needs to be connected to the SDK or network flow protocol of the camera manufacturer; the acquisition of external data sources can realize real-time or quasi-real-time data interaction based on RESTful API, MQTT message queue or WebSocket.

[0037] The core of the detection unit 20 lies in computer vision and image processing algorithms, for example, it may include a feature extraction model based on a convolutional neural network (CNN), a picture similarity calculation module based on traditional feature point matching, and automatic determination logic for overlapping areas.

[0038] The fusion unit 30 may be based on a big data platform or an AI analysis platform, and may process spatiotemporal data from multiple sources, and conduct a comprehensive comparison of vehicle trajectories, vehicle speeds, and lane usage, and ultimately derive information indicating illegal behavior.

[0039] The early warning unit 40 can push abnormal information or potential risk information to the traffic control management terminal or other user terminals in the first time based on event-driven message services.

[0040] In this way, the system provided by this application can realize full-time, all-round monitoring and early warning of road traffic violations based on the functional collaboration of the above-mentioned units. Through multi-source data fusion and intelligent analysis of AI algorithms, the system has advantages in improving traffic management efficiency, reducing manual inspection pressure, and reducing traffic accident risks. The system is particularly suitable for urban roads, highways and other scenes with dense traffic flow.

[0041] As an optional implementation manner, generating the detection result includes: Performing time stamp alignment on the video data collected by the multiple cameras to generate multiple image frame sets; For the image frame set, extract image features using a preset model to generate a feature vector for evaluating picture similarity; Based on the feature vector, determining the picture similarity between multiple image frames in the same image frame set; In response to the picture similarity score being greater than or equal to a first threshold, marking the corresponding image frame as repeated video information; For the image frames whose picture similarity scores are greater than or equal to the second threshold and less than the first threshold, performing difference analysis and generating complementary video information; The image frames marked as repeated video information and complementary video information are integrated to generate the detection result, and the detection result is output to the fusion unit 30.

[0042] In the specific implementation, the system first needs to perform timestamp alignment on the original video streams collected by multiple cameras to form a comparable set of image frames. To ensure the time synchronization of image frames between different cameras, the Network Time Protocol (NTP) or other distributed synchronization mechanisms can be used during the camera deployment phase or on the server side to uniformly calibrate the video streams of each camera.

[0043] The image frames acquired at different times are grouped according to their corresponding timestamps. Image frames with adjacent timestamps (e.g., at the same shooting time or in the same batch) can be merged into one image frame set.

[0044] In addition, in some cases, alignment error correction can be performed (for example, by interpolation, discarding invalid frames or redundant frames) to ensure that the physical scenes corresponding to the images of each camera have consistent or approximately consistent time windows during subsequent similarity analysis. Through the above timestamp alignment process, image frames from different cameras at the same time or in the same time window can be packaged into several image frame sets, providing a basic data structure for subsequent feature extraction and similarity analysis.

[0045] After acquiring the above-mentioned image frame set, the system will call a pre-configured image feature extraction model to process these frames and generate a feature vector for evaluating the similarity of the pictures.

[0046] In terms of the selection of preset models, the key features of the image can be extracted based on traditional feature point matching algorithms (such as SIFT, SURF, ORB) or convolutional neural network (CNN) models based on deep learning.

[0047] For each image frame, appearance features (such as color histogram, edge, texture, etc.) and / or semantic features (such as road position, vehicle bounding box, license plate information, etc.) are extracted, and these information are packaged as feature vectors and output to subsequent modules. After the feature vector generation is completed, the detection unit 20 performs pairwise or multi-pair comparisons on multiple frames (from different cameras) in the same image frame set to obtain a picture similarity score.

[0048] Among them, formulas such as cosine similarity, Euclidean distance, and Mahalanobis distance can be used to measure the similarity between feature vectors; metric learning in deep learning models can also be used to measure similarity. The similarity calculated for each pair (or group) of frames is output as a numerical result, such as a score between 0 and 1, or presented in percentage form.

[0049] In a specific implementation, once it is detected that the picture similarity score between certain image frames is greater than or equal to the first threshold, it means that the group of pictures is very likely to belong to the same road section, the same vehicle (or the same target) under the same perspective or the same scene. The system directly marks the corresponding image frame as duplicate video information. Further, this information can be stored in a database, and the corresponding timestamp, camera ID, etc. can be recorded so that the subsequent fusion unit 30 can perform cross-camera tracking and consistency confirmation.

[0050] For image frames with similarity scores between the second threshold and the first threshold, it means that they do not overlap highly with the reference frame, but still have many common features (it may be the same vehicle from different perspectives or similar scenes from a partially occluded perspective). At this point, the system will further perform a difference analysis; In specific implementations, the difference index can be calculated based on target contours, deformation, vehicle feature points (such as license plates, roof shapes), local textures and other information; the vehicle speed and position in the external data source can also be referenced to match the frame information to confirm whether it is the same vehicle / object. When the difference analysis results confirm that the group of frames are complementary in terms of perspective, vehicle position, etc., the system marks it as complementary video information so that the vehicle trajectory can be supplemented or corrected in subsequent fusion analysis.

[0051] After completing the above-mentioned marking operation, the detection unit 20 integrates the image frames corresponding to the repeated video information and the complementary video information to form a final detection result. The detection result includes: A list of frame pairs with high similarity (duplicate video information) to facilitate subsequent cross-camera consistent identification of vehicles; The frame pairs or frame groups (complementary video information) obtained after difference analysis can make up for the monitoring blind spots or occlusion problems under a single camera.

[0052] Exemplarily, the system may create a structured data record including a corresponding image frame ID, a similarity score, a difference analysis parameter, and a label indicating whether it is marked as “duplicate” or “complementary”.

[0053] Finally, the detection unit 20 sends the above-mentioned integrated detection results to the fusion unit 30 to combine more external data for in-depth illegal behavior identification.

[0054] The output may be in the form of a database interface, a message queue or a Web API, to ensure that the fusion unit 30 can receive the detection result in the first place and perform subsequent cross-camera vehicle tracking, violation determination and early warning processing.

[0055] In actual deployment, if there are low bandwidth or network delay problems, local edge computing nodes can be used to quickly determine the similarities and differences directly, and then only send the final marking information to the cloud, reducing data transmission and improving processing efficiency.

[0056] In this way, by setting the first threshold and the second threshold and adopting the difference analysis step, it is possible to effectively distinguish between highly similar picture frames (truly repeated information) and frames with certain similarity but complementary information under different viewing angles (complementary video information), thereby improving the overall detection accuracy. For frames between the second threshold and the first threshold, the difference analysis can be used to further determine whether there are problems such as partial occlusion and angle offset, rather than simply ignoring such frames to avoid losing possible illegal clues. The repeated video information and the complementary video information are integrated and output to the fusion unit 30, which is convenient for subsequent vehicle cross-camera recognition, trajectory splicing, and combining other multi-source data for higher-level illegal behavior judgment and early warning.

[0057] As an optional implementation, the detection unit 20 also includes an automatic calibration subunit, which can obtain the spatial mapping relationship between multiple cameras based on feature points in the target road area and perform necessary spatial correction on the video data before calculating the image similarity.

[0058] In the specific implementation, a number of feature points for calibration are first arranged in the target road area, such as road markings, traffic signs, curb corners or specially placed markers. By mapping these feature points or combining them with drone aerial photography, lidar scanning and other methods, their position information in the world coordinate system can be accurately obtained.

[0059] After receiving the images taken by different cameras, the automatic calibration subunit uses computer vision algorithms to identify the above-mentioned feature points in the images and matches them with the real coordinates, thereby solving the external and internal parameters of each camera through multi-view geometry or camera calibration algorithms (such as Zhang Zhengyou calibration method). Based on this calibration result, the automatic calibration subunit can perform projection transformation and distortion correction on the images captured by each camera, so that the images of the same vehicle or road section under different cameras can be mapped to the same reference coordinate system or top-down plane.

[0060] In this way, when performing similarity analysis, the differences caused by the camera installation position, shooting angle or lens distortion can be eliminated, thereby improving the accuracy of identifying overlapping images in the coverage area.

[0061] The present application further proposes another optional implementation, that is, in the detection unit 20, the system can make a first dynamic adjustment to the first threshold and the second threshold based on multi-source traffic data. The so-called first threshold and the second threshold are used to distinguish whether the similarity score between image frames is sufficient to be marked as duplicate video information or complementary video information, but in the actual environment, factors such as traffic flow, weather conditions and vehicle speed often affect the quality and difficulty of image recognition. For this reason, when the detection unit 20 receives real-time traffic flow, weather and Internet of Vehicles information from an external data source, the threshold can be automatically corrected.

[0062] For example, when the traffic volume increases sharply, the density of vehicles in the picture increases and occlusions become more frequent. The system can appropriately lower the similarity threshold to allow a more tolerant judgment range for frames that may belong to the same scene. In severe weather conditions, such as heavy rain, dense fog or dust, the picture quality is significantly impaired, and the threshold can be further lowered or raised in combination with meteorological data to reduce misjudgment caused by noise interference.

[0063] In addition, the real-time driving speed of vehicles in the Internet of Vehicles information also affects the judgment of picture similarity. If it is detected that vehicles generally pass at high speed, the vehicle position in the picture frame changes faster. The system can also adjust the threshold in time to avoid missing frames with shorter time domain overlap.

[0064] In this way, adaptive adjustment of the similarity threshold is achieved in complex and changeable actual road environments, thereby taking into account both accuracy and real-time performance and improving the robustness of multi-camera fusion monitoring.

[0065] It can be understood that the configuration of the above-mentioned automatic calibration sub-unit enables the images of different cameras to obtain unified spatial correction before entering the similarity analysis process, fundamentally reducing the errors caused by installation angles and lens differences; and the mechanism of dynamic adjustment of thresholds based on multi-source traffic data maintains the rationality and adaptability of similarity division when dealing with bad weather, sudden increases in traffic volume or abnormal vehicle speeds.

[0066] In an optional embodiment, the detection unit 20 first partitions the target road area based on the above detection result, which includes repeated video information and complementary video information obtained by analyzing the similarity or difference of different camera images based on the first threshold and the second threshold.

[0067] At this time, if the picture similarity score between certain image frames is greater than or equal to the second threshold, it means that there is a certain degree of overlap or complementary area in the monitoring range between the corresponding cameras. These cameras can be regarded as the same partition and classified into the same camera partition.

[0068] In other words, the detection unit 20 utilizes the camera IDs associated with these high-similarity or relatively high-similarity frames to aggregate cameras with similar or obvious overlapping monitoring ranges into a partition, thereby achieving a preliminary division of the target road area in space.

[0069] After completing the partitioning, the detection unit 20 calculates the partition environment complexity index within each camera partition based on the local multi-source traffic data.

[0070] Compared with the data of the entire area, local multi-source traffic data usually refers to the road conditions actually monitored or counted within the camera partition, including local traffic flow, local weather conditions (such as local heavy rain or dense fog sections), average vehicle speed and congestion level within the partition.

[0071] For example, if the traffic volume in the partition is prominent and the weather is bad, the camera image may be densely populated with vehicles and have obvious noise, which will pose challenges to similarity analysis and target recognition; in this case, the partition environment complexity index will take a higher value. On the other hand, if the traffic volume in the partition is not large, the weather is fine, and the vehicles are driving smoothly during the current period, the complexity index will be reduced accordingly.

[0072] After calculating the environment complexity index of each partition, the detection unit 20 may perform a second dynamic adjustment on the first threshold and the second threshold according to the index.

[0073] It should be noted that, unlike the first dynamic adjustment based on global multi-source traffic data, the second dynamic adjustment pays more attention to the real-time status of local partitions. Each partition can independently modify the judgment threshold originally used to divide duplicate video information and complementary video information according to its environmental complexity index.

[0074] Specifically, if the environmental complexity index of a certain partition is high, it means that it is necessary to be relatively more flexible in judging the similarity of the images to avoid being overly strict under high noise and high occlusion conditions, which may lead to missed detections or false detections; conversely, if the environmental complexity index of a certain partition is low, a stricter threshold can be set for similarity judgment, thereby reducing redundant calculations or unnecessary misjudgments in the area.

[0075] It should be noted that the first threshold and the second threshold between different camera partitions are independent of each other, and each partition can have its own threshold configuration. This can better match the actual situation of different road sections and different monitoring coverage areas.

[0076] For example, in different sections of the same city, some sections have high traffic flow and are easily affected by bad weather, while other sections have stable traffic flow and relatively mild weather. If a unified threshold strategy is forcibly used, it will cause misjudgment or oversensitivity in some sections. By subdividing the target road area and adjusting the threshold locally, the detection unit 20 can adapt to more types of road environments, greatly improving the accuracy and adaptability of the multi-camera fusion monitoring system.

[0077] In this way, by considering cameras with a screen similarity score greater than or equal to the second threshold as the same partition, calculating the environmental complexity index of each partition, and making a second dynamic adjustment to the first threshold and the second threshold accordingly, the detection unit 20 can implement a more flexible similarity determination mechanism at the local level. This mechanism not only enhances the robustness of the system in scenarios such as high traffic, bad weather or complex terrain, but also provides more accurate basic support for subsequent illegal behavior identification and real-time warning.

[0078] For example, see Figure 2 , Figure 2 A camera partition diagram provided in an embodiment of the present application; a number of cameras are deployed for monitoring at a section of a city where a main road and a branch road intersect. After automatically calibrating the installation positions, shooting angles, and coverage areas of multiple cameras (or manually calibrating them in advance), the detection unit 20 can perform similarity calculations on the video frames captured by these cameras. For ease of explanation, this example involves a total of five cameras, numbered C1, C2, C3, C4, and C5.

[0079] According to the above-mentioned embodiment, the detection unit 20 performs similarity analysis on the image frame set in the same time window. Statistics show that: The similarity scores between C1 and C2 and C3 are higher than the second threshold at most times; There are also a large number of similarity scores between C4 and C5 that are greater than or equal to the second threshold, but the similarity scores between C4, C2, and C3 are generally low.

[0080] Because there is overlap or complementary information between C1, C2, and C3, and the similarity score is stably greater than the second threshold, the detection unit 20 divides C1, C2, and C3 into camera partition A; For C4 and C5, although the similarity scores between them are high, the intersection with the camera coverage area in partition A (C1, C2, C3) is not large, and the similarity scores are not enough to meet the second threshold judgment standard, so C4 and C5 are divided into camera partition B.

[0081] As a result, the target road area is effectively divided into at least two relatively independent camera monitoring partitions, namely partition A and partition B. The division result can be recorded in the system in a data structure or visual form, for example: Partition A: includes cameras C1, C2, and C3; Partition B: Contains cameras C4 and C5.

[0082] For each camera partition, the detection unit 20 calculates an environmental complexity index (ECI) after combining the local multi-source traffic data. The index can take a value range of 0~1 or 0~100. The higher the value, the more complex the current environment of the partition. The following is a calculation example for each of the two partitions: Calculation of environmental complexity index for partition A: For local traffic flow, in the area monitored by zone A (such as the main road section), the traffic flow obtained by roadside radar or microwave detector is 2,400 vehicles / hour (peak period). Compared with the local road design flow standard, this value has exceeded the threshold of 80% of the saturation flow, so it can be determined that the traffic flow complexity is high.

[0083] As for the weather conditions, the meteorological data report shows that it is currently raining moderately. Visibility is moderate, but waterlogging on the road will cause vehicles to slightly deviate from their driving trajectory, and raindrops and light reflections will interfere with the picture.

[0084] As for the average vehicle speed, according to the statistics of the vehicle network information (V2X system), the average speed of vehicles in zone A is about 45 km / h, which is slightly lower than usual, indicating that the road has a certain congestion trend, but it has not yet been seriously congested.

[0085] The system performs weighted or rule-based operations on the above data and obtains the environmental complexity index ECI_A of partition A in the current period as 0.75 (when the value is between 0 and 1, 0.75 indicates medium to high complexity).

[0086] Calculation of the environmental complexity index for partition B: As for local traffic flow, the branch roads covered by Zone B have relatively small traffic flow, only 800 vehicles per hour, and are relatively smooth.

[0087] As for the weather conditions, it is also in moderate rainy weather, but due to the good drainage facilities of the branch road, the interference of water accumulation is not obvious. The impact of weather on visibility is similar to that of Zone A.

[0088] As for the average vehicle speed, the Internet of Vehicles statistics show that the average speed of vehicles in zone B is about 60 km / h, and there is no serious congestion.

[0089] During calculation, the system found that the partition was mainly affected by moderate rain and the traffic volume was not large. Therefore, after comprehensive weighting, the environmental complexity index ECI_B of partition B was 0.40 (the overall environmental interference was low).

[0090] Since the environmental complexity indexes of different partitions are significantly different, the detection unit 20 will make differential adjustments to the threshold strategies of the two partitions, thereby reducing missed detections or misjudgments in complex environments and avoiding wasting too many resources in simple scenarios.

[0091] For Partition A, threshold adjustment: Exemplarily, the initial first threshold T1_base and the second threshold T2_base (assuming that T1_base=0.80 and T2_base=0.60 respectively).

[0092] Combined with ECI_A=0.75, the system determines that zone A is in a high congestion and moderate rain environment, and the vehicle overlap phenomenon in the image may be obvious and there is a lot of noise interference. In order to improve the ability to capture potential duplicate or complementary images, the threshold can be slightly lowered, for example, the updated first threshold T1_A=0.78; the updated second threshold T2_A=0.58; In this way, partition A can more easily identify relatively similar image frames in the picture similarity score and regard them as repeated or complementary information, thus avoiding missed detections caused by traditional fixed thresholds.

[0093] For partition B, threshold adjustment: Also based on T1_base=0.80, T2_base=0.60. T1_base is the first threshold of partition B, and T2_base is the second threshold of partition B; Since ECI_B=0.40, the environmental interference is not great, and the system can appropriately increase or maintain the threshold, for example, the updated first threshold T1_B=0.82 in partition B; the updated second threshold T2_B=0.61; Compared with partition A, partition B has better image quality and greater vehicle dispersion, making it easier to distinguish between truly repeated and accidentally similar scenes. Therefore, using a slightly higher threshold can reduce misjudgment and improve efficiency.

[0094] In this way, each partition has a threshold setting that suits its actual situation, realizing a localized and refined screen similarity determination strategy. The threshold adjustments between partitions are independent of each other, avoiding the drawbacks of a one-size-fits-all approach that easily misses detection in high-complexity scenes and over-captures redundant information in low-complexity scenes.

[0095] This embodiment demonstrates the technology in urban roads, but it can also be extended to highways, bridges, tunnels or other scenes with multi-camera coverage. As long as there is a similarity analysis and partitioning requirement, this method can be effectively applied.

[0096] It is understandable that the specific calculation method of the aforementioned partition environment complexity index, the setting of the initial threshold, and the threshold update range or adjustment rules for each partition can be achieved through various machine learning and big data analysis methods. For example, the system can be trained based on a large amount of historical monitoring data and real road condition records using deep neural networks, random forests or other AI models to automatically learn the optimal threshold adjustment strategy; it can also combine expert rules and heuristic algorithms to perform more targeted optimization of the weight distribution in the complexity index calculation.

[0097] As an optional implementation manner, the detection unit 20 is further configured as: When overlapping areas are detected between different partitions, the corresponding partitions are marked as adjacent partitions; For the adjacent partitions, the similarity determination and difference analysis results are collaboratively corrected.

[0098] In actual road monitoring scenarios, although the detection unit 20 will divide the target road area into multiple camera partitions based on similarity scores and other results; however, in scenarios such as urban interchanges, complex road networks or multi-story elevated roads, different partitions are not always independent of each other, and often some areas are monitored simultaneously under the coverage of two or more partitions.

[0099] To this end, the present application adds corresponding processing logic in the detection unit 20. When the system recognizes that there are overlapping areas between different partitions, these partitions will be marked as adjacent partitions. At the same time, in order to ensure the consistency of recognition across partition boundaries, the system will coordinate the similarity judgment and difference analysis results.

[0100] In the specific implementation process, the detection unit 20 first identifies which partitions have overlapping images with high similarity in space or time based on the camera coverage, image similarity information, and previously generated repeated video information and complementary video information. If the cameras of two partitions have multiple image frames with a similarity score greater than or equal to a predetermined threshold (such as a second threshold) at the same time or adjacent times, the system determines that the monitoring areas of the two partitions have a significant overlapping relationship, and thus marks them as adjacent partitions. The system will maintain a partition relationship table internally to record the adjacent conditions of each partition, the corresponding overlapping area coordinates or overlapping frame information.

[0101] Next, the detection unit 20 will coordinately correct the similarity determination and difference analysis results within these adjacent partitions. Specifically, each partition usually has independent threshold parameters and difference analysis mechanisms to adapt to the partition's own traffic flow, weather conditions or camera configuration. However, when a vehicle or picture crosses adjacent partitions, if the two partitions have conflicting or inconsistent determination results on their similarity, occlusion status or vehicle feature information (for example, it is marked as complementary video information in partition A, but is determined to be duplicate video information in partition B), the system needs to conduct a unified review within the overlapping range of adjacent partitions, and use multi-source data or other feature points for comparison and cross-validation.

[0102] During the collaborative correction process, the system can determine which partitions have more accurate recognition results by weighting confidence, recalculating similarity scores, etc. If external vehicle location, speed, road geographic information, etc. confirm that the two partitions are shooting the same vehicle or the same road section, the system will correct the previous divergent similarity judgments or difference results, and update the marks of duplicate video information or complementary video information when necessary.

[0103] In this way, even if adjacent partitions use different threshold strategies or there are significant differences in shooting angles, collaborative correction can be used to maintain consistency in cross-partition recognition and reduce the risk of repeatedly identifying the same vehicle or misidentifying it as different objects.

[0104] For example, at the intersection of the city's elevated road and the ground auxiliary road, partition A mainly monitors the elevated lanes, while partition B is responsible for the ground auxiliary road. There is a certain overlap between the two in a certain section of the ramp area. At this time, a vehicle driving out at high speed will appear briefly in the picture of partition A, and then continue to enter the field of view of partition B. Since the installation angles and focal lengths of the cameras of the two are different, there may be some differences in the imaging of the vehicle's appearance, resulting in inconsistent determinations of the vehicle's category or similarity scores within each partition. The system triggers a collaborative correction process by identifying the overlapping areas of adjacent partitions here, and uniformly compares the frame sequences of the vehicle during the crossing of adjacent partitions. If it is confirmed to be the same vehicle, partition A or partition B will adjust its recognition results accordingly and update the screen mark to form a more complete cross-partition trajectory for the vehicle.

[0105] In this way, the present application effectively avoids the problems of repeated identification, misjudgment or information fragmentation across partition boundaries, and further enhances the overall accuracy and coordination of multi-camera fusion monitoring. After marking adjacent partitions and performing collaborative correction, the detection unit 20 can output more reliable and consistent repeated video information and complementary video information, providing a more robust data basis for the subsequent fusion unit 30 and the early warning unit 40. Combined with the aforementioned technical means such as dynamic adjustment of thresholds, partition environment complexity index, and automatic calibration sub-units, the present application provides a more complete solution for the accurate detection and real-time early warning of various traffic violations in urban roads, elevated interchanges, and complex road network environments.

[0106] As an optional implementation, the collaborative correction includes: Based on the overlapping degree of coverage areas between the adjacent partitions, confidence weighting processing is performed on the similarity determination and the difference analysis results; In response to detecting that the same target vehicle and / or the same frame image generates a conflict determination in the adjacent partitions, recalculating the similarity score and / or the difference score through cross-validation; In response to completing the recalculation, the repeated video information and the complementary video information are updated and marked, and the updated detection result is output.

[0107] In the specific implementation, under the processing framework of adjacent partitions, when it is confirmed that there is overlap in the monitoring areas between two or more camera partitions, it is necessary to perform more refined collaborative correction on the similarity judgment and difference analysis results across partitions.

[0108] Specifically, the detection unit 20 first performs confidence weighting processing on the obtained similarity determination and difference analysis results based on the overlapping degree of coverage areas between adjacent partitions.

[0109] For example, if partition A and partition B have a large overlapping monitoring area and have maintained highly consistent judgment results in multiple time periods in the past, it means that both have strong monitoring capabilities for adjacent areas; at this time, they can be given higher and similar confidence weights to ensure that no party is favored during fusion.

[0110] On the contrary, if the camera clarity or coverage of a certain partition is relatively weak, its weight in the adjacent partition will be relatively reduced to avoid the overall result being affected by the misjudgment of a single path.

[0111] After completing the initial confidence weighting, if the system further detects that the same target vehicle or the same frame has a conflict judgment between adjacent partitions, it will enter the cross-validation phase.

[0112] For example, partition A marks a vehicle as complementary video information, while partition B considers it to be duplicate video information; or partition A gives a low similarity score, while partition B gives a higher score, and there is a clear disagreement between the two.

[0113] At this time, the detection unit 20 can call external data or deeper visual features to perform cross-validation and recalculate the similarity score or the difference score to eliminate deviations caused by factors such as partition threshold differences, different occlusion perspectives, or data synchronization errors.

[0114] Exemplary, specific practices may include: Perform secondary comparison on the local features of the vehicle (license plate shape, body color / texture, feature point distribution); Check the car network GPS information, vehicle radar marking data or road geometry information to confirm whether it is the same vehicle appearing at the same time and in the same geographical location; According to the continuous motion trajectory of the previous and next frames, it is determined whether the recognition results of partition A and partition B are consistent in time and space.

[0115] After completing the cross-validation, the detection unit 20 will correct or update the duplicate video information and complementary video information tags previously generated in partition A and partition B based on the updated similarity score and / or difference score, and output the updated detection result.

[0116] The update process may include: If it is confirmed that it is the same vehicle and should be marked as duplicate video information, the original mark of the complementary information will be deleted; conversely, if it is confirmed that it is a different vehicle or the perspectives are complementary, the complementary video information will be retained and the duplicate mark will be revoked.

[0117] To avoid the subsequent recurrence of the same or similar conflicts, the system can record the cross-validation results in the database and adjust the confidence factor for partition A or partition B, making it easier to reach consensus in the same or similar situations in the future.

[0118] The final updated detection results will be packaged and sent to the fusion unit 30 to ensure that the input data for subsequent illegal behavior analysis or trajectory tracking is comprehensive and correct.

[0119] Through the above collaborative correction steps, the system will make more consistent and accurate identification conclusions for the same vehicle or the same frame at the junction of adjacent partitions, avoiding conflicting judgments or repeated identification caused by differences in thresholds and different degrees of occlusion within the partition. At the same time, the introduction of confidence weighting and cross-validation mechanisms can make full use of the complementary information of different cameras in adjacent partitions, improve the reliability and accuracy of multi-camera fusion monitoring in complex environments, and provide more robust data support for subsequent traffic violation identification and warning.

[0120] In a further embodiment of the present application, multi-source traffic data includes not only radar detection, meteorological data, vehicle networking information, etc., but also road geometry information of the target road area. The so-called road geometry information refers to the accurate description or vectorized map data of information such as road direction, lane line distribution, road curvature, number of lanes, slope, speed limit signs, etc. This information can come from a geographic information system (GIS), a high-precision map service, or a road network database of a traffic management department.

[0121] By introducing road geometry information, the system can accurately align and compare the vehicle trajectory with the actual road layout, so as to better judge whether there are behaviors such as speeding, illegal lane changes, driving in the wrong direction, occupying the emergency lane, etc.

[0122] In a specific implementation, repeated video information can be used to establish a vehicle trajectory benchmark. In an area where multiple cameras overlap, if a target vehicle is detected with a similarity score greater than or equal to a first threshold in different camera images, or if it is confirmed to be the same vehicle through collaborative correction of adjacent partitions, the relevant image frames are marked as repeated video information. These repeated video information can provide a stable and highly consistent image basis, and can therefore be regarded as a main line trajectory for vehicle cross-camera tracking.

[0123] The system can confirm the consistency of the target vehicle at the intersection of multiple cameras based on timestamps, vehicle appearance features or external Internet of Vehicles information, thereby forming a continuous record of the vehicle's driving path. This process can effectively reduce the uncertainty caused by obstructed vision or other noise from a single camera, making repeated video information a reliable benchmark for vehicle trajectory analysis.

[0124] In some scenarios, the vehicle may be photographed from different perspectives, or be affected by factors such as partial occlusion and lighting changes, making it difficult for a single camera to continuously identify it.

[0125] However, if the similarity score is between the second threshold and the first threshold, and the difference analysis confirms that these image frames belong to complementary video information, it means that these frames can supplement the vehicle position from different perspectives or in a partially occluded state.

[0126] Based on this, the fusion unit 30 can superimpose the complementary video information to infer the complete or more complete movement trajectory of the target vehicle in terms of time sequence. For example, when a vehicle temporarily enters a ramp or a blind spot and then returns to the main lane, the complementary video information can help to splice the missing path and reduce the loss of the vehicle.

[0127] After completing the fusion processing of repeated video information and complementary video information, a relatively complete vehicle trajectory data set is obtained, which includes information such as the vehicle's temporal position, speed, and camera area.

[0128] Next, the system will align this vehicle trajectory data with the road geometry information. This can include: If the vehicle trajectory is represented by pixel coordinates or camera coordinate system, it can be first projected or transformed into a global coordinate system or a map coordinate system via the automatic calibration subunit (as described above); Based on the road geometry information, identify which lane or ramp the vehicle is in, and determine the relationship between its travel path and the road lane lines, dividers, speed limit signs, etc.; By comparing the road's prescribed speed, curve radius, prohibited lane change section, emergency lane and other information, determine whether the vehicle trajectory conflicts with the prescribed traffic rules.

[0129] Through the above comparison process, a comparison result can be produced to clarify whether the target vehicle has any signs of violating traffic regulations. For example, if the vehicle's driving speed on a certain road section exceeds the speed threshold specified by the speed limit sign, or crosses the lane line multiple times in a prohibited lane change area, the comparison result will show potential violations.

[0130] Based on the comparison results, if the system determines that the target vehicle does indeed have a preset violation (such as speeding, illegal lane changing, driving in the opposite direction, occupying the emergency lane, etc.), the system will automatically extract relevant vehicle identification information, such as license plate number, vehicle model, color, as well as the timestamp and geographic location of the violation.

[0131] At the same time, the system will generate corresponding violation indication information according to relevant supervision or law enforcement needs. The indication information may include an encrypted vehicle identity data, violation type, time and location description, violation screenshots or short video clips, etc., which will be reported to the traffic law enforcement department in real time or sent to the traffic management backend for subsequent law enforcement or punishment.

[0132] Under this implementation, because the system introduces road geometry information and combines the advantages of repeated video information and complementary video information, it can more accurately locate the vehicle's state in the road space and maintain a complete grasp of the vehicle's trajectory in complex road conditions and multi-camera scenarios.

[0133] For management departments, this solution significantly improves the ability and accuracy of capturing various types of illegal behaviors, while reducing reliance on manual inspections and post-event video screenings, thereby achieving more efficient and intelligent road traffic supervision and early warning.

[0134] In this way, the present application establishes a trajectory baseline based on image frames marked as repeated video information, and improves the vehicle movement position in combination with image frames marked as complementary video information. The vehicle trajectory data is then aligned and compared using road geometry information to determine whether there is a specific traffic violation, and generates corresponding indication information after confirming the violation. This technical process not only improves the integrity of the multi-camera monitoring system in cross-camera tracking, but also significantly enhances the accuracy of violation detection and positioning.

[0135] See also Figure 3 , Figure 3 A flowchart of a method for generating complementary video information provided in an embodiment of the present application; as an optional implementation, the generating complementary video information includes S101 to S104, wherein: S101: performing foreground extraction processing on the image frame to determine a suspicious target area; S102: Matching the suspicious target area with the vehicle position and / or lane information in the multi-source traffic data to generate a difference parameter; S103: In response to the difference parameter being greater than or equal to a preset determination parameter, marking the corresponding image frame as complementary video information; S104: In response to the difference parameter being smaller than the preset determination parameter, updating the corresponding image frame to repeated video information.

[0136] In some cases, similarity scores alone cannot clearly determine whether an image frame is just a repeated shot of the same scene, or a complementary shot whose perspective difference can provide additional information. To this end, the present application proposes a technical solution for foreground extraction processing of image frames, that is, when it is detected that the similarity score falls between the second threshold and the first threshold (or other interval range set by the system), further foreground / background separation is performed to find suspicious target areas and perform difference analysis in combination with multi-source traffic data.

[0137] In a specific implementation, in the early image preprocessing stage, the detection unit 20 will use background subtraction, motion target detection or a semantic segmentation method based on deep learning to extract the contour area of ​​a vehicle or other suspicious moving targets for each frame of the image.

[0138] For example, commonly used algorithms for background subtraction include Mixture of Gaussians (MOG2), KNN method, etc.; deep learning methods can use models such as U-Net or Mask R-CNN to segment the foreground and background. On this basis, the detection unit 20 can obtain several foreground areas, which are suspicious target areas, including important objects such as vehicles and pedestrians.

[0139] After extracting the suspicious target area, the detection unit 20 will further compare the area with the vehicle position and lane information from external sources. The vehicle position here can come from the real-time GPS coordinates of the Internet of Vehicles system, the detection data of the roadside radar, or the vehicle coordinates calculated after integrating the calibration of each camera; the lane information includes the specific lane number, lane line position, current road speed limit, etc.

[0140] If the spatial position, shape or motion trajectory of the foreground target is significantly inconsistent with the vehicle positioning data or corresponding lane information at that moment, it means that this image frame may provide a different perspective or new available information, which helps to improve the trajectory of the target vehicle in scenes with occlusion or poor perspective.

[0141] On the contrary, if the detection results of the foreground target under different cameras are highly consistent with the multi-source traffic data, it may mean that the frame only repeatedly captures the same vehicle and the same lane position, and there is no substantial difference from the content captured by other cameras.

[0142] During this process, the system calculates a difference parameter to quantify the degree of difference between the suspicious target area and the existing vehicle or lane information.

[0143] For example, if the detected license plate position, size, trajectory inference speed, etc. of the foreground target deviates greatly from the external data and there is no other reasonable explanation (such as measurement error, time synchronization deviation), the difference value will be high; if the two are highly overlapped or well matched, the difference value will be very low.

[0144] When the difference parameter is greater than or equal to the preset judgment parameter threshold, it means that the suspicious target area provides information that is different from conventional or known data in time and space. Therefore, the corresponding image frame can be marked as complementary video information, which means that the vehicle position, outline or other elements are supplementarily identified from different perspectives, which can help the system understand the driving condition of the target vehicle more comprehensively.

[0145] If the difference parameter is less than the preset judgment parameter, it means that the frame is actually basically consistent with the scenes captured by other cameras and no effective additional information is introduced. At this time, the system can update it to repeated video information to avoid repeatedly occupying subsequent calculation or analysis resources for actually the same or similar pictures.

[0146] For example, taking the night time as an example, a camera captures a vehicle changing lanes at a long distance. Due to poor lighting, the image similarity algorithm cannot temporarily determine whether it is just an extension of the same picture as the vehicle captured by another camera. At this time, if the foreground extraction finds that the vehicle is near the lane line and its movement trend is slightly different from the GPS trajectory reported by the Internet of Vehicles, the difference parameter will be evaluated as high, which means that the frame can make up for the information loss caused by the night lighting difference, and is therefore marked as complementary video information. On the contrary, in the case of good lighting or the vehicle is obviously in the same position and posture, the system detects a small difference and updates it to duplicate information to maintain data consistency and simplicity.

[0147] This solution based on foreground extraction and difference parameter determination ensures that the system does not rely too much on a single similarity score, but can more flexibly and accurately identify images that complement vehicle detection under different cameras or different perspectives. Combined with the aforementioned threshold dynamic adjustment, automatic calibration sub-units, and adjacent partition collaborative correction mechanisms, the system can fully mine complementary information in complex road environments, occluded scenes, or large perspective differences, thereby further improving the overall accuracy and robustness of cross-camera vehicle recognition and violation detection.

[0148] Those skilled in the art can understand that in the above method of the specific implementation, the order of writing each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic. It should be understood that determining B based on A does not mean determining B based only on A, but B can also be determined based on A and / or other information.

[0149] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

Claims

1. Traffic incident violation warning system based on AI and multi-source data fusion, characterized by: The system comprises: An acquisition unit, which acquires video data collected by multiple cameras deployed in a target road area and multi-source traffic data provided by at least one external data source; A detection unit, based on the video data, determines the camera images with overlapping coverage areas and generates a detection result; wherein the detection result includes repeated video information and complementary video information; A fusion unit, which performs fusion analysis on the detection result and the multi-source traffic data to generate traffic violation indication information; The warning unit generates a warning signal based on the traffic violation indication information; wherein the warning signal is used to indicate potential and / or ongoing traffic violation events.

2. The traffic incident violation warning system based on AI and multi-source data fusion according to claim 1 is characterized in that: Generating the detection result comprises: Performing time stamp alignment on the video data collected by the multiple cameras to generate multiple image frame sets; For the image frame set, extract image features using a preset model to generate a feature vector for evaluating picture similarity; Based on the feature vector, determining the picture similarity between multiple image frames in the same image frame set; In response to the picture similarity score being greater than or equal to a first threshold, marking the corresponding image frame as repeated video information; For the image frames whose picture similarity scores are greater than or equal to the second threshold and less than the first threshold, performing difference analysis and generating complementary video information; The image frames marked as repeated video information and complementary video information are integrated to generate the detection result, and the detection result is output to the fusion unit.

3. The traffic incident violation warning system based on AI and multi-source data fusion according to claim 2 is characterized in that: The detection unit also includes: The automatic calibration subunit obtains the spatial mapping relationship of the multiple cameras based on the feature points of the target road area, and performs spatial correction on the video data before calculating the picture similarity.

4. The traffic incident violation warning system based on AI and multi-source data fusion according to claim 3 is characterized in that: The detection unit performs a first dynamic adjustment on the first threshold and the second threshold based on the multi-source traffic data; The multi-source traffic data includes: traffic volume, weather conditions and vehicle speed; Among them, the traffic flow is obtained through radar detection data, the weather conditions are obtained through meteorological data, and the vehicle speed is obtained through Internet of Vehicles information.

5. The traffic incident violation warning system based on AI and multi-source data fusion according to claim 4 is characterized in that: The detection unit is also configured as: Based on the detection result, the target road area is divided into a plurality of camera partitions; wherein the cameras corresponding to the image frames whose picture similarity scores are greater than or equal to a second threshold are regarded as the same partition; In each of the camera partitions, based on the local multi-source traffic data, calculating the partition environment complexity index of the camera partition; Based on the partition environment complexity index, performing a second dynamic adjustment on the first threshold and the second threshold; The first threshold and the second threshold of different camera partitions are independent of each other.

6. The traffic incident violation warning system based on AI and multi-source data fusion according to claim 5 is characterized in that: The detection unit is also configured as: When overlapping areas are detected between different partitions, the corresponding partitions are marked as adjacent partitions; For the adjacent partitions, the similarity determination and difference analysis results are collaboratively corrected.

7. The traffic incident violation warning system based on AI and multi-source data fusion according to claim 6 is characterized in that: The collaborative correction includes: Based on the overlapping degree of coverage areas between the adjacent partitions, confidence weighting processing is performed on the similarity determination and the difference analysis results; In response to detecting that the same target vehicle and / or the same frame image generates a conflict determination in the adjacent partitions, recalculating the similarity score and / or the difference score through cross-validation; In response to completing the recalculation, the repeated video information and the complementary video information are updated and marked, and the updated detection result is output.

8. The traffic incident violation warning system based on AI and multi-source data fusion according to claim 7 is characterized in that: The multi-source traffic data also includes: road geometry information; the fusion analysis of the detection result and the multi-source traffic data to generate traffic violation indication information includes: Based on the image frames marked as repeated video information, the consistency of the continuous tracking trajectory of the target vehicle in the overlapping area covered by multiple cameras is confirmed, and the repeated video information is used as the vehicle trajectory benchmark; Based on the image frames marked with complementary video information, the moving position of the target vehicle under different viewing angles or occlusion conditions is identified to generate vehicle trajectory data; Aligning and comparing the vehicle trajectory data with the road geometry information to generate a comparison result; Based on the comparison result, determining whether the target vehicle has a preset illegal behavior; In response to the target vehicle having the preset illegal behavior, corresponding vehicle identification information and timestamp are extracted, and indication information for indicating the illegal behavior is generated.

9. The traffic incident violation warning system based on AI and multi-source data fusion according to claim 2 is characterized in that: Generating complementary video information comprises: Performing foreground extraction processing on the image frame to determine a suspicious target area; Matching the suspicious target area with the vehicle position and / or lane information in the multi-source traffic data to generate a difference parameter; In response to the difference parameter being greater than or equal to a preset determination parameter, marking the corresponding image frame as complementary video information; In response to the difference parameter being smaller than the preset determination parameter, the corresponding image frame is updated to repeated video information.

Images