Multi-model fusion target vehicle identification analysis method and system based on machine vision

Abstract

The invention discloses a multi-model fusion target vehicle identification analysis method and system based on machine vision, and the method comprises the steps: A, capturing and collecting a road vehicle video / image through an unmanned plane, and transmitting the video / image to a streaming media server in a wireless manner; b, an improved target vehicle detection model is adopted, a multi-task cooperative detection framework is constructed, and vehicle illegal parking is judged by applying a multi-model fusion detection mechanism; c, designing a lightweight detection network, reducing the problems of high reasoning delay and large resource occupation in the multi-model cooperative detection process, and reducing the complexity of a detection algorithm; and D, carrying out intelligent alarm and event pushing. According to the invention, the method can improve the recognition accuracy of vehicle illegal parking behaviors in a complex scene, and improves the real-time performance and stability of a single-model monitoring system for vehicle target detection.

Description

Technical Field

[0001] This application relates to computer vision and deep learning technologies, and in particular to a multi-model fusion target vehicle recognition and analysis method and system based on machine vision. Background Technology

[0002] With the improvement of people's living standards and the continuous increase in car ownership, urban road traffic pressure is intensifying, and random and illegal parking has become one of the major causes of traffic congestion. Therefore, identifying, judging, and regulating illegal parking is of great significance for maintaining road traffic order, ensuring driving safety, improving urban management efficiency, and improving residents' quality of life.

[0003] Currently, vehicle illegal parking detection technology has become an important component of intelligent traffic management systems. However, existing vehicle illegal parking detection / recognition technologies still face many challenges, among which low detection / recognition efficiency is one of the most prominent issues, mainly manifested in the following aspects:

[0004] First, the existing vehicle illegal parking detection / identification technology mainly relies on two traditional methods: human supervision and fixed video surveillance. However, these two methods have certain limitations, as follows: First, the shortcomings of human supervision are mainly: (1) Limited coverage: It is difficult to achieve real-time coverage of all areas by relying on manual patrols, especially in urban areas with dense roads and changing illegal parking points, there are obvious blind spots in supervision; (2) Low patrol efficiency: Manual patrols need to check one by one, which is inefficient and easily leads to a time difference between the occurrence and handling of illegal parking, making it difficult to discover and collect evidence in a timely manner; (3) High labor costs: A large number of law enforcement personnel are required, and under long-term operation, human resources are tight, management costs are high, and maintenance is difficult; (4) High work intensity and easy to make subjective misjudgments: Law enforcement personnel are prone to fatigue due to long-term high-intensity work, which may lead to subjective problems such as missed detection and misjudgment, affecting the accuracy and fairness of law enforcement. Secondly, the shortcomings of fixed video surveillance methods are mainly: (1) Limited monitoring area: The camera deployment position is fixed, which can only cover some key areas and cannot cope with situations such as temporary changes in roads and relocation of illegal parking hotspots; (2) Blind spots and dead zones: The camera is affected by building obstruction, changes in light, and dense traffic, which can easily form blind spots and cannot obtain a full picture; (3) Reliance on manual inspection and low judgment efficiency: Video data requires real-time manual monitoring or post-event playback analysis, which is labor-intensive, inefficient, and prone to missed judgments and misjudgments; (4) Poor system scalability and high adjustment cost: Once the monitoring area needs to be adjusted or a new area needs to be expanded, the hardware needs to be redeployed, which involves high equipment and construction costs and lacks flexibility.

[0005] Secondly, although vehicle intelligent recognition technology is constantly being innovated, vehicle illegal parking intelligent detection technology still has many shortcomings, specifically: (1) The target feature extraction method is backward and the generalization ability is poor: Traditional methods usually rely on manually extracting image features for matching, which is difficult to adapt to the changes in the diversity of vehicle appearance and the complexity of the scene, resulting in low recognition accuracy in practical applications; (2) It is difficult to cope with complex traffic environments: Due to the variety of vehicle types, different shooting angles, and serious occlusion, traditional detection methods are prone to false detection or missed detection in multi-target dense scenes, which cannot meet the refined needs of urban traffic management; (3) The detection model is single and lacks multi-dimensional analysis capabilities: Existing systems mostly use a single detection model, which makes it difficult to achieve comprehensive perception of vehicle behavior, spatial position and scene semantics, limiting the system's adaptability and intelligence level in dynamic traffic environments.

[0006] In conclusion, traditional human-based monitoring (such as manual inspections and ground vehicle patrols) and fixed video surveillance for identifying illegal parking have many limitations. These limitations, constrained by manpower, resources, and environmental factors, result in low inspection efficiency, limited coverage, and safety hazards, thus limiting their effectiveness in detecting illegal parking. Furthermore, traditional identification methods have poor generalization capabilities, struggle to cope with complex traffic environments, and lack multi-dimensional analytical capabilities, thus failing to meet the real-time, comprehensive, and accurate management needs of modern cities. Summary of the Invention

[0007] In view of this, the main objective of this application is to provide a multi-model fusion target vehicle recognition and analysis method and system based on machine vision. The method aims to utilize a constructed target vehicle detection model to identify vehicles in videos / images, and combine this with a vehicle tracking algorithm to perform cross-frame unique identifier matching on detected target vehicles, achieving continuous tracking of the target vehicles in the time dimension, thereby determining the motion attributes of the target vehicles. Furthermore, by introducing a parking space detection model to extract parking space location information from the image, and by calculating the spatial overlap area between the target vehicle detection box and the parking space detection box, combined with the aforementioned target vehicle motion state attributes, a comprehensive judgment is made as to whether the vehicle is illegally parked. This aims to improve the accuracy of identifying illegally parked vehicles in complex scenarios, and to enhance the real-time performance and stability of vehicle target detection in single-model monitoring systems.

[0008] Another objective of this application is to embed a multi-target detection model and utilize a multi-threaded parallel processing mechanism for multi-model collaborative detection. This enables the system to simultaneously execute its respective detection tasks using vehicle detection, parking space detection, and pedestrian / driver detection models when processing the same frame of image. Furthermore, through an intelligent alarm and event push mechanism, the system performs multi-dimensional correlation analysis on the detection results, thereby achieving comprehensive recognition and joint analysis of multiple types of targets in videos / images and further improving the target detection accuracy and robustness in complex scenarios.

[0009] To achieve the above objectives, this application adopts the following technical solution:

[0010] A multi-model fusion target vehicle recognition and analysis method based on machine vision includes:

[0011] A. The steps of using drones to capture and collect road vehicle videos / images and transmitting them wirelessly to a streaming media server;

[0012] B. Adopt an improved target vehicle detection model and construct a multi-task collaborative detection framework, and use a multi-model fusion detection mechanism to determine illegally parked vehicles;

[0013] C. Design a lightweight detection network to reduce the high inference latency and large resource consumption in the multi-model collaborative detection process, and reduce the complexity of the detection algorithm;

[0014] D. Steps for implementing intelligent alarms and event push notifications.

[0015] Preferably, step C is followed by:

[0016] E. Optimize the aforementioned models collaboratively and separate them from the UAV flight mission, then feed back the optimized model parameters to step B to further optimize the target vehicle detection model and / or the multi-target collaborative detection model; and collect and analyze the inference results of each model in real time during the UAV cruise.

[0017] Wherein: The UAV described in step A performs operations including take-off, landing, trajectory planning, altitude adjustment and hovering control through a preset or dynamically planned flight trajectory.

[0018] Step B includes:

[0019] B1. Introduce an algorithm model for detecting illegally parked vehicles and a parking space detection algorithm model. By matching the spatial overlap of the target boxes of the detected vehicles and the target boxes of the parking spaces, the system integrates a vehicle tracking algorithm to perform cross-frame trajectory matching and identity preservation of the targets. Combined with the trajectory persistence analysis in the time dimension, it realizes intelligent identification of illegal behaviors such as static vehicles occupying non-parking spaces.

[0020] B2. The system embeds a pedestrian / driver detection model to perform image-level feature extraction and recognition for high-risk events such as traffic accidents.

[0021] Preferably, after step B, the method further includes: receiving one or more optimized model parameters based on confidence adjustment information, false alarm / missed alarm identification and correction feedback information, and task-customized model re-optimization information from the UAV dynamic feedback subsystem, and re-optimizing the target vehicle detection model and / or the multi-target collaborative detection model.

[0022] Specifically, the lightweight detection network design described in step C requires comprehensive evaluation based on floating-point computation time, memory access time, and network parallelism metrics, and involves the following steps:

[0023] C1. Divide the input features into two parts. Perform true depth-separable calculation on one part, concatenate the calculation results with the other part through channels, and finally perform channel shuffling to complete the information exchange steps.

[0024] C2. Remove group convolutions to avoid point-by-point addition operations;

[0025] C3. When downsampling and channel doubling are required, the Channel Split operation step has been removed in ShuffleNet V2.

[0026] Step D includes: when an abnormal event of illegal parking of a vehicle is detected, an alarm is triggered to generate alarm information, and an alarm image frame / image is saved locally. The image frame includes a detection box for drawing the abnormal event and is asynchronously uploaded to the MinIO object storage and pushed to the front-end alarm platform or the monitoring terminal in real time.

[0027] A machine vision-based multi-model fusion target vehicle recognition and analysis system includes:

[0028] The video / image acquisition subsystem is used to capture and collect road vehicle video / images using drones and transmit them wirelessly to a streaming media server.

[0029] The target detection and multi-task collaborative detection subsystem uses an improved target vehicle detection model and constructs a multi-target collaborative detection model to determine illegally parked vehicles using a multi-model fusion detection mechanism.

[0030] The lightweight detection network subsystem is designed to reduce the high inference latency and high resource consumption in the multi-model collaborative detection process by designing a lightweight detection network, thereby reducing the complexity of the detection algorithm.

[0031] The intelligent alarm and event push subsystem is used for intelligent alarm and event push.

[0032] Preferably, it also includes:

[0033] The UAV dynamic feedback subsystem is used to optimize the model collaboration and separate it from the UAV flight mission. The optimized model parameters are then fed back to the target detection and multi-task collaborative detection subsystem to further optimize the target vehicle detection model and / or the multi-target collaborative detection model. It is also used to collect and analyze the inference results of each model in real time during the UAV cruise to form a closed-loop process of "detection-feedback-optimization".

[0034] The target detection and multi-task collaborative detection subsystem further includes:

[0035] The vehicle detection algorithm module is used to train the vehicle detection model and implement various enhancement strategies to improve the generalization ability and detection accuracy of the vehicle detection model. After the vehicle detection model is trained, it is also used to deploy the vehicle detection model on an edge device or a small server to perform real-time vehicle detection for each frame of the video stream and store the detection results.

[0036] The target tracking and trajectory smoothing module is used to achieve temporal tracking of target vehicles based on the detection results and by combining the ByTrack tracking algorithm; and to introduce a window smoothing mechanism for the historical position information of each target vehicle to improve the stability of the target vehicle trajectory.

[0037] The vehicle speed analysis and judgment module is used to obtain the historical position and timestamp data of the target vehicle from the target tracking module and calculate the displacement speed of the target vehicle between two adjacent frames.

[0038] The parking space target detection module is used to train a model using a public dataset of parking spaces. Based on this, it implements various data augmentation strategies and uses the trained parking space target detection algorithm to identify and detect each frame of the video stream in real time, storing the detected parking space target information.

[0039] The vehicle and parking space matching and judgment module is used to perform intersection-union ratio calculation on the target vehicle and parking space target detected in each frame and judge the spatial overlap relationship between the two: if the target vehicle frame has a valid overlap with any parking space frame, it is judged as legal parking; if the target vehicle frame does not overlap with any parking space frame, it is considered that the vehicle is parked in a non-parking space area, which meets the second judgment condition of illegal parking.

[0040] The parking duration determination module is used to monitor the duration of the parking state when the target vehicle meets both the conditions of "low speed" and "not occupying a parking space". If the state continues to exceed the set threshold, the target vehicle is finally determined to be illegally parked.

[0041] The multi-model fusion target vehicle recognition and analysis method and system based on machine vision of the present invention has the following beneficial effects compared with the prior art:

[0042] 1) This invention combines deep machine learning technology with the application of drone inspection videos / images to automatically analyze vehicle status, thereby achieving intelligent identification of illegal parking behavior.

[0043] 2) This invention relates to a multi-model fusion target vehicle recognition and analysis method and system based on machine vision. It utilizes an improved YOLOv11n model to identify vehicles in videos / images and combines this with a vehicle tracking algorithm to perform cross-frame unique identifier matching on detected target vehicles, achieving continuous tracking of the target vehicles in the time dimension and thus determining their motion attributes. Furthermore, it introduces a parking space detection model to extract parking space location information from the image and calculates the spatial overlap between the target vehicle detection box and the parking space detection box. Combined with the aforementioned target vehicle motion state attributes, it comprehensively determines whether the vehicle is illegally parked. Compared to traditional target vehicle detection / recognition technologies, this significantly improves the accuracy of identifying illegally parked vehicles in complex scenarios, while also enhancing the real-time performance and stability of single-model monitoring systems for vehicle target detection.

[0044] 3) This invention relates to a multi-model fusion target vehicle recognition and analysis method and system based on machine vision. By embedding a multi-target detection model (multi-model fusion) and utilizing a multi-threaded parallel processing mechanism for collaborative detection of multiple target models, it can perform vehicle detection, parking space detection, and pedestrian / driver's seat detection models in parallel when processing the same frame of image. Furthermore, through an intelligent alarm and event push mechanism, it performs multi-dimensional correlation analysis on the detection results, achieving comprehensive recognition and joint analysis of multiple types of targets in videos / images. Therefore, it further improves the accuracy and robustness of target vehicle detection in complex scenes.

[0045] 4) The multi-model fusion target vehicle recognition and analysis method and system based on machine vision of the present invention can also perform multi-dimensional correlation analysis on the detection results through the collaborative vehicle illegal parking and multi-target recognition model detection mechanism of multi-model fusion, and realize the perception and rapid early warning of real-time road traffic scenes through intelligent alarm and event push mechanism. Furthermore, it can provide technical support and guarantee for improving the timely response capability in the road traffic management process through intelligent alarm linkage. Attached Figure Description

[0046] Figure 1 This is a flowchart of the multi-model fusion target vehicle recognition and analysis method based on machine vision, according to an embodiment of the present invention.

[0047] Figure 2 for Figure 1 The diagram shows a multi-model fusion collaborative vehicle illegal parking and a multi-task collaborative detection mechanism.

[0048] Figure 3 a~ Figure 3 d is a comparison diagram of the lightweight network designs of ShuffleNetV1 and ShuffleNetV2 in the lightweight detection algorithm network design of the embodiments of the present invention;

[0049] Figure 4 This is a schematic diagram of the architecture of the multi-model fusion target vehicle recognition and analysis system based on machine vision, according to an embodiment of the present invention.

[0050] Figures 5a-5b These are schematic diagrams illustrating the results of identifying illegally parked vehicles on the road. Detailed Implementation

[0051] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0052] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0053] The present invention relates to a multi-model fusion target vehicle identification and analysis method and system based on machine vision, which mainly combines drone technology, especially the application technology of carrying high-definition cameras and artificial intelligence algorithms, to achieve efficient and flexible inspection of road traffic conditions, especially of illegally parked vehicles, using drones.

[0054] Figure 1 This is a flowchart of a multi-model fusion target vehicle recognition and analysis method based on machine vision, according to an embodiment of the present invention. Figure 1 As shown, this multi-model fusion target vehicle recognition and analysis method based on machine vision mainly includes the following steps:

[0055] Step 11: Using drones to capture and collect video / images of road vehicles and transmitting them to a streaming media server via wireless communication.

[0056] In this embodiment, a drone equipped with a high-definition camera and image acquisition module is used to capture video / images of vehicles on the ground. These images are initially processed by an image acquisition card before being transmitted to a ground receiving end, and then to a streaming media server for further processing in step 12. Preferably, the captured vehicle video / images transmitted to the streaming media server can be transmitted and displayed in real time. The streaming media server uses standard protocols such as Real-Time Streaming Protocol (RSTP) and Real-Time Messaging Protocol (RTMP) to transmit video data.

[0057] Furthermore, the UAV used in this embodiment of the invention possesses complete flight control capabilities, including support for takeoff, landing, trajectory planning, altitude adjustment, and hovering control. It can guide the UAV to perform inspections within a designated area along the optimal path using a preset flight route or dynamically planned flight trajectory, ensuring full-scene coverage and efficient operation. Additionally, based on the UAV dynamic feedback subsystem (see reference...), Figure 4 The information fed back by the UAV dynamic feedback subsystem includes: the detection confidence level of the current target vehicle's location, the location of suspected missed detection areas, image quality assessment indicators (such as blurriness, lighting conditions, etc.), and environmental complexity level. The dynamic adjustment of analysis parameters and image acquisition strategies includes: (1) adjusting flight speed, flight altitude, or hovering time to further improve the image acquisition quality of the planned patrol path or key patrol areas; (2) controlling the angle of the UAV camera gimbal to align with low confidence areas or areas where detection has failed, thereby improving the model's recognition while also increasing the coverage of subsequent image acquisition; (3) dynamically adjusting the image acquisition frequency, resolution, and exposure parameters to adapt to different lighting conditions or target states; and (4) increasing the image acquisition density of specific areas to provide more sufficient training samples for subsequent model fine-tuning.

[0058] By using information feedback or interaction, a closed-loop information interaction is achieved between the UAV dynamic feedback subsystem and the video / image acquisition subsystem, thereby improving the overall data quality acquisition capability and model adaptive optimization efficiency of the system.

[0059] Ideally, the system can also be configured to automatically hover at key locations when performing detection tasks in specific areas, cooperating with target detection / recognition models to perform image acquisition and target recognition with higher confidence. For example, when the system detects suspected illegal parking hotspots or needs to determine whether there is a driver in the driver's seat, it can trigger refined hovering shooting and multi-angle reshooting strategies, effectively improving the recognition rate and detection stability of small targets.

[0060] Step 12: Adopt an improved target detection model and construct a multi-task collaborative detection framework, and use a multi-model fusion detection mechanism to determine illegally parked vehicles.

[0061] In this embodiment, an improved YOLOv11n model is used as the basic target detection module to train vehicle detection and parking space detection models. Combined with vehicle trajectories obtained through tracking algorithms, a comprehensive judgment is made as to whether a vehicle is illegally parked. Based on this, a pedestrian / driver recognition model is further integrated to construct a multi-task collaborative detection framework, enabling comprehensive perception and intelligent analysis of illegal parking and related events in road traffic. The process of constructing the multi-task collaborative detection framework specifically includes:

[0062] Step 121: Introduce the illegal parking vehicle detection algorithm model and the parking space detection algorithm model. By matching the spatial overlap of the target boxes of the detected vehicles and the target boxes of the parking spaces, the system integrates the vehicle tracking algorithm to perform cross-frame trajectory matching and identity preservation of the target vehicles. Combined with the trajectory persistence analysis in the time dimension, it realizes the intelligent identification of illegal behaviors such as static vehicles occupying non-parking spaces.

[0063] Step 122: The system embeds a pedestrian / driver detection model to perform image-level feature extraction and recognition for high-risk events such as traffic accidents.

[0064] The multi-model fusion detection mechanism not only realizes the integrated processing of functions such as vehicle detection and illegal parking identification, but also supports multi-angle collaborative judgment, making the system more adaptable to different scenarios and more business-scalable, and significantly improving the intelligence level of the traffic supervision system in responding to emergencies and ensuring public safety.

[0065] Figure 2 for Figure 1 The diagram below illustrates a multi-model fusion collaborative vehicle illegal parking and multi-task collaborative detection mechanism. (See attached diagram.) Figure 2 The specific implementation process of this embodiment is described below:

[0066] 1) Train a vehicle detection model using the public dataset VisDrone2019 to obtain an accurate target vehicle detection model under UAV vision;

[0067] 2) Train a parking space detection model using a public parking dataset to obtain an accurate parking space detection model under UAV vision;

[0068] 3) Then, the vehicle tracking and matching algorithm is used to calculate the historical change trajectory of the vehicle's position in order to determine whether the vehicle is moving or stationary;

[0069] 4) Use illegal parking algorithm logic to determine whether illegal parking exists. For example, based on the vehicle trajectory, if the stationary time is greater than a certain time threshold (such as 180s), the target vehicle can be judged to be illegally parked; or if the overlap between the vehicle detection box and the parking space detection box is less than a certain value or equal to 0, the vehicle is judged to be illegally parked.

[0070] Preferably, step 12 is further followed by: receiving data from the UAV dynamic feedback subsystem (referencing...). Figure 4 The step of further optimizing the target vehicle detection model and / or the multi-task collaborative detection model by means of one or more feedback information such as confidence adjustment information, false alarm / missed alarm identification and correction feedback information, and task customization re-optimization information.

[0071] Step 13: Design a lightweight detection network to reduce the problems of high inference latency and large resource consumption in the multi-model collaborative detection process, and reduce the complexity of the detection algorithm.

[0072] This embodiment refers to Figure 3 a~ Figure 3 The comparison diagram of the lightweight network designs of ShuffleNetV1 and ShuffleNetV2 shown in Figure d illustrates the role and working principle of the lightweight detection network and its network design.

[0073] To address the issues of high inference latency and resource consumption caused by multiple models co-detecting in low-altitude UAV images, this invention primarily focuses on lightweight improvements through network structure optimization and module replacement.

[0074] A lightweight YOLOv11n design based on ShuffleNet V2 addresses the issue that FLOPs (Floating Point Operations) are not a direct measure of computational complexity and do not necessarily equate to algorithm speed. Networks with similar FLOPs can vary significantly in speed; using FLOPs alone is insufficient, as memory access consumption and GPU parallelism must also be considered. Therefore, a lightweight network design based on ShuffleNetV2 was developed. Specifically, building upon the problems of ShuffleNetV1, it addresses how to maintain the majority of convolutions in fully convolutional or grouped convolutions to ensure equal input and output channels. This is addressed by proposing the Channel Split operation, and the ShuffleNetV2 unit is constructed. Figure 3 a, Figure 3 As shown in b, these correspond to the Units in ShuffleNet V1; as... Figure 3 c. Figure 3 d represents the Units corresponding to the improved ShuffleNetV2 version.

[0075] The basic unit of ShuffleNetV2 has the following three characteristics:

[0076] 1) A novel Channel Split operation is proposed, as shown above. Figure 3 As shown in diagram c, the input features are divided into two parts. One part undergoes true depthwise separability computation, and the computation result is combined with the other part using the channel concat function. Finally, a channel shuffle operation is performed to complete the information exchange. The channel split operation divides the input tensor into two or more subsets along the channel dimension, used to optimize the neural network structure to reduce computational complexity and improve efficiency. The concat function operation is used to join two or more arrays; performing this operation does not change the existing arrays, but only returns a copy of the joined arrays, returning a new array generated by adding all ArrayX parameters to an ArrayObject. If the parameter for the concat operation is an array, then elements of the array are added, not the array itself. Here, ArrayObject is a built-in class for implementing dynamic array operations, supporting the storage of data in array form and the manipulation of data like objects.

[0077] The channel shuffle operation aims to enhance the feature interaction and information fusion capabilities between different network branches. Unlike traditional ShuffleNet which uses grouped convolutions, this invention focuses on solving the feature processing and interaction problems in multi-branch structures. By performing the channel shuffle operation, the channel dimensions of the feature map are rearranged and combined, enabling the channels of different branches to be fused together in subsequent calculations.

[0078] The purpose and technical effects of channel shuffling in this invention are mainly reflected in: (1) Achieving efficient feature interaction across branches: Channel shuffling rearranges the output channels of each branch, enabling the features extracted from different branches to be cross-combined in subsequent layers, breaking the channel independence between branches and significantly enhancing the information flow capability within the model. (2) Improving the model's feature fusion capability and discrimination performance: The shuffling operation enables the model to capture joint representations across branches, thereby improving the network's ability to model complex targets, multi-scale information, etc., and effectively improving recognition accuracy without significantly increasing computational costs. (3) Improving expressive power while maintaining structural lightweightness: This operation is achieved only through tensor dimension adjustment, without involving additional convolution calculations or parameter increases, maintaining the overall computational efficiency and inference speed of the network, and is very suitable for real-time task deployment in edge computing scenarios. (4) Enhancing the synergy and scalability of network modules: As a structurally lightweight but functionally critical operation, channel shuffling can provide more diverse inputs for subsequent network layers, enhancing the modular synergy capability of the overall network structure.

[0079] 2) The entire process does not use 1×1 convolutions and avoids point-by-point addition operations.

[0080] 3) When downsampling and channel doubling are required, ShuffleNetV2 removes the Channel Split operation. This means that the number of channels will double when the string is merged using the Concat function, such as... Figure 3 As shown in d.

[0081] In a specific embodiment of the present invention, the ShuffleNetV2 lightweight network design specifically includes:

[0082] Existing lightweight methods often use floating-point operations (FLOPs) as the primary evaluation metric for model performance. FLOPs refer to the number of floating-point computations required for a single forward propagation, and are used to measure model complexity. However, since networks with similar FLOPs can have different speeds, using only FLOPs as a metric is insufficient. Therefore, it is necessary to combine memory access cost (MAC) and network parallelism for a comprehensive evaluation. Accordingly, this invention conducted extensive experiments using ShuffleNetV2 to analyze the factors affecting network speed and proposed four basic rules for building high-performance networks:

[0083] G1: When the number of input and output feature channels of a convolutional layer is equal, the MAC is the smallest, and the model is the fastest at this time;

[0084] G2: Too many convolutional groups will increase MAC, which will slow down the model.

[0085] G3: Network fragmentation reduces parallelizability, which means that the fewer branches in the model, the faster the model will be.

[0086] Although G4: ElementWise operations have a low FLOPs value, their MAC is high, so elementWise operations should be minimized as much as possible.

[0087] The embodiments of the present invention are based on the four rules G1 to G4 mentioned above, with reference to... Figure 3 a, Figure 3 As shown in b, it can be seen that networks prior to ShuffleNetV1 violated this rule in three ways: First, they used 1×1 group convolutions and 1×1 pointwise convolutions in Bottleneck, resulting in different numbers of input and output channels, violating rules G1 and G2; Second, the network used a large number of group convolutions, resulting in too many groups, violating rule G3; Third, there were a large number of pointwise addition operations in the network, violating rule G4.

[0088] Accordingly, in response to the aforementioned technical defects or deficiencies, the present invention proposes the following improvements in this embodiment, such as... Figure 3 c~ Figure 3 As shown in d:

[0089] 1) A novel Channel Split operation is proposed, as shown above. Figure 3As shown in c, the input features are divided into two parts. One part is subjected to true depth separability calculation, and the calculation result is combined with the other part through channel concat. Finally, the channel shuffling operation is performed to complete the information exchange.

[0090] 2) Removal, i.e., not using group convolution: The entire process does not use 1×1 group convolution, and also avoids point-by-point addition operations;

[0091] 3) When downsampling and channel doubling are required, the ChannelSplit operation has been removed in ShuffleNetV2. This will double the number of channels during the final Concat operation, such as... Figure 3 As shown in d.

[0092] Therefore, according to Figure 3 As shown in Figure c, based on the ShuffleNetV1 Block, the ShuffleNetV2 Block introduces the simple operator ChannelSplit to achieve the above purpose, such as... Figure 3 As shown in c. At the beginning of each block, the input feature map's c channels are split into two branches: cc channels and c' channels. To minimize fragmentation in the G3 network, one branch remains unchanged (shortcut connection), while the other branch contains three convolutions with the same number of channels to satisfy G1. Unlike V1, the two 1×1 convolutions in the V2 block no longer use grouped convolutions, partly to satisfy G2, and partly because the initial channel splitting operation already achieved the grouping effect.

[0093] Finally, the results of the two branches are concatenated, ensuring that the input and output channels are identical for the convolutional block, conforming to the G1 principle. Similar to ShuffleNetV1, channel shuffling is used to guarantee information exchange between the two branches.

[0094] Combination Figure 3 c. Figure 3 The convolutional blocks shown in d are stacked together to form the final ShuffleNetV2 model. Let c′ = c / 2. The stacked network is similar to the ShuffleNetV1 model. The detailed network structure is shown in Table 1 below.

[0095] Table 1:

[0096]

[0097] The difference between ShuffleNet V1 and ShuffleNet V2 blocks lies in the fact that ShuffleNet V2 adds a 1×1 convolution before the global averaged pooling layer to mix up features, while ShuffleNet V1 does not. Like ShuffleNet V1, the number of channels in ShuffleNetV2 blocks is scaled proportionally by 0.5×1× to generate ShuffleNet V2 networks of varying complexity, labeled as ShuffleNet V2 0.5×, ShuffleNetV2 1×, etc.

[0098] The lightweight network design described in step 13 of this embodiment can effectively solve the problems of high inference latency and large computational resource consumption caused by multi-model collaborative detection in low-altitude UAV image processing. It can leverage the lightweight design principles of ShuffleNetV2 and optimize the structure and improve the modules based on four key network design rules (G1 to G4), effectively achieving the goal of constructing an efficient and low-complexity target detection network proposed in this invention. The beneficial technical effects are specifically reflected in the following aspects: (1) Significantly reducing the computational overhead and inference latency of the model: This embodiment introduces Channel Split operation and depthwise separable convolution to replace the traditional convolution structure. Combined with the optimized module design in ShuffleNetV2, it can significantly reduce redundant computation and memory access overhead, thereby effectively reducing FLOPs and MAC while ensuring detection accuracy, improving the overall inference speed, and meeting the needs of edge devices with high real-time requirements, i.e., corresponding to the G1 rule. (2) Enhance the feature information interaction capability between different branches: Channel shuffle operation is used to rearrange the channels between branches, breaking the channel isolation phenomenon between branches, so that the features extracted by different paths can be fully integrated in the subsequent processing stage, effectively improving the network's expressive ability and multi-target detection accuracy, which corresponds to the G3 rule. (3) Avoid group convolution and fragmented design to improve computational efficiency and parallelism: The embodiment of the present invention adopts the optimized network structure, cancels 1×1 group convolution and point-by-point addition and other high MAC operations, reduces the stacking of network fragmented modules, improves the coherence and execution efficiency of the computation graph, enhances the network's parallel processing capability on hardware platforms such as GPUs, and reduces resource consumption, which corresponds to the G2, G3 and G4 rules. (4) Have good scalability and deployment flexibility: The embodiment of the present invention supports adjusting the channel scaling ratio (such as 0.5×, 1×, etc.) based on different computing power conditions, and can flexibly build models of different complexity according to application scenarios, adapting to various deployment environments from lightweight edge devices to high-performance servers, which enhances the scalability of the technology implementation. (5) Improve the overall performance of the multi-model collaborative detection system: The lightweight structural design and cross-branch feature fusion mechanism adopted in the embodiments of the present invention enable multiple detection models to run efficiently in parallel in the same system, significantly reduce system load, improve the real-time performance and accuracy of image processing, and provide strong support for applications such as low-altitude UAV image analysis.

[0099] Step 14: Steps for implementing intelligent alarms and event push notifications.

[0100] In this embodiment of the invention, when an abnormal event such as illegal parking is detected, an alarm is triggered, generating alarm information. The alarm supports generating alarm images and JSON alarm information. At this time, the system saves the alarm image locally; this frame includes a detection box depicting the abnormal event, and asynchronously uploads it to the MinIO object storage, pushing it in real-time to the front-end alarm platform or monitoring terminal for processing. Figure 5a , Figure 5b As shown.

[0101] In practical applications, the alarm information can be set to be sent every 3 seconds for processing by the backend platform. The alarm information is organized by category and event to form structured data, including: timestamp, video stream source (drone number), event type (illegally parked vehicle, fire, etc.), number of abnormal events, coordinates of detected objects, confidence level of detected objects, and alarm image storage path, etc.

[0102] Correspondingly, in drone video surveillance systems, the recorded information for abnormal events also includes some key fields:

[0103] Timestamp: Represents the actual time when an event was triggered, used to accurately record the time when an abnormal event occurred.

[0104] Video stream source: The unique drone ID in the system, used to distinguish the detection information of different drones, and is used throughout the entire system.

[0105] Event type: refers to the category of abnormal events detected, such as illegal parking alarms, fire alarms, etc., used to classify and manage alarm information.

[0106] Number of abnormal events: This indicates the number of abnormal targets detected in the current frame, such as the number of illegally parked vehicles and the number of fire sources, providing a basis for subsequent statistics and processing.

[0107] Detected object coordinates: Records the location information of abnormal targets in the image. The drone can use these coordinates to locate the target and continuously track it by adjusting the camera lens.

[0108] Object detection confidence level: This indicates the reliability of the detection result. Higher confidence levels improve the accuracy of identification, thereby reducing false alarms and false negatives.

[0109] Alarm image storage path: This refers to the path of alarm images stored in MinIO. The backend can obtain screenshots of abnormal events through this path and display relevant image information in the system in real time.

[0110] To further improve the operational efficiency and detection accuracy of the multi-model collaborative detection system, the model collaboration process is first optimized through model structure optimization and task decoupling design. Specifically, as described in step 13, a lightweight network is designed to reduce the high inference latency and resource consumption during the model collaborative detection process, thereby reducing the complexity of the detection algorithm. Simultaneously, the operational logic of each detection model is decoupled from the UAV flight control system to prevent model inference from interfering with the flight control task, thus improving the overall stability of the system.

[0111] Preferably, step 13 is followed by:

[0112] Step 15: Optimize the aforementioned models collaboratively and separate them from the UAV flight mission; then feed back the optimized model parameters to Step 12 to further optimize the target vehicle detection model and / or the multi-task collaborative detection model; and collect and analyze the inference results of each model during the UAV cruise in real time to form a closed-loop process of "detection-feedback-optimization".

[0113] Specifically, in the embodiments of the present invention, a dynamic feedback mechanism is added in step 15. By collecting and analyzing the inference results of each model during the UAV's cruise in real time, a closed-loop process of "detection-feedback-optimization" is formed, thereby further improving the adaptability and accuracy of the multi-model detection system. This is specifically reflected in the following aspects:

[0114] Step 151: Automatic Confidence Adjustment: During the drone's cruise, the system dynamically adjusts the confidence threshold for targets in each frame based on the model's current detection capabilities and scene complexity. For example, when the model's detection stability is low or environmental conditions are harsh (such as low light or occlusion), the confidence threshold is appropriately lowered to improve recall; in scenarios with relatively stable detection performance or a high number of false alarms, the threshold is increased to suppress false detections, thereby achieving more accurate target selection and optimized sample selection, providing higher-quality feedback data for subsequent model retraining.

[0115] Step 152: False Positive / False Negative Identification and Correction Mechanism: The system combines continuous analysis of the UAV flight trajectory with cross-frame target tracking results to automatically identify false positives and false negatives in the detection process. For example, when the identified target category is inconsistent with the expectation, the system marks the relevant samples as "anomalies" and incorporates them into the feedback optimization process. Through sample supplementation, label correction, and fine-tuning retraining, the system specifically addresses false positives and false negatives, improving the model's stability and accuracy.

[0116] Step 153: Task-Customized Model Re-optimization: To address the differences in tasks performed by the UAV in different regions (such as towns, mountains, and highways), the system collects image data for each region during cruise and, combined with environmental attributes and target distribution, constructs region-specific datasets for model training and optimization. Based on the trained multi-scene-specific models, the system can dynamically switch or select the most suitable model version during actual task execution, according to the current task scene and image features. This enables adaptive detection in diverse environments, effectively improving the efficiency and accuracy of target detection.

[0117] Step 154: Video Acquisition Feedback Linkage Step: To implement the image acquisition control mechanism based on model feedback, the UAV dynamic feedback subsystem also establishes an interactive communication connection with the video / image acquisition subsystem, inputting the feedback information generated during model analysis into the UAV flight control system and the video / image acquisition subsystem in real time. Specifically, the feedback information includes, but is not limited to: the detection confidence level of the current target area, the location of suspected missed detection areas, image quality assessment indicators (such as blurriness, lighting conditions), and environmental complexity level.

[0118] It is important to note that steps 151 to 154 above are not sequential; they can be executed individually, in pairs, or concurrently in all three. This process, employing steps 151 to 154, not only allows each detection model to process in parallel through multi-threading, effectively improving overall processing efficiency, but also enables the system to fuse the detection results of each model in each frame and perform unified rendering based on information such as the target's coordinates and category. Since each model targets different recognition scenarios, false detections are prone to occur during continuous drone flight. To further reduce the false detection rate, the system can also be strategically configured based on different waypoints. For example, when flying to areas where illegally parked vehicles may exist, parking space detection results can provide a judgment area for the illegal parking detection model, effectively narrowing the detection range and improving recognition accuracy; flame and smoke detection are triggered only in specific areas to save computational resources, thereby improving overall system efficiency.

[0119] In addition, in key monitoring areas, drones can adjust their flight altitude and hover to optimize image quality and model input, thereby improving detection effectiveness and reducing the false negative rate.

[0120] Figure 4 This is a schematic diagram of the architecture of a multi-model fusion target vehicle recognition and analysis system based on machine vision, according to an embodiment of the present invention. Figure 4 As shown, this machine vision-based multi-model fusion target vehicle recognition and analysis system mainly includes the following subsystems:

[0121] The video / image acquisition subsystem is used to capture and collect video / images of road vehicles using drones and transmit them wirelessly to a streaming media server.

[0122] The target detection and multi-task collaborative detection subsystem uses an improved target vehicle detection model and a multi-task collaborative detection framework to determine illegally parked vehicles using a multi-model fusion detection mechanism.

[0123] A lightweight detection network subsystem is designed to reduce the high inference latency and resource consumption in the multi-model collaborative detection process, thereby reducing the complexity of the detection algorithm.

[0124] The intelligent alarm and event push subsystem is used for intelligent alarm and event push.

[0125] In embodiments of the present invention, such as Figure 4 The machine vision-based multi-model fusion target vehicle recognition and analysis system shown includes a target detection and multi-task collaborative detection subsystem. (Refer to...) Figure 2 It further includes the following functional modules:

[0126] To accurately identify illegally parked vehicles, this subsystem integrates multiple algorithm modules, including a vehicle detection algorithm module, a target trajectory and trajectory smoothing processing module, a vehicle speed analysis and judgment module, a parking space target detection module, a vehicle and parking space matching judgment module, and a parking space persistence judgment module. Through joint analysis of multiple conditions, it ultimately determines whether a vehicle constitutes an illegal parking violation. The overall system adopts a modular architecture design, possessing high scalability and concurrent processing capabilities, making it suitable for edge computing devices or small server deployment scenarios. The roles of each functional module in this embodiment are explained as follows:

[0127] The vehicle detection algorithm module:

[0128] 1) The system trains the vehicle detection model based on a publicly available vehicle dataset, dividing the dataset into a training set, a validation set, and a test set. On this basis, various enhancement strategies are implemented, such as image flipping, rotation, scaling, and adding noise, to improve the model's generalization ability and detection accuracy.

[0129] 2) After the model is trained, the system deploys the detection model on the edge or a small server to perform real-time vehicle detection for each frame of the video stream and stores the detection results, including the bounding box, category and confidence score.

[0130] The target tracking and trajectory smoothing processing module:

[0131] 1) Based on the detection results, the system combines the ByTrack tracking algorithm to achieve temporal tracking of target vehicles. The ByTrack algorithm uses the Hungarian algorithm and IoU for inter-frame association, assigning a unique Track_ID to each target, so that tracking continuity can be maintained even if there is short-term occlusion or detection failure.

[0132] 2) To improve trajectory stability, the system introduces a window smoothing mechanism for the historical position information of each target vehicle. If the number of historical positions does not reach the window size, the most recent position is returned; if it does, the average of all positions within the window is taken as the current smoothed position.

[0133] The vehicle speed analysis and judgment module:

[0134] 1) The system obtains the historical position and timestamp data of the target vehicle from the target tracking module and calculates the displacement velocity of the target vehicle between two adjacent frames.

[0135] 2) If valid historical data exists, calculate the speed and perform smoothing; otherwise, set the speed to 0.

[0136] 3) If the speed after smoothing remains below the threshold (e.g., <2px / frame), it is determined to be in a stationary state, which satisfies the first condition for illegal parking identification.

[0137] Preferably, when using the vehicle speed analysis and judgment module to calculate the speed of the target vehicle, the flight speed of the UAV needs to be taken into account. Specifically, there are the following situations: First, when the UAV is in a prohibited state, i.e., hovering in the air, the displacement speed of the target vehicle between two adjacent frames is the speed of the target vehicle; Second, when the UAV is in a cruise state, when using the displacement speed of the target vehicle between two adjacent frames to calculate the speed of the target vehicle, it is also necessary to subtract or add the displacement speed of the UAV relative to the target vehicle.

[0138] The parking space target detection module, similar to the vehicle target detection algorithm module, uses a public parking space dataset for model training. Based on this, it implements various data augmentation strategies and uses the trained parking space target detection algorithm to identify and detect each frame of the video stream in real time, storing the detected parking space target information.

[0139] The vehicle and parking space matching and judgment module performs IoU (Intersection over Union) calculation on the detected vehicle and parking space targets in each frame to determine their spatial overlap relationship.

[0140] 1) If the vehicle frame has a valid overlap with any parking space frame (IoU > threshold (0.2)), it is considered a legal parking;

[0141] 2) If the vehicle frame does not overlap with any of the parking space frames, the vehicle is considered to be parked in a non-parking space area, which meets the second condition for illegal parking.

[0142] The duration determination module further monitors the duration of a vehicle's parking status if it simultaneously meets the conditions of "low speed" and "not occupying a parking space." If this status persists for more than a set threshold (180 seconds), the system considers the possibility of a drone detecting a suspicious illegally parked vehicle and places the drone in a hovering state for further assessment. For example, it can be combined with a driver identification module to make a comprehensive judgment based on existing illegal parking criteria such as whether a driver is present in the driver's seat. For instance, if a vehicle remains stationary for more than 180 seconds without a driver in the driver's seat when the parking space is not occupied, the vehicle can ultimately be determined to be illegally parked.

[0143] The systems all adopt a modular design, with all models implemented through class encapsulation. Users only need to configure a few parameters to quickly load and execute various recognition tasks, facilitating expansion and maintenance. Regarding model execution, the system introduces a multi-threading mechanism to enable the concurrent operation of multiple models, ensuring that each model does not interfere with the others and effectively improving the overall concurrent processing capability of the system.

[0144] To reduce the load on model inference and improve real-time response efficiency, the system adopts a frame skipping (frequency skipping) strategy, processing only key frames, thereby significantly improving processing speed while ensuring recognition accuracy. Furthermore, during the model training phase, all models uniformly adopt lightweight network architectures such as ShuffleNetV2, effectively reducing the computational complexity and resource consumption of the models, further enhancing the system's real-time processing capabilities in resource-constrained environments such as edge devices.

[0145] Ideally, the system can also embed other target detection models. For example, it can perform image-level feature extraction and recognition for high-risk events such as traffic accidents and sudden fires, supporting real-time alarms for fire sources and smoke areas. Based on the output of the fire and smoke model, the system determines the fire area; if the confidence level is higher than a set threshold (e.g., >0.6), a fire warning is triggered.

[0146] By integrating multiple models, the system not only achieves integrated processing of functions such as vehicle detection and illegal parking identification, but also supports multi-angle collaborative judgment, making the system more adaptable to different scenarios and more business-scalable, and significantly improving the intelligence level of the traffic supervision system in responding to emergencies and ensuring public safety.

[0147] Better, such as Figure 4 The aforementioned machine vision-based multi-model fusion target vehicle recognition and analysis system further includes a UAV dynamic feedback subsystem.

[0148] The UAV dynamic feedback subsystem is primarily used to optimize the collaborative model and separate it from the UAV flight mission; to feed back the optimized model parameters to the target vehicle detection model and / or the multi-task collaborative detection model for further optimization; and to collect and analyze the inference results of each model during the UAV's cruise in real time, so as to form a closed-loop process of "detection-feedback-optimization". The UAV dynamic feedback subsystem specifically includes:

[0149] The automatic confidence adjustment module dynamically adjusts the confidence threshold of targets in each frame of an image during drone navigation, based on the model's current detection capabilities and scene complexity. For example, when the model's detection stability is low or environmental conditions are harsh (such as low light or occlusion), the confidence threshold is appropriately lowered to improve recall; in scenarios with relatively stable detection performance or a high number of false positives, the threshold is increased to suppress false detections, thereby achieving more accurate target selection and optimizing sample selection, providing higher-quality feedback data for subsequent model retraining.

[0150] The false positive / false negative identification and correction mechanism module combines continuous analysis of the UAV flight trajectory with cross-frame target tracking results to automatically identify false positives and false negatives in the detection process. For example, when the identified target category is inconsistent with the expectation, the system marks the relevant samples as "anomalies" and incorporates them into the feedback optimization process. Through sample supplementation, label correction, and fine-tuning retraining, it addresses false positives and false negatives in a targeted manner, improving the model's stability and accuracy.

[0151] The task-customized model re-optimization module is designed to address the varying mission requirements of UAVs in different regions (such as towns, mountains, and highways). During navigation, the system collects image data for each region and, combined with environmental attributes and target distribution, constructs region-specific datasets for model training and optimization. Based on the trained multi-scene-specific models, the system can dynamically switch or select the most suitable model version during actual mission execution, according to the current mission scenario and image features. This enables adaptive detection in diverse environments, effectively improving the efficiency and accuracy of target detection.

[0152] Video Acquisition Feedback Linkage Module: To realize the image acquisition and control mechanism based on model feedback, the UAV dynamic feedback subsystem also establishes an interactive communication connection with the video / image acquisition subsystem. This connection is used to input the feedback information generated during model analysis into the UAV flight control system and the video / image acquisition subsystem in real time. Specifically, the feedback information includes, but is not limited to: the detection confidence level of the current target area, the location of suspected missed detection areas, image quality assessment indicators (such as blurriness, lighting conditions), and environmental complexity level.

[0153] To reduce false positive rates and improve detection accuracy, the system employs a waypoint-based strategic detection configuration and task scheduling mechanism in its actual deployment, achieving the goal of "on-demand detection and fine-tuning." This is specifically reflected in the following aspects:

[0154] First, area determination based on waypoint strategy and invocation of different models.

[0155] The system divides the task area based on preset flight routes and key waypoints. Combining Geographic Information System (GIS) data with historical detection data, it configures corresponding detection strategies for each waypoint. In areas with high rates of illegal parking (such as scenic area entrances and narrow passages), the system automatically activates a parking space detection model, generating parking space area masks and only performing illegal parking checks within designated parking spaces, thus avoiding misjudgments of non-parking areas. After applying this strategy, the false detection rate of the illegal parking detection model decreased from 12.6% to 3.2%, significantly improving the recognition accuracy.

[0156] Second, the adaptive flight control of UAVs improves detection accuracy.

[0157] Small targets are difficult to identify when viewed from below a drone. Controlling the drone's altitude reduces the false detection rate. In key monitoring areas, the drone adaptively adjusts its flight altitude and short-term hovering, optimizing image resolution and shooting angle to reduce occlusion and motion blur. This significantly improves target edge sharpness, enhances the model's feature extraction capabilities, reduces the false detection rate from 7.8% to 1.4%, and improves overall detection confidence by over 15%.

[0158] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A multi-model fusion target vehicle recognition and analysis method based on machine vision, characterized in that, include: A. The steps of using drones to capture and collect road vehicle videos / images and transmitting them wirelessly to a streaming media server; B. Adopt an improved target vehicle detection model and construct a multi-task collaborative detection framework, and use a multi-model fusion detection mechanism to determine illegally parked vehicles; C. Design a lightweight detection network to reduce the high inference latency and large resource consumption in the multi-model collaborative detection process, and reduce the complexity of the detection algorithm; D. Steps for implementing intelligent alarms and event push notifications.

2. The multi-model fusion target vehicle recognition and analysis method based on machine vision according to claim 1, characterized in that, Step C is followed by: E. Optimize the aforementioned models collaboratively and separate them from the UAV flight mission; then feed back the optimized model parameters to step B to further optimize the target vehicle detection model and / or the multi-target collaborative detection model; and collect and analyze the inference results of each model in real time during the UAV cruise.

3. The multi-model fusion target vehicle recognition and analysis method based on machine vision according to claim 1, characterized in that, The UAV described in step A performs operations including takeoff, landing, trajectory planning, altitude adjustment, and hovering control through a preset or dynamically planned flight path.

4. The multi-model fusion target vehicle recognition and analysis method based on machine vision according to claim 1, characterized in that, Step B includes: B1. Introduce an algorithm model for detecting illegally parked vehicles and a parking space detection algorithm model. By matching the spatial overlap of the target boxes of the detected vehicles and the target boxes of the parking spaces, the system integrates a vehicle tracking algorithm to perform cross-frame trajectory matching and identity preservation of the targets. Combined with the trajectory persistence analysis in the time dimension, it realizes intelligent identification of illegal behaviors such as static vehicles occupying non-parking spaces. B2. The system embeds a pedestrian / driver detection model to perform image-level feature extraction and recognition for high-risk events such as traffic accidents.

5. The multi-model fusion target vehicle recognition and analysis method based on machine vision according to claim 4, characterized in that, Step B1 is followed by: The step of receiving one or more optimized model parameters based on confidence adjustment information, false alarm / missed alarm identification and correction feedback information, and task-customized model re-optimization information from the UAV dynamic feedback subsystem, and re-optimizing the target vehicle detection model and / or the multi-target collaborative detection model.

6. The multi-model fusion target vehicle recognition and analysis method based on machine vision according to claim 1, characterized in that, The lightweight detection network design described in step C requires comprehensive evaluation based on floating-point computation time, memory access time, and network parallelism metrics, and the following steps must be performed: C1. Divide the input features into two parts. Perform true depth-separable calculation on one part, concatenate the calculation results with the other part through channels, and finally perform channel shuffling to complete the information exchange steps. C2. Remove group convolutions to avoid point-by-point addition operations; C3. When downsampling and channel doubling are required, the Channel Split operation step has been removed in ShuffleNet V2.

7. The multi-model fusion target vehicle recognition and analysis method based on machine vision according to claim 1, characterized in that, Step D includes: when an abnormal event of illegal parking of a vehicle is detected, an alarm is triggered to generate alarm information, and an alarm image frame / image is saved locally. The image frame includes a detection box for drawing the abnormal event and is asynchronously uploaded to the MinIO object storage and pushed to the front-end alarm platform or the monitoring terminal in real time.

8. A multi-model fusion target vehicle recognition and analysis system based on machine vision, characterized in that, include: The video / image acquisition subsystem is used to capture and collect road vehicle video / images using drones and transmit them wirelessly to a streaming media server. The target detection and multi-task collaborative detection subsystem uses an improved target vehicle detection model and constructs a multi-target collaborative detection model to determine illegally parked vehicles using a multi-model fusion detection mechanism. The lightweight detection network subsystem is designed to reduce the high inference latency and high resource consumption in the multi-model collaborative detection process by designing a lightweight detection network, thereby reducing the complexity of the detection algorithm. The intelligent alarm and event push subsystem is used for intelligent alarm and event push.

9. The multi-model fusion target vehicle recognition and analysis system based on machine vision according to claim 8, characterized in that, Also includes: The UAV dynamic feedback subsystem is used to optimize the model collaboration and separate it from the UAV flight mission. The optimized model parameters are then fed back to the target detection and multi-task collaborative detection subsystem to further optimize the target vehicle detection model and / or the multi-target collaborative detection model. And it is used to collect and analyze the inference results of each model in real time during the drone's cruise, so as to form a closed-loop process of "detection-feedback-optimization".

10. The multi-model fusion target vehicle recognition and analysis system based on machine vision according to claim 8 or 9, characterized in that, The target detection and multi-task collaborative detection subsystem further includes: The vehicle detection algorithm module is used to train the vehicle detection model and implement various enhancement strategies to improve the generalization ability and detection accuracy of the vehicle detection model. After the vehicle detection model is trained, it is also used to deploy the vehicle detection model on an edge device or a small server to perform real-time vehicle detection for each frame of the video stream and store the detection results. The target tracking and trajectory smoothing module is used to achieve temporal tracking of target vehicles based on the detection results and by combining the ByTrack tracking algorithm; and to introduce a window smoothing mechanism for the historical position information of each target vehicle to improve the stability of the target vehicle trajectory. The vehicle speed analysis and judgment module is used to obtain the historical position and timestamp data of the target vehicle from the target tracking module and calculate the displacement speed of the target vehicle between two adjacent frames. The parking space target detection module is used to train a model using a public dataset of parking spaces. Based on this, it implements various data augmentation strategies and uses the trained parking space target detection algorithm to identify and detect each frame of the video stream in real time, storing the detected parking space target information. The vehicle and parking space matching and judgment module is used to perform intersection-union ratio calculation on the target vehicle and parking space target detected in each frame and judge the spatial overlap relationship between the two: if the target vehicle frame has a valid overlap with any parking space frame, it is judged as legal parking; if the target vehicle frame does not overlap with any parking space frame, it is considered that the vehicle is parked in a non-parking space area, which meets the second judgment condition of illegal parking. The parking duration determination module is used to monitor the duration of the parking state when the target vehicle meets both the conditions of "low speed" and "not occupying a parking space". If the state continues to exceed the set threshold, the target vehicle is finally determined to be illegally parked.

Images