Same target fusion method, device and electronic equipment for multi-anomaly detection

Abstract

The application provides a same target fusion method and device for multi-abnormality detection and electronic equipment. The application determines a self-learning strategy for various classification algorithms and detection algorithms; establishes a plurality of detection target teacher models and a sensor list according to the self-learning strategy, trains an optimized abnormal event detection and recognition algorithm model according to the detection target teacher models and the sensor list; and realizes fusion detection and recognition of the same target through a self-adaptive weighted fusion strategy, solves the problem of low accuracy of abnormal event detection and recognition after the fusion of various algorithms, and improves the accuracy and performance of abnormal event detection and recognition. The application can improve the accuracy and performance of abnormal event detection and recognition, support intelligent applications such as city brain, intelligent security, and city fine management, and the like.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and electronic device for fusing the same target in multiple anomaly detection. Background Technology

[0002] As AI is increasingly applied across industries, AI application scenarios are becoming more personalized and fragmented. This means that AI application products need to run in parallel and integrate diverse AI algorithms, and their interactions will become more frequent. The efficient and accurate integration of diverse algorithms is the future trend of AI algorithm application development.

[0003] In existing anomaly detection methods, when AI application products run in parallel and integrate various AI algorithms, the frequent interaction between the algorithms leads to the problem that the accuracy of multiple algorithms analyzing videos simultaneously to detect and identify abnormal events is lower than that of a single algorithm. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, and electronic device for fusing the same target in multiple anomaly detections, in order to solve the problem in existing anomaly detection methods where, when AI application products run in parallel and integrate various AI algorithms, frequent algorithm interactions result in lower accuracy in detecting and identifying abnormal events when multiple algorithms analyze videos simultaneously compared to a single algorithm.

[0005] To achieve the aforementioned objective, the present invention first provides a method for fusing the same target in multiple anomaly detections, comprising the following steps:

[0006] For various classification and detection algorithms, determine the self-learning strategy;

[0007] Multiple target teacher models and a perceptron list are established based on the self-learning strategy, and an abnormal event detection and recognition algorithm model is trained and optimized based on the target teacher models and the perceptron list.

[0008] When a new anomaly detection and recognition algorithm model is introduced, it is determined whether the algorithm of the anomaly detection and recognition algorithm model is a new algorithm. If it is a new algorithm, an algorithm number is automatically added. Based on the new algorithm, an algorithm instance with the corresponding algorithm number is generated. When generating the algorithm instance, the anomaly detection and recognition algorithm model with the corresponding algorithm number is loaded. Based on the algorithm number, the algorithm instance inference interface of the corresponding algorithm is called. The algorithm instance inference interface is based on a multi-algorithm marking method that moves video memory to marked memory. The algorithm inference interface implements algorithm inference and algorithm post-processing logic, and feeds back the anomaly detection and recognition results and location information corresponding to the video frame image. The anomaly detection and recognition results and location information fed back by each algorithm simultaneously are weighted and adaptively fused to obtain a fused anomaly detection and recognition result with the same target result.

[0009] Furthermore, for diverse classification and detection algorithms, self-learning strategies are determined, including:

[0010] For various classification algorithms, the learning object algorithm is incorporated based on actual test indicators;

[0011] For various detection algorithms, a threshold queue is designed based on their prediction results and arranged in descending order of their confidence.

[0012] The prediction results of the thresholds above the threshold queue are included in the accuracy P of the detection algorithm corresponding to the threshold, and the average of its accuracy is calculated as the learning value S of the learning object algorithm.

[0013] Based on the learning value S of the learning object algorithm, the teacher algorithm is determined to form an initial list of teacher algorithms. According to the principle of prioritizing the inclusion of learning object algorithms, these algorithms are used as pre-training networks for the same target detection algorithm network. The loss function of the same target detection algorithm is adjusted under supervision, and its algorithm weights are updated and determined to form a self-learning strategy.

[0014] Furthermore, the formula for calculating the learned value S of the learning object algorithm is as follows:

[0015]

[0016] Where accuracy P = positive detection / total detection, Wn is the threshold queue, L is the length of the threshold queue, and i is the threshold of the threshold queue;

[0017] Furthermore, based on the self-learning strategy, multiple target teacher models and a perceptron list are established. An abnormal event detection and recognition algorithm model is trained and optimized based on the target teacher models and the perceptron list, including:

[0018] Establish a list of algorithm types, and for each type of algorithm, establish a list of target types. For each target type, establish an initial target teacher model for detection. Based on the self-learning strategy, if an algorithm model has a higher accuracy than the target teacher model for a corresponding target type, replace the corresponding target teacher model with the algorithm model.

[0019] Collect, filter, and classify target sample data, and train and test the target sample data to obtain an abnormal event detection and recognition algorithm model;

[0020] Referring to the prediction results of the teacher model for the target and the perceptron list for each teacher model for the same target, the current algorithm loss function corresponding to the target is modified, the abnormal event detection and recognition algorithm model for the target is tested and optimized, and the optimized abnormal event detection and recognition algorithm model is obtained.

[0021] Furthermore, the anomaly detection and identification results and location information simultaneously fed back by each algorithm are weighted and adaptively fused to obtain fused anomaly detection and identification results with the same target result, including:

[0022] The abnormal event detection and identification results and location information simultaneously fed back by each algorithm are weighted and adaptively fused according to the following formula:

[0023]

[0024] Where, ω i For the adaptively adjusted weights corresponding to algorithm number i, D i B(x) represents the detection box of the same target result corresponding to algorithm number i, and B(x) represents the detection box of the abnormal event detection and recognition result after the same target is fused.

[0025] Furthermore, the multi-algorithm marking method for marking memory transfers in video memory includes:

[0026] The video frame image processing task thread pool, event exception algorithm thread pool, blocking queue and its global memory space are developed based on the standard C / C++11 library. Each algorithm shares the global blocking queue memory space.

[0027] The video frame images and their corresponding algorithm numbers are marked in the memory space. The video frame images are packaged into an image block of 16 images each time. The number of image blocks corresponding to the number of GPU cards is moved each time and placed on each GPU card.

[0028] Based on hardware acceleration, the image blocks are processed in batches, the results of the same detection target are predicted, and the results are fed back to the corresponding image of the image block marked in the memory space.

[0029] Furthermore, the video frame image processing task thread pool consists of single-task threads corresponding to the video number, and the blocking queue consists of a single queue corresponding to the video number. The single-task thread decodes the video and then skips frames to extract video frame images.

[0030] Video frame images are stored in a single queue corresponding to the video number, and a global identifier and its reference index are marked for the single queue, enabling multiple algorithms to share the single queue and memory. If a corresponding algorithm uses the single queue, the reference index is incremented by 1. When the reference index is 0, the single queue is cleared by a single task thread. When the number of frame images of a video with a corresponding number exceeds the capacity of the single queue for that video number, the single task thread for that video number waits in a blocking manner.

[0031] Furthermore, the event exception algorithm thread pool consists of single algorithm threads with corresponding algorithm numbers. For each additional single algorithm thread, the global identifier reference index of the single queue is incremented by 1.

[0032] A single algorithm thread continuously retrieves video frame images from a single queue marked with corresponding video IDs in memory. Based on the video ID, frame ID, algorithm ID, and parameters identified by the video frame image, it uniquely determines the corresponding point, frame image, and algorithm of the video. Through a standardized algorithm interface, the numbering algorithm determined by the single algorithm thread analyzes and returns the results of abnormal events.

[0033] The present invention also provides a fusion device for multiple anomaly detection of the same target, comprising:

[0034] The self-learning strategy unit is used to determine the self-learning strategy for a variety of classification and detection algorithms.

[0035] An optimized model unit is used to establish multiple detection target teacher models and a perceptron list according to the self-learning strategy, and to train and optimize the abnormal event detection and recognition algorithm model according to the detection target teacher models and the perceptron list.

[0036] The real-time fusion unit is used to determine whether the algorithm of the newly accessed abnormal event detection and recognition algorithm model is a new algorithm when a new abnormal event detection and recognition algorithm model is introduced; if it is a new algorithm, an algorithm number is automatically added; an algorithm instance with the corresponding algorithm number is generated according to the new algorithm; when generating the algorithm instance, the abnormal event detection and recognition algorithm model with the corresponding algorithm number is loaded; according to the algorithm number, the algorithm instance inference interface of the corresponding algorithm is called; the algorithm instance inference interface is based on the multi-algorithm marking method of marking memory and moving video memory, and implements algorithm inference and algorithm post-processing logic in the algorithm instance inference interface, and feeds back the abnormal event detection and recognition results and location information corresponding to the video frame image; the abnormal event detection and recognition results and location information fed back by each algorithm simultaneously are weighted and adaptively fused to obtain a fused abnormal event detection and recognition result with the same target result.

[0037] The present invention also provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs any of the steps of the methods described above.

[0038] The beneficial effects of this invention are as follows: This invention provides a method, apparatus, and electronic device for fusing the same target in multiple anomaly detections. It determines a self-learning strategy for various classification and detection algorithms; establishes multiple detection target teacher models and a perceptron list based on the self-learning strategy; and trains and optimizes the anomaly detection and recognition algorithm model based on the detection target teacher models and the perceptron list. Through an adaptive weighted fusion strategy, it achieves fusion detection and recognition of the same target, solving the problem of low accuracy in anomaly detection and recognition after fusing various algorithms, and improving the accuracy and performance of anomaly detection and recognition. Attached Figure Description

[0039] To further understand the features and technical content of this invention, please refer to the following detailed description and accompanying drawings. However, the drawings are provided for reference and illustration only and are not intended to limit the invention.

[0040] In the attached diagram,

[0041] Figure 1 This is a flowchart of the multi-anomaly detection method for fusing the same target in this invention.

[0042] Figure 2 This is a flowchart illustrating the specific process of the multi-anomaly detection method for fusing the same target in this invention.

[0043] Figure 3 This is a flowchart of the self-learning strategy of the multi-anomaly detection same target fusion method of the present invention;

[0044] Figure 4 This is a detailed flowchart of step S1 of the multi-anomaly detection same target fusion method of the present invention;

[0045] Figure 5 This is a detailed flowchart of step S2 of the multi-anomaly detection same target fusion method of the present invention;

[0046] Figure 6 A flowchart of some steps in a multi-algorithm labeling method for moving video memory to mark memory;

[0047] Figure 7 A flowchart of some steps in a multi-algorithm labeling method for moving video memory to mark memory;

[0048] Figure 8 A flowchart of some steps in a multi-algorithm labeling method for moving video memory to mark memory;

[0049] Figure 9 This is a flowchart illustrating the implementation of the same-target fusion method for multiple anomaly detection according to the present invention.

[0050] Figure 10 This is a schematic diagram of the same target fusion device for multiple anomaly detection according to the present invention;

[0051] Figure 11 This is a schematic diagram of the electronic device of the present invention. Detailed Implementation

[0052] To further illustrate the technical means and effects of the present invention, the following detailed description is provided in conjunction with the preferred embodiments of the present invention and their accompanying drawings.

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0055] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0056] Please see Figure 1 and Figure 2 This invention first provides a method for fusing the same target in multiple anomaly detections, comprising the following steps:

[0057] Step S1: Determine the self-learning strategy for various classification and detection algorithms.

[0058] Specifically, such as Figure 3 and Figure 4 As shown, for various classification and detection algorithms, self-learning strategies are determined, including:

[0059] Step S11: For various classification algorithms, incorporate the learning object algorithm based on the measured indicators;

[0060] For various classification algorithms, an initial minimum accuracy strategy for the learning target algorithm is adopted. When the learning value of the ResNet34 algorithm reaches 73%, it is included as the current learning target algorithm. The learning value of the current learning target algorithm is determined to be the ResNet34 learning value, which is 73%. Based on the principle of prioritizing the inclusion of ResNet and ResNext series algorithms as learning target algorithms, the weight value of the included learning target algorithm is 0.9, and it is recorded in the initial teacher algorithm list. The minimum accuracy strategies for other classification algorithms included as learning target algorithms are as follows: ResNet50 corresponds to 76%, ResNeXt50-32x4d corresponds to 93%, MobileNetv2 corresponds to 71%, and MNASNet1.0 corresponds to 73%, with corresponding weight values ​​of 0.9, 0.86, 0.84, and 0.82, respectively.

[0061] Step S12: For various detection algorithms, design a threshold queue by arranging their prediction results in descending order of confidence.

[0062] Threshold queues, for example: 0, 0.2, 0.4, 0.6, 0.8, 1.

[0063] Step S13: The prediction results of the thresholds above the threshold of the threshold queue are included in the accuracy P of the detection algorithm corresponding to the threshold, and the average value of its accuracy is calculated as the learning value S of the learning object algorithm.

[0064] The prediction results for values ​​exceeding the threshold of the threshold queue are included in the accuracy P of the detection algorithm corresponding to that threshold, thus obtaining 6 sets of accuracy. The formula for calculating the learning value S of the learning object algorithm is as follows:

[0065]

[0066] Where accuracy P = positive detection / total detection, Wn is the threshold queue, L is the length of the threshold queue, and i is the threshold of the threshold queue.

[0067] The minimum accuracy strategy for initializing the learning target algorithm is as follows: when the learning value of the YOLOv5x algorithm reaches 50%, YOLOv5x is included as the current learning target algorithm. The learning value of the current learning target algorithm is determined to be the YOLOv5x learning value, which is 50%. According to the principle of prioritizing the inclusion of YOLOv5x as the learning target algorithm, the weight value of the included learning target algorithm is 0.9, and it is recorded in the initial teacher algorithm list. The minimum accuracy strategies for other detection algorithms included in the learning target algorithm are as follows: YOLOX-x corresponds to 51%, YOLOX-m corresponds to 46%, and YOLOv5m corresponds to 45%, with corresponding weight values ​​of 0.88, 0.86, and 0.84, respectively.

[0068] Step S14: Based on the learning value S of the learning object algorithm, determine the teacher algorithm, form an initial teacher algorithm list, and use it as a pre-training network for the same target detection algorithm network according to the principle of prioritizing the inclusion of learning object algorithms. Supervise and adjust the loss function of the same target detection algorithm, update and determine its algorithm weight, and form a self-learning strategy.

[0069] Step S2: Establish multiple target teacher detection models and a perceptron list according to the self-learning strategy, and train and optimize the abnormal event detection and recognition algorithm model according to the target teacher detection models and the perceptron list.

[0070] Specifically, such as Figure 5 As shown, multiple target teacher models and a perceptron list are established based on the self-learning strategy. An abnormal event detection and recognition algorithm model is trained and optimized based on the target teacher models and the perceptron list, including:

[0071] Step S21: Establish an algorithm type list, establish a target type list for each type of algorithm, and establish an initial detection target teacher model for each target type; based on the self-learning strategy, if an algorithm model has a higher accuracy than the detection target teacher model for the corresponding target type, replace the corresponding detection target teacher model with the algorithm model.

[0072] Step S22: Collect, filter and classify target sample data, and train and test the target sample data to obtain an abnormal event detection and recognition algorithm model.

[0073] Step S23: Referring to the prediction results of the detection target teacher model and the perceptron list of each detection target teacher model corresponding to the same target, the current algorithm loss function corresponding to the target is modified, the abnormal event detection and recognition algorithm model of the target is tested and optimized, and the optimized abnormal event detection and recognition algorithm model is obtained.

[0074] Step S3: When a new abnormal event detection and recognition algorithm model is accessed, it is determined whether the algorithm of the abnormal event detection and recognition algorithm model is a new algorithm; if it is a new algorithm, an algorithm number is automatically added; according to the new algorithm, an algorithm instance with the corresponding algorithm number is generated; when generating the algorithm instance, the abnormal event detection and recognition algorithm model with the corresponding algorithm number is loaded; according to the algorithm number, the algorithm instance inference interface of the corresponding algorithm is called; the algorithm instance inference interface is based on the multi-algorithm marking method of marking memory and moving video memory, and implements algorithm inference and algorithm post-processing logic in the algorithm instance inference interface, and feeds back the abnormal event detection and recognition results and location information corresponding to the video frame image; the abnormal event detection and recognition results and location information fed back by each algorithm simultaneously are weighted and adaptively fused to obtain the fused abnormal event detection and recognition result with the same target result.

[0075] Specifically, the anomaly detection and identification results and location information simultaneously fed back by each algorithm are weighted and adaptively fused to obtain fused anomaly detection and identification results with the same target result, including:

[0076] The abnormal event detection and identification results and location information simultaneously fed back by each algorithm are weighted and adaptively fused according to the following formula:

[0077]

[0078] Where, ω i For the adaptively adjusted weights corresponding to algorithm number i, D i B(x) represents the detection box of the same target result corresponding to algorithm number i, and B(x) represents the detection box of the abnormal event detection and recognition result after the same target is fused.

[0079] Please see Figure 9 This invention develops a multi-algorithm labeling method that involves moving video memory between memory and GPU memory. It performs preprocessing, inference, and post-processing on video frames, providing feedback on identical detection targets in the video frame images, including target IDs and location information. A fusion strategy is used to determine whether identical detection targets should be fused, and a weighted adaptive fusion formula for identical targets is used to calculate the fused detection bounding box. Based on memory, it has the ability to globally share multiple algorithms, reducing the time spent moving between memory and GPU memory for each algorithm, avoiding repetitive operations on video frame images, and the time consumed by copying and transferring, thus reducing the computational burden on hardware devices and improving the performance of multi-algorithm analysis. Figure 6 As shown, the multi-algorithm marking method for marking memory movement of video memory includes:

[0080] Step S31: Develop a video frame image processing task thread pool, an event exception algorithm thread pool, a blocking queue, and their global memory space based on the standard C / C++11 library. Each algorithm shares the global blocking queue memory space.

[0081] Step S32: Mark the video frame images and corresponding algorithm numbers in the memory space, and pack the video frame images into an image block of 16 images each time. Move the image blocks corresponding to the number of GPU cards each time on each GPU card.

[0082] Step S33: Based on hardware acceleration, batch process the image blocks, predict the results of the same detection target, and feed them back to the corresponding image marked in the memory space corresponding to the image block.

[0083] like Figure 7 As shown, the multi-algorithm marking method for marking memory movement of video memory also includes:

[0084] Step S41: The video frame image processing task thread pool consists of single task threads corresponding to the video number, and the blocking queue consists of single queues corresponding to the video number. The single task thread decodes the video and then skips frames to extract video frame images.

[0085] Step S42: Store the video frame images into a single queue corresponding to the video number, mark the global identifier of the single queue and its reference index, so that multiple algorithms can share the single queue and memory; if a corresponding algorithm uses the single queue, increment the reference index by 1; when the reference index is 0, clear the single queue by a single task thread; when the number of frame images of the corresponding video number exceeds the capacity of the single queue of the corresponding video number, the single task thread of the corresponding video number will wait in a blocking manner.

[0086] like Figure 8 As shown, the multi-algorithm marking method for marking memory movement of video memory also includes:

[0087] Step S51: The event exception algorithm thread pool consists of single algorithm threads with corresponding algorithm numbers. For each additional single algorithm thread, the global identifier reference index of the single queue is incremented by 1.

[0088] In step S52, a single algorithm thread continuously retrieves video frame images from a single queue marked with the corresponding video number in memory. Based on the video number, frame number, algorithm number, and parameters identified by the video frame image, the corresponding point, frame image, and algorithm are uniquely determined. Through a standardized algorithm interface, the numbering algorithm determined by the single algorithm thread analyzes and returns the results of abnormal events.

[0089] The following specific examples illustrate the solution of this invention: Before training the AI ​​model, a list of algorithm types, such as 01 detection algorithms and 02 classification algorithms, is established. Based on the aforementioned initial learning strategy, a list of learning values ​​and weights for the following algorithms is formed: 01 type algorithm YOLOv5x with a learning value of 50% and a weight of 0.9; 01 type algorithm YOLOX-x with a learning value of 51% and a weight of 0.88; 01 type algorithm YOLOX-m with a learning value of 46% and a weight of 0.86; 01 type algorithm YOLOv5m with a learning value of 45% and a weight of 0.84; 02 type algorithm ResNet50 with a learning value of 76% and a weight of 0.9; 02 type algorithm ResNeXt50-32x4d with a learning value of 93% and a weight of 0.86; 02 type algorithm MobileNetv2 with a learning value of 71% and a weight of 0.84; and 02 type algorithm MNASNet1.0 with a corresponding weight of 73% and a weight of 0.84.

[0090] Within each of the above algorithm types, establish a target type list including: 001 Pedestrians, 002 Motor Vehicles, 003 Non-motorized Vehicles, 004 Faces, 005 Strollers, 006 Handbags, 007 Suitcases, 008 Cargo Boxes, 009 Trash Cans, 010 Wheelchairs, 011 Strollers, 012 Safety Helmets, 013 Chef's Hats, 014 Name Badge Straps, 015 Fire Equipment, 016 Masks, and 017 Canes. For each target type, if the initial learning strategy is met, establish an initial algorithm model number corresponding to that target and use it as the initial teacher algorithm model for that target. If any algorithm model has a learning value greater than the initial teacher algorithm model for a given target type, replace the corresponding initial teacher algorithm model number with this algorithm model number.

[0091] To address the AI ​​application needs of categories 001 (pedestrians), 005 (strollers), 009 (trash cans), and 010 (wheelchairs), detection algorithm models for these target types were designed. Millions of sample data points for these detection targets were collected, filtered, and categorized. The corresponding algorithm loss functions for each detection target were modified. Through training, initial teacher algorithm models for abnormal event detection and recognition of these targets were obtained. For the 001 pedestrian target type, the initial teacher algorithm network for pedestrian detection was used as a pre-training network for algorithms related to pedestrian lingering, illegal gate access, escalator reversal, crowd density, and regional passenger flow monitoring. For the 005 stroller, 009 trash can, and 010 wheelchair target types, the initial teacher algorithm networks for stroller, trash can, and wheelchair detection were used as pre-training networks for stroller riding on escalators, trash can overflow, and wheelchair detection, respectively.

[0092] Based on the pre-trained network described above, the prediction results of various algorithm models for the same objective were statistically analyzed. For the pedestrian target type 001, the algorithm models were as follows: 80% learning value for pedestrian lingering, weight 0.9 (72%); 82% learning value for illegal gate passage, weight 0.88 (72.16%); and 84% learning value for escalator reversal, weight 0.86 (72.24%). Based on the ranking of these comprehensive values, the optimal algorithm model was determined to be escalator reversal. The pedestrian target feedback from the escalator reversal algorithm model was used to modify the loss functions for the corresponding pedestrian target's pedestrian lingering and illegal gate passage algorithms. Through testing, the algorithm models for pedestrian lingering and illegal gate passage were optimized, resulting in the optimized algorithm models for these objectives.

[0093] Based on the aforementioned algorithm models related to pedestrian target type 001, such as 01 pedestrian lingering, 02 illegal gate crossing, and 03 escalator reversing, before video analysis, algorithm instances with algorithm model numbers 01, 02, and 03 are created. The corresponding abnormal event detection and recognition algorithm models for pedestrian lingering, illegal gate crossing, and escalator reversing with numbers 01, 02, and 03 are loaded. During video analysis, for the preprocessing, inference, and postprocessing of video frames, a multi-algorithm labeling method based on labeled memory and video memory is adopted to provide feedback on pedestrian targets in video frame images of lingering, illegal gate crossing, and escalator reversing. The results are: "object_id":"0212", "rectangle": [638, 232, 92, 253]; "object_id":"0212", "rectangle": [640, 232, 92, 253]; "object_id":"0212", "rectangle": [642, 232, 92, 253].

[0094] Determine whether the pedestrian targets mentioned above were accurately detected by the algorithm. If the confidence level of the pedestrian target is less than the threshold, then the pedestrian target is considered a false detection, and the above pedestrian targets are filtered out. The determination formula is as follows:

[0095] C i (x) < c (filter); C i (x)≥cD i-n (x) < d (for backup); C i (x)≥cD i-n (x)≥d fusion

[0096] Where C i (x) represents the confidence level of the pedestrian target at position i in the algorithm analysis, c is the confidence threshold, and D is the confidence level. i-n (x) represents the intersection of the pedestrian target in the detection boxes of algorithms i to n, and d is the threshold of the detection box intersection.

[0097] If the confidence level of the pedestrian target is greater than or equal to the threshold, and the intersection of the pedestrian target detection boxes is less than the threshold, it is marked in memory for later use; if the confidence level of the pedestrian target is greater than or equal to the threshold, and the intersection of the pedestrian target detection boxes is greater than or equal to the threshold, then a fusion strategy is adopted. When the weights for personnel lingering, illegal passage through the turnstile, and escalator reversal are 1, 0, and 0, respectively, the fused detection boxes are obtained according to the same target weighted adaptive fusion formula as "object_id": "0212", "rectangle": [638, 232, 92, 253].

[0098] The anomaly detection method of this invention is based on a multi-anomaly detection learning strategy. It establishes a multi-target teacher model perceptron, trains and calibrates corresponding target algorithm models, and achieves fusion detection and recognition of the same target through an adaptive weighted fusion strategy. This improves the accuracy and performance of anomaly event detection and recognition, supporting intelligent applications such as city brain, smart security services, and refined urban management. The main effects are as follows:

[0099] We design learning strategies for various anomaly detection algorithms, discover learnable target algorithms, and establish multiple detection target teacher model perceptrons. Based on the perceived targets of each detection target perceptron, we calibrate the prediction results of the same target of the current algorithm and train to obtain the calibrated algorithm model of the multiple detection target teacher algorithms.

[0100] When various algorithms are used to detect and identify abnormal events, an adaptive weighted fusion strategy for abnormal events is established based on the above algorithm modules through real-time prediction results. This enables the fusion detection and identification of multiple events and multiple algorithms for the same target, further improving the accuracy of abnormal event detection and identification.

[0101] The multi-algorithm labeling method, which uses memory to move video memory, enables various algorithms to share video frame images, perform batch labeling and algorithm inference, reduce the time spent moving between memory and video memory, avoid repetitive operations on video frame images, and reduce the time spent copying and transmitting, thereby reducing the computational burden on hardware devices and improving the performance of multi-algorithm analysis.

[0102] Please see Figure 10 The present invention provides a fusion device for multiple anomaly detection of the same target, comprising:

[0103] Self-learning strategy unit 10 is used to determine self-learning strategies for various classification and detection algorithms;

[0104] The optimization model unit 20 is used to establish multiple detection target teacher models and a perceptron list according to the self-learning strategy, and to train and optimize the abnormal event detection and recognition algorithm model according to the detection target teacher models and the perceptron list.

[0105] The real-time fusion unit 30 is used to determine whether the algorithm of the newly accessed abnormal event detection and recognition algorithm model is a new algorithm when a new abnormal event detection and recognition algorithm model is introduced; if it is a new algorithm, an algorithm number is automatically added; an algorithm instance with the corresponding algorithm number is generated according to the new algorithm; when generating the algorithm instance, the abnormal event detection and recognition algorithm model with the corresponding algorithm number is loaded; according to the algorithm number, the algorithm instance inference interface of the corresponding algorithm is called; the algorithm instance inference interface is based on the multi-algorithm marking method of marking memory and moving video memory, and implements algorithm inference and algorithm post-processing logic in the algorithm instance inference interface, and feeds back the abnormal event detection and recognition results and location information corresponding to the video frame image; the abnormal event detection and recognition results and location information fed back by each algorithm simultaneously are weighted and adaptively fused to obtain a fused abnormal event detection and recognition result with the same target result.

[0106] Please see Figure 11 The present invention provides an electronic device, including: a memory 200 and a processor 100, wherein the memory 200 stores a computer program, and when the computer program is executed by the processor 100, the processor 100 performs the steps of the method.

[0107] This invention also provides a storage medium storing a computer program. When executed by a processor, the computer program implements some or all of the steps in the various embodiments of the multi-anomaly detection same-target fusion method provided by this invention. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0108] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0109] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, for the embodiments of the same target fusion device for multiple anomaly detection, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0110] The embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention.

Claims

1. A method for fusing the same target in multiple anomaly detections, characterized in that, Includes the following steps: For various classification and detection algorithms, determine the self-learning strategy; Multiple target teacher models and a perceptron list are established based on the self-learning strategy. An abnormal event detection and recognition algorithm model is trained and optimized based on the target teacher models and perceptron list. Specifically, an algorithm type list is established, a target type list is established for each type of algorithm, and an initial target teacher model is established for each target type. Based on the self-learning strategy, if an algorithm model has a higher accuracy than the target teacher model for a corresponding target type, the algorithm model replaces the corresponding target teacher model. To meet the business needs of AI applications, a target detection algorithm model for a target type list is designed; target sample data is collected, filtered, and classified; and training and testing are performed based on the target sample data to obtain an initial teacher algorithm model for abnormal event detection and recognition of the detected targets. Referring to the prediction results of the teacher model for the target and the perceptron list for each teacher model for the same target, the current algorithm loss function corresponding to the target is modified, the abnormal event detection and recognition algorithm model for the target is tested and optimized, and the optimized abnormal event detection and recognition algorithm model is obtained. When a new abnormal event detection and recognition algorithm model is introduced, it is determined whether the algorithm of the abnormal event detection and recognition algorithm model is a new algorithm. If it is a new algorithm, an algorithm number is automatically added. Based on the new algorithm, an algorithm instance with the corresponding algorithm number is generated. When generating the algorithm instance, the abnormal event detection and recognition algorithm model with the corresponding algorithm number is loaded. Based on the algorithm number, the algorithm instance inference interface of the corresponding algorithm is called. The algorithm instance inference interface is based on a multi-algorithm labeling method that moves video memory to marked memory. The algorithm inference interface implements algorithm inference and algorithm post-processing logic, and feeds back the abnormal event detection and recognition results and location information corresponding to the video frame image. The abnormal event detection and recognition results and location information fed back by each algorithm simultaneously are weighted and adaptively fused to obtain a fused abnormal event detection and recognition result with the same target result. Multi-algorithm marking methods for marking memory movement of video memory include: The video frame image processing task thread pool, event exception algorithm thread pool, blocking queue and its global memory space are developed based on the standard C / C++11 library. Each algorithm shares the global blocking queue memory space. The video frame images and their corresponding algorithm numbers are marked in the memory space. The video frame images are packaged into an image block of 16 images each time. The number of image blocks corresponding to the number of GPU cards is moved each time and placed on each GPU card. Based on hardware acceleration, the image blocks are processed in batches, the results of the same detection target are predicted, and the results are fed back to the corresponding image of the image block marked in the memory space.

2. The method for fusing the same target in multiple anomaly detection as described in claim 1, characterized in that, For various classification and detection algorithms, a self-learning strategy is determined, including: For various classification algorithms, the learning object algorithm is incorporated based on actual test indicators; For various detection algorithms, a threshold queue is designed based on their prediction results and arranged in descending order of their confidence. The prediction results of the thresholds above the threshold queue are included in the accuracy P of the detection algorithm corresponding to the threshold, and the average of its accuracy is calculated as the learning value S of the learning object algorithm. Based on the learning value S of the learning object algorithm, the teacher algorithm is determined to form an initial list of teacher algorithms. According to the principle of prioritizing the inclusion of learning object algorithms, these algorithms are used as pre-training networks for the same target detection algorithm network. The loss function of the same target detection algorithm is adjusted under supervision, and its algorithm weights are updated and determined to form a self-learning strategy.

3. The method for fusing the same target in multiple anomaly detection as described in claim 2, characterized in that, The formula for calculating the learning value S of the learning object algorithm is as follows: Where accuracy P = positive detection / total detection, Wn is the threshold queue, L is the length of the threshold queue, and i is the threshold of the threshold queue.

4. The method for fusing the same target in multiple anomaly detection as described in claim 1, characterized in that, The anomaly detection and identification results and location information simultaneously fed back by various algorithms are weighted and adaptively fused to obtain fused anomaly detection and identification results with the same target result, including: The abnormal event detection and identification results and location information simultaneously fed back by each algorithm are weighted and adaptively fused according to the following formula: Where ωi is the adaptively adjusted weight corresponding to algorithm number i, Di(x) is the detection box of the same target result corresponding to algorithm number i, and B(x) is the detection box of the abnormal event detection and recognition result after the fusion of the same target.

5. The method for fusing the same target in multiple anomaly detection as described in claim 1, characterized in that, The video frame image processing task thread pool consists of single task threads corresponding to the video number, and the blocking queue consists of single queues corresponding to the video number. The single task thread decodes the video and then skips frames to extract video frame images. Video frame images are stored in a single queue corresponding to the video number, and a global identifier and its reference index are marked for the single queue, enabling multiple algorithms to share the single queue and memory. If a corresponding algorithm uses the single queue, the reference index is incremented by 1. When the reference index is 0, the single queue is cleared by a single task thread. When the number of frame images of a video with a corresponding number exceeds the capacity of the single queue for that video number, the single task thread for that video number waits in a blocking manner.

6. The method for fusing the same target in multiple anomaly detection as described in claim 5, characterized in that, The event exception algorithm thread pool consists of single algorithm threads with corresponding algorithm numbers. For each additional single algorithm thread, the global identifier reference index of the single queue is incremented by 1. A single algorithm thread continuously retrieves video frame images from a single queue marked with corresponding video IDs in memory. Based on the video ID, frame ID, algorithm ID, and parameters identified by the video frame image, it uniquely determines the corresponding point, frame image, and algorithm of the video. Through a standardized algorithm interface, the numbering algorithm determined by the single algorithm thread analyzes and returns the results of abnormal events.

7. A fusion device for multiple anomaly detection of the same target, characterized in that, include: The self-learning strategy unit is used to determine the self-learning strategy for a variety of classification and detection algorithms. An optimization model unit is used to establish multiple detection target teacher models and a perceptron list according to the self-learning strategy, and to train and optimize anomaly detection and recognition algorithm models according to the detection target teacher models and perceptron lists; wherein, an algorithm type list is established, a target type list is established for each type of algorithm, and an initial detection target teacher model is established for each target type; based on the self-learning strategy, if an algorithm model has a higher accuracy than the detection target teacher model for a corresponding target type, the algorithm model replaces the corresponding detection target teacher model; To meet the business needs of AI applications, a target detection algorithm model for a target type list is designed; target sample data is collected, filtered, and classified; and training and testing are performed based on the target sample data to obtain an initial teacher algorithm model for abnormal event detection and recognition of the detected targets. Referring to the prediction results of the teacher model for the target and the perceptron list for each teacher model for the same target, the current algorithm loss function corresponding to the target is modified, the abnormal event detection and recognition algorithm model for the target is tested and optimized, and the optimized abnormal event detection and recognition algorithm model is obtained. The real-time fusion unit is used to determine whether the algorithm of a newly accessed anomaly detection and recognition algorithm model is a new algorithm when a new anomaly detection and recognition algorithm model is introduced; if it is a new algorithm, it automatically adds an algorithm number; based on the new algorithm, it generates an algorithm instance with the corresponding algorithm number; when generating the algorithm instance, it loads the anomaly detection and recognition algorithm model with the corresponding algorithm number; based on the algorithm number, it calls the algorithm instance inference interface of the corresponding algorithm; the algorithm instance inference interface is based on a multi-algorithm marking method that moves video memory to a marked memory, and implements algorithm inference and algorithm post-processing logic in the algorithm instance inference interface, and feeds back the anomaly detection and recognition results and location information corresponding to the video frame images; it also processes the anomaly detection and recognition results fed back by various algorithms simultaneously. The results and location information are weighted and adaptively fused to obtain the fused abnormal event detection and recognition results for the same target result; a multi-algorithm labeling method for moving video memory is used, including: developing a video frame image processing task thread pool, an event exception algorithm thread pool, a blocking queue, and its global memory space based on the standard C / C++11 library, with each algorithm sharing the global blocking queue memory space; labeling video frame images and their corresponding algorithm numbers in the memory space, packaging the video frame images into image blocks of 16 images each time, and moving the corresponding number of image blocks to each GPU card each time; based on hardware acceleration, batch processing of the image blocks, predicting the same detection target result, and feeding it back to the corresponding image of the image block marked in the memory space.

8. An electronic device, characterized in that, include: A memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1-6.

Images