A detection bounding box smoothing method for multi-frame target synthesis

Abstract

This invention discloses a method for smoothing detection boxes of multi-frame target integration. The steps are as follows: (1) calculate the weight of the detection result of the current frame; (2) take the results of the previous three frames and the result of the next frame respectively to calculate the weight of the detection result relative to the current frame; normalize the five weights obtained in steps (1) and (2) to obtain normalized weight coefficients; multiply each normalized coefficient by the result of the corresponding frame and accumulate them to obtain the smoothed output of the current frame; repeat steps (1) to (4) until the video stream data ends. In video stream detection tasks, compared with the neural network target detection model that directly outputs the target category, confidence and box position, the method of this invention can generate smoother detection results, suppress the jitter and missed detection of the output target box, and make the neural network target detection model have higher accuracy and stability in video detection tasks, thus improving the practical value of the algorithm in various fields.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method for smoothing the output target detection box of a neural network target detection model in a video detection task. Background Technology

[0002] Object detection refers to identifying the location coordinates of a target in an image or video and its corresponding known category. Currently, in the context of big data, artificial intelligence technology has achieved rapid development in various fields. Neural network-based object detection technology has shown significant advantages over traditional image algorithms in areas such as traffic monitoring, military reconnaissance, and early warning, and has achieved remarkable practical application results.

[0003] Currently, most commonly used neural network-based object detection algorithms for video stream data detection employ single-frame image-based algorithms. The output target is independent of each video frame, depending only on the current input frame. This involves feeding the current frame image into a neural network object detection model, where it undergoes a series of operations such as convolution, pooling, fusion, and downsampling to obtain feature maps at different scales. These feature maps are then processed through classification, location regression, and non-maximum suppression to ultimately output the target's category and location information within the image.

[0004] In practical applications, neural network target detection models based on single-frame information, when performing detection and recognition tasks on video stream data, will generate different feature maps if adjacent frames are not completely identical in input, thus leading to different categories, confidence levels, and location information for the target. In videos, target detection results exhibit fluctuations and jitter, and in some frames, due to low target confidence, consecutively appearing targets may be missed.

[0005] In video stream detection tasks, single-frame object detection models suffer from issues such as detection box jitter and missed detections, which significantly impact the accuracy and stability of the detection results. To improve the accuracy and robustness of object detection models in video stream detection tasks, it is necessary to fully consider the relationship between targets in consecutive frames within the video. By leveraging the characteristic that targets do not change much in a very short time between consecutive frames, the output of the current frame can be smoothed to suppress the jitter and missed detection phenomena. Summary of the Invention

[0006] This invention fully leverages the detection efficiency of the neural network object detection model. By utilizing the detection results of multiple frames, the output of the neural network object detection model in the current frame is smoothed, suppressing target box jitter and missed detections, thereby improving the model's detection performance and stability.

[0007] The technical solution adopted by this invention to solve its technical problem is: a detection box smoothing method for multi-frame target synthesis, comprising the following steps:

[0008] (1) Calculate the weight of the detection result of the current frame;

[0009] (2) Take the results of the first three frames and the result of the next frame respectively to calculate the weight of the detection result relative to the current frame;

[0010] (3) Normalize the five weights obtained in steps (1) and (2) to obtain normalized weight coefficients;

[0011] (4) Multiply each normalization coefficient by the result of the corresponding frame and sum them up to obtain the smoothed output of the current frame.

[0012] (5) Repeat steps (1) to (4) to process the next frame.

[0013] In the multi-frame target synthesis detection box smoothing method described above, the output result of the neural network target detection model in step (1) of the current t-th frame is represented as (cls) t ,conf t ,x t ,y t ,w t ,h t ), where cls represents the category, conf represents the classification confidence, and (x, y, w, h) is the location of the target in the image, representing the bounding box of the target. t The center point coordinates, width, and height are given, and the current frame weight is w. t =1×conf t .

[0014] The multi-frame target synthesis detection box smoothing method, in step (2), outputs the target detection result (cls) for the (t-1)th frame. t-1 conf t-1 x t-1 y t-1 w t-1 h t-1 The weighting coefficient is the detection box in frame t-1. t-1 relative to the detection box in frame t t The product of the IOU and the geometric mean of the confidence level is expressed as: Where IOU is the intersection-union ratio of the two circumscribed rectangles. Following the method described above, calculate the weight values ​​w for frames t-2, t-3, and t+1. t-3 w t-3 and w t+1 .

[0015] The method for smoothing detection boxes in multi-frame target synthesis, step (3) normalizes the five obtained weight values ​​using the following formula:

[0016] In the multi-frame target synthesis detection box smoothing method described above, step (4) involves multiplying the normalized weight coefficients with the output results of the corresponding frames and then summing the results to obtain the smoothed output result for the current frame.

[0017]

[0018] Smoothed output (cls) t conf t x t y t w t h t ) smooth Replacement model output (cls) t conf t x t y t w t h t This is the final output of the current frame.

[0019] The beneficial effects of this invention are: In video stream detection tasks, compared with neural network object detection models that directly output the object category, confidence level and box position, the method of this invention can generate smoother detection results, suppress jitter and missed detection phenomena in the output object box, and make the neural network object detection model have higher accuracy and stability in video detection tasks, thus improving the practical value of the algorithm in various fields. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the smoothing calculation of the multi-frame integrated detection box in this invention;

[0021] Figure 2 It is the detection box in frame t-1. t-1 relative to the detection box in frame t t IOU diagram. Detailed Implementation

[0022] To further illustrate the objectives and technical solutions of this invention, the invention will be described in more detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are for illustrative purposes only and do not constitute a limitation thereof.

[0023] Reference Figure 1 , Figure 2As shown, the present invention discloses a method for smoothing detection boxes in multi-frame target synthesis, comprising the following steps:

[0024] (1) Calculate the weights based on the detection results of the current frame.

[0025] The output of the neural network model in the current frame t is represented in the following way:

[0026] (cls t ,conf t ,x t ,y t ,w t ,h t ),

[0027] Where cls represents the category, conf represents the classification confidence, and (x, y, w, h) is the location of the target in the image, where cls represents the bounding box of the target, and conf represents the bounding box of the target. t The coordinates of the center point, as well as the width and height.

[0028] The weight of the current frame is: w t =1×conf t .

[0029] (2) Take the results of the first three frames and the result of the last frame, and calculate the weights relative to the current frame respectively.

[0030] The target detection output (cls) for frame t-1 t-1 ,conf t-1 ,x t-1 ,y t-1 ,w t-1 ,h t-1 The weighting coefficient is the detection box in frame t-1. t-1 relative to the detection box in frame t t The product of the OOU and the geometric mean of the confidence level is expressed as:

[0031]

[0032] Where OIU is the intersection-union ratio of the two bounding rectangles:

[0033]

[0034] Following the method described above, calculate the weight values ​​w for the (t-2)th frame, (t-3)th frame, and (t+1)th frame, respectively. t-3 w t-3 and w t+1 .

[0035] (3) Normalize the five weights obtained in (1) and (2) to obtain the normalized weight coefficients:

[0036]

[0037] (4) Each normalization coefficient is multiplied by the result of the corresponding frame and then summed to obtain the smoothed output of the current frame.

[0038] The normalized weight coefficients are multiplied by the output of the corresponding frame and then summed. The resulting output is used as the smoothed output of the current frame.

[0039]

[0040] Smoothed output (cls) t ,conf t ,x t ,y t ,w t ,h t ) smooth Replacement model output (cls) t ,conf t ,x t ,y t ,w t ,h t This is the final output of the current frame.

[0041] (5) The processing of the next frame repeats the process (1) to (4): The calculation of subsequent frames repeats the process (1) to (4) to achieve smoothing of the output results of each frame in the video image. In the video target detection task, the jitter and missed detection of the output detection box are suppressed, and the accuracy and stability of target detection are improved.

[0042] Those skilled in the art will readily understand that the above description is merely a preferred use case of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for smoothing detection bounding boxes in multi-frame target synthesis, characterized in that: Includes the following steps: (1) Calculate the weight of the detection result of the current frame: the weight of the current frame of the neural network object detection model. t A particular output result of a frame is represented as ( cls t , conf t , x t , y t , w t , h t ),in cls Indicates the category to which it belongs. conf Indicates the confidence level of the classification, ( x , y , w , h ) represents the bounding rectangle of the target. box t The center point coordinates, width, and height, and the current frame weight are... w t =1× conf t ; (2) Calculate the detection result weight relative to the current frame by taking the results of the first three frames and the result of the next frame respectively: For the first frame... t -1 frame output result (𝑐l𝑠) t-1 , 𝑐𝑜𝑛𝑓 t-1 , 𝑥 t-1 , 𝑦 t-1 , t-1 , ℎ t-1 ), whose weighting coefficient is the first t -1 frame detection box box t-1 Compared to the first t Frame detection box box t The product of 𝐼𝑂𝑈 and the geometric mean of the confidence level is expressed as: , where 𝐼𝑂𝑈 is the intersection-union ratio of the two bounding rectangles. Calculate the first one respectively t -2 frames, the t -3 frames and the t+ Weight value of 1 frame 𝑤 t-2 , 𝑤 t-3 and 𝑤 t+1 ; (3) Normalize the five weights obtained in steps (1) and (2) to obtain the normalized weight coefficients; (4) Multiply each normalization coefficient by the result of the corresponding frame and sum them up to obtain the smoothed output of the current frame; (5) Repeat steps (1) to (4) to process the next frame to achieve smoothing of the output results of each frame in the video image.

2. The detection box smoothing method for multi-frame target synthesis according to claim 1, characterized in that, Step (3) normalizes the five obtained weight values ​​using the following formula: .

3. The detection box smoothing method for multi-frame target synthesis according to claim 2, characterized in that, The smoothed output result in step (4) = ， Smoothed output results ( cls t , conf t , x t , y t , w t , h t ) smooth Replace the model output results ( cls t , conf t , x t , y t , w t , h t This is the final output of the current frame.

Images