Method and system for improving small-size target video image recognition accuracy in large field of view and storage medium

Abstract

The invention discloses a method and system for improving the recognition accuracy of a small-size target video image in a large field of view, and a storage medium. The method comprises the following steps: inputting a feature map; performing size unification operation on the input feature map; assigning an excitation value to each channel of the feature map through an attention mechanism; outputting the enhanced feature map to the feature extraction network of the next layer; and introducing a multi-scale feature fusion module based on an attention mechanism into the feature extraction network, and inputting a fused feature map into a detection network for detection. According to the invention, the problem of low detection and recognition rate caused by low target pixel proportion in a large-range monitoring video adopted by an existing monitoring system is solved.

Description

Technical Field

[0001] The present invention relates to the field of big data and image recognition technology, and in particular to a method, system and storage medium for improving the recognition accuracy of small-size target video images under a large field of view. Background Art

[0002] Video image recognition technology has always been a research hotspot in the field of computer vision and has been widely used and developed rapidly in recent years. It uses machine learning and artificial intelligence methods to enable computers to recognize and understand the content in digital images.

[0003] At present, video image recognition technology has been applied in many fields in China. In the industrial field, it is used to detect product quality problems, such as defects and damage. In the medical field, it is used to help doctors diagnose and treat diseases at an early stage. In the transportation field, it is used in intelligent transportation systems to identify vehicles and pedestrians and improve the efficiency and safety of transportation. In addition, image recognition technology is also widely used in security, education, finance and other fields.

[0004] In massive generalized video data, the intelligent processing of video images in complex scenes suffers from low pixel occupancy of small-sized targets under a large field of view, resulting in low target detection accuracy and large differences in imaging conditions. Summary of the invention

[0005] In view of this, one of the objects of the present invention is to provide a method for improving the recognition accuracy of small-sized target video images under a large field of view, mainly for solving the problem of low detection accuracy of small-sized targets.

[0006] One of the purposes of the present invention is achieved through the following technical solutions: The method for improving the recognition accuracy of a small-size target video image under a large field of view comprises the following steps: Input feature map; Perform a uniform size operation on the input feature map; Assign an incentive value to each channel of the feature map through the attention mechanism; Output the enhanced feature map to the feature extraction network of the next layer; A multi-scale feature fusion module based on the attention mechanism is introduced into the feature extraction network, and the fused feature map is input into the detection network for detection.

[0007] Furthermore, the unification operation is performed through a Resize operation, and the formula of the Resize operation is: ; Among them, H, W, and C represent the length, width, and dimension of the input feature map, respectively. After a series of operations, a one-dimensional feature array with a length of C is obtained.

[0008] Furthermore, the process of assigning an excitation value to each channel of the feature map is as follows: By modeling the correlation of each channel, the excitation formula is as follows: ; In the above formula, the parameters W1 and W2 are trained respectively by ReLU and Sigmoid activation functions, the obtained excitation values ​​are assigned to each channel, and the dimension of the final output S is C.

[0009] Furthermore, the feature extraction network adopts ResNet50, and the five scale feature maps output by the five convolution stages in ResNet50 are fused. The influence of the feature map is analyzed and weights are assigned to it through the attention mechanism, and then the fused feature map is input into the detection network for detection.

[0010] Furthermore, the influence of the feature graph is achieved by an average influence weighted algorithm, which includes the following steps: First, assuming there are m samples and each sample contains n feature maps, the influence factor matrix X and network output Y can be expressed as: ; ; Secondly, it is necessary to fine-tune the i-th influencing factor, and the fine-tuning value is set to δ. At this time, we can get: ; ; Next, calculate the average influence of the feature map, that is, the MIV value, which represents the relative influence of the current feature map. The calculation formula is as follows: ; Finally, calculate the contribution C corresponding to each feature map i And the average impact value C: ; .

[0011] Furthermore, the detection is performed by selecting the detection frame with the highest confidence and calculating the overlapping area between the frame and the remaining frames. If the calculated result is higher than the set threshold, it means that the overlap rate is too large, and the frame needs to be removed. Then, the step is repeated until all detection frames are calculated.

[0012] Furthermore, a method combining dynamic target detection and conventional target detection is used to detect small-sized targets from video frames and form feature maps. The specific steps are as follows: First, the multi-frame frame difference method is used to detect moving targets in the video stream. The frame difference method is to perform differential operations on continuous image frames. After denoising through morphological methods, it is determined whether there are moving targets in the target area. When a moving target appears, it is sent to the target classification module and classified by the trained feature target classifier. When the target suddenly stops during the movement and is in a static state, it will be detected by the end-to-end detection and classification module; In order to improve the dynamic target detection effect, the dynamic target detection module adopts the dynamic target detection method of Gaussian background modeling. Gaussian background modeling converts the motion target detection problem of sequence images into a binary classification problem based on the current background estimation, divides all pixels into two categories: background and motion foreground, and then post-processes the classification results to obtain the final detection result.

[0013] Furthermore, when there is overlap between the candidate boxes, first determine whether the overlapping area between the detection box and the remaining boxes is greater than the set threshold. If this is the case, the old confidence is multiplied by the attenuation factor to obtain a new confidence. Otherwise, proceed to the next round of screening.

[0014] A second object of the present invention is to provide a computer system, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the compression method and / or restoration method as described above when executing the computer program.

[0015] A third object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the compression method and / or restoration method as described above.

[0016] The beneficial effects of the present invention are: Aiming at the key scientific problem of low accuracy in small-size target detection, the present invention proposes a global feature alignment algorithm based on multi-scale feature fusion, utilizes a cross-layer feature fusion module to fuse target features of different scales into a new feature map, reduces the inter-domain differences caused by the different sizes of foreground targets, and utilizes a background suppression module to reduce background noise interference, focusing the network's attention on the alignment of foreground target features, thereby solving the problem of low detection and recognition rate due to the low proportion of target pixels in large-scale surveillance videos used in existing systems, and improving the current detection capabilities and deep-level data value in the field of smart transportation.

[0017] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description and the preceding claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which: Figure 1 The general flow chart of the method of the present invention is as follows; Figure 2 This is a schematic diagram of the Resize operation; Figure 3 Schematic diagram of the combination of ResNet50 network and multi-scale feature fusion module; Figure 4 Schematic diagram of the confidence optimization strategy; Figure 5 A schematic diagram showing the detection effect; Figure 6 Flowchart of the multi-scale feature fusion algorithm that introduces the attention mechanism; Figure 7 This is the actual effect diagram of dynamic target detection. DETAILED DESCRIPTION

[0019] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0020] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than for limiting the protection scope of the present invention.

[0021] Embodiment 1 like Figure 1 As shown, the method for improving the recognition accuracy of small-size target video images under a large field of view of the present invention comprises the following steps: Step S1: input feature map; Step S2: Perform a uniform size operation on the input feature map; Figure 2 As shown, in this embodiment, the operation of unifying the size is performed through a Resize operation, and the formula of the Resize operation is: ; Among them, H, W, and C represent the length, width, and dimension of the input feature map, respectively. After a series of operations, a one-dimensional feature array with a length of C is obtained.

[0022] Step S3: assigning an incentive value to each channel of the feature map through the attention mechanism to achieve the purpose of enhancing the attention of the key channel. In this embodiment, the process of assigning an incentive value to each channel of the feature map is as follows: By modeling the correlation of each channel, the excitation formula is as follows: ; In the above formula, the parameters W1 and W2 are trained respectively by ReLU and Sigmoid activation functions, the obtained excitation values ​​are assigned to each channel, and the dimension of the final output S is C.

[0023] Step S4: output the enhanced feature map to the feature extraction network of the next layer; Step S5: Introduce a multi-scale feature fusion module based on the attention mechanism into the feature extraction network, and input the fused feature map into the detection network for detection.

[0024] The feature extraction network uses ResNet50, which fuses the five scale feature maps output by the five convolution stages in ResNet50. SENet is a channel attention mechanism with the following structure: Figure 2 As shown in the figure, the influence of the feature map is analyzed through the attention mechanism and weights are assigned to it. The SE attention module performs global average pooling on the input feature map and reduces the eigenvalues ​​of each channel to a global vector. This step aims to capture the global information of each channel. The channel attention weight obtained in the previous step is multiplied by the original input feature map. This step is used to adjust the eigenvalues ​​of each channel, emphasize the information of important channels, and suppress the information of unimportant channels. The SE attention module is combined with ResNet, and the fused feature map is input into the detection network for detection. The attention module is combined with ResNet, and the combined structure is as follows: Figure 3 As shown in the figure, the architecture of the original Residual module (left) and the SE-ResNet module (right), the parameters and computational complexity of the combined ResNet50 network will increase to varying degrees. When the dimensionality reduction coefficient r is 16, the number of parameters increases by about 10%, but the computational complexity increases by less than 1%, so it will not significantly affect the calculation speed.

[0025] Embodiment 2 The conventional multi-scale feature fusion network does not distinguish the importance of each feature map (i.e., the influence in step S5) during the fusion process. In order to solve the above problem, based on the first embodiment, this embodiment proposes a global feature alignment algorithm to solve the problem of influence analysis of feature maps. The algorithm is implemented using the average influence (MIV) weighted algorithm, which is used to evaluate different feature maps of the network, and specifically includes the following steps: First, suppose there are m samples, each of which contains n feature maps, then the influence factor matrix X and the network output Y can be expressed as: ; ; Secondly, it is necessary to fine-tune the i-th influencing factor, and the fine-tuning value is set to δ (usually set to ±10%). At this time, we can get: ; ; Next, calculate the average influence of the feature map, that is, the MIV value, which represents the relative influence of the current feature map. The calculation formula is as follows: ; Finally, calculate the contribution C corresponding to each feature map i And the average impact value C: ; .

[0026] Embodiment 3 In this embodiment, the detection in step S5 is further explained. The detection is performed by screening out the detection frame with the highest confidence, and calculating the overlapping area (IOU) between the frame and the remaining frames. If the calculated result is higher than the set threshold, it means that the overlap rate is too large, and the frame needs to be removed. Then the step is repeated until all the detection frames are calculated. The key factor is the setting of the IOU threshold. If the threshold is too large, the suppression effect cannot be fully achieved. It cannot be set too low, which may cause multiple target objects to be merged together and affect the detection effect. Although the NMS algorithm can effectively suppress the generation of duplicate candidate frames, it will have the following problems: in the case of overlapping candidate frames, if the candidate frame containing multiple targets is removed due to excessive repetition rate, it may easily lead to missed detection of the target, thus having a greater impact on the detection performance. At this time, it is necessary to reduce the confidence instead of removing the candidate frame. Figure 4 Optimization shown.

[0027] The specific steps are as follows: for overlapping boxes that are originally required to be removed, by reducing their confidence, many candidate boxes containing other targets can be retained, which can avoid the impact caused by too many overlapping areas to a certain extent. The specific method is to first determine whether the IOU is greater than the set threshold. If it meets the situation, the old confidence needs to be multiplied by the attenuation factor to obtain a new confidence. Otherwise, the next round of screening is carried out. The attenuation factor conforms to the Gaussian distribution. The closer to the center of the Gaussian distribution, the more attenuation, that is, the attenuation factor is proportional to the IOU.

[0028] By reducing the confidence level so that the target box is retained, it helps to improve the target detection effect. Figure 5 shown.

[0029] The flow chart of the multi-scale feature fusion algorithm that introduces the attention mechanism is as follows Figure 6 As shown in the figure, the input feature map is first resized to a uniform size, and then an incentive value is assigned to each channel of the feature map through the attention mechanism to enhance the attention of the key channels, and finally the enhanced feature map is output to the next layer of the network.

[0030] In the present invention, the target features are integrated into the new feature graph in order to be able to more focusedly search for features that are highly correlated with the output results, and to improve the quality of the output by increasing the feature graphs with high influence and weakening the feature graphs with low influence. At the same time, the input subset can be purposefully selected to reduce the data dimension, thereby reducing the amount of calculation.

[0031] Because under the condition of large field of view, the pixel proportion of small targets in the image will be very small, and the imaging area of ​​the target in the frame image sent to the algorithm module is very small. In addition, due to the influence of light background, it is very difficult to detect small targets directly from the video frame, resulting in low accuracy. Therefore, the present invention designs a method combining dynamic target detection + conventional target detection, which is used to detect small-sized targets from video frames and form a feature map. The specific method is as follows: first, a multi-frame frame difference method is used to detect moving targets in the video stream. The frame difference method is to perform differential operations on continuous image frames. After denoising by morphological methods, it is determined whether there are moving targets in the target area. When a moving target appears, it is sent to the target classification module and the target is classified by the trained feature target classifier. When the target suddenly stops during the movement and is in a stationary state, it will be detected by the end-to-end detection and classification module.

[0032] In order to improve the dynamic target detection effect, the dynamic target detection module adopts the dynamic target detection method of Gaussian background modeling. Gaussian background modeling is also called background estimation. Its main purpose is to transform the problem of moving target detection of sequence images into a binary classification problem based on the current background estimation, and divide all pixels into two categories: background and motion foreground. Then, the classification results are post-processed to obtain the final detection result. The Gaussian background model is a method for extracting and updating the background and foreground in the process of moving target detection. The single distribution Gaussian background model believes that for a background image, the distribution of the brightness of a specific pixel satisfies the Gaussian distribution, that is, for the background image B, the brightness of each point (x, y) satisfies B(x, y) ~ N(μ, d).

[0033] ; That is, each point (x, y) contains two attributes, mean μ (i.e. average brightness value) and variance d. The mean μ and variance d of each point in the video sequence image over a period of time are calculated as the background model. For any image G containing a foreground, the following formula is used to calculate each point (x, y) on the image.

[0034] ; T is a constant threshold. If the value meets the formula, the point is considered to be a background point, otherwise it is a foreground point. Then the background is updated, and each frame of the image participates in the background update: ; Among them, p is a constant used to reflect the background update rate. The larger p is, the slower the background is updated. Under normal circumstances, the change of d after the background is updated is very small, so d is generally not updated after the background is updated.

[0035] The actual effect of using this method for dynamic target detection Figure 7 shown.

[0036] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0037] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0038] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0039] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for improving the recognition accuracy of small-size target video images under a large field of view, characterized in that: The method comprises the following steps: Input feature map; Perform a uniform size operation on the input feature map; Assign an incentive value to each channel of the feature map through the attention mechanism; Output the enhanced feature map to the feature extraction network of the next layer; A multi-scale feature fusion module based on the attention mechanism is introduced into the feature extraction network, and the fused feature map is input into the detection network for detection.

2. The method for improving the recognition accuracy of small-size target video images under a large field of view according to claim 1, characterized in that: The unification operation is performed through the Resize operation, and the formula of the Resize operation is: ； Among them, H, W, and C represent the length, width, and dimension of the input feature map, respectively. After a series of operations, a one-dimensional feature array with a length of C is obtained.

3. The method for improving the recognition accuracy of small-size target video images under a large field of view according to claim 1 or 2, characterized in that: The process of assigning an excitation value to each channel of the feature map is as follows: By modeling the correlation of each channel, the excitation formula is as follows: ； In the above formula, the parameters W1 and W2 are trained respectively by ReLU and Sigmoid activation functions, the obtained excitation values ​​are assigned to each channel, and the dimension of the final output S is C.

4. The method for improving the recognition accuracy of small-size target video images under a large field of view according to claim 1, characterized in that: The feature extraction network adopts ResNet50, and the five scale feature maps output by the five convolution stages in ResNet50 are fused. The influence of the feature map is analyzed and weights are assigned to it through the attention mechanism, and then the fused feature map is input into the detection network for detection.

5. The method for improving the recognition accuracy of small-size target video images under a large field of view according to claim 4, characterized in that: The influence of the feature graph is achieved by an average influence weighted algorithm, which includes the following steps: First, suppose there are m samples, each of which contains n feature maps, then the influence factor matrix X and the network output Y can be expressed as: ； ； Secondly, it is necessary to fine-tune the i-th influencing factor, and the fine-tuning value is set to δ. represents the influencing factor matrix after fine-tuning, Represents the network output after fine-tuning, and we can get: ； ； Next, calculate the average influence of the feature map. This value represents the relative influence of the current feature map. The calculation formula is as follows: ； In the above formula, That is, the MIV value, is the characteristic value of the network training variable, To initialize the network feature value, i represents the number of training times, and j represents the learning rate; Finally, calculate the contribution C corresponding to each feature map i And the average impact value C: ； 。 6. The method for improving the recognition accuracy of small-size target video images under a large field of view according to claim 4, characterized in that: Detection is performed by selecting the detection frame with the highest confidence and calculating the overlapping area between the frame and the remaining frames. If the calculated result is higher than the set threshold, it means that the overlap rate is too large, and the frame needs to be removed. Then, the step is repeated until all detection frames are calculated.

7. The method for improving the recognition accuracy of small-size target video images under a large field of view according to claim 5, characterized in that: When candidate boxes overlap, first determine whether the overlapping area between the detection box and the remaining boxes is greater than the set threshold. If so, multiply the old confidence by the attenuation factor to get a new confidence. Otherwise, proceed to the next round of screening.

8. The method for improving the recognition accuracy of small-size target video images under a large field of view according to claim 1, characterized in that: A method combining dynamic target detection and conventional target detection is used to detect small-sized targets from video frames and form feature maps. The specific steps are as follows: First, the multi-frame frame difference method is used to detect moving targets in the video stream. The frame difference method is to perform differential operations on continuous image frames. After denoising through morphological methods, it is determined whether there are moving targets in the target area. When a moving target appears, it is sent to the target classification module and classified by the trained feature target classifier. When the target suddenly stops during the movement and is in a static state, it will be detected by the end-to-end detection and classification module; A dynamic target detection module is set up, and a dynamic target detection method based on Gaussian background modeling is used to improve the detection effect of dynamic targets. Gaussian background modeling converts the motion target detection problem of sequence images into a binary classification problem based on the current background estimation, and divides all pixels into two categories: background and motion foreground. The classification results are then post-processed to obtain the final detection results.

9. A computer system comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the method for improving the recognition accuracy of small-size target video images under a large field of view as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for improving the recognition accuracy of a small-size target video image under a large field of view as described in any one of claims 1 to 8 is implemented.

Images