Method, device, and medium for detecting moving targets based on feature mask and YOLO model

Abstract

The present application discloses a method, device, and medium for detecting moving targets based on a feature mask and a YOLO model. The method comprises: performing image scaling on continuous frame images according to an image scaling algorithm to obtain a target continuous frame image; performing background extraction on the target continuous frame image according to a background subtraction algorithm to obtain a foreground image; obtaining a YOLO model including a feature mask generation module and a moving target detection module; inputting the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask; inputting the target foreground feature mask and the target continuous frame image into the moving target detection module for target detection to obtain a moving target detection result. The present application performs moving target detection on continuous frame images based on the foreground feature mask and the YOLO model, so that the background area of ​​the continuous frame image is masked out, and only the foreground area participates in the neural network operation, thereby improving the computing speed of the neural network running on the hardware terminal performing the moving target detection operation.

Description

Technical Field

[0001] The present application relates to, but is not limited to, the field of image processing technology, and in particular to a method, device, and medium for detecting a moving target based on a feature mask and a YOLO model. Background Art

[0002] Thanks to the significant improvement in the accuracy of convolutional neural networks compared to traditional computer vision algorithms, convolutional neural networks have been widely used in various fields of computer vision, such as intelligent security, autonomous driving, drones, medical care, and the military. However, the high accuracy of convolutional neural networks comes at the cost of significant computing and storage resources. General-purpose central processing units (CPUs) are no longer capable of handling convolution multiplication and addition calculations. A common practice is to use graphics processing units (GPUs) to accelerate the calculation of convolutional neural networks, but GPUs have the problem of high power consumption and high price. Therefore, it is necessary to propose a moving target detection method that can improve the computing speed of convolutional neural networks on hardware terminals while reducing the operating power consumption of the hardware terminals. Summary of the Invention

[0003] The embodiments of the present application provide a method, device, and medium for detecting moving targets based on feature masks and a YOLO model, which can improve the computing speed of a hardware terminal running a neural network while reducing the operating power consumption of the hardware terminal.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting moving objects based on a feature mask and a YOLO model, including:

[0005] Acquire continuous frame images, and perform image scaling on the continuous frame images according to a preset image scaling algorithm to obtain target continuous frame images;

[0006] Performing background extraction on the target continuous frame images according to a preset background subtraction algorithm to obtain a foreground image of the target continuous frame images;

[0007] Obtain a pre-trained YOLO model, wherein the YOLO model includes a feature mask generation module and a moving target detection module;

[0008] Inputting the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask;

[0009] The target foreground feature mask and the target continuous frame image are input into the moving target detection module for target detection processing to obtain a moving target detection result.

[0010] In some embodiments, the moving target detection module includes at least a first convolutional layer and a second convolutional layer connected in sequence, the feature mask generation module includes a first maximum pooling layer and a second maximum pooling layer connected in sequence, the target foreground feature mask includes at least a first foreground feature mask and a second foreground feature mask, the first foreground feature mask is the foreground feature mask of the first convolutional layer, and the second foreground feature mask is the foreground feature mask of the second convolutional layer, and the foreground image is input into the feature mask generation module for image processing to obtain the target foreground feature mask, including:

[0011] Determine a first convolution parameter and a second convolution parameter, where the first convolution parameter is a convolution parameter of the first convolution layer, and the second convolution parameter is a convolution parameter of the second convolution layer;

[0012] Based on the first convolution parameters, inputting the foreground image into the first maximum pooling layer for maximum pooling processing to obtain a first foreground feature mask;

[0013] Based on the second convolution parameters, the first foreground feature mask is input into the second maximum pooling layer for maximum pooling processing to obtain a second foreground feature mask.

[0014] In some embodiments, the feature mask generation module further includes a third maximum pooling layer arranged after the second maximum pooling layer. After inputting the first foreground feature mask into the maximum pooling layer for maximum pooling processing based on the second convolution parameter to obtain the second foreground feature mask, the method further includes:

[0015] Determining attribute information of a network layer that is sorted after the second convolutional layer;

[0016] When the attribute information indicates that the network layer is a convolutional layer, determining a third convolution parameter of the network layer, and inputting the second foreground feature mask into the third maximum pooling layer for maximum pooling processing based on the third convolution parameter to obtain a third foreground feature mask, where the third foreground feature mask is the foreground feature mask of the network layer;

[0017] or,

[0018] When the attribute information indicates that the network layer is an upsampling layer, the first foreground feature mask and the second foreground feature mask are determined as the target foreground feature masks.

[0019] In some embodiments, the moving target detection module further includes an upsampling layer, a first preset network layer, a second preset network layer, and a third preset network layer, wherein the first preset network layer is located between the first convolutional layer and the second convolutional layer, and the second preset network layer is arranged after the second convolutional layer. The target foreground feature mask and the target continuous frame image are input into the moving target detection module for target detection processing to obtain a moving target detection result, including:

[0020] Inputting the target continuous frame image and the first foreground feature mask into the first convolution layer for convolution processing to obtain a first feature map image;

[0021] Inputting the first feature map image into the first preset network layer for image processing to obtain a first intermediate feature image;

[0022] Inputting the first intermediate feature image and the second foreground feature mask into the second convolutional layer for convolution processing to obtain a second feature map image;

[0023] Inputting the second feature map image into the second preset network layer for image processing to obtain a second intermediate feature image;

[0024] When the network layer ordered after the second preset network layer is an upsampling layer, inputting the second intermediate feature map image into the upsampling layer for upsampling processing to obtain a third intermediate feature image;

[0025] The third intermediate feature image is input into the third preset network layer for image processing to obtain the moving target detection result.

[0026] In some embodiments, the first feature map image includes a plurality of pixels, the first foreground feature mask includes a plurality of mask values, each pixel corresponds to one mask value, and after inputting the target continuous frame image and the first foreground feature mask into the first convolution layer for convolution processing to obtain the first feature map image, the method further includes:

[0027] Determine a target pixel from the plurality of pixel points, wherein the mask value corresponding to the target pixel point is 1;

[0028] Calculate the output feature map values ​​of all channels corresponding to each target pixel;

[0029] The first feature map image is re-determined according to each of the output feature map values.

[0030] In some embodiments, the background subtraction algorithm is a Gaussian mixture model method.

[0031] In some embodiments, before scaling the continuous frame images according to a preset image scaling algorithm to obtain target continuous frame images, the method further includes:

[0032] The continuous frame images are preprocessed according to a preset image preprocessing rule to obtain preprocessed continuous frame images.

[0033] In a second aspect, an embodiment of the present application provides a moving target detection device based on a feature mask and a YOLO model, comprising:

[0034] An image acquisition module is used to acquire continuous frame images and scale the continuous frame images according to a preset image scaling algorithm to obtain target continuous frame images;

[0035] a foreground image extraction module, configured to perform background extraction on the target continuous frame images according to a preset background subtraction algorithm to obtain a foreground image of the target continuous frame images;

[0036] A model acquisition module is used to acquire a pre-trained YOLO model, wherein the YOLO model includes a feature mask generation module and a moving target detection module;

[0037] a target foreground feature mask determination module, the target foreground feature mask determination module being configured to input the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask;

[0038] The moving target detection result determination module is used to input the target foreground feature mask and the target continuous frame image into the moving target detection module for target detection processing to obtain a moving target detection result.

[0039] In a third aspect, an embodiment of the present application provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the computer program, the motion target detection method based on feature mask and YOLO model as described in the first aspect is implemented.

[0040] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the motion target detection method based on feature mask and YOLO model as described in the first aspect.

[0041] The embodiment of the present application provides a method, device, and medium for detecting a moving target based on a feature mask and a YOLO model. The method includes: obtaining continuous frame images, performing image scaling on the continuous frame images according to a preset image scaling algorithm to obtain a target continuous frame image; performing background extraction on the target continuous frame images according to a preset background subtraction algorithm to obtain a foreground image of the target continuous frame image; obtaining a pre-trained YOLO model, the YOLO model including a feature mask generation module and a moving target detection module; inputting the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask; inputting the target foreground feature mask and the target continuous frame images into the moving target detection module for target detection processing to obtain a moving target detection result. The embodiment of the present application performs moving target detection on continuous frame images based on the foreground feature mask and the YOLO model, so that only the foreground area of ​​the continuous frame images participates in the neural network operation, while the background area is eliminated by the mask and skips the neural network operation. This can improve the computing speed of the neural network running on the hardware terminal that performs the moving target detection operation while reducing the operating power consumption of the hardware terminal. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flowchart of a method for detecting moving objects based on a feature mask and a YOLO model, provided in one embodiment of the present application;

[0043] Figure 2 is a flowchart of steps for calculating a target foreground feature mask provided by another embodiment of the present application;

[0044] Figure 3 is a flowchart of steps for calculating a target foreground feature mask provided by another embodiment of the present application;

[0045] Figure 4 This is a flowchart of the steps for obtaining a moving target detection result provided by another embodiment of the present application;

[0046] Figure 5 is a flowchart of steps for obtaining a first feature map image provided by another embodiment of the present application;

[0047] Figure 6 is a flowchart of steps for performing image preprocessing on consecutive frame images provided by another embodiment of the present application;

[0048] Figure 7 1 is a module diagram of a moving target detection device based on a feature mask and a YOLO model provided by another embodiment of the present application;

[0049] Figure 8 is a hardware schematic diagram of an electronic device provided by another embodiment of the present application;

[0050] Figure 9 is a schematic diagram of target continuous frame images provided by another embodiment of the present application;

[0051] Figure 10 yes Figure 9 Schematic diagram of the corresponding background image;

[0052] Figure 11 yes Figure 9 -(c) Schematic diagram of the corresponding foreground image;

[0053] Figure 12 This is a network structure diagram of the YOLO model provided in another embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0055] It is understood that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and the like in the specification, claims, or accompanying drawings are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0056] The embodiment of the present application provides a method, device, and medium for detecting a moving target based on a feature mask and a YOLO model. The method includes: obtaining continuous frame images, performing image scaling on the continuous frame images according to a preset image scaling algorithm to obtain a target continuous frame image; performing background extraction on the target continuous frame images according to a preset background subtraction algorithm to obtain a foreground image of the target continuous frame image; obtaining a pre-trained YOLO model, the YOLO model including a feature mask generation module and a moving target detection module; inputting the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask; inputting the target foreground feature mask and the target continuous frame images into the moving target detection module for target detection processing to obtain a moving target detection result. The embodiment of the present application performs moving target detection on continuous frame images based on the foreground feature mask and the YOLO model, so that only the foreground area of ​​the continuous frame images participates in the neural network operation, while the background area is eliminated by the mask and skips the neural network operation. This can improve the computing speed of the neural network running on the hardware terminal that performs the moving target detection operation while reducing the operating power consumption of the hardware terminal.

[0057] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0058] like Figure 1 As shown, Figure 1 This is a flowchart of a method for detecting a moving target based on a feature mask and a YOLO model provided by an embodiment of the present application. The embodiment of the present application provides a method for detecting a moving target based on a feature mask and a YOLO model, which includes but is not limited to the following steps:

[0059] Step S110, acquiring continuous frame images, and performing image scaling on the continuous frame images according to a preset image scaling algorithm to obtain target continuous frame images;

[0060] Step S120, performing background extraction on the target continuous frame images according to a preset background subtraction algorithm to obtain a foreground image of the target continuous frame images;

[0061] Step S130, obtaining a pre-trained YOLO model, the YOLO model including a feature mask generation module and a moving target detection module;

[0062] Step S140: input the foreground image into a feature mask generation module for image processing to obtain a target foreground feature mask;

[0063] Step S150: Input the target foreground feature mask and the target continuous frame images into a moving target detection module for target detection processing to obtain a moving target detection result.

[0064] Understandably, thanks to their significantly improved accuracy compared to traditional computer vision algorithms, convolutional neural networks have been widely applied in various fields of computer vision, such as intelligent security, autonomous driving, drones, healthcare, and the military. However, the high accuracy of convolutional neural networks comes at the cost of significant computing and storage resources. CPUs are no longer capable of processing convolutional multiplication and addition calculations. GPUs are commonly used to accelerate convolutional neural network computations, but GPUs are power-hungry and expensive. Based on this, an embodiment of the present application provides a motion target detection method based on a feature mask and a YOLO model, including: obtaining continuous frame images, scaling the continuous frame images according to a preset image scaling algorithm, and obtaining target continuous frame images; performing background extraction on the target continuous frame images according to a preset background subtraction algorithm, and obtaining a foreground image of the target continuous frame images; obtaining a pre-trained YOLO model, the YOLO model includes a feature mask generation module and a motion target detection module; inputting the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask; inputting the target foreground feature mask and the target continuous frame images into the motion target detection module for target detection processing to obtain a motion target detection result. It should be noted that the embodiment of the present application does not limit the specific content of the motion target detection result, and may include the position information of the motion target and the category information of the motion target. The embodiment of the present application performs motion target detection on continuous frame images based on the foreground feature mask and the YOLO model, so that only the foreground area of ​​the continuous frame images participates in the neural network operation, while the background area is masked out and skips the neural network operation. While realizing motion target detection, it can improve the computing speed of the neural network running on the hardware terminal that performs the motion target detection operation, and reduce the operating power consumption of the hardware terminal.

[0065] It is understandable that since the image input resolution corresponding to the YOLO model is fixed, for example, the image input resolution of the YOLO model in the embodiment of the present application is 416*416, and the resolution of the initial continuous frame image depends on the image acquisition device, for example, the resolution of the initial continuous frame image acquired from the image acquisition device is 1280*720, it is necessary to scale the continuous frame image with a resolution of 1280*720 according to a preset image scaling algorithm to obtain the target continuous frame image that meets the image input resolution of the YOLO model. It should be noted that the embodiment of the present application does not limit the specific image scaling algorithm, which can be a bilinear interpolation algorithm or a nearest point interpolation algorithm, and those skilled in the art can select it according to actual conditions.

[0066] It should be noted that the embodiment of the present application is not limited to a specific background subtraction algorithm, and a commonly used background modeling technology may be used. For example, the embodiment of the present application uses a mixed Gaussian model method to perform background modeling on a target continuous frame image, where the target continuous frame image includes multiple frames of time-continuous images, such as Figure 9 As shown, Figure 9 Representing consecutive frames of two cars driving on the road, where only the cars are moving between frames, while the crosswalks, lane lines, and road surface are all stationary. Figure 9 -(a), 9-(b) and 9-(c) are three frames of target continuous frame images sorted by time. The mixed Gaussian model method is used to analyze multiple frames, such as Figure 9 After iterative updates of 9-(a), 9-(b) and 9-(c), a more stable background image will be obtained. Figure 10 As shown, the embodiment of the present application will also update the background image according to the preset weight; based on this, for each new frame of the target continuous frame image, by comparing it with the background image, the foreground image of the frame image can be obtained as shown in FIG. Figure 11 As shown, Figure 11 The white area in the middle is the pixel position occupied by the vehicle. The value corresponding to the pixel position in the white area is 1, indicating a moving area, and the value of the pixel position corresponding to the remaining black area is 0, indicating a stationary area.

[0067] In addition, in one embodiment, the moving target detection module includes at least a first convolutional layer and a second convolutional layer connected in sequence, the feature mask generation module includes at least a first maximum pooling layer and a second maximum pooling layer connected in sequence, and the target foreground feature mask includes at least a first foreground feature mask and a second foreground feature mask, the first foreground feature mask is the foreground feature mask of the first convolutional layer, and the second foreground feature mask is the foreground feature mask of the second convolutional layer. Figure 2 , Figure 1 Step S140 in the illustrated embodiment further includes but is not limited to the following steps:

[0068] Step S210, determining a first convolution parameter and a second convolution parameter, where the first convolution parameter is a convolution parameter of the first convolution layer, and the second convolution parameter is a convolution parameter of the second convolution layer;

[0069] Step S220: input the foreground image into the first maximum pooling layer for maximum pooling processing based on the first convolution parameter to obtain a first foreground feature mask;

[0070] Step S230: Based on the second convolution parameter, the first foreground feature mask is input into the second maximum pooling layer to perform maximum pooling processing to obtain a second foreground feature mask.

[0071] In addition, in one embodiment, the feature mask generation module further includes a third maximum pooling layer arranged after the second maximum pooling layer, referring to Figure 3 , in execution Figure 2 After step S230 in the illustrated embodiment, the method for detecting moving objects based on feature masks and the YOLO model provided in the embodiment of the present application includes but is not limited to the following steps:

[0072] Step S310, determining attribute information of the network layer arranged after the second convolutional layer;

[0073] Step S320: When the attribute information representation network layer is a convolutional layer, a third convolution parameter of the network layer is determined. Based on the third convolution parameter, the second foreground feature mask is input into a third maximum pooling layer for maximum pooling to obtain a third foreground feature mask. The third foreground feature mask serves as the foreground feature mask of the network layer.

[0074] or,

[0075] Step S330: When the attribute information representation network layer is an upsampling layer, the first foreground feature mask and the second foreground feature mask are determined as target foreground feature masks.

[0076] It can be understood that when running the moving target detection module of the YOLO model, each time the convolution operation of the convolution layer is completed, it is necessary to determine the attribute information of the next network layer sorted after the current convolution layer. When the attribute information represents that the network layer is a convolution layer, the convolution operation is repeated based on the scheme of the above embodiment to obtain the foreground feature mask. For example, when the network layer sorted after the second convolution layer is a convolution layer, the third convolution parameter of the network layer is determined. Based on the third convolution parameter, the second foreground feature mask is input to the third maximum pooling layer for maximum pooling processing to obtain the third foreground feature mask; when the attribute information represents that the network layer is an upsampling layer, the maximum pooling layer will not be able to continue to calculate the receptive field of the subsequent convolution layer, so the generation of the foreground feature mask is terminated. The current target foreground feature mask only includes the first foreground feature mask and the second foreground feature mask, that is, when the network layer sorted after the second convolution layer is an upsampling layer, the step of traversing and calculating the foreground feature mask is terminated, and all network layers sorted after the upsampling layer have no foreground feature mask involved in the operation.

[0077] In addition, in one embodiment, reference Figure 12 The moving target detection module 1210 further includes an upsampling layer, a first preset network layer, a second preset network layer, and a third preset network layer. The first preset network layer is located between the first convolution layer and the second convolution layer in the feature mask generation module 1220, and the second preset network layer is arranged after the second convolution layer. Figure 4 , Figure 1Step S150 includes but is not limited to the following steps:

[0078] Step S410: inputting the target continuous frame image and the first foreground feature mask into the first convolution layer for convolution processing to obtain a first feature map image;

[0079] Step S420: inputting the first feature map image into a first preset network layer for image processing to obtain a first intermediate feature image;

[0080] Step S430: Input the first intermediate feature image and the second foreground feature mask into the second convolution layer for convolution processing to obtain a second feature map image;

[0081] Step S440: inputting the second feature map image into a second preset network layer for image processing to obtain a second intermediate feature image;

[0082] Step S450: When the network layer ordered after the second preset network layer is an upsampling layer, the second intermediate feature map image is input to the upsampling layer for upsampling processing to obtain a third intermediate feature image;

[0083] Step S460: Input the third intermediate feature image into the third preset network layer for image processing to obtain a moving target detection result.

[0084] It can be understood that after calculating the target foreground feature mask (the target foreground feature mask includes the first foreground feature mask and the second foreground feature mask), the target continuous frame image and the first foreground feature mask are input into the first convolution layer for convolution processing to obtain a first feature mapping image, the first feature mapping image is input into the first preset network layer for image processing to obtain a first intermediate feature image, the first intermediate feature image and the second foreground feature mask are input into the second convolution layer for convolution processing to obtain a second feature mapping image, when the network layer arranged after the second preset network layer is an upsampling layer, the second intermediate feature mapping image is input into the upsampling layer for upsampling processing to obtain a third intermediate feature image, the third intermediate feature image is input into the third preset network layer for image processing to obtain a moving target detection result, so that only the foreground area of ​​the continuous frame image participates in the neural network operation, and the background area is eliminated by the mask and skips the neural network operation, which can effectively improve the efficiency of the YOLO model in calculating the moving target detection result.

[0085] It should be noted that the embodiments of the present application do not limit the specific content of the motion target detection results, which may include continuous frame images of the target carrying the prediction box and the motion target position identifier.

[0086] In addition, in one embodiment, the first feature map image includes a plurality of pixels, the first foreground feature mask includes a plurality of mask values, and each pixel corresponds to a mask value. Figure 5 , Figure 4 After step S410, the moving target detection method based on feature mask and YOLO model provided in the embodiment of the present application further includes but is not limited to the following steps:

[0087] Step S510, determining a target pixel from a plurality of pixels, where the mask value corresponding to the target pixel is 1;

[0088] Step S520, calculating the output feature map values ​​of all channels corresponding to each target pixel;

[0089] Step S530: re-determine the first feature map image according to each output feature map value.

[0090] It should be noted that the specific method of determining the target pixel point from multiple pixel points can be to traverse all the pixel points of the first feature mapping image based on the spatial dimension, and filter them according to the mask values ​​corresponding to each pixel point. In the embodiment of the present application, the mask value corresponding to the pixel points contained in the first feature mapping image can be 0 or 1, and the mask value corresponding to the target pixel point is 1.

[0091] It can be understood that based on the spatial dimension, all the pixels of the first feature map image are traversed. The spatial dimension represents the pixel position information of the first feature map image, which can be expressed as (i, j). Each pixel corresponds to a mask value, which can be expressed as msk(i, j). The value range of i is (0, width-1) and the value range of j is (0, height-1). The values ​​of width and height are determined according to the image size of the first feature map image. When traversing to a certain pixel position (i, j), if the mask value msk(i, j) corresponding to the pixel position is 1, it means that the receptive field corresponding to the pixel position contains the original input image, that is, the motion pixel of the foreground image. The pixel corresponding to the mask value of msk(i, j)=1 is determined as the target pixel, and the calculation is performed. The output feature map values ​​out(i, j, k) of all channels corresponding to the target pixel point, where the output channel number is channel, and the range of k is 0 to (channel-1), and the first feature map image is re-determined according to each output feature map value; when traversing to a certain pixel position (i, j), if the mask value msk(i, j) corresponding to the pixel point corresponding to the pixel position is 0 = 0, it means that the receptive field corresponding to the pixel position only contains the static pixels of the foreground image, and the convolution operation of the pixel point is skipped, and the traversal continues to the next pixel point; if the pixel position (width-1, height-1) of the first feature map is traversed based on the spatial dimension, it means that all pixel positions in the first feature map output by the first convolution layer have been traversed; otherwise, the traversal continues to the next pixel position. By calculating the foreground feature mask of the convolutional layer as a basis for determining whether each pixel in the target feature map image output by the convolutional layer can skip the convolution calculation step, the convolution layer can skip the calculation of the output results of the original input image, that is, the static area in the foreground image, in each convolutional layer. The skipped calculation can save memory data read and write bandwidth resources, network operation time, and power consumption caused by both when the hardware terminal runs the YOLO model. When the number of channels in the intermediate network layer of the YOLO model is large, the method of performing convolution processing based on the foreground feature mask in the above embodiment can mask the data of all channels, which can save more than 50% of the read and write data bandwidth in some scenarios. In addition, by inputting the foreground feature mask, the multiplication and accumulation calculation of the bypassed convolution kernel is omitted, thereby speeding up the calculation speed of the hardware terminal running the YOLO model. These can achieve lower power consumption and cost for hardware terminals for real-time moving target detection.

[0092] It should be noted that the embodiment of the present application does not limit the specific content of the first convolution parameter and the second convolution parameter. The convolution parameters of the convolution layer of the feature mask generation module may include the convolution kernel size and the convolution step size. In addition, the specific feature information of the feature map image output by each convolution layer in the moving target detection module in this embodiment depends on the local area of ​​the original image input to the YOLO model. The size of the local area is called the receptive field, and the receptive field is obtained according to the following formula:

[0093]

[0094] Among them, r n+1 is the receptive field of the n+1th convolutional layer of the moving target detection module, r n is the receptive field of the nth convolutional layer of the moving target detection module, k n+1 is the convolution kernel size, k n+1 The value of is an integer greater than or equal to 1; s i is the convolution step size, s i The value of is an integer greater than or equal to 1, n is an integer greater than or equal to 0, r0 = 1, s0 = 1. Those skilled in the art can set the number of convolutional layers of the moving target detection module according to actual conditions, and no limitation is imposed here.

[0095] It's important to note that the computational complexity of a max pooling layer is much smaller than that of a convolutional layer: The computational complexity of calculating the output feature map image for a convolutional layer is (kernel_w * kernel_h * channel_in) multiplications and (kernel_w * kernel_h * channel_in – 1) additions, where kernel_w is the horizontal kernel size, kernel_h is the vertical kernel size, and channel_in is the number of input channels. In the YOLO model, channel_in ranges from 32 to 1024. The computational complexity of a max pooling layer is only (kernel_w * kernel_h) comparisons. Therefore, using the foreground feature mask calculated by an additional max pooling layer to reduce convolutional layer computations for static regions can effectively reduce the YOLO model's computation time, thereby saving hardware terminal memory resources and power consumption.

[0096] In addition, refer to Figure 6 In one embodiment, when executing Figure 1 Before step S110 in the illustrated embodiment, the method for detecting moving objects based on feature masks and the YOLO model provided in the embodiment of the present application further includes but is not limited to the following steps:

[0097] Step S610 , performing image preprocessing on the continuous frame images according to a preset image preprocessing rule to obtain preprocessed continuous frame images.

[0098] It can be understood that the main purpose of image preprocessing for continuous frame images is to eliminate irrelevant information in the continuous frame images, restore useful real information, enhance the detectability of relevant information and simplify the data to the maximum extent, thereby improving the reliability of subsequent continuous frame image applications.

[0099] It should be noted that the embodiments of the present application do not limit the specific method of image preprocessing for continuous frame images. The continuous frame images can be subjected to image random number rotation processing, normalization processing or image binarization processing, etc., and technical personnel in this field can select the specific method of image preprocessing according to actual conditions.

[0100] In addition, refer to Figure 7 , Figure 7 : is a module diagram of a moving target detection device based on a feature mask and a YOLO model provided by another embodiment of the present application. In one embodiment, the present application embodiment provides a moving target detection device 700 based on a feature mask and a YOLO model. The moving target detection device 700 based on a feature mask and a YOLO model includes:

[0101] The image acquisition module 710 is used to acquire continuous frame images and scale the continuous frame images according to a preset image scaling algorithm to obtain target continuous frame images;

[0102] The foreground image extraction module 720 is used to perform background extraction on the target continuous frame image according to a preset background subtraction algorithm to obtain a foreground image of the target continuous frame image;

[0103] Model acquisition module 730, the model acquisition module 730 is used to obtain a pre-trained YOLO model, the YOLO model includes a feature mask generation module and a moving target detection module;

[0104] The target foreground feature mask determination module 740 is used to input the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask;

[0105] The moving target detection result determination module 750 is used to input the target foreground feature mask and the target continuous frame image into the moving target detection module for target detection processing to obtain the moving target detection result.

[0106] The specific implementation of the motion target detection device based on the feature mask and the YOLO model is basically the same as the specific embodiment of the motion target detection method based on the feature mask and the YOLO model described above, and will not be repeated here.

[0107] In addition, reference Figure 8 , Figure 8 This is a hardware structure diagram of an electronic device provided by another embodiment of the present application. One embodiment of the present application also provides an electronic device 800, which includes: a memory 810, a processor 820, and a computer program stored in the memory 810 and executable on the processor 820.

[0108] The processor 820 and the memory 810 may be connected via a bus or other means.

[0109] The non-transient software program and instructions required to implement the method for detecting moving targets based on feature masks and YOLO models in the above embodiment are stored in the memory 810. When executed by the processor 820, the method for detecting moving targets based on feature masks and YOLO models in the above embodiment is executed, for example, the method described above is executed. Figure 1 Steps S110 to S150 of the method, Figure 2 Steps S210 to S230 of the method, Figure 3 Steps S310 to S330 of the method, Figure 4 Steps S410 to S460 of the method, Figure 5 Steps S510 to S530 of the method and Figure 6 Method step S610 in .

[0110] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0111] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are executed by a processor or controller, for example, by a processor in the embodiment of the electronic device 800, so that the processor can execute the method for detecting moving targets based on feature masks and YOLO models in the above embodiment, for example, executing the above-described Figure 1 Steps S110 to S150 of the method, Figure 2 Steps S210 to S230 of the method, Figure 3 Steps S310 to S330 of the method, Figure 4 Steps S410 to S460 of the method, Figure 5 Steps S510 to S530 of the method and Figure 6Method step S610 in. It will be appreciated by those skilled in the art that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

Claims

1. A moving target detection method based on feature mask and YOLO model, characterized in that: include: Acquire continuous frame images, and perform image scaling on the continuous frame images according to a preset image scaling algorithm to obtain target continuous frame images; Performing background extraction on the target continuous frame images according to a preset background subtraction algorithm to obtain a foreground image of the target continuous frame images; Obtain a pre-trained YOLO model, wherein the YOLO model includes a feature mask generation module and a moving target detection module; Inputting the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask; Inputting the target foreground feature mask and the target continuous frame images into the moving target detection module for target detection processing to obtain a moving target detection result; The moving target detection module includes at least a first convolutional layer and a second convolutional layer connected in sequence, the feature mask generation module includes a first maximum pooling layer and a second maximum pooling layer connected in sequence, the target foreground feature mask includes at least a first foreground feature mask and a second foreground feature mask, the first foreground feature mask is the foreground feature mask of the first convolutional layer, and the second foreground feature mask is the foreground feature mask of the second convolutional layer, the foreground image is input into the feature mask generation module for image processing to obtain the target foreground feature mask, including: Determine a first convolution parameter and a second convolution parameter, where the first convolution parameter is a convolution parameter of the first convolution layer, and the second convolution parameter is a convolution parameter of the second convolution layer; Based on the first convolution parameters, inputting the foreground image into the first maximum pooling layer for maximum pooling processing to obtain a first foreground feature mask; Based on the second convolution parameters, inputting the first foreground feature mask into the second maximum pooling layer for maximum pooling processing to obtain a second foreground feature mask; The feature mask generation module further includes a third maximum pooling layer arranged after the second maximum pooling layer. After inputting the first foreground feature mask into the maximum pooling layer for maximum pooling processing based on the second convolution parameter to obtain a second foreground feature mask, the method further includes: Determining attribute information of a network layer that is sorted after the second convolutional layer; When the attribute information indicates that the network layer is a convolutional layer, determining a third convolution parameter of the network layer, and inputting the second foreground feature mask into the third maximum pooling layer for maximum pooling processing based on the third convolution parameter to obtain a third foreground feature mask, where the third foreground feature mask is the foreground feature mask of the network layer; or, When the attribute information indicates that the network layer is an upsampling layer, the first foreground feature mask and the second foreground feature mask are determined as the target foreground feature masks.

2. The moving target detection method based on feature mask and YOLO model according to claim 1, wherein: The moving target detection module further includes an upsampling layer, a first preset network layer, a second preset network layer, and a third preset network layer, wherein the first preset network layer is located between the first convolutional layer and the second convolutional layer, and the second preset network layer is arranged after the second convolutional layer. The target foreground feature mask and the target continuous frame image are input into the moving target detection module for target detection processing to obtain a moving target detection result, including: Inputting the target continuous frame image and the first foreground feature mask into the first convolution layer for convolution processing to obtain a first feature map image; Inputting the first feature map image into the first preset network layer for image processing to obtain a first intermediate feature image; Inputting the first intermediate feature image and the second foreground feature mask into the second convolutional layer for convolution processing to obtain a second feature map image; Inputting the second feature map image into the second preset network layer for image processing to obtain a second intermediate feature image; When the network layer ordered after the second preset network layer is an upsampling layer, the second intermediate feature image is input into the upsampling layer for upsampling to obtain a third intermediate feature image; The third intermediate feature image is input into the third preset network layer for image processing to obtain the moving target detection result.

3. The moving target detection method based on feature mask and YOLO model according to claim 2, characterized in that: The first feature map image includes a plurality of pixels, the first foreground feature mask includes a plurality of mask values, each pixel corresponds to one mask value, and after inputting the target continuous frame image and the first foreground feature mask into the first convolution layer for convolution processing to obtain the first feature map image, the method further includes: Determine a target pixel from the plurality of pixel points, wherein the mask value corresponding to the target pixel point is 1; Calculate the output feature map values ​​of all channels corresponding to each target pixel; The first feature map image is re-determined according to each of the output feature map values.

4. The moving target detection method based on feature mask and YOLO model according to claim 1, wherein: The background subtraction algorithm is a Gaussian mixture model method.

5. The moving target detection method based on feature mask and YOLO model according to claim 1, wherein: Before scaling the continuous frame images according to a preset image scaling algorithm to obtain target continuous frame images, the method further includes: The continuous frame images are preprocessed according to a preset image preprocessing rule to obtain preprocessed continuous frame images.

6. A moving target detection device based on feature mask and YOLO model, characterized in that: include: An image acquisition module is used to acquire continuous frame images and scale the continuous frame images according to a preset image scaling algorithm to obtain target continuous frame images; a foreground image extraction module, configured to perform background extraction on the target continuous frame images according to a preset background subtraction algorithm to obtain a foreground image of the target continuous frame images; A model acquisition module is used to acquire a pre-trained YOLO model, wherein the YOLO model includes a feature mask generation module and a moving target detection module; a target foreground feature mask determination module, the target foreground feature mask determination module being configured to input the foreground image into the feature mask generation module for image processing to obtain a target foreground feature mask; a moving target detection result determination module, the moving target detection result determination module being used to input the target foreground feature mask and the target continuous frame image into the moving target detection module for target detection processing to obtain a moving target detection result; The moving target detection module includes at least a first convolutional layer and a second convolutional layer connected in sequence, the feature mask generation module includes a first maximum pooling layer and a second maximum pooling layer connected in sequence, the target foreground feature mask includes at least a first foreground feature mask and a second foreground feature mask, the first foreground feature mask is the foreground feature mask of the first convolutional layer, and the second foreground feature mask is the foreground feature mask of the second convolutional layer, and the foreground image is input into the feature mask generation module for image processing to obtain the target foreground feature mask, including: Determine a first convolution parameter and a second convolution parameter, where the first convolution parameter is a convolution parameter of the first convolution layer, and the second convolution parameter is a convolution parameter of the second convolution layer; Based on the first convolution parameters, inputting the foreground image into the first maximum pooling layer for maximum pooling processing to obtain a first foreground feature mask; Based on the second convolution parameters, inputting the first foreground feature mask into the second maximum pooling layer for maximum pooling processing to obtain a second foreground feature mask; The feature mask generation module further includes a third maximum pooling layer arranged after the second maximum pooling layer, and after inputting the first foreground feature mask into the maximum pooling layer for maximum pooling processing based on the second convolution parameter to obtain a second foreground feature mask, further includes: Determining attribute information of a network layer that is sorted after the second convolutional layer; When the attribute information indicates that the network layer is a convolutional layer, determining a third convolution parameter of the network layer, and inputting the second foreground feature mask into the third maximum pooling layer for maximum pooling processing based on the third convolution parameter to obtain a third foreground feature mask, where the third foreground feature mask is the foreground feature mask of the network layer; or, When the attribute information indicates that the network layer is an upsampling layer, the first foreground feature mask and the second foreground feature mask are determined as the target foreground feature masks.

7. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting a moving target based on a feature mask and a YOLO model as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the moving target detection method based on feature mask and YOLO model according to any one of claims 1 to 5.

Images