Moving target segmentation method and device and terminal equipment

Abstract

The invention provides a moving target segmentation method and device, and the method comprises the steps: carrying out the layer-by-layer coding of a monitoring image after the monitoring image containing a moving target object in a monitoring video is obtained in real time, obtaining a first feature map, determining a weight map corresponding to at least one decoding layer based on a preset rule, and carrying out the segmentation of the moving target object based on the weight map. And decoding the first feature map layer by layer to obtain a segmented image of the target object in the monitoring image. Therefore, the first feature map is decoded layer by layer based on the weight map, the feature expression of the moving target object is highlighted, and the corresponding features of the background elements are suppressed, so that the accuracy and generalization of moving target segmentation are improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a moving target segmentation method, device and terminal equipment. Background Art

[0002] With the continuous development of computer vision technology, moving target segmentation plays an extremely important role in security and other fields. However, due to the complexity of monitoring scenes (for example, in some offline store scenes, there are many people and goods, and the background is messy), there is a phenomenon of inaccurate segmentation of moving targets (such as people). Summary of the invention

[0003] This application proposes a moving target segmentation method and device to improve the accuracy of moving target segmentation. The technical solution of this application is as follows:

[0004] According to a first aspect of an embodiment of the present application, an embodiment of the present application provides a moving target segmentation method, including:

[0005] Acquire surveillance images containing moving target objects in surveillance videos in real time;

[0006] Encode the monitoring image layer by layer to obtain a first feature map;

[0007] Based on a preset rule, determining a weight map corresponding to at least one decoding layer;

[0008] Based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the monitoring image.

[0009] In the present application, after obtaining a surveillance image containing a moving target object in a surveillance video in real time, the surveillance image can be encoded layer by layer to obtain a first feature map, and then, based on a preset rule, a weight map corresponding to at least one decoding layer is determined, and then, based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the surveillance image. Thus, the first feature map is decoded layer by layer based on the weight map, highlighting the feature expression of the moving target object and suppressing the corresponding features of the background elements, thereby improving the accuracy and generalization of the moving target segmentation.

[0010] In a possible implementation manner of the embodiment of the first aspect of the present application, determining a weight map corresponding to at least one decoding layer based on a preset rule includes at least one of the following steps:

[0011] Determine a weight map corresponding to the first decoding layer according to a ratio between a first size of a second feature map output by a first encoding layer corresponding to the first decoding layer and a second size of the monitoring image, and a first pixel coordinate corresponding to a spatial position of the moving target object in the monitoring image;

[0012] Perform spatial attention calculation and channel attention calculation on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the weight map corresponding to the second decoding layer.

[0013] In a possible implementation manner of the embodiment of the first aspect of the present application, determining a weight map corresponding to the first decoding layer according to a ratio between a first size of a second feature map output by a first encoding layer corresponding to the first decoding layer and a second size of a monitoring image, and a first pixel coordinate corresponding to a spatial position of a moving target object in the monitoring image, includes:

[0014] Determine a second pixel coordinate corresponding to the first pixel coordinate in a second feature map output by a first encoding layer corresponding to the first decoding layer;

[0015] Determining a maximum displacement of a first pixel coordinate within a preset time period;

[0016] The maximum displacement is scaled based on the ratio to determine a target displacement;

[0017] Taking the second pixel coordinate as the center and based on the target displacement, determining a reference range of the moving target object in the second feature map;

[0018] Determine a weight value on each third pixel coordinate in the weight map according to a distance between each third pixel coordinate and the second pixel coordinate, wherein the third pixel coordinate is a coordinate of a pixel in the second feature map that is within a reference range;

[0019] Determine that the weight value at each fourth pixel coordinate in the weight map is zero, wherein the fourth pixel coordinate is the coordinate of a pixel in the second feature map that is outside the reference range.

[0020] In a possible implementation manner of the embodiment of the first aspect of the present application, spatial attention calculation and channel attention calculation are performed on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the weight map corresponding to the second decoding layer, including:

[0021] Performing maximum pooling and average pooling of the channel dimension on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the first reference weight map and the second reference weight map;

[0022] activating the sum of the first reference weight map and the second reference weight map to determine a third reference weight map;

[0023] Performing average pooling of the width dimension and the height dimension on the third feature map to determine a fourth reference weight map and a fifth reference weight map;

[0024] Transpose and sum the fourth reference weight map and the fifth reference weight map and activate them to determine a sixth reference weight map;

[0025] The sum of the third reference weight map and the sixth reference weight map is determined as the weight map.

[0026] In a possible implementation manner of the embodiment of the first aspect of the present application, the monitoring image includes an infrared grayscale image and a visible light image in the same field of view, and the monitoring image is encoded layer by layer to obtain a first feature map, including:

[0027] The channels corresponding to the infrared grayscale image and the channels corresponding to the visible light image are randomly mixed to perform fusion encoding on the infrared grayscale image and the visible light image to obtain a first feature map.

[0028] In a possible implementation manner of the first aspect of the present application, decoding the first feature map based on the weight map to obtain a segmented image of the moving target object in the surveillance image includes:

[0029] Based on the weight map associated with the i-th decoding layer, the feature map output by the i-th decoding layer is corrected to obtain a corrected feature map, where i is a natural number greater than or equal to 1 and less than or equal to N, and N is the total number of decoding layers included in the decoding module;

[0030] The corrected feature map is decoded using the i+1th decoding layer until a segmented image of the moving target object in the surveillance image is obtained.

[0031] In a possible implementation manner of the embodiment of the first aspect of the present application, based on the weight map associated with the i-th decoding layer, the feature map output by the i-th decoding layer is corrected to obtain the corrected feature map, including:

[0032] The sum of the feature map output by the encoding layer corresponding to the i-th decoding layer and the feature map output by the i-th decoding layer is determined as the reference feature of the i-th decoding layer;

[0033] Based on the weight map associated with the i-th decoding layer, the reference feature is corrected to obtain a corrected feature map.

[0034] According to a second aspect of the embodiment of the present application, the embodiment of the present application provides a moving target segmentation device, including:

[0035] An acquisition module is used to acquire in real time a surveillance image containing a moving target object in a surveillance video;

[0036] The encoding module is used to encode the monitoring image layer by layer to obtain a first feature map;

[0037] A determination module, used to determine a weight map corresponding to at least one decoding layer based on a preset rule;

[0038] The decoding module is used to decode the first feature map layer by layer based on the weight map to obtain a segmented image of the moving target object in the monitoring image.

[0039] In a possible implementation manner of the second aspect of the present application, the above-mentioned determination module is used to:

[0040] Determine a weight map corresponding to the first decoding layer according to a ratio between a first size of a second feature map output by a first encoding layer corresponding to the first decoding layer and a second size of the monitoring image, and a first pixel coordinate corresponding to a spatial position of the moving target object in the monitoring image;

[0041] Perform spatial attention calculation and channel attention calculation on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the weight map corresponding to the second decoding layer.

[0042] In a possible implementation manner of the second aspect of the present application, the above-mentioned determination module is used to:

[0043] Determine a second pixel coordinate corresponding to the first pixel coordinate in a second feature map output by a first encoding layer corresponding to the first decoding layer;

[0044] Determining a maximum displacement of a first pixel coordinate within a preset time period;

[0045] The maximum displacement is scaled based on the ratio to determine a target displacement;

[0046] Taking the second pixel coordinate as the center and based on the target displacement, determining a reference range of the moving target object in the second feature map;

[0047] Determine a weight value on each third pixel coordinate in the weight map according to a distance between each third pixel coordinate and the second pixel coordinate, wherein the third pixel coordinate is a coordinate of a pixel in the second feature map that is within a reference range;

[0048] Determine that the weight value at each fourth pixel coordinate in the weight map is zero, wherein the fourth pixel coordinate is the coordinate of a pixel in the second feature map that is outside the reference range.

[0049] In a possible implementation manner of the second aspect of the present application, the above-mentioned determination module is used to:

[0050] Performing maximum pooling and average pooling of the channel dimension on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the first reference weight map and the second reference weight map;

[0051] activating the sum of the first reference weight map and the second reference weight map to determine a third reference weight map;

[0052] Performing average pooling of the width dimension and the height dimension on the third feature map to determine a fourth reference weight map and a fifth reference weight map;

[0053] Transpose and sum the fourth reference weight map and the fifth reference weight map and activate them to determine a sixth reference weight map;

[0054] The sum of the third reference weight map and the sixth reference weight map is determined as the weight map.

[0055] In a possible implementation manner of the second aspect of the present application, the monitoring image includes an infrared grayscale image and a visible light image in the same field of view, and the encoding module is used to:

[0056] The channels corresponding to the infrared grayscale image and the channels corresponding to the visible light image are randomly mixed to perform fusion encoding on the infrared grayscale image and the visible light image to obtain a first feature map.

[0057] In a possible implementation manner of the second aspect of the present application, the decoding module is used to:

[0058] Based on the weight map associated with the i-th decoding layer, the feature map output by the i-th decoding layer is corrected to obtain a corrected feature map, where i is a natural number greater than or equal to 1 and less than or equal to N, and N is the total number of decoding layers included in the decoding module;

[0059] The corrected feature map is decoded using the i+1th decoding layer until a segmented image of the moving target object in the surveillance image is obtained.

[0060] In a possible implementation manner of the second aspect of the present application, the decoding module is used to:

[0061] The sum of the feature map output by the encoding layer corresponding to the i-th decoding layer and the feature map output by the i-th decoding layer is determined as the reference feature of the i-th decoding layer;

[0062] Based on the weight map associated with the i-th decoding layer, the reference feature is corrected to obtain a corrected feature map.

[0063] According to a third aspect of an embodiment of the present application, a terminal device is provided, including:

[0064] processor;

[0065] a memory for storing processor-executable instructions;

[0066] The processor is configured to execute instructions to implement the moving target segmentation method as described in the first aspect of the embodiment.

[0067] According to the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by the processor of the terminal device, the terminal device can execute the motion target segmentation method as described in the above-mentioned first embodiment.

[0068] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which implements the motion target segmentation method of the above-mentioned embodiment when the computer program is executed by a processor.

[0069] The technical solution provided by the embodiment of the present application brings at least the following beneficial effects: after obtaining a surveillance image containing a moving target object in a surveillance video in real time, the surveillance image can be encoded layer by layer to obtain a first feature map, and then, based on a preset rule, a weight map corresponding to at least one decoding layer is determined, and then, based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the surveillance image. Thus, the first feature map is decoded layer by layer based on the weight map, highlighting the feature expression of the moving target object and suppressing the corresponding features of the background elements, thereby improving the accuracy and generalization of the moving target segmentation.

[0070] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application, and do not constitute an improper limitation on the present application.

[0072] Figure 1 A schematic diagram of a flow chart of a moving target segmentation method provided in the first embodiment of the present application;

[0073] Figure 2 A schematic diagram of a network structure provided for this application;

[0074] Figure 3 A schematic flow chart of another moving target segmentation method provided in the second embodiment of the present application;

[0075] Figure 4 A schematic flow chart of another moving target segmentation method provided in the third embodiment of the present application;

[0076] Figure 5 A schematic flow chart of another moving object segmentation method provided in the fourth embodiment of the present application;

[0077] Figure 6 A schematic diagram of a network structure provided for this application;

[0078] Figure 7 A schematic diagram of the structure of a moving target segmentation device provided in the fifth embodiment of the present application;

[0079] Figure 8 The present invention is a block diagram of a terminal device for segmenting a moving target according to an exemplary embodiment. DETAILED DESCRIPTION

[0080] In order to enable ordinary persons in the art to better understand the technical solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0081] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0082] In the process of moving target segmentation, using the network to train the background vector space requires additional long-term sequence video frames as the network training input. At the same time, in the subsequent network segmentation process, the background template often relies on the correction of the previous segmentation results, which is not ideal in long-term detection and segmentation tasks.

[0083] In addition, the accuracy of moving target segmentation may be affected by many factors. For example, first, affected by the changes in light in the monitoring environment (such as cloudy days, rain and fog, night, etc.), the monitoring image is not clear, which seriously affects the actual use scope and effect of the moving target segmentation algorithm. Second, in the moving target segmentation, when the moving target object moves to some background environments that are highly similar to its appearance, the moving target object is mistaken for the background, resulting in unsatisfactory segmentation effect. Third, due to the change in the moving target object's posture (turning, squatting, etc.), intermittent movement, and being blocked by other objects, the appearance of the moving target object changes, resulting in a large segmentation error.

[0084] The moving target segmentation method and device of the embodiment of the present application are described below with reference to the accompanying drawings. The moving target segmentation method provided by the present application can be executed by the moving target segmentation device (hereinafter referred to as the segmentation device) provided by the present application.

[0085] Figure 1 A flow chart of a moving target segmentation method provided in an embodiment of the present application includes the following steps:

[0086] Step 101: Acquire a surveillance image containing a moving target object in a surveillance video in real time.

[0087] In the present application, the moving target object can be monitored in real time by a monitoring device to generate a monitoring video, and then a monitoring image containing the moving target object can be extracted from the monitoring video.

[0088] The moving target object may be any person or thing specified by the user, and this application does not impose any limitation on this.

[0089] Optionally, the monitoring image may include an infrared grayscale image and / or a visible light image. When the monitoring image includes an infrared grayscale image and a visible light image (also referred to as multi-source data), the infrared grayscale image and the visible light image are images in the same field of view.

[0090] In the present application, the moving target object can be monitored in real time by an infrared monitoring device to generate an infrared monitoring video. Afterwards, the infrared monitoring video and the monitoring video to which the visible light image belongs can be synchronized in time to obtain an external grayscale image and a visible light image containing the moving target object in real time. When the sizes of the infrared grayscale image and the visible light image are different, the infrared grayscale image and the visible light image can be scaled to the same size. Thus, the moving target object is segmented based on the infrared grayscale image and the visible light image of the same size, which is conducive to improving the accuracy of moving target segmentation.

[0091] Step 102: Encode the monitoring image layer by layer to obtain a first feature map.

[0092] In the present application, multiple coding layers can be used to encode the monitoring image layer by layer to obtain the first feature map output by the last coding layer, so as to accurately extract the features of the moving target object in the monitoring image, thereby improving the accuracy of moving target segmentation.

[0093] In addition, when each encoding layer encodes the surveillance image, the size of the output feature map is different.

[0094] Optionally, when the monitoring image includes an infrared grayscale image and a visible light image, the channels corresponding to the infrared grayscale image and the channels corresponding to the visible light image may be randomly mixed, and then fusion encoding is performed to obtain the first feature map.

[0095] It is understandable that the infrared grayscale image contains clear posture information of the moving target object. The channels corresponding to the infrared grayscale image and the channels corresponding to the visible light image are randomly mixed, and then fusion encoding is performed to obtain the first feature map, so that the first feature map contains rich visual information and posture information, so that the accuracy of moving target segmentation is not affected by factors such as changes in ambient light, high similarity between the moving target object and the background environment, changes in motion posture, intermittent motion, and occlusion by other objects. This improves the accuracy, robustness and generalization ability of moving target segmentation and expands the application scenarios of the moving target segmentation algorithm.

[0096] Step 103: Determine a weight map corresponding to at least one decoding layer based on a preset rule.

[0097] The weight map is used to assign different weight coefficients to each pixel in the feature map output by the decoding layer, so as to highlight the feature expression corresponding to the moving target object and suppress the features corresponding to the background pixels, thereby improving the accuracy of moving target segmentation.

[0098] In addition, the moving target segmentation method in the present application includes multiple encoding layers and multiple decoding layers, and the encoding layer is used to encode the monitoring image layer by layer to obtain a first feature map. The decoding layer is used to decode the first feature map to obtain a segmented image of the moving target object in the monitoring image. The multiple encoding layers and the multiple decoding layers are in a mirror relationship, and each decoding layer corresponds to an encoding layer, and the size of the feature map output by the decoding layer and the encoding layer in a mirror relationship is the same.

[0099] In the present application, the weight map corresponding to the first decoding layer can be determined according to the ratio between the first size of the second feature map output by the first coding layer corresponding to the first decoding layer (i.e., the coding layer in a mirror relationship with the first decoding layer) and the second size of the monitoring image, and the first pixel coordinates corresponding to the spatial position of the moving target object in the monitoring image. The weight map contains the displacement information of the moving target object. Among them, the spatial position can be a mobile hotspot (Wireless Fidelity, wifi) position or a global positioning system (Global Positioning System, GPS) position, etc., and this application does not limit this.

[0100] Alternatively, a spatial attention calculation is performed on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine a weight map corresponding to the second decoding layer.

[0101] Alternatively, a channel attention calculation is performed on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine a weight map corresponding to the second decoding layer.

[0102] Alternatively, a spatial attention calculation and a channel attention calculation are performed on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine a weight map corresponding to the second decoding layer. It can be understood that the weight map contains spatial position information of the moving target object.

[0103] Alternatively, the weight map corresponding to the first decoding layer is determined according to the ratio between the first size of the second feature map output by the first encoding layer corresponding to the first decoding layer and the second size of the monitoring image, and the first pixel coordinates corresponding to the spatial position of the moving target object in the monitoring image. The spatial attention calculation and the channel attention calculation are performed on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the weight map corresponding to the second decoding layer.

[0104] In addition, the first decoding layer and the second decoding layer are any one or more decoding layers that decode the first feature map.

[0105] Optionally, the first decoding layer and the second decoding layer may be the same or different, and this application does not impose any limitation on this.

[0106] Optionally, when the first decoding layer and the second decoding layer are the same, the weight map X determined according to the ratio between the first size of the feature map output by the encoding layer corresponding to the same decoding layer and the second size of the monitoring image, and the first pixel coordinate corresponding to the spatial position of the moving target object in the monitoring image may be d ", and "perform spatial attention calculation and channel attention calculation on the feature map output by the encoding layer corresponding to the same decoding layer to determine the weight map" perform multi-source feature fusion (such as adding two weight maps, or multiplying two weight maps, etc.) to determine the weight map of the same decoding layer.

[0107] like Figure 2 As shown, the above two weight graphs can be fused (i.e., multi-source fusion module) by referring to the following formula:

[0108]

[0109] Among them, CA represents the channel attention module, SA represents the spatial attention module, and X s represents the feature map of the encoding layer output corresponding to the same decoding layer as above, G represents the original input feature, represents element-wise multiplication, Conv() represents the convolution operation, and σ represents the activation function.

[0110] Step 104: Based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the monitoring image.

[0111] In the present application, the first feature map can be decoded layer by layer using multiple decoding layers. After the decoding layer corresponding to the weight map decodes and outputs the feature map, the feature map can be multiplied by the corresponding weight map to obtain a corrected feature map. The corrected feature map is then input to the next decoding layer for decoding until a segmented image of the moving target object in the surveillance image is obtained.

[0112] When the weight maps corresponding to the multiple decoding layers are determined, after each decoding layer in the multiple decoding layers decodes and outputs the feature map, the feature map output by each decoding layer is corrected based on the weight map associated with each decoding layer to obtain the corrected feature map corresponding to each decoding layer. The corrected feature map corresponding to each decoding layer is decoded by a decoding layer after each decoding layer until a segmented image of the moving target object in the monitoring image is obtained.

[0113] Optionally, the calculation method for decoding the first feature map layer by layer may refer to the following formula:

[0114]

[0115] Among them, MR i () represents the i-th multi-source fusion module, which is used to fuse the feature map output by the encoding layer corresponding to the j-th decoding layer and the multiple weight maps corresponding to the j-th decoding layer. j () indicates the jth decoding layer. X t represents the feature map of the jth decoding layer input, Indicates channel splicing.

[0116] Therefore, the multi-scale fusion of shallow features and deep features is used to improve the accuracy of multi-scale moving target object segmentation.

[0117] It can be understood that the weight map contains the displacement information and / or spatial position information of the moving target object. The weight map assists the coding layer to complete the segmentation of the moving target object, so that the accuracy of the moving target segmentation algorithm is not affected by the displacement and size change caused by the movement of the moving target object. This improves the accuracy and generalization of the moving target segmentation and expands the application scenarios of the moving target segmentation algorithm.

[0118] In addition, a unified end-to-end framework using multi-source data and multi-source spatial feature fusion mechanism avoids relying on long-term video frame pre-trained background models and additional labeled ground truth as network input, which can be directly applied to complex monitoring scenarios such as intermittent motion, frequent posture changes, and occlusion by other objects, thereby improving the accuracy of moving target segmentation in complex scenarios.

[0119] Optionally, the first feature map can be decoded layer by layer using a dilated convolution to expand the receptive field of view, thereby improving the accuracy of moving target segmentation.

[0120] In the present application, after obtaining a surveillance image containing a moving target object in a surveillance video in real time, the surveillance image can be encoded layer by layer to obtain a first feature map, and then, based on a preset rule, a weight map corresponding to at least one decoding layer is determined, and then, based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the surveillance image. Thus, the first feature map is decoded layer by layer based on the weight map, highlighting the feature expression of the moving target object and suppressing the corresponding features of the background elements, thereby improving the accuracy and generalization of the moving target segmentation.

[0121] Figure 3 A flow chart of a moving target segmentation method provided in an embodiment of the present application includes the following steps:

[0122] Step 301: Acquire a surveillance image containing a moving target object in a surveillance video in real time.

[0123] Step 302: Encode the monitoring image layer by layer to obtain a first feature map.

[0124] In the present application, the specific implementation process of step 301-step 302 can be found in the detailed description of any embodiment of the present application, and will not be repeated here.

[0125] Step 303, determining a second pixel coordinate corresponding to the first pixel coordinate in a second feature map output by a first encoding layer corresponding to the first decoding layer, wherein the first pixel coordinate is a pixel coordinate corresponding to the spatial position of the moving target object in the monitoring image.

[0126] The spatial location may be a wifi or GPS location, etc., and this application does not impose any restrictions on this.

[0127] In the present application, the spatial position of the moving target object can be obtained by a positioning device, and then the preset coordinate conversion matrix can be multiplied by the spatial position to determine the first pixel coordinate corresponding to the spatial position of the moving target object in the monitoring image.

[0128] In the present application, the monitoring image can be scaled to the same size as the second feature map, and the reference coordinates corresponding to the first pixel coordinates in the scaled monitoring image can be obtained. After that, the pixel coordinates corresponding to the reference coordinates in the second feature map can be determined as the second pixel coordinates corresponding to the first pixel coordinates.

[0129] Step 304: determine the maximum displacement of the first pixel coordinate within a preset period of time.

[0130] In the present application, the coordinates of the first pixel in the monitoring image at each moment in the preset time period before the current moment can be determined. Afterwards, the maximum displacement of the first pixel coordinate in the preset time period can be determined based on the distance between the first pixel coordinate in the monitoring image at the current moment and the first pixel coordinate in the monitoring image at each moment in the preset time period.

[0131] Step 305: scaling the maximum displacement based on the ratio to determine a target displacement.

[0132] In the present application, the ratio between the first size of the second feature map and the second size of the monitoring image can be determined, and then the maximum displacement is scaled based on the ratio to determine the target displacement.

[0133] Optionally, the target displacement can be pre-set in the system.

[0134] Step 306, taking the second pixel coordinate as the center and based on the target displacement, determine a reference range of the moving target object in the second feature map.

[0135] In the present application, a circle formed by taking the second pixel coordinate as the center and the target displacement as the radius in the second feature map can be determined as the reference range of the moving target object in the second feature map. Alternatively, a rectangle formed by taking the second pixel coordinate as the center and twice the target displacement as the diagonal length in the second feature map can be determined as the reference range of the moving target object in the second feature map. This application does not limit this.

[0136] In this application, the calculation method of the rectangular reference range can refer to the following formula:

[0137] (x-Δx,y-Δy),(x+Δx,y+Δy)

[0138] Where (x, y) is the second pixel coordinate, Δx and Δy are the target displacements in the x-axis and y-axis directions respectively. (x-Δx, y-Δy) is the coordinate of the upper left corner of the rectangle, and (x+Δx, y+Δy) is the coordinate of the lower right corner of the rectangle.

[0139] It can be understood that since the second pixel coordinate is generated based on the spatial position conversion of the moving target object, and the target displacement is the displacement corresponding to the actual displacement that may be generated by the movement of the moving target object in the second feature map, the reference range corresponding to the movement of the moving target object in the second feature map is determined based on the target displacement with the second pixel coordinate as the center, thereby improving the accuracy of the reference range.

[0140] 307 , determining a weight value on each third pixel coordinate in the weight map according to a distance between each third pixel coordinate and the second pixel coordinate, wherein the third pixel coordinate is a coordinate of a pixel in the second feature map that is within a reference range.

[0141] In the present application, the third pixel in the reference range of the second feature map may be the pixel corresponding to the moving target object after it moves, and the closer the third pixel is to the second pixel coordinate, the greater the possibility that it is the pixel corresponding to the moving target object after it moves. Therefore, the weight corresponding to each third pixel coordinate in the weight map can be determined according to the distance between the third pixel coordinate and the second pixel coordinate.

[0142] Step 308 , determining that the weight value at each fourth pixel coordinate in the weight map is zero, wherein the fourth pixel coordinate is the coordinate of a pixel in the second feature map that is outside the reference range.

[0143] In the present application, the fourth pixel outside the reference range in the second feature map is less likely to be the pixel corresponding to the moving target object after it moves. The weight corresponding to each fourth pixel coordinate in the weight map may be determined to be zero to avoid the influence of the fourth pixel outside the reference range on the moving target segmentation, thereby improving the accuracy of the moving target segmentation.

[0144] It can be understood that the weight map is a displacement weight map of the moving target object.

[0145] In this application, the calculation method of the weight map can refer to the following formula:

[0146]

[0147] Among them, (x, y) is the second pixel coordinate, (x i ,y i ) is the pixel coordinate within the reference range, ξ, ψ are the preset hyperparameters, Δx, Δy are the target displacements in the x-axis and y-axis directions respectively.

[0148] It can be understood that the weight map is adaptively calculated in combination with the scene prior position signal (wifi, GPS, etc.) to improve the accuracy and stability of moving target recognition and solve the problem of inaccurate moving target segmentation caused by moving target object loss and misjudgment.

[0149] Step 309: Based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the monitoring image.

[0150] In the present application, the specific implementation process of step 309 can be found in the detailed description of any embodiment of the present application and will not be repeated here.

[0151] In the present application, the first pixel coordinates corresponding to the spatial position of the moving target object in the monitoring image and the second pixel coordinates corresponding to the second feature map output by the first encoding layer corresponding to the first decoding layer can be determined, and the maximum displacement of the first pixel coordinates within a preset time period can be determined. After that, the maximum displacement is scaled based on the ratio to determine the target displacement, and the reference range of the moving target object in the second feature map is determined based on the target displacement with the second pixel coordinate as the center. Then, the weight value of each third pixel coordinate in the weight map is determined based on the distance between each third pixel coordinate and the second pixel coordinate within the reference range, and the weight value of each fourth pixel coordinate outside the reference range in the weight map is determined to be zero. This improves the accuracy of the weight map containing displacement information, thereby improving the accuracy and stability of moving target segmentation.

[0152] Figure 4 A flow chart of a moving target segmentation method provided in an embodiment of the present application includes the following steps:

[0153] Step 401: Acquire a surveillance image containing a moving target object in a surveillance video in real time.

[0154] Step 402: Encode the monitoring image layer by layer to obtain a first feature map.

[0155] In the present application, the specific implementation process of step 401-step 402 can be found in the detailed description of any embodiment of the present application, and will not be repeated here.

[0156] Step 403, performing maximum pooling and average pooling in the channel dimension on the third feature map output by the second encoding layer corresponding to the second decoding layer, and determining a first reference weight map and a second reference weight map.

[0157] In the present application, the third feature map output by the second encoding layer corresponding to the second decoding layer is subjected to maximum pooling in the channel dimension to determine the first reference weight map. The third feature map output by the second encoding layer corresponding to the second decoding layer is subjected to average pooling in the channel dimension to determine the second reference weight map.

[0158] Step 404: Activate the sum of the first reference weight map and the second reference weight map to determine a third reference weight map.

[0159] In the present application, the first reference weight map and the second reference weight map can be added together, and the weight reference map obtained after the addition is activated using an activation function to obtain a third reference weight map. Thus, by performing multiple pooling operations on the third feature map in the channel dimension to obtain accurate channel feature information, the accuracy of the third reference weight map in the channel dimension is improved.

[0160] In this application, let the third characteristic graph be X t∈R C×H×W , where C, H, and W are the number of channels, height, and width of the third feature map, respectively. The calculation method of the third reference weight map can be found in the following formula:

[0161]

[0162] Among them, AvgPool() represents average pooling, MaxPool() represents maximum pooling, and SH() represents a shared convolutional layer module. represents the concatenation operation, and σ represents the activation function.

[0163] Step 405 , performing average pooling of the width dimension and the height dimension on the third feature map to determine a fourth reference weight map and a fifth reference weight map.

[0164] In the present application, the third feature map may be average pooled in the width dimension to determine the fourth reference weight map, and the third feature map may be average pooled in the height dimension to determine the fifth reference weight map.

[0165] Step 406 , performing transposition summation and activating on the fourth reference weight map and the fifth reference weight map to determine a sixth reference weight map.

[0166] In the present application, the fourth reference weight map and the fifth reference weight map can be transposed and summed, and the weight reference map obtained after the transposed and summed is activated using an activation function to obtain a sixth reference weight map. Thus, by performing a pooling operation on the width dimension and the height dimension on the third feature map to obtain accurate spatial feature information, the accuracy of the sixth reference weight map in the spatial dimension is improved.

[0167] In this application, let the third characteristic graph be X t ∈R C×H×W , where C, H, and W are the number of channels, height, and width of the third feature map, respectively. The calculation method of the sixth reference weight map can be found in the following formula:

[0168]

[0169] in, represents the fourth reference weight map, X h Represents the fifth reference weight map, σ represents the activation function, and Conv() represents the convolution operation.

[0170] Step 407: determine the sum of the third reference weight map and the sixth reference weight map as the weight map.

[0171] It can be understood that the weight map is a spatial position weight map of the moving target object. Step 408: Based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the monitoring image.

[0172] In the present application, the specific implementation process of step 408 can be found in the detailed description of any embodiment of the present application, and will not be repeated here.

[0173] In the present application, the third feature map output by the second encoding layer corresponding to the second decoding layer can be subjected to maximum pooling and average pooling in the channel dimension to determine the first reference weight map and the second reference weight map, and the sum of the first reference weight map and the second reference weight map is activated to determine the third reference weight map, and then the third feature map is subjected to average pooling in the width dimension and average pooling in the height dimension to determine the fourth reference weight map and the fifth reference weight map, and the fourth reference weight map and the fifth reference weight map are transposed, summed and activated to determine the sixth reference weight map, and then the sum of the third reference weight map and the sixth reference weight map is determined as the weight map. Thus, multiple pooling operations in the channel dimension are performed on the third feature map to obtain accurate channel feature information, and pooling operations in the width dimension and the height dimension are performed on the third feature map to obtain accurate spatial feature information, thereby improving the accuracy of the weight map containing spatial position information, and thereby improving the accuracy and generalization of moving target segmentation.

[0174] Figure 5 A flow chart of a moving target segmentation method provided in an embodiment of the present application includes the following steps:

[0175] Step 501: Acquire a surveillance image containing a moving target object in a surveillance video in real time.

[0176] Step 502: Encode the monitoring image layer by layer to obtain a first feature map.

[0177] Step 503: Determine a weight map corresponding to at least one decoding layer based on a preset rule.

[0178] In the present application, the specific implementation process of steps 501 to 503 can be found in the detailed description of any embodiment of the present application and will not be repeated here.

[0179] Step 504, based on the weight map associated with the i-th decoding layer, correct the feature map output by the i-th decoding layer to obtain a corrected feature map, where i is a natural number greater than or equal to 1 and less than or equal to N, and N is the total number of decoding layers included in the decoding module.

[0180] In the present application, in the process of decoding the first feature map layer by layer, after the feature map output by the i-1 decoding layer is decoded by the i-th decoding layer, and the feature map decoded by the i-th decoding layer is obtained, the weight map associated with the i-th decoding layer can be multiplied with the feature map output by the i-th decoding layer to correct the feature map decoded by the i-th decoding layer and obtain the corrected feature map.

[0181] Optionally, in the process of decoding the first feature map layer by layer, the feature map output by the i-1 decoding layer is decoded at the i-th decoding layer, and after the feature map decoded by the i-th decoding layer is obtained, the sum of the feature map output by the encoding layer corresponding to the i-th decoding layer and the feature map output by the i-th decoding layer can be determined as the reference feature of the i-th decoding layer. Figure 6 As shown, the feature map output by the encoding layer corresponding to the i-th decoding layer and the feature map output by the i-th decoding layer can be jump-connected to determine the reference feature of the i-th decoding layer. Afterwards, the weight map associated with the i-th decoding layer can be multiplied by the reference feature of the i-th decoding layer to obtain a corrected feature map. Thus, the features output by the decoding layer and the features output by the encoding layer are fused to enhance the feature extraction capability, thereby improving the accuracy of moving target segmentation.

[0182] Step 505: Use the (i+1)th decoding layer to decode the corrected feature map until a segmented image of the moving target object in the monitoring image is obtained.

[0183] In the present application, after obtaining a monitoring image containing a moving target object in a monitoring video in real time, the monitoring image is encoded layer by layer to obtain a first feature map, and based on a preset rule, a weight map corresponding to at least one decoding layer is determined. After that, based on the weight map associated with the i-th decoding layer, the feature map output by the i-th decoding layer is corrected to obtain a corrected feature map, and the corrected feature map is decoded using the i+1-th decoding layer until a segmented image of the moving target object in the monitoring image is obtained. Thus, auxiliary decoding of the first feature map is performed based on the weight map, highlighting the features corresponding to the moving target object, while shielding the features corresponding to the background elements. This improves the accuracy and generalization of moving target segmentation.

[0184] Figure 7 is a block diagram of a moving target segmentation device according to an exemplary embodiment. Figure 7 The device includes an acquisition module 710, an encoding module 720, a determination module 730, and a decoding module 740.

[0185] An acquisition module 710 is used to acquire a surveillance image containing a moving target object in a surveillance video in real time;

[0186] The encoding module 720 is used to encode the monitoring image layer by layer to obtain a first feature map;

[0187] A determination module 730, configured to determine a weight map corresponding to at least one decoding layer based on a preset rule;

[0188] The decoding module 740 is used to decode the first feature map layer by layer based on the weight map to obtain a segmented image of the moving target object in the monitoring image.

[0189] In a possible implementation of the embodiment of the present application, the determination module 730 is used to:

[0190] Determine a weight map corresponding to the first decoding layer according to a ratio between a first size of a second feature map output by a first encoding layer corresponding to the first decoding layer and a second size of the monitoring image, and a first pixel coordinate corresponding to a spatial position of the moving target object in the monitoring image;

[0191] Perform spatial attention calculation and channel attention calculation on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the weight map corresponding to the second decoding layer.

[0192] In a possible implementation of the embodiment of the present application, the determination module 730 is used to:

[0193] Determine a second pixel coordinate corresponding to the first pixel coordinate in a second feature map output by a first encoding layer corresponding to the first decoding layer;

[0194] Determining a maximum displacement of a first pixel coordinate within a preset time period;

[0195] The maximum displacement is scaled based on the ratio to determine a target displacement;

[0196] Taking the second pixel coordinate as the center and based on the target displacement, determining a reference range of the moving target object in the second feature map;

[0197] Determine a weight value on each third pixel coordinate in the weight map according to a distance between each third pixel coordinate and the second pixel coordinate, wherein the third pixel coordinate is a coordinate of a pixel in the second feature map that is within a reference range;

[0198] Determine that the weight value at each fourth pixel coordinate in the weight map is zero, wherein the fourth pixel coordinate is the coordinate of a pixel in the second feature map that is outside the reference range.

[0199] In a possible implementation of the embodiment of the present application, the determination module 730 is used to:

[0200] Performing maximum pooling and average pooling of the channel dimension on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the first reference weight map and the second reference weight map;

[0201] activating the sum of the first reference weight map and the second reference weight map to determine a third reference weight map;

[0202] Performing average pooling of the width dimension and the height dimension on the third feature map to determine a fourth reference weight map and a fifth reference weight map;

[0203] Transpose and sum the fourth reference weight map and the fifth reference weight map and activate them to determine a sixth reference weight map;

[0204] The sum of the third reference weight map and the sixth reference weight map is determined as the weight map.

[0205] In a possible implementation of the embodiment of the present application, the monitoring image includes an infrared grayscale image and a visible light image in the same field of view, and the encoding module 720 is used to:

[0206] The channels corresponding to the infrared grayscale image and the channels corresponding to the visible light image are randomly mixed to perform fusion encoding on the infrared grayscale image and the visible light image to obtain a first feature map.

[0207] In a possible implementation of the embodiment of the present application, the decoding module 740 is used to:

[0208] Based on the weight map associated with the i-th decoding layer, the feature map output by the i-th decoding layer is corrected to obtain a corrected feature map, where i is a natural number greater than or equal to 1 and less than or equal to N, and N is the total number of decoding layers included in the decoding module;

[0209] The corrected feature map is decoded using the i+1th decoding layer until a segmented image of the moving target object in the surveillance image is obtained.

[0210] In a possible implementation of the embodiment of the present application, the decoding module 740 is used to:

[0211] The sum of the feature map output by the encoding layer corresponding to the i-th decoding layer and the feature map output by the i-th decoding layer is determined as the reference feature of the i-th decoding layer;

[0212] Based on the weight map associated with the i-th decoding layer, the reference feature is corrected to obtain a corrected feature map.

[0213] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0214] In the present application, after obtaining a surveillance image containing a moving target object in a surveillance video in real time, the surveillance image can be encoded layer by layer to obtain a first feature map, and then, based on a preset rule, a weight map corresponding to at least one decoding layer is determined, and then, based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the surveillance image. Thus, the first feature map is decoded layer by layer based on the weight map, highlighting the feature expression of the moving target object and suppressing the corresponding features of the background elements, thereby improving the accuracy and generalization of the moving target segmentation.

[0215] Figure 8 The present invention is a block diagram of a terminal device for segmenting a moving target according to an exemplary embodiment.

[0216] like Figure 8 As shown, the terminal device 800 includes:

[0217] The memory 810 and the processor 820, a bus 830 connecting different components (including the memory 810 and the processor 820), the memory 810 stores a computer program, and when the processor 820 executes the program, the moving target segmentation method described in the embodiment of the present application is implemented.

[0218] Bus 830 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0219] The terminal device 800 typically includes a variety of electronic device readable media, which can be any available media that can be accessed by the terminal device 800, including volatile and non-volatile media, removable and non-removable media.

[0220] The memory 810 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 840 and / or cache memory 850. The terminal device 800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 860 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical medium) may be provided. In these cases, each drive may be connected to bus 830 via one or more data medium interfaces. Memory 810 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0221] A program / utility 880 having a set (at least one) of program modules 870 may be stored, for example, in the memory 810, such program modules 870 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 870 generally perform the functions and / or methods of the embodiments described herein.

[0222] The terminal device 800 can also communicate with one or more external devices 890 (e.g., keyboard, pointing device, display 891, etc.), and can also communicate with one or more devices that enable users to interact with the terminal device 800, and / or communicate with any device that enables the terminal device 800 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through an input / output (I / O) interface 892. In addition, the terminal device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 893. As shown in the figure, the network adapter 893 communicates with other modules of the terminal device 800 through a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the terminal device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0223] The processor 820 executes various functional applications and data processing by running the programs stored in the memory 810 .

[0224] It should be noted that the implementation process and technical principles of the terminal device of this embodiment refer to the aforementioned explanation of the moving target segmentation method of the embodiment of the present application, and will not be repeated here.

[0225] In the present application, after obtaining a surveillance image containing a moving target object in a surveillance video in real time, the surveillance image can be encoded layer by layer to obtain a first feature map, and then, based on a preset rule, a weight map corresponding to at least one decoding layer is determined, and then, based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the surveillance image. Thus, the first feature map is decoded layer by layer based on the weight map, highlighting the feature expression of the moving target object and suppressing the corresponding features of the background elements, thereby improving the accuracy and generalization of the moving target segmentation.

[0226] In an exemplary embodiment, the present application also provides a computer-readable storage medium including instructions, such as a memory including instructions, and the above instructions can be executed by a processor of a terminal device to complete the above method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0227] In order to implement the above embodiments, the present application also provides a computer program product. When the computer program is executed by a processor of a terminal device, the terminal device can execute the moving target segmentation method as described above.

[0228] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0229] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A moving target segmentation method, characterized in that: include: Acquire surveillance images containing moving target objects in surveillance videos in real time; Encoding the monitoring image layer by layer to obtain a first feature map; Based on a preset rule, determining a weight map corresponding to at least one decoding layer; Based on the weight map, the first feature map is decoded layer by layer to obtain a segmented image of the moving target object in the monitoring image.

2. The method according to claim 1, characterized in that Determining a weight map corresponding to at least one decoding layer based on a preset rule comprises at least one of the following steps: Determine a weight map corresponding to the first decoding layer according to a ratio between a first size of a second feature map output by a first encoding layer corresponding to the first decoding layer and a second size of the monitoring image, and a first pixel coordinate corresponding to a spatial position of the moving target object in the monitoring image; Perform spatial attention calculation and channel attention calculation on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the weight map corresponding to the second decoding layer.

3. The method according to claim 2, characterized in that The step of determining a weight map corresponding to the first decoding layer according to a ratio between a first size of a second feature map output by a first encoding layer corresponding to the first decoding layer and a second size of the monitoring image and a first pixel coordinate corresponding to a spatial position of the moving target object in the monitoring image comprises: Determine a second pixel coordinate corresponding to the first pixel coordinate in a second feature map output by a first encoding layer corresponding to a first decoding layer; Determining a maximum displacement of the first pixel coordinates within a preset time period; Scaling the maximum displacement based on the ratio to determine a target displacement; Taking the second pixel coordinate as the center and based on the target displacement, determining a reference range of the moving target object in the second feature map; Determine a weight value on each of the third pixel coordinates in the weight map according to a distance between each of the third pixel coordinates and the second pixel coordinate, wherein the third pixel coordinate is a coordinate of a pixel in the second feature map that is within the reference range; Determine that the weight value at each of the fourth pixel coordinates in the weight map is zero, wherein the fourth pixel coordinate is the coordinate of a pixel in the second feature map that is outside the reference range.

4. The method according to claim 2, characterized in that The performing spatial attention calculation and channel attention calculation on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the weight map corresponding to the second decoding layer includes: Performing maximum pooling and average pooling of the channel dimension on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the first reference weight map and the second reference weight map; activating the sum of the first reference weight map and the second reference weight map to determine a third reference weight map; Performing average pooling of the width dimension and the height dimension on the third feature map to determine a fourth reference weight map and a fifth reference weight map; Transpose, sum, and activate the fourth reference weight map and the fifth reference weight map to determine a sixth reference weight map; The sum of the third reference weight map and the sixth reference weight map is determined as the weight map.

5. The method according to claim 1, characterized in that The monitoring image includes an infrared grayscale image and a visible light image in the same field of view, and encoding the monitoring image layer by layer to obtain a first feature map includes: The channels corresponding to the infrared grayscale image and the channels corresponding to the visible light image are randomly mixed to perform fusion encoding on the infrared grayscale image and the visible light image to obtain a first feature map.

6. The method according to any one of claims 1 to 5, characterized in that: The decoding of the first feature map based on the weight map to obtain a segmented image of the moving target object in the monitoring image includes: Based on the weight map associated with the i-th decoding layer, correct the feature map output by the i-th decoding layer to obtain a corrected feature map, where i is a natural number greater than or equal to 1 and less than or equal to N, and N is the total number of decoding layers included in the decoding module; The corrected feature map is decoded using the (i+1)th decoding layer until a segmented image of the moving target object in the monitoring image is obtained.

7. The method according to claim 6, characterized in that The step of correcting the feature map output by the i-th decoding layer based on the weight map associated with the i-th decoding layer to obtain the corrected feature map includes: Determine the sum of a feature map output by the encoding layer corresponding to the i-th decoding layer and a feature map output by the i-th decoding layer as a reference feature of the i-th decoding layer; Based on the weight map associated with the i-th decoding layer, the reference feature is corrected to obtain a corrected feature map.

8. A moving target segmentation device, characterized in that: include: An acquisition module is used to acquire in real time a surveillance image containing a moving target object in a surveillance video; An encoding module, used to encode the monitoring image layer by layer to obtain a first feature map; A determination module, used to determine a weight map corresponding to at least one decoding layer based on a preset rule; A decoding module is used to decode the first feature map layer by layer based on the weight map to obtain a segmented image of the moving target object in the monitoring image.

9. The device according to claim 8, characterized in that The determining module is used to: Determine a weight map corresponding to the first decoding layer according to a ratio between a first size of a second feature map output by a first encoding layer corresponding to the first decoding layer and a second size of the monitoring image, and a first pixel coordinate corresponding to a spatial position of the moving target object in the monitoring image; Perform spatial attention calculation and channel attention calculation on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the weight map corresponding to the second decoding layer.

10. The device according to claim 9, characterized in that The determining module is used to: Determine a second pixel coordinate corresponding to the first pixel coordinate in a second feature map output by a first encoding layer corresponding to a first decoding layer; Determining a maximum displacement of the first pixel coordinates within a preset time period; Scaling the maximum displacement based on the ratio to determine a target displacement; Taking the second pixel coordinate as the center and based on the target displacement, determining a reference range of the moving target object in the second feature map; Determine a weight value on each of the third pixel coordinates in the weight map according to a distance between each of the third pixel coordinates and the second pixel coordinate, wherein the third pixel coordinate is a coordinate of a pixel in the second feature map that is within the reference range; Determine that the weight value at each of the fourth pixel coordinates in the weight map is zero, wherein the fourth pixel coordinate is the coordinate of a pixel in the second feature map that is outside the reference range.

11. The device according to claim 9, characterized in that The determining module is used to: Performing maximum pooling and average pooling of the channel dimension on the third feature map output by the second encoding layer corresponding to the second decoding layer to determine the first reference weight map and the second reference weight map; activating the sum of the first reference weight map and the second reference weight map to determine a third reference weight map; Performing average pooling of the width dimension and the height dimension on the third feature map to determine a fourth reference weight map and a fifth reference weight map; Transpose, sum, and activate the fourth reference weight map and the fifth reference weight map to determine a sixth reference weight map; The sum of the third reference weight map and the sixth reference weight map is determined as the weight map.

12. The device according to claim 8, characterized in that The monitoring image includes an infrared grayscale image and a visible light image in the same field of view, and the encoding module is used to: The channels corresponding to the infrared grayscale image and the channels corresponding to the visible light image are randomly mixed to perform fusion encoding on the infrared grayscale image and the visible light image to obtain a first feature map.

13. The device according to any one of claims 8 to 12, characterized in that: The decoding module is used for: Based on the weight map associated with the i-th decoding layer, correct the feature map output by the i-th decoding layer to obtain a corrected feature map, where i is a natural number greater than or equal to 1 and less than or equal to N, and N is the total number of decoding layers included in the decoding module; The corrected feature map is decoded using the (i+1)th decoding layer until a segmented image of the moving target object in the monitoring image is obtained.

14. The device according to claim 13, characterized in that The decoding module is used for: Determine the sum of a feature map output by the encoding layer corresponding to the i-th decoding layer and a feature map output by the i-th decoding layer as a reference feature of the i-th decoding layer; Based on the weight map associated with the i-th decoding layer, the reference feature is corrected to obtain a corrected feature map.

15. A terminal device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the moving target segmentation method as described in any one of claims 1-7.

16. A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of a terminal device, the terminal device is enabled to execute the moving target segmentation method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the moving target segmentation method described in any one of claims 1 to 7 is implemented.

Images