Property abnormity monitoring method and device, equipment and storage medium

Abstract

The invention discloses a property abnormity monitoring method and device, equipment and a storage medium, and the method comprises the steps: recognizing a first monitoring object from a monitoring video stream, acquiring a first feature map corresponding to the first monitoring object from a target video frame of the first monitoring object which is identified for the first time, and respectively inputting the first feature map into a global channel sensing module and a local space sensing module to obtain a global sensing weight and a local space sensing weight; fusing the global sensing weight features and the local space sensing weight features to obtain a first target feature map; obtaining a target tracking result based on the first target feature map and the video stream; and when the first monitoring object is abnormal, constructing a target tracking path based on the target tracking result, and sending the target tracking result and the target tracking path to a target account associated with the property monitoring system. According to the invention, multi-scale feature extraction is carried out on the original image of the monitored object, and the monitored object is accurately tracked under the conditions of low resolution and complex environmental factors.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of monitoring technology, and in particular to a property abnormality monitoring method, device, equipment, and storage medium. Background Art

[0002] With the development of technology, the property monitoring system has incorporated artificial intelligence algorithms for image recognition and target tracking, which can track the target monitoring object based on the video stream collected by the video acquisition device, and send the target tracking results to the property management personnel to conduct timely abnormal monitoring of the target monitoring object to ensure the safety of the owner. However, the video acquisition equipment of the property monitoring system is limited by cost, and the video acquisition equipment in different monitoring areas is installed in different locations, so that the quality of the acquired monitoring images is affected by complex backgrounds such as low resolution, lighting changes, and occlusion, thereby affecting the accuracy of target tracking for the target monitoring object, making it difficult to ensure effective abnormal monitoring of the target monitoring object. Summary of the invention

[0003] The embodiments of the present application provide a property abnormality monitoring method, device, equipment, and storage medium, which effectively ensure accurate target tracking of the target monitoring object, thereby effectively monitoring the abnormalities of the target monitoring object.

[0004] In a first aspect, an embodiment of the present application provides a property abnormality monitoring method, which is applied to a property monitoring system, wherein the property monitoring system is pre-set with a monitoring camera, a global channel perception module, a local space perception module, a feature fusion module, and a target tracking module, and the method includes: Acquire a surveillance video stream from the surveillance camera, and when a first surveillance object is determined from a video frame of the surveillance video stream, determine a video frame in which the first surveillance object is first identified as a target video frame, and acquire a first feature map corresponding to the first surveillance object in the target video frame; Inputting the first feature map into the global channel perception module to obtain a global perception weight, and inputting the first feature map into the local space perception module to obtain a local space perception weight; Inputting the global perception weight and the local spatial perception weight into the feature fusion module to obtain an enhanced first target feature map; Inputting the first target feature map and the video frame located after the generation timestamp of the target video frame into the target tracking module to obtain a target tracking result; When an abnormality is detected in the first monitored object based on the target tracking result, a target tracking path is constructed based on the target tracking result, and an alarm information generated based on the target tracking result and the target tracking path is sent to the target account associated with the property monitoring system.

[0005] In some embodiments, the global channel perception module includes a plurality of first convolutional layers, a global average pooling layer, a ReLU activation function layer, and a first Sigmoid activation function layer, and the first feature map is input into the global channel perception module to obtain a global perception weight, including: Inputting the features obtained by convolving the first feature map through the first convolutional layer into the global average pooling layer for pooling processing to obtain a first intermediate feature map; Inputting the features obtained by convolving the first intermediate feature map through the first convolutional layer into the ReLU activation function layer for activation to obtain a second intermediate feature map; Inputting the feature obtained by convolving the second intermediate feature map through the first convolutional layer into the first Sigmoid activation function layer for activation to obtain a first activation result; The product result of the first activation result and the first feature map is added to the first feature map to obtain the global perception weight.

[0006] In some embodiments, the local space perception module includes a plurality of second convolutional layers and a second Sigmoid activation function layer, the convolution kernels of each of the second convolutional layers are different from each other, and the first feature map is input into the local space perception module to obtain the local space perception weight, including: Inputting the first feature maps into each of the second convolutional layers for convolution processing respectively to obtain each third intermediate feature map; Multiplying a splicing result obtained by splicing all the third intermediate feature maps by a preset weight matrix to obtain a first product result; Inputting the first product result into the second Sigmoid activation function layer for activation to obtain a second activation result; The product result of the second activation result and the first feature map is added to the first feature map to obtain the local spatial perception weight.

[0007] In some embodiments, the global perception weight inputs the global perception weight and the local spatial perception weight into the feature fusion module to obtain an enhanced first target feature map, including: Obtaining a first balance parameter, a second balance parameter, and a third balance parameter; Multiplying the first balancing parameter by the global perception weight to obtain a first product result; Multiplying the second balance parameter by the local space perception weight to obtain a second product result; multiplying the third balance parameter by the first characteristic map to obtain a third product result; Adding the first product result, the second product result and the third product result to obtain a fused feature map; The fused feature map is passed through a third convolution layer, a fourth convolution layer, and a fifth convolution layer, respectively, to obtain a fourth intermediate feature map, a fifth intermediate feature map, and a sixth intermediate feature map, respectively, wherein the convolution kernels of the third convolution layer, the fourth convolution layer, and the fifth convolution layer are different from each other; Upsampling the fourth intermediate feature map to obtain an upsampling result, and downsampling the sixth intermediate feature map to obtain a downsampling result; The upsampling result, the fifth intermediate feature map and the downsampling result are concatenated to obtain the first target feature map.

[0008] In some embodiments, the first target feature map and the video frame located after the generation timestamp of the target video frame are input into the target tracking module to obtain the target tracking result, including: Determine an area in the target video frame that surrounds the first target feature map as a first template area, and determine a first template feature map of the first template area as a target template feature map, wherein the target template feature map includes the first target feature map; Determining a reference center position of a matching area based on a reference position of a second feature map of the first monitored object in a frame previous to a current video frame, and determining a reference size of the matching area based on a size and a moving speed of the second feature map, wherein the current video frame is one of all video frames located after a generation timestamp of the target video frame; Extracting a matching feature map from the current video frame based on the reference center position and the reference size, wherein a size of the matching feature map is larger than a size of the target template feature map; Calculate the similarity between the target template feature map and each position in the matching feature map, determine the position with the largest similarity in the matching feature map as the target center position where the first monitored object is located in the current video frame, and determine the first tracking result of the current video frame based on the target center position and the target template feature map; The target tracking result is generated based on the first tracking results of all video frames located after the generation timestamp of the target video frame.

[0009] In some embodiments, determining a first tracking result of the current video frame based on the target center position and the target template feature map includes: Performing global maximum pooling processing on the target template feature map to obtain a seventh intermediate feature map; Increasing the spatial dimension of the seventh intermediate feature map to obtain a new seventh intermediate feature map, and splicing the new seventh intermediate feature map with the matching feature map to obtain a splicing result, wherein the new seventh intermediate feature map has the same size as the matching feature map; The splicing result and the target middle position are input into the MLP structure to obtain the first tracking result.

[0010] In some embodiments, the property monitoring system includes a server, and the monitoring video stream is displayed in a monitoring display module of a visualization interface of the server. After determining a first tracking result of a current video frame based on the target center position and the template feature map, the method further includes: Generate a bounding box according to the second monitoring object selected by the user in the monitoring display module, and determine the image of the bounding box in the current video frame as a third feature map; When it is detected that the second monitored object is the same object as the first monitored object, the third feature map is input into the global channel perception module, the local space perception module and the feature fusion module to obtain a second target feature map; Determine an area surrounding the second target feature map in the current video frame as a second template area, and extract a second template feature map of the second template area; Determine a time interval between generating the first template feature graph and generating the second template feature graph; A new target template feature map is determined based on an exponential moving average algorithm, the time interval, the first template feature map, and the second template feature map.

[0011] In a second aspect, an embodiment of the present application provides a control device, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the property abnormality monitoring method as described in the first aspect.

[0012] In a third aspect, an embodiment of the present application further provides an electronic device, comprising the control device of the second aspect.

[0013] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the property abnormality monitoring method as described in the first aspect.

[0014] The embodiments of the present application provide a property abnormality monitoring method, device, equipment, and storage medium, the method comprising: obtaining a monitoring video stream from the monitoring camera, and when a first monitoring object is determined from a video frame of the monitoring video stream, determining a video frame in which the first monitoring object is first identified as a target video frame, and obtaining a first feature map corresponding to the first monitoring object in the target video frame; inputting the first feature map into the global channel perception module to obtain a global perception weight, and inputting the first feature map into the local space perception module to obtain a local space perception weight; inputting the global perception weight and the local space perception weight into the feature fusion module to obtain an enhanced first target feature map; inputting the first target feature map and a video frame located after a generation timestamp of the target video frame into the target tracking module to obtain a target tracking result; when an abnormality is detected in the first monitoring object based on the target tracking result, constructing a target tracking path based on the target tracking result, and sending an alarm information generated based on the target tracking result and the target tracking path to a target account associated with the property monitoring system. According to the solution provided in the embodiment of the present application, multi-scale features are extracted from the original image corresponding to the monitored object. Under low resolution and complex environmental factors, the details of the monitored object can still be captured based on the multi-scale features to ensure accurate target tracking of the target monitored object, thereby effectively monitoring the target monitored object for abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of the steps of a property abnormality monitoring method provided by an embodiment of the present application; Figure 2 It is a structural diagram of a control device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0017] It is understood that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flow chart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0018] With the development of technology, the property monitoring system has incorporated artificial intelligence algorithms for image recognition and target tracking, which can track the target monitoring object based on the video stream collected by the video acquisition device, and send the target tracking results to the property management personnel to conduct timely abnormal monitoring of the target monitoring object to ensure the safety of the owner. However, the video acquisition equipment of the property monitoring system is limited by cost, and the video acquisition equipment in different monitoring areas is installed in different locations, so that the quality of the acquired monitoring images is affected by complex backgrounds such as low resolution, lighting changes, and occlusion, thereby affecting the accuracy of target tracking for the target monitoring object, making it difficult to ensure effective abnormal monitoring of the target monitoring object.

[0019] To solve the above-mentioned problems, an embodiment of the present application provides a property abnormality monitoring method, device, equipment, and storage medium, the method comprising: obtaining a monitoring video stream from the monitoring camera, when a first monitoring object is determined from a video frame of the monitoring video stream, determining a video frame in which the first monitoring object is first identified as a target video frame, and obtaining a first feature map corresponding to the first monitoring object in the target video frame; inputting the first feature map into the global channel perception module to obtain a global perception weight, and inputting the first feature map into the local space perception module to obtain a local space perception weight; inputting the global perception weight and the local space perception weight into the feature fusion module to obtain an enhanced first target feature map; inputting the first target feature map and a video frame located after the generation timestamp of the target video frame into the target tracking module to obtain a target tracking result; when an abnormality is detected in the first monitoring object based on the target tracking result, constructing a target tracking path based on the target tracking result, and sending an alarm information generated based on the target tracking result and the target tracking path to the target account associated with the property monitoring system. According to the solution provided in the embodiment of the present application, multi-scale features are extracted from the original image corresponding to the monitored object. Under low resolution and complex environmental factors, the details of the monitored object can still be captured based on the multi-scale features to ensure accurate target tracking of the target monitored object, thereby effectively monitoring the target monitored object for abnormalities.

[0020] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0021] refer to Figure 1 , Figure 1 The present invention provides a method for monitoring abnormal properties in accordance with an embodiment of the present invention. The method is applied to a property monitoring system. The property monitoring system is pre-installed with a monitoring camera, a global channel perception module, a local space perception module, a feature fusion module, and a target tracking module. The method includes but is not limited to the following steps: Step S10, obtaining a surveillance video stream from a surveillance camera, and when a first surveillance object is determined from video frames of the surveillance video stream, determining a video frame in which the first surveillance object is first identified as a target video frame, and obtaining a first feature map corresponding to the first surveillance object in the target video frame.

[0022] Specifically, the embodiments of the present application do not limit the specific method of determining the first monitoring object from the monitoring video stream. In this embodiment, the system server monitoring personnel selects the suspicious first monitoring object from the acquired monitoring video stream, and can also identify the first monitoring object from the monitoring video stream based on a preset AI target recognition model.

[0023] Specifically, in this embodiment, the video frame of the first monitoring object determined from the monitoring video stream is not necessarily the target video frame in which the first monitoring object is first identified in the monitoring video stream. For example, when the first monitoring object is determined from the monitoring video stream by a monitoring personnel of a system server selecting it from the acquired monitoring video stream, the monitoring personnel selects the first monitoring object after observing the video stream for a period of time, and the first monitoring object has existed in the monitoring area of ​​the monitoring camera for a period of time. After determining the first monitoring object, this embodiment traces back to the reference video frame before the video frame with the current timestamp in the monitoring video stream, and takes the video frame in which the first monitoring object is first identified in the reference video frame as the target video frame. This can determine the position change of the first reference object in the monitoring area, and provide an effective data basis for subsequent target tracking and monitoring path planning.

[0024] Specifically, those skilled in the art may select an image segmentation method as needed to obtain a first feature map corresponding to the first monitored object from a target video frame, and no further restrictions are imposed herein.

[0025] Step S20, input the first feature map into the global channel perception module to obtain the global perception weight, and input the first feature map into the local space perception module to obtain the local space perception weight.

[0026] It can be understood that this embodiment inputs the first feature map corresponding to the first monitored object into the global channel perception module and the local space perception module respectively, so as to extract the multi-scale detail features (global perception weight and local space perception weight) of the first monitored object in the current imaging environment. In this way, under the conditions of low resolution and complex environmental factors (light changes, multiple occlusions, and fast movement of the monitored object), the first monitored object can still be accurately tracked based on the multi-scale detail features, thereby effectively performing abnormal monitoring of the target monitored object.

[0027] Specifically, in some embodiments, the global channel perception module includes a plurality of first convolutional layers, a global average pooling layer, a ReLU activation function layer and a first Sigmoid activation function layer. Figure 1 Step S20 of inputting the first feature map into the global channel perception module to obtain the global perception weight includes but is not limited to the following steps: Step S21, inputting the features obtained by convolving the first feature map through the first convolutional layer into the global average pooling layer for pooling processing to obtain a first intermediate feature map; Step S22, inputting the features obtained by convolving the first intermediate feature map through the first convolutional layer into the ReLU activation function layer for activation to obtain a second intermediate feature map; Step S23, inputting the features obtained by convolving the second intermediate feature map through the first convolutional layer into the first Sigmoid activation function layer for activation to obtain a first activation result; Step S24, adding the product result of the first activation result and the first feature map to the first feature map to obtain a global perception weight.

[0028] Specifically, the global channel perception module of this embodiment uses an adaptive channel attention mechanism to extract the global channel features of the first feature map, that is, the global perception weight. The global channel perception module specifically includes multiple first convolutional layers, global average pooling layers, ReLU activation function layers and first Sigmoid activation function layers. Each first convolutional layer is a channel group convolutional layer. Different first convolutional layers are preset with corresponding reduction rates. Different reduction rates will result in different output channel sizes of the corresponding output feature maps. Different first convolutional layers are independent of each other, so that the global channel perception module can be more focused on learning the features of different channel groups. The network layers in the global channel perception module are connected through residual connections to enhance the feature extraction capability of the global channel perception module while reducing the amount of calculation.

[0029] It should be noted that, in this embodiment, steps S21 to S24 implement inputting the first feature map into the global channel perception module, and obtaining the global perception weight can be obtained according to the following formula: F_g(X) = X + σ(W2 * GConv(ReLU(W1 * GAP(X) + b1)) + b2) * X; Among them, F_g(X) is the global perception weight, X is the first feature map of the input, σ is the first Sigmoid activation function layer, W1 and W2 are learnable weight matrices, GConv() is the first convolutional layer, GAP(X) is, b1 and b2 are learnable biases, and ReLU() is the ReLU activation function layer.

[0030] It can be understood that in this embodiment, the feature obtained by convolving the first feature map through the first convolution layer is input into the global average pooling layer for pooling processing to obtain a first intermediate feature map; the feature obtained by convolving the first intermediate feature map through the first convolution layer is input into the ReLU activation function layer for activation to obtain a second intermediate feature map; the feature obtained by convolving the second intermediate feature map through the first convolution layer is input into the first Sigmoid activation function layer for activation to obtain a first activation result; the product result of the first activation result and the first feature map is added to the first feature map to obtain the global perception weight, thereby providing an effective data basis for subsequent acquisition of accurate target feature maps.

[0031] Specifically, in some embodiments, the local space perception module includes a plurality of second convolutional layers and a second Sigmoid activation function layer, and the convolution kernels of the second convolutional layers are different from each other. Figure 1 Step S20 of inputting the first feature map into the local space perception module to obtain the local space perception weight includes but is not limited to the following steps: Step S25, inputting the first feature map into each second convolutional layer for convolution processing to obtain each third intermediate feature map; Step S26, multiplying a splicing result obtained by splicing all the third intermediate feature maps by a preset weight matrix to obtain a first product result; Step S27, inputting the first product result into the second Sigmoid activation function layer for activation to obtain a second activation result; Step S28, adding the product result of the second activation result and the first feature map to the first feature map to obtain a local spatial perception weight.

[0032] Specifically, this embodiment uses a spatial attention mechanism to extract local spatial features of the first feature map, that is, local spatial perception weights. The local spatial perception module includes multiple second convolutional layers and second Sigmoid activation function layers. The convolution kernels of each second convolutional layer are different. For example, the number of second convolutional layers in this embodiment can be 3, and the specific number is not limited here. The convolution kernel size of the second convolutional layer A is 1x1, the convolution kernel size of the second convolutional layer B is 3x3, and the convolution kernel size of the second convolutional layer A is 5x5. The second convolutional layer B and the second convolutional layer C are hollow convolution layers. Hollow convolution is introduced to increase the receptive field while maintaining computational efficiency.

[0033] Specifically, the steps of obtaining the local space perception weight corresponding to steps S25 to S28 of this embodiment are implemented according to the following formula: F_l(X) = X + σ(W3 * Concat(Conv1x1(X), DConv3x3(X, d=2), DConv5x5(X,d=3)) + b3) * X; Among them, F_l(X) is the local spatial perception weight, X is the first feature map of the input, σ is the second Sigmoid activation function layer, W3 is the learnable weight matrix, Conv1x1(X) is the second convolutional layer A, DConv3x3(X, d=2) is the second convolutional layer B, DConv5x5(X, d=3) is the second convolutional layer C, d is the expansion rate, b3 is the learnable bias, and Concat() is the feature concatenation operation.

[0034] It can be understood that this embodiment obtains each third intermediate feature map by inputting the first feature map into each second convolution layer for convolution processing; multiplying the splicing result obtained by splicing all the third intermediate feature maps by a preset weight matrix to obtain a first product result; inputting the first product result into the second Sigmoid activation function layer for activation to obtain a second activation result; adding the product result of the second activation result and the first feature map to the first feature map to obtain a local spatial perception weight, which can provide an effective data basis for the subsequent acquisition of accurate target feature maps.

[0035] Specifically, in some embodiments, Figure 1 Step S30 of inputting the first feature map into the local space perception module to obtain the local space perception weight includes but is not limited to the following steps: Step S31, obtaining a first balance parameter, a second balance parameter and a third balance parameter; Step S32, multiplying the first balance parameter by the global perception weight to obtain a first product result; Step S33, multiplying the second balance parameter by the local space perception weight to obtain a second product result; Step S34, multiplying the third balance parameter by the first characteristic graph to obtain a third product result Step S35, adding the first product result, the second product result and the third product result to obtain a fused feature map; Step S36, passing the fused feature map through the third convolution layer, the fourth convolution layer and the fifth convolution layer respectively, to obtain a fourth intermediate feature map, a fifth intermediate feature map and a sixth intermediate feature map respectively, and the convolution kernels of the third convolution layer, the fourth convolution layer and the fifth convolution layer are different from each other; Step S37, upsampling the fourth intermediate feature map to obtain an upsampling result, and downsampling the sixth intermediate feature map to obtain a downsampling result; Step S38, concatenating the upsampling result, the fifth intermediate feature map and the downsampling result to obtain a first target feature map.

[0036] Specifically, the steps of obtaining the first target feature map corresponding to steps S31 to S38 of this embodiment are implemented according to the following formula: F_out = α * F_g(X) + β * F_l(X) + γ * X; F_final = Concat(Upsample(F_out_conv5), F_out_conv4, Downsample(F_out_conv3)); Among them, F_out is the fusion feature map, α is the first balance parameter, β is the second balance parameter, γ is the third balance parameter, F_final is the first target feature map, F_out_conv5 is the sixth intermediate feature map, F_out_conv4 is the fifth intermediate feature map, and F_out_conv3 is the fourth intermediate feature map. In this way, the features of the first feature image at different scales (local spatial perception weight, global perception weight) are fused, and the features of the fused features at different convolution scales are aggregated to obtain the first target feature map, so that the first target feature map has enough feature information of different scales, thereby providing an effective data basis for accurate target tracking of the first monitoring object.

[0037] Step S40, inputting the first target feature map and the video frame located after the generation timestamp of the target video frame into the target tracking module to obtain the target tracking result.

[0038] Specifically, in some embodiments, Figure 1 Step S40 includes but is not limited to the following steps: Step S41, determining an area in a target video frame that surrounds the first target feature map as a first template area, and determining the first template feature map of the first template area as a target template feature map, wherein the target template feature map includes the first target feature map; Step S42, determining a reference center position of the matching area based on a reference position of the second feature map of the first monitored object in a frame previous to the current video frame, and determining a reference size of the matching area based on a size and a moving speed of the second feature map, wherein the current video frame is one of all video frames located after a generation timestamp of the target video frame; Step S43, extracting a matching feature map from the current video frame based on the reference center position and the reference size, wherein the size of the matching feature map is larger than the size of the target template feature map; Step S44, calculating the similarity between each position in the target template feature map and the matching feature map, determining the position with the largest similarity in the matching feature map as the target center position where the first monitored object is located in the current video frame, and determining the first tracking result of the current video frame based on the target center position and the target template feature map; Specifically, this embodiment determines a target template feature map, which is used to identify, retrieve, match and record each frame of a surveillance video stream in which a first surveillance object is detected within a period of time, thereby forming a motion path of the first surveillance object, providing an effective data basis for subsequent construction of a surveillance path for the first surveillance object.

[0039] Specifically, the matching feature map of the present embodiment is used to match the target template feature map, based on the center position of the first monitored object in the previous frame of the current video frame, and the target center position of the first monitored object in the current video frame is determined based on the matching result with the target template feature map. The matching area corresponding to the matching feature map of the present embodiment is adjusted at any time according to the moving speed of the first monitored object in the current video stream and the actual size, so as to ensure the accuracy of the target tracking result.

[0040] Specifically, before performing the matching operation between the target template feature map and the matching feature map, it is necessary to first determine the matching area and the corresponding matching feature map, and determining the matching area requires determining the reference center position. In this embodiment, the center position of each video frame is used as the reference center position of the next video frame, and the movement direction of the first monitored object is predicted in combination with the actual path conditions of the monitored area. The orientation of the monitoring camera is adjusted based on the movement direction, and the position range of the first monitored object that is most likely to appear in the monitored area in the next frame is selected as the matching area by using the reference center position and the predicted movement direction, and the matching feature map of the matching area is extracted. The matching feature map here is obtained from the video frame obtained by the monitoring camera after adjusting the orientation. Compared with the existing operation of performing sequential full-pixel feature matching using the target template feature map and the matching feature map of a fixed range in the monitoring screen of the monitoring area with a fixed viewing angle and the matching feature map of a fixed range, this embodiment can improve the matching efficiency while improving the matching success rate.

[0041] Specifically, this embodiment also adjusts the size of the matching area by confirming the size of the first monitored object.

[0042] Specifically, this embodiment calculates the similarity between each position in the target template feature map and the matching feature map, and determines the position with the largest similarity in the matching feature map as the target center position where the first monitored object is located in the current video frame, according to the following formula: L_ final = ReLU(L(m,n) / (||φ(m)|| * ||φ(n)||)); L(m,n)=F.conv2d(φ(n), φ(m)); φ(x)=ResNet18_features(x); Among them, L_final is the similarity, m is the target template feature map, n is the matching feature map, φ(x) is the ResNet feature extraction network, and L(m,n) is the convolution cross-correlation operation.

[0043] Specifically, step S44 determines the first tracking result of the current video frame based on the target center position and the target template feature map, including but not limited to the following steps: Step S441, performing global maximum pooling processing on the target template feature map to obtain a seventh intermediate feature map; Step S442, increasing the spatial dimension of the seventh intermediate feature map to obtain a new seventh intermediate feature map, and splicing the new seventh intermediate feature map with the matching feature map to obtain a splicing result, wherein the new seventh intermediate feature map has the same size as the matching feature map; Step S443: input the splicing result and the target middle position into the MLP structure to obtain a first tracking result.

[0044] Specifically, the MLP structure of this embodiment includes 3 fully connected layers and a Sigmoid activation function layer. Each fully connected layer is residually connected through a ReLU activation function layer and a batch normalization layer. The last fully connected layer is connected to the Sigmoid activation function layer to convert the learned linear values ​​into weights between 0 and 1, which are used to emphasize important features and suppress unimportant features. The final refined features can better distinguish between targets and interferences, especially in low-resolution or complex backgrounds, thereby improving the tracking performance under low-light or low-resolution camera conditions and reducing tracking drift caused by background interference (such as similar objects).

[0045] In addition, in a specific embodiment, the property monitoring system includes a server, and a monitoring video stream is displayed in a monitoring display module of a visual interface of the server. After performing step S44 of determining the first tracking result of the current video frame based on the target center position and the target template feature map, the property abnormality monitoring method of the embodiment of the present application also includes but is not limited to the following steps: Step S444, generating a bounding box according to the second monitoring object selected by the user in the monitoring display module, and determining the image of the bounding box in the current video frame as a third feature map; Step S445, when it is detected that the second monitored object is the same object as the first monitored object, the third feature map is input into the global channel perception module, the local space perception module and the feature fusion module to obtain a second target feature map; Step S446, determining the area surrounding the second target feature map in the current video frame as the second template area, and extracting the second template feature map of the second template area; Step S447, determining the time interval between generating the first template feature graph and generating the second template feature graph; Step S448, determining a new target template feature map based on the exponential moving average algorithm, the time interval, the first template feature map and the second template feature map.

[0046] It is understandable that the first monitored object may change its appearance within a period of time. For example, if the first monitored object is a person, the clothes will be changed; if it is a car, the license plate will be changed. These situations will affect the accuracy of the target tracking results. Based on this, each time a new monitored object (the second monitored object) is determined in this embodiment, it will match and identify the features of the currently tracked monitored object (the first monitored object). When it is detected that the second monitored object is the same object as the first monitored object, the features of the second monitored object after the change of clothes are extracted again to obtain a second target feature map, and then the area surrounding the second target feature map in the current video frame is determined as the second template area, and the second template feature map of the second template area is extracted. Based on the exponential moving average algorithm, the time interval, the first template feature map and the second template feature map, a new target template feature map is determined, and the new and old templates are integrated to ensure the stability of the detail features, that is, the target template feature map is updated in a timely manner, thereby effectively ensuring the accuracy of the subsequent target tracking results.

[0047] Step S45 : generating a target tracking result based on the first tracking results of all video frames located after the generation timestamp of the target video frame.

[0048] It can be understood that since the first tracking result of this embodiment is the tracking result corresponding to one of the video frames of the monitoring video stream that identifies the first monitored object, the first tracking results of all video frames located after the generation timestamp of the target video frame are fused in chronological order to generate a target tracking result, which can provide an effective data basis for subsequent abnormal monitoring.

[0049] Step S50, when an abnormality is detected in the first monitored object based on the target tracking result, a target tracking path is constructed based on the target tracking result, and an alarm information generated based on the target tracking result and the target tracking path is sent to a target account associated with the property monitoring system.

[0050] Specifically, the target account in this embodiment may be a contact account of a system server administrator or a contact account of an owner corresponding to the monitoring area.

[0051] Specifically, the target tracking path of this embodiment is a predicted motion path of the first monitored object with an abnormality.

[0052] It can be understood that when an abnormality is detected in the first monitored object based on the target tracking results, the system constructs a target tracking path based on the target tracking results, generates alarm information based on the target tracking results and the target tracking path, and sends the alarm information to the target account associated with the property monitoring system, which enables timely abnormal monitoring of the first monitored object and response measures to ensure the safety of the owner.

[0053] like Figure 2 As shown, Figure 2 2 is a structural diagram of a control device provided in an embodiment of the present application. The present invention also provides a control device 200, including: The processor 210 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; The memory 220 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 220 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 220, and the processor 210 calls and executes the property abnormality monitoring method of the embodiment of this application; Input / output interface 230, used to implement information input and output; Communication interface 240, used to realize communication interaction between the apparatus and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); bus 250 , which transmits information between various components of the device (e.g., processor 210 , memory 220 , input / output interface 230 , and communication interface 240 ); The processor 210 , the memory 220 , the input / output interface 230 , and the communication interface 240 are connected to each other in communication within the device via the bus 250 .

[0054] In addition, an embodiment of the present application further provides an electronic device, including the control device 200 of the above embodiment.

[0055] In addition, an embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the above-mentioned property abnormality monitoring method is implemented.

[0056] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0057] It will be appreciated by those skilled in the art that all or some of the steps and systems in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0058] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A property abnormality monitoring method, characterized in that: Applied to a property monitoring system, the property monitoring system is preset with a monitoring camera, a global channel perception module, a local space perception module, a feature fusion module and a target tracking module, and the method includes: Acquire a surveillance video stream from the surveillance camera, and when a first surveillance object is determined from a video frame of the surveillance video stream, determine a video frame in which the first surveillance object is first identified as a target video frame, and acquire a first feature map corresponding to the first surveillance object in the target video frame; Inputting the first feature map into the global channel perception module to obtain a global perception weight, and inputting the first feature map into the local space perception module to obtain a local space perception weight; Inputting the global perception weight and the local spatial perception weight into the feature fusion module to obtain an enhanced first target feature map; Inputting the first target feature map and the video frame located after the generation timestamp of the target video frame into the target tracking module to obtain a target tracking result; When an abnormality is detected in the first monitored object based on the target tracking result, a target tracking path is constructed based on the target tracking result, and an alarm information generated based on the target tracking result and the target tracking path is sent to the target account associated with the property monitoring system.

2. The property abnormality monitoring method according to claim 1, characterized in that: The global channel perception module includes a plurality of first convolutional layers, a global average pooling layer, a ReLU activation function layer and a first Sigmoid activation function layer. The first feature map is input into the global channel perception module to obtain a global perception weight, including: Inputting the features obtained by convolving the first feature map through the first convolutional layer into the global average pooling layer for pooling processing to obtain a first intermediate feature map; Inputting the features obtained by convolving the first intermediate feature map through the first convolutional layer into the ReLU activation function layer for activation to obtain a second intermediate feature map; Inputting the feature obtained by convolving the second intermediate feature map through the first convolution layer into the first Sigmoid activation function layer for activation to obtain a first activation result; The product result of the first activation result and the first feature map is added to the first feature map to obtain the global perception weight.

3. The property abnormality monitoring method according to claim 1, characterized in that: The local space perception module includes a plurality of second convolutional layers and a second Sigmoid activation function layer, the convolution kernels of each of the second convolutional layers are different from each other, the first feature map is input into the local space perception module, and the local space perception weight is obtained, including: Inputting the first feature maps into each of the second convolutional layers for convolution processing respectively to obtain each third intermediate feature map; Multiplying a splicing result obtained by splicing all the third intermediate feature maps by a preset weight matrix to obtain a first product result; Inputting the first product result into the second Sigmoid activation function layer for activation to obtain a second activation result; The product result of the second activation result and the first feature map is added to the first feature map to obtain the local spatial perception weight.

4. The property abnormality monitoring method according to any one of claims 1 to 3, characterized in that: The global perception weight inputs the global perception weight and the local spatial perception weight into the feature fusion module to obtain an enhanced first target feature map, including: Obtaining a first balance parameter, a second balance parameter, and a third balance parameter; Multiplying the first balancing parameter by the global perception weight to obtain a first product result; Multiplying the second balance parameter by the local space perception weight to obtain a second product result; multiplying the third balance parameter by the first characteristic map to obtain a third product result; Adding the first product result, the second product result and the third product result to obtain a fused feature map; The fused feature map is passed through a third convolution layer, a fourth convolution layer, and a fifth convolution layer, respectively, to obtain a fourth intermediate feature map, a fifth intermediate feature map, and a sixth intermediate feature map, respectively, wherein the convolution kernels of the third convolution layer, the fourth convolution layer, and the fifth convolution layer are different from each other; Upsampling the fourth intermediate feature map to obtain an upsampling result, and downsampling the sixth intermediate feature map to obtain a downsampling result; The upsampling result, the fifth intermediate feature map and the downsampling result are concatenated to obtain the first target feature map.

5. The property abnormality monitoring method according to claim 1, characterized in that: Inputting the first target feature map and the video frame located after the generation timestamp of the target video frame into the target tracking module to obtain a target tracking result, including: Determine an area in the target video frame that surrounds the first target feature map as a first template area, and determine a first template feature map of the first template area as a target template feature map, wherein the target template feature map includes the first target feature map; Determining a reference center position of a matching area based on a reference position of a second feature map of the first monitored object in a frame previous to a current video frame, and determining a reference size of the matching area based on a size and a moving speed of the second feature map, wherein the current video frame is one of all video frames located after a generation timestamp of the target video frame; Extracting a matching feature map from the current video frame based on the reference center position and the reference size, wherein a size of the matching feature map is larger than a size of the target template feature map; Calculate the similarity between the target template feature map and each position in the matching feature map, determine the position with the largest similarity in the matching feature map as the target center position where the first monitored object is located in the current video frame, and determine the first tracking result of the current video frame based on the target center position and the target template feature map; The target tracking result is generated based on the first tracking results of all video frames located after the generation timestamp of the target video frame.

6. The property abnormality monitoring method according to claim 5, characterized in that: Determining a first tracking result of the current video frame based on the target center position and the target template feature map includes: Performing global maximum pooling processing on the target template feature map to obtain a seventh intermediate feature map; Increasing the spatial dimension of the seventh intermediate feature map to obtain a new seventh intermediate feature map, and splicing the new seventh intermediate feature map with the matching feature map to obtain a splicing result, wherein the new seventh intermediate feature map has the same size as the matching feature map; The splicing result and the target middle position are input into the MLP structure to obtain the first tracking result.

7. The property abnormality monitoring method according to claim 5, characterized in that: The property monitoring system includes a server, and the monitoring video stream is displayed in a monitoring display module of a visualization interface of the server. After determining a first tracking result of a current video frame based on the target center position and the template feature map, the method further includes: Generate a bounding box according to the second monitoring object selected by the user in the monitoring display module, and determine the image of the bounding box in the current video frame as a third feature map; When it is detected that the second monitored object is the same object as the first monitored object, the third feature map is input into the global channel perception module, the local space perception module and the feature fusion module to obtain a second target feature map; Determine an area surrounding the second target feature map in the current video frame as a second template area, and extract a second template feature map of the second template area; Determine a time interval between generating the first template feature graph and generating the second template feature graph; A new target template feature map is determined based on an exponential moving average algorithm, the time interval, the first template feature map, and the second template feature map.

8. A control device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the property abnormality monitoring method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: Comprising the control device as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the property abnormality monitoring method as described in any one of claims 1 to 7.

Images