Property monitoring method, device and equipment based on improved YOLOV5, and storage medium

Abstract

The invention provides an improved YOLOV5-based property monitoring method and device, equipment and a storage medium, and the method comprises the steps: inputting a target video frame into an input convolution layer to obtain an initial feature map, and inputting the N + 1th layer of the extracted feature map into an Nth shallow fusion module when the number of layers of a trunk convolution module is greater than or equal to 2; inputting the feature fusion images of the deep fusion module and the shallow fusion module into a prediction module to obtain a target detection result; object position information and object action information are determined based on a target object frame generated by the target detection result, and then a monitoring prediction result is determined. According to the technical scheme of the embodiment of the invention, the shallow layer features of the same layer of the multi-layer feature map can be fused in the shallow layer fusion module, more detailed shallow layer feature information is extracted from the video frame, the detection accuracy of a small target is improved, the range accuracy of a target object frame is improved, and the detection efficiency is improved. Automatic early warning of abnormal conditions is achieved through high-reliability object recognition, and the monitoring reliability is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a property monitoring method, device, equipment, and storage medium based on improved YOLOV5. Background Art

[0002] With the acceleration of urbanization and the continuous expansion of community scale, the problem of security monitoring in property management has become increasingly prominent. Especially in major issues related to residents' life safety and property safety, how to accurately and efficiently detect small target graphics, such as pedestrians and vehicles, has become the key to improving the level of community safety. Video surveillance systems mainly rely on frame-level image processing. At present, convolutional neural network technology has been introduced in property monitoring. The YOLOv5 model can perform real-time target detection in resource-constrained scenarios and is a commonly used model in property monitoring.

[0003] However, some property monitoring targets are usually small in the video screen, such as people or equipment that needs to be monitored. The conventional YOLOv5 model does not have an ideal detection effect on small targets. Not only is it easy to lose the target, but it may also generate a large target frame in the monitoring screen. Manual monitoring is still required, and manual warnings are issued when abnormalities occur. It is easy to make mistakes and omissions, and the reliability of monitoring cannot be guaranteed. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a property monitoring method, device, equipment, and storage medium based on improved YOLOV5, which can improve the detection accuracy of small targets, realize automatic early warning of abnormal situations, and improve monitoring reliability.

[0005] In a first aspect, an embodiment of the present invention provides a property monitoring method based on an improved YOLOV5, which is applied to a property monitoring system, wherein the property monitoring system is provided with an improved YOLOV5 model, wherein the improved YOLOV5 model includes an input convolution layer, a backbone network, a neck network and a detection head, wherein the backbone network is provided with a plurality of backbone convolution modules, wherein the neck network includes a plurality of shallow fusion modules and a deep fusion module, wherein the detection head includes a plurality of prediction modules, wherein the deep fusion module is connected to the backbone convolution module of the penultimate layer, wherein the shallow fusion module is connected to the remaining backbone convolution modules, wherein the backbone convolution module of the first layer is not connected to the shallow fusion module, wherein the shallow feature fusion module and the deep fusion module are each connected to one of the prediction modules, wherein the method includes:

[0006] Input the target video frame into the input convolution layer to obtain an initial feature map, input the initial feature map into the backbone network for feature extraction, wherein when the number of layers of the backbone convolution module is greater than or equal to 2, input the extracted feature map into the backbone convolution module of the next layer, and input the N+1th layer of the extracted feature map into the shallow fusion module of the Nth layer, wherein N is a positive integer;

[0007] Inputting the feature fusion graphs of the deep fusion module and the shallow fusion module into the corresponding prediction module, and obtaining the target detection result based on the feature prediction results output by each prediction module;

[0008] At least one target object frame is generated in the target video frame based on the target detection result, object position information is determined based on the target object frame, action recognition is performed in the target object frame to obtain object action information, and a monitoring prediction result is determined based on the object position information and the object action information.

[0009] According to some embodiments of the present invention, the multi-layer backbone convolution modules of the backbone network include a first convolution module, a second convolution module, a third convolution module, a fourth convolution module and a fifth convolution module, and the structures of the backbone convolution modules are the same. An attention module is also provided between every two layers of the backbone convolution modules, and the attention module is used to improve the channel attention weight and the spatial attention weight. The multi-layer shallow fusion module includes a first fusion module and a second fusion module, the second convolution module is connected to the first fusion module, the third convolution module is connected to the second fusion module, and the fourth convolution module is connected to the deep fusion module. The initial feature map is input into the backbone network for feature extraction, including:

[0010] Input the initial feature map into the first convolution module, extract the first trunk feature map through the first convolution module, input the first trunk feature map into the second convolution module after passing through the attention module;

[0011] Extracting a second main feature map through the second convolution module, inputting the second main feature map into the third convolution module after passing through the attention module, and inputting the second layer feature map of the second main feature map into the first fusion module;

[0012] Extracting a third main feature map through the third convolution module, inputting the third main feature map into the fourth convolution module after passing through the attention module, inputting the second layer feature map of the third main feature map into the first fusion module, and inputting the third layer feature map of the third main feature map into the second fusion module;

[0013] Extracting a fourth main feature map through the fourth convolution module, inputting the fourth main feature map into the fifth convolution module after passing through the attention module, inputting the second layer feature map of the fourth main feature map into the first fusion module, inputting the third layer feature map of the fourth main feature map into the second fusion module, and inputting the fourth main feature map into the deep fusion module;

[0014] A fifth trunk feature map is extracted through the fifth convolution module, a second layer feature map of the fifth trunk feature map is input into the first fusion module, and a third layer feature map of the fifth trunk feature map is input into the second fusion module.

[0015] According to some embodiments of the present invention, the neck network further includes an excitation squeezing module and a spatial feature refinement module, the fifth convolution module is connected to the excitation squeezing module, the spatial feature refinement module includes an average pooling layer, a maximum pooling layer, a first convolution layer, and a second convolution layer, the convolution kernel of the first convolution layer is 1×1, and the convolution kernel of the second convolution layer is 7×7, before the feature fusion maps of the deep fusion module and the shallow fusion module are input into the corresponding prediction module, the method further includes:

[0016] The weights of the feature channels of the fifth trunk feature map are adjusted by the excitation squeezing module to obtain a fused input feature map;

[0017] Inputting the fused input feature map into the average pooling layer, the maximum pooling layer and the first convolutional layer respectively;

[0018] Performing feature splicing based on the average pooling feature map output by the average pooling layer, the maximum pooling feature map output by the maximum pooling layer, and the first convolution feature map output by the first convolution layer to obtain a spliced ​​feature map;

[0019] Inputting the feature splicing map into the second convolutional layer to obtain a second convolutional feature map output by the second convolutional layer;

[0020] The second convolutional feature map is input into a preset activation function to generate a feature weight map, and the spatial feature map is obtained by element-by-element multiplication of the feature weight map and the fusion input feature map, and the spatial feature map is input into the deep fusion module.

[0021] According to some embodiments of the present invention, the deep fusion module includes a sixth convolution module, a first upsampling layer and a third fusion layer, the second fusion module includes a seventh convolution module, a second upsampling layer and a second fusion layer, the first fusion module includes an eighth convolution module, a third upsampling layer and a first fusion layer, the structure of the sixth convolution module, the seventh convolution module and the eighth convolution module is the same as the trunk convolution module, and the feature fusion map of each of the deep fusion module and the shallow fusion module is input into the corresponding prediction module, including:

[0022] After the spatial feature map passes through the sixth convolution module and the first upsampling layer in sequence, the spatial feature map is fused with the fourth trunk feature map in the third fusion layer to form the deep fusion feature map;

[0023] After the deep fusion feature map passes through the seventh convolution module and the second upsampling layer in sequence, the second fusion layer is fused with the third layer feature map of the fifth trunk feature map, the third layer feature map of the fourth trunk feature map, and the third layer feature map of the third trunk feature map into a second shallow fusion feature map;

[0024] After the second shallow fusion feature map passes through the eighth convolution module and the third upsampling layer in sequence, the first fusion layer is fused with the second layer feature map of the fifth trunk feature map, the second layer feature map of the fourth trunk feature map, the second layer feature map of the third trunk feature map, and the second layer feature map of the second trunk feature map into a first shallow fusion feature map;

[0025] The spatial feature map, the deep fusion feature map, the second shallow fusion feature map and the first shallow fusion feature map are input into the corresponding prediction module.

[0026] According to some embodiments of the present invention, determining a monitoring prediction result based on the object position information and the object action information includes:

[0027] Extracting an object image from the target object frame, and performing image recognition based on the object image to determine the object type, wherein the object type is a pedestrian, a vehicle, or property equipment;

[0028] Determining the monitoring prediction result based on the object type and the object action information;

[0029] The monitoring prediction results are saved based on the object type classification.

[0030] According to some embodiments of the present invention, the property monitoring system includes a plurality of registered identity information, each of the registered identity information is pre-associated with at least one object permission, each of the object permission is associated with at least one optional behavior information, and when the object type is a pedestrian or a vehicle, the monitoring prediction result is determined based on the object type and the object action information, including:

[0031] Determining at least one candidate authority from preset authorities based on the object location information, wherein the property monitoring system includes a plurality of preset authorities, each of the preset authorities being associated with a plurality of location information;

[0032] Determining object identity information based on the object image, wherein when the object type is a pedestrian, the object identity information is obtained based on face recognition performed on the object image, and when the object type is a vehicle, the object identity information is obtained based on license plate recognition performed on the object image;

[0033] Determine an intersection of the candidate permission and the object permission associated with the object identity information as a first permission;

[0034] Determine object behavior information based on the object identity information and the object action information, and determine a second permission based on the object behavior information;

[0035] When the second permission does not belong to the first permission, generating a first warning message;

[0036] A first event record is generated based on the object identity information, the second authority, and the first alarm information.

[0037] According to some embodiments of the present invention, when the object type is property equipment, determining the monitoring prediction result based on the object type and the object action information includes:

[0038] determining a target device type based on the object image, wherein the target device type is associated with a plurality of selectable device actions;

[0039] determining a target device action based on the object action information, and generating second alarm information when the target device action does not belong to the optional device action;

[0040] A second event record is generated based on the target device type, the target device action, and the second alarm information.

[0041] In the second aspect, an embodiment of the present invention provides a property monitoring device based on improved YOLOV5, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the property monitoring method based on improved YOLOV5 as described in the first aspect above.

[0042] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a property monitoring device based on the improved YOLOV5 as described in the second aspect above.

[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the property monitoring method based on the improved YOLOV5 as described in the first aspect above.

[0044] According to the property monitoring method based on improved YOLOV5 according to the embodiment of the present invention, there are at least the following beneficial effects: inputting the target video frame into the input convolution layer to obtain an initial feature map, inputting the initial feature map into the backbone network for feature extraction, wherein, when the number of layers of the backbone convolution module is greater than or equal to 2, inputting the extracted feature map into the next layer of the backbone convolution module, and inputting the N+1th layer of the extracted feature map into the Nth layer of the shallow fusion module, wherein N is a positive integer; inputting the feature fusion maps of the deep fusion module and the shallow fusion module respectively into the corresponding prediction modules, and obtaining the target detection result based on the feature prediction results output by each of the prediction modules; generating at least one target object frame in the target video frame based on the target detection result, determining the object position information based on the target object frame, performing action recognition in the target object frame to obtain the object action information, and determining the monitoring prediction result based on the object position information and the object action information. According to the technical solution of the embodiment of the present invention, the shallow features of the same layer of multi-layer feature maps can be fused in the shallow fusion module, and more detailed shallow feature information can be extracted from the video frame, thereby improving the detection accuracy of small targets and the range accuracy of the target object frame. Automatic early warning of abnormal situations is achieved through high-reliability object recognition, thereby improving monitoring reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a network diagram of an improved YOLOV5 model provided by an embodiment of the present invention;

[0046] Figure 2 is a flow chart of a property monitoring method based on improved YOLOV5 provided by another embodiment of the present invention;

[0047] Figure 3 is a complete flow chart of a property monitoring method based on improved YOLOV5 provided by another embodiment of the present invention;

[0048] Figure 4 It is a structural diagram of a property monitoring device based on improved YOLOV5 provided in another embodiment of the present invention. DETAILED DESCRIPTION

[0049] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0050] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0051] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0052] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0053] The embodiment of the present invention provides a property monitoring method, device, equipment, and storage medium based on improved YOLOV5, wherein the property monitoring method based on improved YOLOV5 includes: inputting the target video frame into the input convolution layer to obtain an initial feature map, inputting the initial feature map into the backbone network for feature extraction, wherein when the number of layers of the backbone convolution module is greater than or equal to 2, inputting the extracted feature map into the next layer of the backbone convolution module, and inputting the N+1th layer of the extracted feature map into the Nth layer of the shallow fusion module, wherein N is a positive integer; inputting the feature fusion maps of the deep fusion module and the shallow fusion module respectively into the corresponding prediction module, and obtaining the target detection result based on the feature prediction results output by each of the prediction modules; generating at least one target object frame in the target video frame based on the target detection result, determining the object position information based on the target object frame, performing action recognition in the target object frame to obtain the object action information, and determining the monitoring prediction result based on the object position information and the object action information. According to the technical solution of the embodiment of the present invention, the shallow features of the same layer of multi-layer feature maps can be fused in the shallow fusion module, and more detailed shallow feature information can be extracted from the video frame, thereby improving the detection accuracy of small targets and the range accuracy of the target object frame. Automatic early warning of abnormal situations is achieved through high-reliability object recognition, thereby improving monitoring reliability.

[0054] First, refer to Figure 1 , Figure 1 A network schematic diagram of an improved YOLOV5 model provided for an embodiment of the present invention, wherein the improved YOLOV5 model includes an input convolutional layer, a backbone network, a neck network and a detection head, wherein the backbone network is provided with multiple layers of backbone convolutional modules, the neck network includes multiple layers of shallow fusion modules and a layer of deep fusion module, the detection head includes multiple prediction modules, the deep fusion module is connected to the second-to-last layer backbone convolutional module, the shallow fusion module is connected to the remaining backbone convolutional modules, the first layer backbone convolutional module is not connected to the shallow fusion module, and the shallow feature fusion module and the deep fusion module are each connected to a prediction module.

[0055] It should be noted that the backbone network of this embodiment adopts the common CSPDarknet53 network, which enhances the feature extraction capability and reduces the amount of calculation through residual units and dense connections. The specific structure of the backbone network includes multiple convolutional layers, activation functions and pooling layers. The neck network adopts FPN (Feature Pyramid Network) + PAN (Path Aggregation Network) network, which can realize multi-scale feature fusion and capture targets of different scales. The neck network is a part connecting the backbone network and the detection head. FPN helps the model capture targets of different scales through multi-scale feature fusion; PAN further optimizes the transmission and fusion of feature information. Through these structures, the model can extract rich feature information at different scales, thereby improving the detection effect. The detection head can be used to predict the category and confidence to which the target object belongs. The detection head of this embodiment includes multiple prediction modules (YOLO HEAD). Those skilled in the art are familiar with the operating mechanism of YOLOHEAD of YOLOV5, and the specific prediction principle is not described in detail here.

[0056] In addition, the multi-layer backbone convolution module of the backbone network includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module and a fifth convolution module. The structures of each backbone convolution module are the same. An attention module is also arranged between every two layers of backbone convolution modules. The attention module is used to improve the channel attention weight and the spatial attention weight. The multi-layer shallow fusion module includes a first fusion module and a second fusion module. The second convolution module is connected to the first fusion module, the third convolution module is connected to the second fusion module, and the fourth convolution module is connected to the deep fusion module.

[0057] It should be noted that if Figure 1 As shown in Figure 1, the backbone convolution module is a common CBS module in deep convolutional networks, which includes a convolutional layer (Conv), a batch layer (BN), and a SiLu (Sigmoid+ReLu) activation function in sequence. The structure of each backbone convolution module is the same, as shown in Figure 1. Figure 1 As shown, the first convolution module is CBS1, the second convolution module is CBS2, and so on. For the convenience of description, this embodiment takes a 5-layer backbone convolution module as an example. Among them, the output of CBS5 is connected to the Spatial Pyramid Pooling-Fast (SPPF) layer, and the specific principle is not repeated here.

[0058] It should be noted that this embodiment introduces an attention mechanism in each backbone convolution module. The attention module of this embodiment is an Efficient Channel and Spatial Attention (ECSA) module. The ECSA module of this embodiment can increase the channel attention weight and the spatial attention weight in turn, filter out redundant semantic information, make the semantics of small targets more prominent, and improve the detection accuracy.

[0059] In addition, the neck network also includes an excitation squeezing module and a spatial feature refinement module. The fifth convolution module is connected to the excitation squeezing module. The spatial feature refinement module includes an average pooling layer, a maximum pooling layer, a first convolution layer and a second convolution layer. The convolution kernel of the first convolution layer is 1×1, and the convolution kernel of the second convolution layer is 7×7.

[0060] It should be noted that the Squeeze AND Excitation (SE) module is a common module in YOLOV5. It can dynamically adjust the weights of each channel through "squeeze" and "excitation" operations to highlight important features. The specific process is as follows: The squeeze module performs global average pooling on the feature map to generate a global feature vector at the channel level. The excitation module performs a nonlinear transformation on the global feature vector through a pair of fully connected layers to generate the weight of each channel, and then redistributes these weights to each channel to enhance important features.

[0061] In addition, the deep fusion module includes a sixth convolution module, a first upsampling layer and a third fusion layer, the second fusion module includes a seventh convolution module, a second upsampling layer and a second fusion layer, the first fusion module includes an eighth convolution module, a third upsampling layer and a first fusion layer, and the structures of the sixth convolution module, the seventh convolution module and the eighth convolution module are the same as the trunk convolution module.

[0062] It should be noted that this embodiment adopts a 5-layer backbone convolution module, a 2-layer shallow fusion module, and a 1-layer deep fusion module. The shallow fusion module includes a CBS module, an upsampling layer, and a fusion layer in sequence. For example, the shallow fusion module 1 includes the eighth convolution module (CBS8), the third upsampling layer (Upsample3), and the first fusion layer (Concat1) in sequence; the shallow fusion module 2 includes the seventh convolution module (CBS7), the second upsampling layer (Upsample2), and the second fusion layer (Concat2) in sequence, and the deep fusion module includes the sixth convolution module (CBS6), the first upsampling layer (Upsample1), and the third fusion layer (Concat3). Among them, Concat1 is connected to the YOLO HEAD 1 of the detection head, Concat2 is connected to the YOLOHEAD 2 of the detection head, and Concat3 is connected to the YOLO HEAD 3 of the detection head.

[0063] The following is based on the Figure 1 The improved YOLOV5 model shown is used to further illustrate the technical solution of the embodiment of the present invention.

[0064] Reference Figure 2 , Figure 2 A flowchart of a property monitoring method based on an improved YOLOV5 is provided in an embodiment of the present invention. The property monitoring method based on the improved YOLOV5 includes but is not limited to the following steps:

[0065] S11, input the target video frame into the input convolution layer to obtain an initial feature map, input the initial feature map into the backbone network for feature extraction, wherein, when the number of layers of the backbone convolution module is greater than or equal to 2, input the extracted feature map into the next layer of backbone convolution module, and input the N+1th layer of the extracted feature map into the Nth layer of shallow fusion module, wherein N is a positive integer;

[0066] S12, inputting the feature fusion graphs of the deep fusion module and the shallow fusion module into the corresponding prediction modules, and obtaining the target detection result based on the feature prediction results output by each prediction module;

[0067] S13, generating at least one target object frame in the target video frame based on the target detection result, determining the object position information based on the target object frame, performing action recognition in the target object frame to obtain the object action information, and determining the monitoring prediction result based on the object position information and the object action information.

[0068] It is worth noting that after acquiring the target video frame, the initial feature map is provided by the input convolution layer. The input convolution layer can adopt the same CBS structure as the backbone convolution module, which will not be elaborated here.

[0069] It should be noted that after obtaining the initial feature map, Figure 1 CBS1 to CBS5 in the backbone network shown perform feature extraction in sequence, and the feature dimension increases layer by layer. The number of layers of feature maps extracted in each layer depends on the number of convolution kernels. The number of layers of feature maps extracted by the backbone convolution module of this embodiment does not exceed the number of layers. For example, the feature map extracted by CBS2 does not exceed 2 layers, and CBS5 does not exceed 5 layers. The number of layers of the feature map can be adjusted by adjusting the number of convolution kernels.

[0070] For example, Figure 1 As shown, two shallow fusion modules are included. The number of features extracted by CBS1 is small, so it is not used for feature fusion. The feature map extracted by CBS2 includes at least two layers, and the second layer feature map is input to the shallow fusion module 1 of the first layer. CBS2 does not include the third layer feature map, so the feature map is not input to the shallow fusion module 2; the feature map extracted by CBS3 includes at least 3 layers, and the second layer feature map is input to the shallow fusion module 1, and the second layer is input to the shallow fusion module 2, and so on for CBS4 and CBS5.

[0071] It should be noted that this embodiment uses a shallow fusion module to fuse shallow features extracted by multiple backbone convolution modules, which can extract more detailed shallow information from the image. Shallow information usually involves smaller targets. After the feature fusion map obtained by the shallow fusion module is input into the corresponding prediction module, the detection performance of small targets can be improved, making the prediction module more adaptable to the shape, texture and other features of small targets. Figure 1 The YOLO HEAD 1 and YOLOHEAD 2 shown are better able to make object predictions based on features such as shape and texture of small objects.

[0072] It should be noted that the deep fusion module of this embodiment does not adopt the mechanism of the shallow fusion module, but fuses high-dimensional features, which can extract deep semantics in the image and improve the accuracy of image recognition.

[0073] It should be noted that the Nth layer of shallow fusion network in this embodiment is used to represent the number of layers of features. For example, the first layer of shallow fusion network corresponds to Figure 1 The Concat1 shown is used to fuse the second layer of shallow features, and the second layer of shallow fusion network corresponds to Concat2, which is used to fuse the third layer of shallow features.

[0074] It should be noted that the prediction module in this embodiment is Figure 1The YOLO HEAD shown predicts the second layer features of each feature map fused by the shallow fusion module 1 through YOLO HEAD 1, predicts the third layer features of each feature map fused by the shallow fusion module 2 through YOLO HEAD 2, predicts the deep features fused by the deep fusion module through YOLO HEAD 4, and predicts the backbone feature map output by CBS5 through YOLO HEAD 5. The target detection result is determined according to the prediction results of each YOLOHEAD.

[0075] It should be noted that the target detection result of this embodiment may include multiple target objects, that is, multiple target object frames can be generated in the video frame. In the property field, the objects of the target object frame can be pedestrians, vehicles, property equipment, etc. After determining the target object frame, the object position information is the position framed by the target object frame. Since the location of property monitoring is fixed, the object position information can be determined based on the target object frame, which will not be elaborated here. Object action information is used to characterize the action of the object, such as pedestrians walking, vehicles parked or in a form, property equipment operating normally or malfunctioning (such as a broken water pipe). The action type of each object can be pre-trained, and the specific action recognition will not be elaborated here. This embodiment ensures the accuracy of action recognition and object recognition while realizing small target detection, thereby improving the accuracy of property monitoring.

[0076] It should be noted that the monitoring prediction results can be determined based on specific actions. For example, if a pedestrian is detected entering an unauthorized area, or a vehicle is detected to be parked in an unauthorized area for a long time, an alarm message is generated and sent to the associated property management personnel, so that the property management personnel can go to the site in time to perform relevant management operations. For another example, the object of property monitoring is property equipment, such as water pipes and electric meters in a residential area. Through small target detection technology, it is determined that cracks appear in the water pipes, and water seepage occurs at the cracks. The monitoring prediction results are determined as the risk of rupture, and the corresponding alarm information is sent to the associated property management personnel. Those skilled in the art are familiar with how to set the accounts of associated property management personnel for different alarm events so that their terminal devices can receive alarm information.

[0077] In addition, in one embodiment, referring to Figure 3 Step S11 specifically includes but is not limited to the following steps:

[0078] S111, inputting the initial feature map into the first convolution module, extracting the first trunk feature map through the first convolution module, inputting the first trunk feature map into the second convolution module after passing through the attention module;

[0079] S112, extracting a second main feature map through the second convolution module, inputting the second main feature map into the third convolution module after passing through the attention module, and inputting the second layer feature map of the second main feature map into the first fusion module;

[0080] S113, extracting a third main feature map through a third convolution module, inputting the third main feature map into a fourth convolution module after passing through an attention module, inputting the second layer feature map of the third main feature map into a first fusion module, and inputting the third layer feature map of the third main feature map into a second fusion module;

[0081] S114, extracting a fourth trunk feature map through a fourth convolution module, inputting the fourth trunk feature map into a fifth convolution module after passing through an attention module, inputting the second layer feature map of the fourth trunk feature map into a first fusion module, inputting the third layer feature map of the fourth trunk feature map into a second fusion module, and inputting the fourth trunk feature map into a deep fusion module;

[0082] S115, extracting a fifth trunk feature map through a fifth convolution module, inputting a second layer feature map of the fifth trunk feature map into a first fusion module, and inputting a third layer feature map of the fifth trunk feature map into a second fusion module.

[0083] It should be noted that the attention module of this embodiment is an ECSA module, which can improve the channel attention weight and spatial attention weight. Taking the first backbone feature map X as an example, after X is input into the ECSA module, the channel attention is calculated by the preset channel attention function Fac(X), and the feature map after the channel attention weight is obtained. Take S'ECA as an example; the spatial channel attention is calculated based on X, and the expression is SSSE=X·σ1(F 1×1 Conv2D(X)), where F 1×1 Conv2D(X) is used to characterize the convolution operation based on the 1×1 convolution kernel, which is used to adjust the channel information of the feature map. σ1 is the preset activation function of the ECSA module. The final output of the ECSA module is S`ECA+SSSE. The same operation is performed after the output of each layer of the backbone convolution module to filter out redundant semantic information and highlight the small targets used for detection. It will not be repeated later.

[0084] It should be noted that in this embodiment, the Nth layer of the backbone feature map is input to the N-1th layer of the shallow fusion network, and the number of layers of each backbone convolution module does not exceed the number of layers it is in. Therefore, the first backbone feature map output by the first convolution module CBS1 has only one layer of feature map, which is not used for feature fusion, and is directly input to the second convolution module CBS2 for further feature extraction. Starting from the second layer of the backbone convolution module, the output backbone feature maps are output at least twice.

[0085] For example, Figure 1 As shown, the second backbone feature map P obtained by CBS2 in2 Input CBS3, Layer 2 P of F2 in2-2 Input to Concat1; CBS3 to obtain the third backbone feature map P in3 Enter CBS4, P in3 The second layer P in3-2 Input to Concat1, P in3 Layer 3 P in3-3 Input to Concat2; CBS4 obtains the fourth backbone feature map P in4 Input CBS5 and Concat3, P in4 The second layer P in4-2 Input to Concat1, P in4 Layer 3 P in4-3 Input to Concat2; CBS5 to obtain the fifth backbone feature map P in5 Input SPPF layer, P in5 The second layer P in5-2 Input to Concat1, P in5 Layer 3 P in5-3 Input to Concat2. In this way, Concat1 can fuse the 2nd (N+1)th layer feature map output by the 2nd to 5th convolutional layers, and Concat2 can fuse the 3rd (N+1)th layer feature map output by the 3rd to 5th convolutional layers, thereby realizing shallow feature fusion of multiple feature maps.

[0086] In addition, in one embodiment, referring to Figure 3 Before step S12, the following steps are also included but not limited to:

[0087] S21, adjusting the weights of each feature channel of the fifth trunk feature map through the excitation squeezing module to obtain a fused input feature map;

[0088] S22, inputting the fused input feature map into the average pooling layer, the maximum pooling layer and the first convolutional layer respectively;

[0089] S23, performing feature splicing based on the average pooling feature map output by the average pooling layer, the maximum pooling feature map output by the maximum pooling layer, and the first convolution feature map output by the first convolution layer to obtain a spliced ​​feature map;

[0090] S24, inputting the feature concatenation map into the second convolutional layer to obtain a second convolutional feature map output by the second convolutional layer;

[0091] S25, input the second convolutional feature map into a preset activation function to generate a feature weight map, perform element-by-element multiplication of the feature weight map and the fusion input feature map to obtain a spatial feature map, and input the spatial feature map into the deep fusion module.

[0092] It should be noted that if Figure 1 As shown, the SE module is a common module in YOLOV5. The principle of using the SE module to strengthen the weight of the feature channel is not described in detail here. in5 After being input into the SE module, the feature map output by the SE module is used as the fused input feature map, and the fused input feature map is used as the input of the neck network to the Spatial Feature Refinement Modul (SFRM). The spatial feature map P obtained by the SFRM module is in5′ As the input of the deep fusion module.

[0093] It should be noted that the formula of the output feature map of SFRM is as follows: a =σ2(f 7×7 ([Avg(F);Max(F);f 1×1 (F)]))×F, where F is the input feature of the ECSA module, F a is the output feature of the ECSA module, f 7×7 is the second convolutional layer, f 1×1 is the first convolutional layer, Avg(F) is the output of the average pooling layer, Max(F) is the output of the maximum pooling layer, [Avg(F); Max(F); f 1×1 (F)] is a feature concatenation map, σ2 is a preset activation function of the SFRM module, and in this embodiment, the feature map obtained by activating σ is used as a feature weight map, the feature weight map is multiplied element by element with the input feature F, and the feature weight is applied to the original feature map to obtain a spatial feature map.

[0094] In addition, in one embodiment, referring to Figure 3 Step S12 specifically includes but is not limited to the following steps:

[0095] S121, after the spatial feature map passes through the sixth convolution module and the first upsampling layer in sequence, it is fused with the fourth trunk feature map in the third fusion layer to form a deep fusion feature map;

[0096] S122, after the deep fusion feature map passes through the seventh convolution module and the second upsampling layer in sequence, it is fused at the second fusion layer with the third layer feature map of the fifth trunk feature map, the third layer feature map of the fourth trunk feature map, and the third layer feature map of the third trunk feature map into a second shallow fusion feature map;

[0097] S123, after the second shallow fusion feature map passes through the eighth convolution module and the third upsampling layer in sequence, it is fused at the first fusion layer with the second layer feature map of the fifth trunk feature map, the second layer feature map of the fourth trunk feature map, the second layer feature map of the third trunk feature map, and the second layer feature map of the second trunk feature map into a first shallow fusion feature map;

[0098] S124, inputting the spatial feature map, the deep fusion feature map, the second shallow fusion feature map and the first shallow fusion feature map into corresponding prediction modules.

[0099] It should be noted that if Figure 1 As shown in the figure, after the spatial feature map is input into the deep fusion module, it first enters the sixth convolution module CBS6 for convolution processing, and then goes through the first upsampling layer (Upsample1) for upsampling and then input into the third fusion layer (Concat3), where it is combined with P in4 To perform feature fusion, the spatial feature map also needs to be resized so that its scale is consistent with P in4 Alignment, the specific adjustment method is not described here, the obtained deep fusion feature map is input into YOLO HEAD 3 for feature prediction. The expression of the deep fusion feature map is as follows: td4 =Conv(ω1×P in4 +ω2×Resize(P in5′ ), where P td4 is a deep fusion feature map, ω1 and ω2 are preset fusion factors, and ω1 is not equal to ω2, Resize(P in5′ ) is used to characterize the P of the spatial feature map in5′ Adjust the size to match P in4 Consistent.

[0100] It should be noted that the principles of the seventh convolution module, the second upsampling layer, the eighth convolution module and the third upsampling layer are not repeated here, so that the sizes of the feature maps used for fusion are aligned.

[0101] It should be noted that the expression of the fusion operation of the second fusion layer is as follows: td3 =Conv(P in3-3 +P in4-3 +P in5-3 +Resize(P td4 ), similarly, the expression of the fusion operation of the first fusion layer is as follows: td2 =Conv(P in2-2 +P in3-2 +P in4-2 +P in5-2 +Resize(P td3 ), where P td3is the second shallow fusion feature map, P td2 It is the first shallow fusion feature map.

[0102] It should be noted that after obtaining each fusion feature map, the corresponding prediction module is input respectively, such as Figure 1 As shown, the spatial feature map P in5′ Input to YOLO HEAD 4, P td4 Input to YOLO HEAD 3,P td3 Input to YOLO HEAD 2,P td2 The prediction principle of YOLO HEAD is well known to those skilled in the art and will not be elaborated here.

[0103] In addition, in one embodiment, referring to Figure 3 Step S13 specifically includes but is not limited to the following steps:

[0104] S31, extracting an object image from the target object frame, and performing image recognition based on the object image to determine the object type, wherein the object type is a pedestrian, a vehicle, or property equipment;

[0105] S32, determining a monitoring prediction result based on the object type and the object action information;

[0106] S33, saving monitoring prediction results based on object type classification.

[0107] It should be noted that multiple target object frames can be generated in the target video frame, each target object frame corresponds to a monitored object. The present embodiment performs object recognition in each target object frame, extracts the outline of the monitored object to determine the object type. Since the improved YOLOV5 model of the present embodiment can improve the detection effect of small targets, the target object frame can more accurately characterize the range of the monitored object, such as reducing unnecessary frame selection range of the target object frame, so that objects other than the monitored object are reduced in the target object frame, thereby improving the accuracy of object recognition. The specific image recognition algorithm is a technology well known to those skilled in the art and is not limited here.

[0108] It should be noted that for different object types, the method of identifying object action information and determining the monitoring prediction results are different. Therefore, it is necessary to identify the object type first. The object types in this embodiment include pedestrians, vehicles or property equipment. The object action information is used to characterize the current action of the target object. For example, if a pedestrian is walking, the destination is predicted based on his walking direction; if the object type is a vehicle, the object action information may be driving or parked; if the object type is property equipment, the object action information may be whether the equipment has any abnormal phenomenon, such as a broken water pipe or a leaking electric meter.

[0109] It should be noted that after obtaining the monitoring prediction results, this embodiment also classifies and saves the monitoring prediction results according to the object type and generates records to facilitate subsequent backtracking of the monitoring events.

[0110] In addition, in one embodiment, referring to Figure 3 The property monitoring system includes multiple registered identity information, each registered identity information is pre-associated with at least one object permission, and each object permission is associated with at least one optional behavior information. When the object type is a pedestrian or a vehicle, step S32 specifically includes but is not limited to the following steps:

[0111] S41, determining at least one candidate authority from preset authorities based on the object location information, wherein the property monitoring system includes a plurality of preset authorities, and each preset authority is associated with a plurality of location information;

[0112] S42, determining the object identity information based on the object image, wherein when the object type is a pedestrian, the object identity information is obtained based on face recognition of the object image, and when the object type is a vehicle, the object identity information is obtained based on license plate recognition of the object image;

[0113] S43, determining the intersection of the candidate permissions and the object permissions associated with the object identity information as the first permission;

[0114] S44, determining object behavior information based on the object identity information and the object action information, and determining a second permission based on the object behavior information;

[0115] S45, when the second authority does not belong to the first authority, generating a first warning message;

[0116] S46: Generate a first event record based on the object identity information, the second authority, and the first alarm information.

[0117] It should be noted that the registered identity information can correspond to the owner or property management personnel in the community. In the field of property management, there are certain differences in the areas that people of different identities can enter. For example, some management areas do not allow owners to enter. The mapping relationship between preset permissions and location information can be recorded by establishing a mapping table. After determining the object location information, the candidate permissions corresponding to the location information are determined by looking up the table. For example, candidate permissions include administrator permissions and owner permissions. The location information corresponding to administrator permissions includes the property office and the community public area, the location information corresponding to owner permissions includes the community public area and the owner's property, and the location corresponding to outsider permissions includes the community public area; for example, when the object type is a vehicle, the location information of the owner's vehicle permissions includes the community public roads and the owner's garage, and the location information of the outsider's vehicle permissions includes the community public roads; when the object location information corresponds to the community public area, administrator permissions, owner permissions and outsider permissions can be matched as candidate permissions.

[0118] It should be noted that simple face recognition can be performed based on the object image to determine the object's identity information, and the vehicle can be identified through simple license plate recognition, which will not be elaborated here.

[0119] It should be noted that after determining the object identity information, the pre-associated object permissions can be determined according to the above description, and the intersection of the two is used as the first permission, that is, the permissions that the monitored object has at the current object location information. For example, if the object identity information is an outsider, the first permission is the public area of ​​the community. For another example, if the object identity information is an outsider vehicle, the first permission is the public road of the community.

[0120] It should be noted that the object behavior information is used to characterize the current operation of the monitored object. After the target object frame is determined in the target video frame, the object behavior information can be obtained by tracking the action of multiple video frames based on the target object frame. For example, it can be determined based on multiple video frames that the monitored object is walking towards the property office, or that the monitored object is driving into the owner's parking space. A simple judgment can be made based on the actions of multiple consecutive frames.

[0121] It should be noted that the object behavior information represents the behavior trend of the monitored object. Therefore, the second permission can be determined based on the object behavior information, and the permission requirements of the monitored object can be determined through behavior prediction. Based on the permission requirements, it can be judged whether the corresponding permission is possessed, and an early warning can be initiated in a timely manner.

[0122] Exemplarily, the object type is a pedestrian, the object location information is the community public area in front of the property office, the matched candidate permissions include outsider permissions, property management personnel permissions and owner permissions, the object identity information is an outsider, the first permission is the outsider permission, and the identified object behavior information is walking towards the property office. The second permission matched based on the object behavior information is the property management personnel permission. The second permission does not belong to the first permission, and a first alarm message is generated to prompt the property management personnel to pay attention to outsiders.

[0123] Exemplarily, the object type is a vehicle, the object location information is a community public road, the matched candidate permissions include the owner's vehicle permission and the outsider vehicle permission, the object identity information is the owner's vehicle, and the identified object behavior information is illegal parking on a public road. There is no corresponding second permission, so it is equivalent to the second permission not belonging to the first permission. A first alarm message is generated to prompt the business management personnel to remind the vehicle to move away.

[0124] In addition, in one embodiment, referring to Figure 3 When the object type is property equipment, step S32 specifically includes but is not limited to the following steps:

[0125] S51, determining a target device type based on the object image, wherein the target device type is associated with a plurality of optional device actions;

[0126] S52, determining a target device action based on the object action information, and generating second alarm information when the target device action does not belong to an optional device action;

[0127] S53: Generate a second event record based on the target device type, the target device action, and the second alarm information.

[0128] It should be noted that when the object type is property equipment, the target equipment type can be determined by simple image recognition. The image recognition algorithm will not be described in detail here. Each equipment type can be associated with multiple optional equipment actions, and the specific action can also be obtained by multi-frame video frame recognition. The optional equipment action can be maintenance personnel maintenance or normal operation action or.

[0129] It should be noted that the target device action is the current action trend of the property equipment. For example, if the monitored object is a water pipe, when a violent vibration is detected in the water pipe, the target device action is a violent vibration. The optional device action does not include violent vibration, and a second alarm message is generated to prompt the property management personnel to monitor.

[0130] like Figure 4 As shown, Figure 4: is a structural diagram of a property monitoring device based on improved YOLOV5 provided by an embodiment of the present invention. The present invention also provides a property monitoring device based on improved YOLOV5, including:

[0131] The processor 401 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0132] The memory 402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 402, and the processor 401 calls and executes the property monitoring method based on the improved YOLOV5 in the embodiment of this application;

[0133] Input / output interface 403, used to implement information input and output;

[0134] Communication interface 404, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0135] Bus 405 , which transmits information between various components of the device (e.g., processor 401 , memory 402 , input / output interface 403 , and communication interface 404 );

[0136] The processor 401 , the memory 402 , the input / output interface 403 and the communication interface 404 are connected to each other in communication within the device via the bus 405 .

[0137] An embodiment of the present application also provides an electronic device, including the property monitoring device based on the improved YOLOV5 as described above.

[0138] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned property monitoring method based on the improved YOLOV5 is implemented.

[0139] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0140] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0141] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A property monitoring method based on improved YOLOV5, characterized in that: Applied to a property monitoring system, the property monitoring system is provided with an improved YOLOV5 model, the improved YOLOV5 model includes an input convolution layer, a backbone network, a neck network and a detection head, the backbone network is provided with a multi-layer backbone convolution module, the neck network includes a multi-layer shallow fusion module and a layer of deep fusion module, the detection head includes a plurality of prediction modules, the deep fusion module is connected to the second to last layer of the backbone convolution module, the shallow fusion module is connected to the remaining backbone convolution modules, the first layer of the backbone convolution module is not connected to the shallow fusion module, the shallow feature fusion module and the deep fusion module are each connected to one of the prediction modules, the method includes: Input the target video frame into the input convolution layer to obtain an initial feature map, input the initial feature map into the backbone network for feature extraction, wherein when the number of layers of the backbone convolution module is greater than or equal to 2, input the extracted feature map into the backbone convolution module of the next layer, and input the N+1th layer of the extracted feature map into the shallow fusion module of the Nth layer, wherein N is a positive integer; Inputting the feature fusion graphs of the deep fusion module and the shallow fusion module into the corresponding prediction module, and obtaining the target detection result based on the feature prediction results output by each prediction module; At least one target object frame is generated in the target video frame based on the target detection result, object position information is determined based on the target object frame, action recognition is performed in the target object frame to obtain object action information, and a monitoring prediction result is determined based on the object position information and the object action information.

2. The property monitoring method based on improved YOLOV5 according to claim 1, characterized in that: The multi-layer backbone convolution modules of the backbone network include a first convolution module, a second convolution module, a third convolution module, a fourth convolution module and a fifth convolution module, and the structures of the backbone convolution modules are the same. An attention module is also provided between every two layers of the backbone convolution modules, and the attention module is used to improve the channel attention weight and the spatial attention weight. The multi-layer shallow fusion modules include a first fusion module and a second fusion module, the second convolution module is connected to the first fusion module, the third convolution module is connected to the second fusion module, and the fourth convolution module is connected to the deep fusion module. The initial feature map is input into the backbone network for feature extraction, including: Input the initial feature map into the first convolution module, extract the first trunk feature map through the first convolution module, input the first trunk feature map into the second convolution module after passing through the attention module; Extracting a second main feature map through the second convolution module, inputting the second main feature map into the third convolution module after passing through the attention module, and inputting the second layer feature map of the second main feature map into the first fusion module; Extracting a third main feature map through the third convolution module, inputting the third main feature map into the fourth convolution module after passing through the attention module, inputting the second layer feature map of the third main feature map into the first fusion module, and inputting the third layer feature map of the third main feature map into the second fusion module; Extracting a fourth main feature map through the fourth convolution module, inputting the fourth main feature map into the fifth convolution module after passing through the attention module, inputting the second layer feature map of the fourth main feature map into the first fusion module, inputting the third layer feature map of the fourth main feature map into the second fusion module, and inputting the fourth main feature map into the deep fusion module; A fifth trunk feature map is extracted through the fifth convolution module, a second layer feature map of the fifth trunk feature map is input into the first fusion module, and a third layer feature map of the fifth trunk feature map is input into the second fusion module.

3. The property monitoring method based on improved YOLOV5 according to claim 2, characterized in that: The neck network further includes an excitation squeezing module and a spatial feature refinement module, the fifth convolution module is connected to the excitation squeezing module, the spatial feature refinement module includes an average pooling layer, a maximum pooling layer, a first convolution layer, and a second convolution layer, the convolution kernel of the first convolution layer is 1×1, and the convolution kernel of the second convolution layer is 7×7, before the feature fusion graphs of the deep fusion module and the shallow fusion module are input into the corresponding prediction module, the method further includes: The weights of the feature channels of the fifth trunk feature map are adjusted by the excitation squeezing module to obtain a fused input feature map; Inputting the fused input feature map into the average pooling layer, the maximum pooling layer and the first convolutional layer respectively; Performing feature splicing based on the average pooling feature map output by the average pooling layer, the maximum pooling feature map output by the maximum pooling layer, and the first convolution feature map output by the first convolution layer to obtain a spliced ​​feature map; Inputting the feature splicing map into the second convolutional layer to obtain a second convolutional feature map output by the second convolutional layer; The second convolutional feature map is input into a preset activation function to generate a feature weight map, and the spatial feature map is obtained by element-by-element multiplication of the feature weight map and the fusion input feature map, and the spatial feature map is input into the deep fusion module.

4. The property monitoring method based on improved YOLOV5 according to claim 3 is characterized in that: The deep fusion module includes a sixth convolution module, a first upsampling layer and a third fusion layer, the second fusion module includes a seventh convolution module, a second upsampling layer and a second fusion layer, the first fusion module includes an eighth convolution module, a third upsampling layer and a first fusion layer, the structures of the sixth convolution module, the seventh convolution module and the eighth convolution module are the same as the trunk convolution module, and the feature fusion maps of the deep fusion module and the shallow fusion module are input into the corresponding prediction module, including: After the spatial feature map passes through the sixth convolution module and the first upsampling layer in sequence, it is fused with the fourth trunk feature map in the third fusion layer to form the deep fusion feature map; After the deep fusion feature map passes through the seventh convolution module and the second upsampling layer in sequence, the second fusion layer is fused with the third layer feature map of the fifth trunk feature map, the third layer feature map of the fourth trunk feature map, and the third layer feature map of the third trunk feature map into a second shallow fusion feature map; After the second shallow fusion feature map passes through the eighth convolution module and the third upsampling layer in sequence, the first fusion layer is fused with the second layer feature map of the fifth trunk feature map, the second layer feature map of the fourth trunk feature map, the second layer feature map of the third trunk feature map, and the second layer feature map of the second trunk feature map into a first shallow fusion feature map; The spatial feature map, the deep fusion feature map, the second shallow fusion feature map and the first shallow fusion feature map are input into the corresponding prediction module.

5. The property monitoring method based on improved YOLOV5 according to claim 1, characterized in that: Determining a monitoring prediction result based on the object position information and the object action information includes: Extracting an object image from the target object frame, and performing image recognition based on the object image to determine the object type, wherein the object type is a pedestrian, a vehicle, or property equipment; Determining the monitoring prediction result based on the object type and the object action information; The monitoring prediction results are saved based on the object type classification.

6. The property monitoring method based on improved YOLOV5 according to claim 5, characterized in that: The property monitoring system includes a plurality of registered identity information, each of the registered identity information is pre-associated with at least one object permission, each of the object permission is associated with at least one optional behavior information, and when the object type is a pedestrian or a vehicle, the monitoring prediction result is determined based on the object type and the object action information, including: Determining at least one candidate authority from preset authorities based on the object location information, wherein the property monitoring system includes a plurality of preset authorities, each of the preset authorities being associated with a plurality of location information; Determining object identity information based on the object image, wherein when the object type is a pedestrian, the object identity information is obtained based on face recognition performed on the object image, and when the object type is a vehicle, the object identity information is obtained based on license plate recognition performed on the object image; Determine an intersection of the candidate permission and the object permission associated with the object identity information as a first permission; Determine object behavior information based on the object identity information and the object action information, and determine a second permission based on the object behavior information; When the second permission does not belong to the first permission, generating a first warning message; A first event record is generated based on the object identity information, the second authority, and the first alarm information.

7. The property monitoring method based on improved YOLOV5 according to claim 5, characterized in that: When the object type is property equipment, determining the monitoring prediction result based on the object type and the object action information includes: determining a target device type based on the object image, wherein the target device type is associated with a plurality of selectable device actions; determining a target device action based on the object action information, and generating second alarm information when the target device action does not belong to the optional device action; A second event record is generated based on the target device type, the target device action, and the second alarm information.

8. A property monitoring device based on improved YOLOV5, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the property monitoring method based on the improved YOLOV5 as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: Including the property monitoring device based on improved YOLOV5 as described in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the property monitoring method based on the improved YOLOV5 as described in any one of claims 1 to 7.

Images