Monitoring object movement identification method, system, terminal and storage medium

Abstract

The present invention relates to the field of image processing technology, and specifically provides a monitoring object movement recognition method, system, terminal and storage medium, including: using a target detection algorithm to identify the monitoring object contour from the monitoring image of the monitoring area, and using a human posture recognition model to extract a first key point set based on the monitoring object contour; using the feature points of the monitoring image and the standard image to calculate the homography matrix; using the homography matrix to map the first key point set to the standard image coordinate system, and calculating the area of ​​the polygon surrounded by the first key point set on the standard image coordinate system; using the area of ​​the polygon surrounded by the key points of the monitoring object in the standard image as the standard area, and calculating the measurement value of the polygon area and the standard area; comparing the measurement value with a set threshold, if the measurement value does not exceed the threshold, it is determined that the monitoring object is moved. The present invention can be applied to dynamic monitoring scenes to improve the accuracy of recognition results in dynamic monitoring scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method, system, terminal and storage medium for identifying movement of a monitored object. Background Art

[0002] In complex environments such as power construction sites, some important equipment cannot be moved. For example, safety fences are set up around substations, high-voltage equipment, lifting work sites, etc. Once moved, safety accidents are very likely to occur. Therefore, these important equipment need to be monitored. The existing monitoring method is to install cameras in the monitoring area and then perform image consistency comparison to confirm whether the monitored object is in the original position. However, this method is expensive, and fixed cameras are also easily damaged at the construction site. Although the use of dynamic monitoring equipment such as automatic tracking cameras or drones can save costs, the existing image consistency comparison method cannot obtain accurate recognition results due to the non-fixed camera field of view and focal length. In addition, the existing image consistency comparison method is also difficult to accurately detect the actual position and area of ​​the fence when the fence shape is changeable, which is easy to cause false alarms. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method, system, terminal and storage medium for identifying the movement of a monitored object to solve the above-mentioned technical problems.

[0004] In a first aspect, the present invention provides a method for identifying movement of a monitored object, comprising:

[0005] Using a target detection algorithm to identify the outline of a monitored object from a monitored image of a monitored area, and using a human posture recognition model to extract a first key point set based on the outline of the monitored object;

[0006] Extracting a first feature point of the monitoring image and a second feature point in the standard image that has a mapping relationship with the first feature point by using a feature extraction algorithm, and calculating a homography matrix that maps the monitoring image to a standard image coordinate system based on the first feature point and the second feature point;

[0007] Mapping the first key point set to a standard image coordinate system using a homography matrix, and calculating the area of ​​a polygon enclosed by the first key point set in the standard image coordinate system;

[0008] The area of ​​the polygon enclosed by the key points of the monitored object in the standard image is taken as the standard area, and the measurement value of the polygon area and the standard area is calculated;

[0009] The measurement value is compared with a set threshold value, and if the measurement value does not exceed the threshold value, it is determined that the monitored object has been moved.

[0010] Furthermore, before using the target detection algorithm to identify the outline of the monitored object from the monitored image of the monitored area, and using the human posture recognition model to extract the first key point set based on the outline of the monitored object, the method also includes:

[0011] Use the image batch scaling tool to scale the size of the surveillance images to the set standard size;

[0012] Use the trained EfficientDet network model to identify human targets from surveillance images of standard size;

[0013] If a human target is identified, the monitoring object outline is continuously identified from the monitoring image of standard size, wherein the monitoring object outline is the minimum circumscribed rectangle of the monitoring object target;

[0014] If no human target is identified, the recognition procedure for the monitoring image is terminated and a prompt message is output.

[0015] Further, using a target detection algorithm to identify the outline of a monitored object from a monitored image of a monitored area, and using a human posture recognition model to extract a first key point set based on the outline of the monitored object, including:

[0016] A human posture recognition model is used to identify key points of images within the contour range of the monitored object. The human posture recognition model adopts a DarkPose model, and the DarkPose model backbone network uses an HRNet neural network, the learning rate is set to 0.001, and the optimizer is Adam;

[0017] The data annotation software is used to annotate the coordinates of all the identified key points, and the coordinates of all the key points are stored as the first key point set.

[0018] Furthermore, a feature extraction algorithm is used to extract a first feature point of the monitoring image and a second feature point in the standard image that has a mapping relationship with the first feature point, and a homography matrix that maps the monitoring image to a standard image coordinate system is calculated based on the first feature point and the second feature point, including:

[0019] Using the OrientedBrief feature extraction algorithm, a plurality of feature points are extracted from the monitoring image and the standard image respectively, wherein the standard image is an image of the monitoring object in an initial state;

[0020] A brute force algorithm is used to match multiple feature points of the monitoring image with multiple feature points of the standard image, and a one-to-one mapping relationship between the multiple feature points of the monitoring image and the multiple feature points of the standard image is established;

[0021] The homography matrix H is calculated using the formula I = H*K, where K is the coordinates of the feature points in the monitoring image and I is the coordinates of the mapped feature points in the standard image;

[0022] Multiple pairs of feature points with mapping relationships are used to calculate multiple homography matrices, and an accurate homography matrix is ​​selected from the multiple homography matrices.

[0023] Furthermore, the area of ​​the polygon enclosed by the key points of the monitored object in the standard image is used as the standard area, and the measurement value of the polygon area and the standard area is calculated, including:

[0024] Using the formula Calculate the metric value, where S ref is the standard area of ​​the monitored object area in the standard image, S n is the area of ​​the polygon enclosed by the first key point set in the standard image coordinate system, and |·| represents the operation of finding the number of pixels.

[0025] Further, comparing the measurement value with a set threshold, and determining that the monitored object is moved if the measurement value does not exceed the threshold, includes:

[0026] By pre-collecting multiple sample images of the monitored object in different states, and processing the multiple sample images respectively to obtain sample measurement values ​​of the multiple sample images relative to the standard image, the threshold is set by analyzing the monitored object states corresponding to the multiple sample measurement values;

[0027] The metric value of the monitoring image relative to the standard image is compared with a set threshold value. If the metric value exceeds the threshold value, it is determined that the monitored object has not been moved.

[0028] Furthermore, the method further comprises:

[0029] Collect surveillance video of the surveillance area, and use an image batch scaling tool to capture multiple frames of surveillance images of standard size from the surveillance video;

[0030] Processing the multiple surveillance images respectively to obtain surveillance object motion recognition results of the multiple surveillance images;

[0031] The monitoring object motion recognition results that account for a larger proportion are output as the final monitoring object motion recognition results.

[0032] In a second aspect, the present invention provides a monitoring object movement identification system, comprising:

[0033] A target recognition unit, configured to recognize the outline of a monitored object from a monitored image of a monitored area using a target detection algorithm, and to extract a first key point set based on the outline of the monitored object using a human posture recognition model;

[0034] a matrix calculation unit, configured to extract, by using a feature extraction algorithm, a first feature point of the monitoring image and a second feature point in the standard image that has a mapping relationship with the first feature point, and calculate a homography matrix that maps the monitoring image to a standard image coordinate system based on the first feature point and the second feature point;

[0035] an area calculation unit, configured to map the first key point set to a standard image coordinate system using a homography matrix, and calculate the area of ​​a polygon enclosed by the first key point set in the standard image coordinate system;

[0036] A metric calculation unit, used to take the area of ​​a polygon enclosed by key points of the monitored object in the standard image as the standard area, and calculate a metric value between the polygon area and the standard area;

[0037] The result determination unit is used to compare the measurement value with a set threshold value, and if the measurement value does not exceed the threshold value, it is determined that the monitored object has been moved.

[0038] Furthermore, the system also includes a preprocessing unit, which is used to:

[0039] Use the image batch scaling tool to scale the size of the surveillance images to the set standard size;

[0040] Use the trained EfficientDet network model to identify human targets from surveillance images of standard size;

[0041] If a human target is identified, the monitoring object outline is continuously identified from the monitoring image of standard size, wherein the monitoring object outline is the minimum circumscribed rectangle of the monitoring object target;

[0042] If no human target is identified, the recognition procedure for the monitoring image is terminated and a prompt message is output.

[0043] Furthermore, the target recognition unit is used to:

[0044] A human posture recognition model is used to identify key points of images within the contour range of the monitored object. The human posture recognition model adopts a DarkPose model, and the DarkPose model backbone network uses an HRNet neural network, the learning rate is set to 0.001, and the optimizer is Adam;

[0045] The data annotation software is used to annotate the coordinates of all the identified key points, and the coordinates of all the key points are stored as the first key point set.

[0046] Furthermore, the matrix calculation unit is used to:

[0047] Using the OrientedBrief feature extraction algorithm, a plurality of feature points are extracted from the monitoring image and the standard image respectively, wherein the standard image is an image of the monitoring object in an initial state;

[0048] A brute force algorithm is used to match multiple feature points of the monitoring image with multiple feature points of the standard image, and a one-to-one mapping relationship between the multiple feature points of the monitoring image and the multiple feature points of the standard image is established;

[0049] The homography matrix H is calculated using the formula I = H*K, where K is the coordinates of the feature points in the monitoring image and I is the coordinates of the mapped feature points in the standard image;

[0050] Multiple pairs of feature points with mapping relationships are used to calculate multiple homography matrices, and an accurate homography matrix is ​​selected from the multiple homography matrices.

[0051] Furthermore, the metric calculation unit is used to:

[0052] Using the formula Calculate the metric value, where S ref is the standard area of ​​the monitored object area in the standard image, S n is the area of ​​the polygon enclosed by the first key point set in the standard image coordinate system, and |·| represents the operation of finding the number of pixels.

[0053] Furthermore, the result determination unit is used to:

[0054] By pre-collecting multiple sample images of the monitored object in different states, and processing the multiple sample images respectively to obtain sample measurement values ​​of the multiple sample images relative to the standard image, the threshold is set by analyzing the monitored object states corresponding to the multiple sample measurement values;

[0055] The metric value of the monitoring image relative to the standard image is compared with a set threshold value. If the metric value exceeds the threshold value, it is determined that the monitored object has not been moved.

[0056] Furthermore, the system further comprises a result correction unit, wherein the result correction unit is used to:

[0057] Collect surveillance video of the surveillance area, and use an image batch scaling tool to capture multiple frames of surveillance images of standard size from the surveillance video;

[0058] Processing the multiple surveillance images respectively to obtain surveillance object motion recognition results of the multiple surveillance images;

[0059] The monitoring object motion recognition results that account for a larger proportion are output as the final monitoring object motion recognition results.

[0060] In a third aspect, a terminal is provided, including:

[0061] processor, memory, wherein:

[0062] The memory is used to store computer programs.

[0063] The processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned terminal method.

[0064] According to a fourth aspect, a computer storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the methods described in the above aspects.

[0065] The monitoring object movement identification method, system, terminal and storage medium provided by the present invention have the following beneficial effects:

[0066] (1) Based on image matching and deep learning technology, we can model targets in dynamic monitoring scenarios, identify movement behaviors, and improve the monitoring capabilities of monitored objects.

[0067] (2) Modeling the monitored object area in the image in a "coarse" to "fine" manner. In the prior art, the target detection method is used to obtain the position and boundary of the monitored object to achieve rough modeling of the monitored object; the present invention adds key point detection of the monitored object on the basis of the existing method to achieve refined modeling of the monitored object and improve the modeling accuracy.

[0068] (3) By using image matching technology, the monitored object area in the current image is corrected to the standard image coordinate system, so that the degree of change of the monitored object area can be calculated in the same coordinate system, so that the method proposed in the present invention can be applied to dynamic monitoring scenarios, expanding the application scenarios of existing methods.

[0069] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0072] Figure 2 It is a schematic diagram of the minimum circumscribed rectangle of a fence and key point markings of a method according to an embodiment of the present invention.

[0073] Figure 3It is a flow chart of a fence movement identification algorithm of a method according to an embodiment of the present invention.

[0074] Figure 4 The present invention is a flowchart of a method for correcting fence key points in a current image according to an embodiment of the present invention.

[0075] Figure 5 It is a schematic diagram of a fence area in a reference image and a current image of a method according to an embodiment of the present invention.

[0076] Figure 6 is a schematic block diagram of a system according to an embodiment of the present invention.

[0077] Figure 7 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0079] In order to solve the problems of low recognition accuracy in existing image recognition methods and only being able to perform consistency matching on images with fixed field of view and focal length, the present invention provides a monitoring object movement recognition method, comprising the following steps:

[0080] By using the drone to regularly cruise and take pictures of the monitored object, the monitored image is obtained. The monitored object is processed using steps S1-S4 to obtain the monitoring result of the current cruise.

[0081] S 1. Using a target detection algorithm to identify the outline of a monitored object from a monitored image of a monitored area, and using a human posture recognition model to extract a first key point set based on the outline of the monitored object.

[0082] S2, using a feature extraction algorithm to extract a first feature point of the monitoring image and a second feature point in the standard image that has a mapping relationship with the first feature point, and calculating a homography matrix that maps the monitoring image to a standard image coordinate system based on the first feature point and the second feature point;

[0083] S3, mapping the first key point set to a standard image coordinate system using a homography matrix, and calculating the area of ​​a polygon enclosed by the first key point set in the standard image coordinate system;

[0084] S4, taking the area of ​​the polygon enclosed by the key points of the monitored object in the standard image as the standard area, and calculating the measurement value of the polygon area and the standard area;

[0085] The measurement value is compared with a set threshold value, and if the measurement value does not exceed the threshold value, it is determined that the monitored object has been moved.

[0086] To facilitate understanding of the present invention, the following further describes the monitoring object motion identification method provided by the present invention based on the principle of the monitoring object motion identification method of the present invention and the process of performing motion identification on the monitoring object in the embodiment.

[0087] For details, please refer to Figure 1 Taking the monitoring object as a safety fence as an example, the monitoring object movement identification method includes:

[0088] (1) Image acquisition and processing: In the power safety fence monitoring area, use automatic tracking cameras, drones and other dynamic monitoring equipment to collect video data. Use OpenCV tools (image batch scaling tools) to decode images from captured video clips or real-time video streams, extract continuous key frame images, and scale the image size to 1080×720.

[0089] (2) Since the safety fence is most likely to be moved only when there are people, in order to reduce the amount of useless calculations, this embodiment designs human target detection: the image obtained in step 1 is input into the EfficientDet network (target detection algorithm), and the detection result of the human target is output. The EfficientDet is a lightweight general target detection network that can be used to detect targets in power scenarios after training. The network is trained using the default hyperparameters of the EfficientDet algorithm. Figure 1 If no human target is detected, the rough modeling, refined modeling and moving identification of the fence will not be performed, and the process returns to step 1.

[0090] Example of human target detection result format:

[0091]

[0092]

[0093] In the target detection result format example, "class" indicates the category, "person" indicates the human body; "confidence" indicates the confidence level, which ranges from [0,1]; "box" indicates the minimum bounding rectangle of the human target, which is in the format of [x min ,y min ,x max ,y max ].

[0094] (3) Rough modeling of fences: Use the EfficientDet object detection network to roughly model the fences and obtain the rough location and area of ​​the fences. Use the default hyperparameters of the EfficientDet algorithm to train the network. Figure 1 If there is a fence target in the image, the EfficientDet model outputs the minimum bounding rectangle of the fence target in the format of [x min ,y min ,x max ,y max ]. If the fence target is not detected, the fence fine-grained modeling and movement recognition are not performed, and the process returns to step 1.

[0095] Example of fence target detection result format:

[0096]

[0097] In the target detection result format example, "class" indicates the category, "safety_fence" indicates the safety fence, "confidence" indicates the confidence level, and its range is [0,1]; "box" indicates the minimum enclosing rectangle of the fence target.

[0098] (4) Refined modeling of fence: Affected by the shooting angle, only target detection cannot accurately detect the actual area of ​​the fence. Refined modeling of the fence can adopt key point detection, semantic segmentation and other technologies. Taking into account the large amount of computation and low execution efficiency of the semantic segmentation algorithm, the present invention adds fence key point detection on the basis of fence target detection to further obtain the refined area of ​​the fence. In this embodiment, DarkPose is used to detect the key points of the fence. DarkPose is a key point detection network based on deep learning. In addition to the algorithm performance, its main difference from OpenPose is that OpenPose is a bottom-up method, while DarkPose is a top-down method. DarkPose needs to determine the target boundary before detecting the key points. This method can avoid interference from other targets and is conducive to improving the accuracy of key point prediction. The target boundary is the minimum enclosing rectangle of the fence in step 3. Reference Figure 2 , the key points of the fence are located at the four corners, namely Figure 2 In the figure, P1 to P4 point to four black dots; a1 and a2 are fence posts; b is the fence crossbar or protection belt. The open source software Labelme is used to annotate the fence key points and generate training and test sets. In this embodiment, the DarkPose backbone network uses HRNet, the learning rate is set to 0.001, the optimizer is Adam, and other training hyperparameters use the default configuration of the algorithm.

[0099] (5) Fence movement recognition: After a human body is detected in the image and the fence is modeled, the fence movement recognition algorithm is executed. During the shooting of dynamic monitoring scenes such as automatic tracking cameras and drone inspections, if the camera moves or zooms, the position and shape of objects in the image will also change accordingly. The present invention adds an image correction algorithm to map the key points of the fence to the reference image coordinate system, so as to compare whether the fence area has changed significantly in the same coordinate system to identify whether the fence has been moved. The fence movement recognition algorithm process is as follows: Figure 3 As shown, the process is as follows:

[0100] (5.1) Obtaining images and fence key points: Reference image I ref It is defined as the image when a human body and a fence target are detected for the first time during real-time monitoring. After the fence removal recognition algorithm is executed, the reference image is cleared. Obtain the reference image I from step (1). ref , current image I. Obtain the fence key points in the current image from step (4), which are represented as P1 = (x1, y1), P2 = (x2, y2), P3 = (x3, y3), and P4 = (x4, y4).

[0101] (5.2) Correcting the fence key points in the current image: The purpose of this step is to correct the key points in the current image to the reference image coordinate system, which includes extracting feature points, matching feature points, calculating homography matrix and correcting fence key points, such as Figure 4 To improve the image matching speed, this embodiment uses the ORB (Oriented Brief) feature extraction algorithm to quickly extract the current image I and the reference image I ref The Brute Force method is used to match I and I ref The feature points in . The homography matrix H is calculated by matching the feature points, and then the fence key points in the current image are corrected. The findHomography function in OpenCV is used to solve the homography matrix H.

[0102] The perspective transformation formula is:

[0103]

[0104] Expanding formula 1, we can get

[0105] x=a 11 u+a 12 v+a 13 (Formula 2)

[0106] y=a 21 u+a 22 v+a 23 (Formula 3)

[0107] z=a 31 u+a 32 v+a 33 (Formula 4)

[0108] The new coordinates after correction can be calculated by equations 2, 3, and 4.

[0109]

[0110]

[0111] Among them, u and v are the original coordinates of the point. From the above derivation process, it can be seen that the new coordinates x′ and y′ after perspective transformation can be calculated through the homography matrix H.

[0112] Correct fence key points: Substitute the fence key points P1 = (x1, y1), P2 = (x2, y2), P3 = (x3, y3), P4 = (x4, y4) in the current image into the new coordinate calculation expression to obtain the corrected fence key point P 1n =(x 1n ,y 1n )、P2=(x 2n ,y 2n ), P 3n =(x 3n ,y 3n ), P 4n =(x 4n ,y 4n ).

[0113] If the camera moves or zooms during the shooting process, after the above correction processing, the coordinate system of the fence key points in the current image can be unified to the coordinate system of the reference image, so that the fence area can be compared in the same coordinate system, making the method proposed in the present invention applicable to dynamic monitoring scenarios.

[0114] (5.3) Calculate the fence area IOU: Existing target detection technologies usually use the minimum bounding rectangle to model the target area. This method easily introduces invalid background areas and cannot accurately model the target area. The present invention uses a key point-based approach to model the target area, selecting the four vertices of the fence to form a quadrilateral as the effective area of ​​the fence. Let the fence area in the reference image be S ref , the corrected fence area is S n ,like Figure 5 shown.

[0115] Calculate the IOU between the fence area in the reference image and the corrected fence area in the current image. The expression is:

[0116]

[0117] In Formula 7, |·| represents the operation of finding the number of pixels.

[0118] According to the pre-set threshold IOU th Determine if the fence has been moved:

[0119] Case 1: IOU ≤ IOU th , the fence was moved;

[0120] Case 2: IOU>IOU th , the fence has not been moved.

[0121] In this embodiment, the threshold IOU th Set to 0.7. Furthermore, the embodiments of the present invention only illustrate the use of a single current image to determine whether the fence has been moved. During the implementation of the project, false detections are inevitable in target detection and key point detection algorithms. In order to reduce the false alarm rate and improve the robustness of the fence movement recognition algorithm, multi-frame detection and the "voting method" can be used for judgment. The specific method is that after continuous detection of k frames, if "the fence has been moved" accounts for the majority, the output result is that the fence has been moved; otherwise, the output result is that the fence has not been moved. During implementation, the value of k should be adjusted according to the camera frame rate f, and the range of k is generally f~10f.

[0122] In addition, the training set of the deep learning model in this embodiment contains 16,000 images, and the test set contains 4,000 images. The hardware configuration is 16G memory, Intel Core i7-9750, NVIDIA GeForce GTX 1070. When processing 1080×720 videos, the speed is about 20fps.

[0123] like Figure 6 As shown, the system 600 includes:

[0124] The target recognition unit 610 is used to recognize the outline of the monitored object from the monitored image of the monitored area by using the target detection algorithm, and extract the first key point set based on the outline of the monitored object by using the human posture recognition model;

[0125] A matrix calculation unit 620 is used to extract a first feature point of the monitoring image and a second feature point in the standard image that has a mapping relationship with the first feature point by using a feature extraction algorithm, and calculate a homography matrix that maps the monitoring image to a standard image coordinate system based on the first feature point and the second feature point;

[0126] An area calculation unit 630, configured to map the first key point set to a standard image coordinate system using a homography matrix, and calculate the area of ​​a polygon enclosed by the first key point set in the standard image coordinate system;

[0127] A metric calculation unit 640, configured to take the area of ​​a polygon enclosed by key points of the monitored object in the standard image as a standard area, and calculate a metric value between the area of ​​the polygon and the standard area;

[0128] The result determination unit 650 is used to compare the measurement value with a set threshold value, and if the measurement value does not exceed the threshold value, it is determined that the monitored object has been moved.

[0129] Optionally, as an embodiment of the present invention, the system further includes a preprocessing unit, wherein the preprocessing unit is used to:

[0130] Use the image batch scaling tool to scale the size of the surveillance images to the set standard size;

[0131] Use the trained EfficientDet network model to identify human targets from surveillance images of standard size;

[0132] If a human target is identified, the monitoring object outline is continuously identified from the monitoring image of standard size, wherein the monitoring object outline is the minimum circumscribed rectangle of the monitoring object target;

[0133] If no human target is identified, the recognition procedure for the monitoring image is terminated and a prompt message is output.

[0134] Optionally, as an embodiment of the present invention, the target identification unit is used to:

[0135] A human posture recognition model is used to identify key points of images within the contour range of the monitored object. The human posture recognition model adopts a DarkPose model, and the DarkPose model backbone network uses an HRNet neural network, the learning rate is set to 0.001, and the optimizer is Adam;

[0136] The data annotation software is used to annotate the coordinates of all the identified key points, and the coordinates of all the key points are stored as the first key point set.

[0137] Optionally, as an embodiment of the present invention, the matrix calculation unit is used for:

[0138] Using the Oriented Brief feature extraction algorithm, a plurality of feature points are extracted from the monitoring image and the standard image respectively, wherein the standard image is an image of the monitoring object in an initial state;

[0139] A brute force algorithm is used to match multiple feature points of the monitoring image with multiple feature points of the standard image, and a one-to-one mapping relationship between the multiple feature points of the monitoring image and the multiple feature points of the standard image is established;

[0140] The homography matrix H is calculated using the formula I = H*K, where K is the coordinates of the feature points in the monitoring image and I is the coordinates of the mapped feature points in the standard image;

[0141] Multiple pairs of feature points with mapping relationships are used to calculate multiple homography matrices, and an accurate homography matrix is ​​selected from the multiple homography matrices.

[0142] Optionally, as an embodiment of the present invention, the metric calculation unit is used to:

[0143] Using the formula Calculate the metric value, where S ref is the standard area of ​​the monitored object area in the standard image, S n is the area of ​​the polygon enclosed by the first key point set in the standard image coordinate system, and |·| represents the operation of finding the number of pixels.

[0144] Optionally, as an embodiment of the present invention, the result determination unit is used to:

[0145] By pre-collecting multiple sample images of the monitored object in different states, and processing the multiple sample images respectively to obtain sample measurement values ​​of the multiple sample images relative to the standard image, the threshold is set by analyzing the monitored object states corresponding to the multiple sample measurement values;

[0146] The metric value of the monitoring image relative to the standard image is compared with a set threshold value. If the metric value exceeds the threshold value, it is determined that the monitored object has not been moved.

[0147] Optionally, as an embodiment of the present invention, the system further includes a result correction unit, wherein the result correction unit is used to:

[0148] Collect surveillance video of the surveillance area, and use an image batch scaling tool to capture multiple frames of surveillance images of standard size from the surveillance video;

[0149] Processing the multiple surveillance images respectively to obtain surveillance object motion recognition results of the multiple surveillance images;

[0150] The monitoring object motion recognition results that account for a larger proportion are output as the final monitoring object motion recognition results.

[0151] Figure 7 The present invention provides a schematic diagram of the structure of a terminal 700 provided in an embodiment of the present invention. The terminal 700 can be used to execute the monitoring object movement identification method provided in an embodiment of the present invention.

[0152] The terminal 700 may include: a processor 710, a memory 720 and a communication unit 730. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0153] The memory 720 can be used to store the execution instructions of the processor 710. The memory 720 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 720 are executed by the processor 710, the terminal 700 can execute some or all of the steps in the following method embodiments.

[0154] The processor 710 is the control center of the storage terminal, and uses various interfaces and lines to connect various parts of the entire electronic terminal. It runs or executes software programs and / or modules stored in the memory 720, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of a plurality of packaged ICs with the same or different functions. For example, the processor 710 can include only a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0155] The communication unit 730 is used to establish a communication channel so that the storage terminal can communicate with other terminals, receive user data sent by other terminals or send user data to other terminals.

[0156] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0157] Therefore, the present invention is based on image matching and deep learning technology to model targets in dynamic monitoring scenes, identify moving behaviors, and improve the monitoring capabilities of monitored objects; the monitoring object area in the image is modeled in a "coarse" to "fine" manner. The prior art uses a target detection method to obtain the position and boundary of the monitored object to achieve rough modeling of the monitored object; the present invention adds key point detection of the monitored object on the basis of the existing method to achieve refined modeling of the monitored object and improve modeling accuracy; the image matching technology is used to correct the monitored object area in the current image to the standard image coordinate system, and the degree of change of the monitored object area is calculated in the same coordinate system, so that the method proposed by the present invention can be applied to dynamic monitoring scenes, expanding the application scenarios of the existing methods. The technical effects that can be achieved by this embodiment can be found in the description above, and will not be repeated here.

[0158] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention.

[0159] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0160] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0161] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0163] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of these shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for identifying the movement of a monitored object, characterized in that: include: Using a target detection algorithm to identify the outline of a monitored object from a monitored image of a monitored area, and using a human posture recognition model to extract a first key point set based on the outline of the monitored object; Extracting a first feature point of the monitoring image and a second feature point in the standard image that has a mapping relationship with the first feature point by using a feature extraction algorithm, and calculating a homography matrix that maps the monitoring image to a standard image coordinate system based on the first feature point and the second feature point; Mapping the first key point set to a standard image coordinate system using a homography matrix, and calculating the area of ​​a polygon enclosed by the first key point set in the standard image coordinate system; The area of ​​the polygon enclosed by the key points of the monitored object in the standard image is taken as the standard area, and the measurement value of the polygon area and the standard area is calculated; Comparing the measurement value with a set threshold, and determining that the monitored object has been moved if the measurement value does not exceed the threshold; The area of ​​a polygon enclosed by key points of the monitored object in the standard image is used as the standard area, and a measurement value of the polygon area and the standard area is calculated, including: Using the formula The metric is calculated, where S ref is the standard area of ​​the monitored object area in the standard image, S n is the area of ​​the polygon enclosed by the first key point set in the standard image coordinate system, and |·| represents the operation of finding the number of pixels.

2. The method according to claim 1, characterized in that Before using the target detection algorithm to identify the outline of the monitored object from the monitored image of the monitored area, and using the human posture recognition model to extract the first key point set based on the outline of the monitored object, the method further includes: Use the image batch scaling tool to scale the size of the surveillance images to the set standard size; Use the trained EfficientDet network model to identify human targets from surveillance images of standard size; If a human target is identified, the monitoring object outline is continuously identified from the monitoring image of standard size, wherein the monitoring object outline is the minimum circumscribed rectangle of the monitoring object target; If no human target is identified, the recognition procedure for the monitoring image is terminated and a prompt message is output.

3. The method according to claim 1, characterized in that Using a target detection algorithm to identify the outline of a monitored object from a monitored image of a monitored area, and using a human posture recognition model to extract a first key point set based on the outline of the monitored object, including: A human posture recognition model is used to identify key points of images within the contour range of the monitored object. The human posture recognition model adopts a DarkPose model, and the DarkPose model backbone network uses an HRNet neural network, the learning rate is set to 0.001, and the optimizer is Adam; The data annotation software is used to annotate the coordinates of all the identified key points, and the coordinates of all the key points are stored as the first key point set.

4. The method according to claim 1, characterized in that A first feature point of the monitoring image and a second feature point in the standard image having a mapping relationship with the first feature point are extracted using a feature extraction algorithm, and a homography matrix for mapping the monitoring image to a standard image coordinate system is calculated based on the first feature point and the second feature point, including: Using the Oriented Brief feature extraction algorithm, a plurality of feature points are extracted from the monitoring image and the standard image respectively, wherein the standard image is an image of the monitoring object in an initial state; A brute force algorithm is used to match multiple feature points of the monitoring image with multiple feature points of the standard image, and a one-to-one mapping relationship between the multiple feature points of the monitoring image and the multiple feature points of the standard image is established; The homography matrix H is calculated using the formula I=H*K, where K is the coordinates of the feature points in the monitoring image and I is the coordinates of the mapped feature points in the standard image; Multiple pairs of feature points with mapping relationships are used to calculate multiple homography matrices, and an accurate homography matrix is ​​selected from the multiple homography matrices.

5. The method according to claim 1, characterized in that Comparing the measurement value with a set threshold, and determining that the monitored object has been moved if the measurement value does not exceed the threshold, includes: By pre-collecting multiple sample images of the monitored object in different states, and processing the multiple sample images respectively to obtain sample measurement values ​​of the multiple sample images relative to the standard image, the threshold is set by analyzing the monitored object states corresponding to the multiple sample measurement values; The metric value of the monitoring image relative to the standard image is compared with a set threshold value. If the metric value exceeds the threshold value, it is determined that the monitored object has not been moved.

6. The method according to claim 1, characterized in that The method further comprises: Collect surveillance video of the surveillance area, and use an image batch scaling tool to capture multiple frames of surveillance images of standard size from the surveillance video; Processing the multiple surveillance images respectively to obtain surveillance object motion recognition results of the multiple surveillance images; The monitoring object motion recognition results that account for a larger proportion are output as the final monitoring object motion recognition results.

7. A monitoring object movement identification system, characterized in that: include: A target recognition unit, configured to recognize the outline of a monitored object from a monitored image of a monitored area using a target detection algorithm, and to extract a first key point set based on the outline of the monitored object using a human posture recognition model; a matrix calculation unit, configured to extract, by using a feature extraction algorithm, a first feature point of the monitoring image and a second feature point in the standard image that has a mapping relationship with the first feature point, and calculate a homography matrix that maps the monitoring image to a standard image coordinate system based on the first feature point and the second feature point; an area calculation unit, configured to map the first key point set to a standard image coordinate system using a homography matrix, and calculate the area of ​​a polygon enclosed by the first key point set in the standard image coordinate system; A metric calculation unit, used to take the area of ​​a polygon enclosed by key points of the monitored object in the standard image as the standard area, and calculate a metric value between the polygon area and the standard area; A result determination unit, used for comparing the measurement value with a set threshold value, and determining that the monitored object is moved if the measurement value does not exceed the threshold value; The area of ​​a polygon enclosed by key points of the monitored object in the standard image is used as the standard area, and a measurement value of the polygon area and the standard area is calculated, including: Using the formula The metric is calculated, where S ref is the standard area of ​​the monitored object area in the standard image, S n is the area of ​​the polygon enclosed by the first key point set in the standard image coordinate system, and |·| represents the operation of finding the number of pixels.

8. A terminal, characterized in that: include: processor; A memory for storing execution instructions of the processor; The processor is configured to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images