Abnormal behavior detection method, device, and medium

Abstract

The present application relates to the field of machine vision, and discloses an abnormal behavior detection method, device, and medium, including: obtaining an image to be processed that includes a monitoring target and is taken by each camera, so as to determine whether the monitoring target has abnormal behavior. Extracting the key points of the monitoring target in the image to be processed, and obtaining a first target image that only includes one monitoring target based on the key points. Calculating the similarity of the first target images taken by each camera based on the key points, and fusing each first target image based on the similarity to obtain a second target image, and determining whether the monitoring target has abnormal behavior based on the second target image. It can be seen that the abnormal behavior detection method provided by the present application, by first extracting the key points and obtaining the first target image based on the key points, does not need to prioritize target matching, thereby improving the image processing speed. Simultaneously, the images to be processed taken by multiple cameras are used to detect the monitoring target, thereby improving the accuracy of behavior detection.

Description

Technical Field

[0001] The present application relates to the field of machine vision, and in particular to a method, device, and medium for detecting abnormal behavior. Background Art

[0002] In scenarios such as etiquette training and examinations, supervisors are usually required to monitor the behavior of people in the scene to determine whether any abnormalities occur and correct them in a timely manner.

[0003] However, due to the large number of people in the scene and their small movements, supervisors may not be able to notice any unusual movements of individuals. For example, in an exam room, due to the large number of examinees, invigilators may not be able to accurately detect subtle movements such as shaking the head or leaning to the side, and thus cannot correct them in time. Currently, multiple supervisors are usually deployed in the exam room to improve supervision effectiveness, but this method wastes a lot of manpower and resources.

[0004] It can be seen that how to provide an abnormal behavior detection method with better supervision effect to reduce the waste of manpower and material resources is a problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, and medium for detecting abnormal behavior to reduce the waste of manpower and material resources while improving the supervision effect.

[0006] To solve the above technical problems, the present application provides a method for detecting abnormal behavior, including:

[0007] Obtain the images to be processed that contain the monitored targets taken by each camera;

[0008] Extracting key points of the monitoring target in the image to be processed, and acquiring a first target image based on the key points, wherein the first target image is an image including only one monitoring target, and the key points are points corresponding to body parts of the monitoring target;

[0009] calculating the similarity of each of the first target images according to the key points, and fusing each of the first target images according to the similarity to obtain a second target image;

[0010] Determine whether the monitored target has abnormal behavior based on the second target image.

[0011] Preferably, extracting key points of the monitored target in the image to be processed and acquiring the first target image according to the key points includes:

[0012] Utilize the CNN network to extract the key points in the image and construct a detection skeleton;

[0013] Acquire detection features of the detection skeleton and detection features of the key points;

[0014] Combining the detection features of the detection skeleton and the classification loss function, matching the sample with the highest category confidence and position confidence;

[0015] Obtaining a prediction frame of the first target image;

[0016] Calculating a multi-task loss function using the detection features of the detection skeleton and the detection features of the key points, and performing a regression operation on the prediction box using the multi-task loss function to obtain the first target image;

[0017] The calculation formula of the multi-task loss function is, where is the target classification loss function, is the target position regression loss function, is the target positioning loss function, is the feature point positioning loss function, is the weight of the target classification loss function, is the weight of the target position regression loss function, is the weight of the target positioning loss function, and is the weight of the feature point positioning loss function.

[0018] Preferably, after the step of obtaining the first target image according to the key points, the method further includes:

[0019] Performing image quality assessment on the first target image using a linear correlation coefficient and an evaluation data set;

[0020] If the image quality of the first target images meets the preset requirement, the step of calculating the similarity of each of the first target images according to the key points is performed.

[0021] Preferably, the calculating the similarity of each of the first target images according to the key points includes:

[0022] Calculating the correlation between the detection features of each key point and the response variable using the mutual information coefficient;

[0023] The similarity of each of the first target images is calculated based on the key points whose correlation is higher than a correlation threshold.

[0024] Preferably, judging whether the monitored target has abnormal behavior according to the second target image includes:

[0025] Acquiring the behavior of the monitored target in each detection period according to the second target image;

[0026] Performing a lightweight posture difference comparison on the behavior action based on the classification model to determine whether there is a first abnormal behavior;

[0027] If the first abnormal behavior does not exist, obtaining a detection skeleton model of the monitored target in the second target image;

[0028] The difference between the detection skeleton model and the standard skeleton model is calculated to determine whether a second abnormal behavior exists.

[0029] Preferably, after the step of determining whether the monitored target has abnormal behavior according to the second target image, the method further includes:

[0030] If the first abnormal behavior or the second abnormal behavior occurs, an alarm message is sent to a management personnel, where the alarm message includes the location information of the monitoring target where the first abnormal behavior or the second abnormal behavior occurs.

[0031] Preferably, the step of obtaining the to-be-processed images containing the monitored targets captured by each camera includes:

[0032] The images to be processed that include the monitored target and are taken by each of the cameras at the same time are obtained.

[0033] To solve the above technical problems, the present application also provides an abnormal behavior detection device, comprising:

[0034] The first acquisition module is used to acquire the image to be processed containing the monitored target taken by each camera;

[0035] a second acquisition module, configured to extract key points of the monitoring target in the image to be processed, and acquire a first target image based on the key points, wherein the first target image is an image including only one monitoring target, and the key points are points corresponding to body parts of the monitoring target;

[0036] a fusion module, configured to calculate the similarity of each of the first target images according to the key points, and fuse the first target images according to the similarity to obtain a second target image;

[0037] A judgment module is used to judge whether the monitored target has abnormal behavior based on the second target image.

[0038] To solve the above technical problems, the present application further provides an abnormal behavior detection device, comprising a memory for storing a computer program;

[0039] A processor is configured to implement the steps of the abnormal behavior detection method when executing the computer program.

[0040] To solve the above technical problems, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the abnormal behavior detection method are implemented.

[0041] The present application provides a method for detecting abnormal behavior, including: obtaining an image to be processed that includes a monitoring target and is taken by each camera, so as to determine whether the monitoring target has abnormal behavior. Extracting the key points of the monitoring target in the image to be processed, and obtaining a first target image that only includes one monitoring target based on the key points, so as to facilitate the subsequent extraction and judgment of the behavior of the monitoring target. Calculating the similarity of the first target images taken by each camera based on the key points, and fusing each first target image based on the similarity to obtain a second target image, and judging whether the monitoring target has abnormal behavior based on the second target image. It can be seen that the abnormal behavior detection method provided by the present application, by first extracting the key points and obtaining the first target image based on the key points, does not need to prioritize the target matching work, thereby improving the image processing speed. Simultaneously, the images to be processed taken by multiple cameras are used to detect the monitoring target, thereby improving the accuracy of behavior detection.

[0042] In addition, the present application also provides an abnormal behavior detection device and medium, which correspond to the above method and have the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] Figure 1 A flowchart of an abnormal behavior detection method provided in an embodiment of the present application;

[0045] Figure 2 A structural diagram of an abnormal behavior detection device provided in an embodiment of the present application;

[0046] Figure 3 This is a structural diagram of another abnormal behavior detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] The core of this application is to provide a method, device, and medium for detecting abnormal behavior.

[0049] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0050] The abnormal behavior detection method provided in this application is applied in examination rooms to monitor whether examinees have any illegal actions such as gesturing and passing notes, so as to reduce the workload of invigilators in the examination rooms and improve the accuracy of behavior detection.

[0051] In an exam room application scenario, multiple cameras installed in the exam room are used to simultaneously capture the exam room, thereby obtaining images to be processed that include each examinee. Key points of the examinee's body in the images to be processed are extracted, and a first target image containing only one examinee is obtained based on the key points, facilitating subsequent extraction and judgment of the examinee's behavior. The similarity of the first target images captured by each camera is calculated based on the key points, and the first target images captured by each camera are fused based on the similarity to obtain a second target image. Based on the second target image, it is determined whether the monitored target has abnormal behavior. Thus, the abnormal behavior detection method provided in this application, by first extracting key points and then obtaining the first target image based on the key points, eliminates the need for prioritizing target matching, thereby improving image processing speed. Simultaneously using the images to be processed captured by multiple cameras to detect the monitored target improves the accuracy of behavior detection. It is understandable that, since a large number of video images need to be processed simultaneously, in order to ensure the processing speed of the system, a distributed server deployment method can be adopted. The server obtains image data captured by a large number of cameras and dynamically allocates the more intensive computing tasks to each distributed server.

[0052] Figure 1 This is a flow chart of an abnormal behavior detection method provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0053] S10: Acquire the image to be processed that contains the monitored target and is captured by each camera.

[0054] It is understood that to ensure recognition accuracy and prevent recognition failure due to movement of the monitored target, each image to be processed is taken at the same moment (within a time error of 1ms). In practice, each camera can be controlled to capture images simultaneously, or video frames at the same moment can be selected from the video captured by the cameras. In this embodiment, the former solution is used.

[0055] In this embodiment, at least two cameras are used. By matching and fusing the images captured by the two cameras, human body matching and depth estimation can be achieved using the principle of multi-viewing. In a specific implementation, multiple cameras (e.g., three, five, etc.) can be used to perform error correction using images captured by multiple cameras. It is understood that the more cameras used, the more accurate the abnormal behavior recognition results, but the higher the cost.

[0056] S11: extract key points of the monitoring target in the image to be processed, and obtain a first target image based on the key points. The first target image is an image that only includes one monitoring target, and the key points are points corresponding to the body parts of the monitoring target.

[0057] It is understandable that in some application scenarios, due to factors such as brightness, obstacles, etc., the image of some areas may be inaccurate, such as the front and back rows of a classroom. To address this issue, the first target image can be screened and preprocessed.

[0058] In this embodiment, a CNN architecture is selected to perform skeleton detection and recognition in combination with key points of human body parts such as hands and face, so as to obtain a prediction frame containing only one monitoring target, and the prediction frame is regressed to obtain a first target image including only one monitoring target.

[0059] S12: Calculating the similarity of each first target image according to the key points, and fusing each first target image according to the similarity to obtain a second target image.

[0060] In the implementation, features are extracted from the CNN detection results of each frame to be processed. The same skeletons are then matched across the images captured by multiple cameras. A ReID-centroids model is used to perform 3D reconstruction and detection of keypoints on these matched skeletons. Multiple features with the same ID are leveraged to improve similarity. The ReID model used in tracking builds on this foundation by integrating multiple open-source datasets and compressing model features to 128 dimensions to improve generalization performance.

[0061] It should be noted that before calculating the similarity, it is necessary to sort and filter the features of each frame image to determine the features that have the greatest impact on the detection results, and to fuse the first target images based on the similarity of these features to obtain the second target image.

[0062] The fusion of the first target images involves obtaining the coordinates of key points on the human body from multiple camera views using multi-view stereo vision, completing matching and depth estimation. This includes calculating the initial camera matrix (including the spatial position, camera orientation, camera resolution, curvature, etc.) and using the camera matrix to reconstruct the 3D coordinates of the target point from multiple cameras.

[0063] S13: Determine whether the monitored target has abnormal behavior based on the second target image.

[0064] In the specific implementation, lightweight posture difference comparison is performed through the classification model. By comparing the behavior of the monitored target with the standard behavior, it is possible to identify whether the monitored target has abnormal behavior. The 3D skeleton model of the monitored target can also be identified to determine whether there is abnormal behavior.

[0065] Furthermore, when abnormal behavior of a monitoring target is detected, the coordinates of the monitoring target with abnormal behavior can be sent to a management personnel so that the management personnel can handle it in a timely manner.

[0066] In this embodiment, an abnormal behavior detection method is provided, including: obtaining an image to be processed that includes a monitoring target and is taken by each camera, so as to determine whether the monitoring target has abnormal behavior. Extracting the key points of the monitoring target in the image to be processed, and obtaining a first target image that only includes one monitoring target based on the key points, so as to facilitate the subsequent extraction and judgment of the behavior of the monitoring target. Calculating the similarity of the first target images taken by each camera based on the key points, and fusing each first target image based on the similarity to obtain a second target image, and judging whether the monitoring target has abnormal behavior based on the second target image. It can be seen that the abnormal behavior detection method provided by the present application, by first extracting the key points and obtaining the first target image based on the key points, does not need to prioritize the target matching work, thereby improving the image processing speed. Simultaneously, the images to be processed taken by multiple cameras are used to detect the monitoring target, thereby improving the accuracy of behavior detection.

[0067] In a specific implementation, the above abnormal behavior detection model needs to obtain detection features of key points of the monitored target, classify the target according to the detection features, and then obtain a first target image containing only the monitored target. The specific process of obtaining the first target image includes:

[0068] Use CNN network to extract key points in the image and build a detection skeleton;

[0069] Obtaining detection features of the monitoring target and detection features of key points of the monitoring target;

[0070] Combine the detection features of the monitored target and the classification loss function to match the sample with the highest category confidence and location confidence;

[0071] Obtain a prediction box of the first target image;

[0072] The multi-task loss function is calculated by monitoring the detection features of the target and the detection features of the key points, and the prediction box is regressed using the multi-task loss function to obtain the detection box;

[0073] The calculation formula of the multi-task loss function is L all =λ cls L cls +λ l1 L l1 +λ giou L giou +λ lan L lan , where L cls is the target classification loss function, L l1 is the target position regression loss function, L giou is the target positioning loss function, L lan is the feature point positioning loss function, λ cls is the weight of the target classification loss function, λ l1 is the weight of the target position regression loss function, λ giou is the weight of the target positioning loss function, λ lan The weight of the loss function for locating the feature points.

[0074] It's understandable that each element in an image has numerous features. This solution aims to detect abnormal behavior in the monitored target, so it only extracts the relevant detection features of the human body detection point and ignores other features. This reduces computational complexity, improves the model's detection efficiency, and reduces interference from noise signals. The sample with the highest confidence level is the one with the lowest loss function value.

[0075] In this embodiment, the feature detection network is first used to extract features of the image to be processed containing the monitoring target. It should be noted that while extracting the detection features of the monitoring target, it is also necessary to extract the detection features of the key points of the monitoring target, where the key points of the monitoring target refer to the points that can best reflect the body features of the monitoring target and the points that help determine the position of the monitoring target, such as hands, face and other human body parts. It can be understood that the above-mentioned prediction box is an inaccurate detection box obtained after sample matching of each data true value in the process of processing the image to be detected by the detection model, which is mainly used to determine the approximate position of the monitoring target. The present invention adopts a minimum cost allocation strategy, combines the classification loss function and the detection features of the monitoring target to perform sample matching on the data true value, where the matched samples are the samples with the highest category confidence and position confidence.

[0076] After obtaining the prediction frame, it is necessary to combine the detection features of the key points of the monitored target and the multi-task loss function to perform regression on the prediction frame to obtain an accurate detection frame and obtain the first target image based on the detection frame. In the multi-task loss function, the calculation method of each loss function is:

[0077] Target classification loss function

[0078] Target position regression loss function

[0079] Target localization loss function

[0080] Feature point positioning loss function

[0081] Among them, Y in formula (1) xyc is the ground truth label, is the label output by the network, and N is the number of samples. α is used to deal with the imbalance of positive and negative samples. γ is used to reduce the loss weight of simple samples and increase the loss weight of difficult samples. In formula (2), L l1 is a smooth loss function, which predicts the center coordinate of the box by regression (c x ,c y ) and width w and height h, calculate the loss function between the predicted box and the real box. In formula (3), L giou is the giou loss function, which calculates the giou loss between the predicted box and the real box by regressing the center coordinates (cx, cy) of the predicted box, its width w, and its height h. lan is the feature point positioning loss function.

[0082] In this embodiment, the detection features of the monitored target and the detection features of the key points of the monitored target are combined, and a regression operation is performed on the estimated detection frame through a multi-task loss function to obtain an accurate detection frame and a first target image, so that the monitored target detection is more accurate and can better complete the task of detecting abnormal actions.

[0083] On the basis of the above embodiment, in order to prevent the acquired first target image from being unclear due to factors such as obstruction by obstacles and insufficient lighting, thereby introducing invalid data and affecting the detection result of abnormal behavior of the monitored target, on the basis of the above embodiment, after the step of acquiring the first target image according to the key points, the following further comprises:

[0084] performing image quality assessment on the first target image using the linear correlation coefficient and the evaluation data set;

[0085] If the image quality of the first target image meets the preset requirement, the step of calculating the similarity of each first target image according to the key points is performed.

[0086] In this embodiment, the quality of the first target image is evaluated, and only when the first target image meets the preset requirements is the first target image used as a valid image and fused with other images, thereby improving the accuracy of abnormal behavior detection.

[0087] As a preferred embodiment, determining whether the monitored target has abnormal behavior according to the second target image includes:

[0088] Acquire the behavior of the monitored target in each detection cycle according to the second target image;

[0089] Performing a lightweight posture difference comparison on the behavioral actions based on the classification model to determine whether there is a first abnormal behavior;

[0090] If the first abnormal behavior does not exist, obtaining a detection skeleton model of the monitored target in the second target image;

[0091] The difference between the detection skeleton model and the standard skeleton model is calculated to determine whether a second abnormal behavior exists.

[0092] It can be understood that the movement amplitude of the first abnormal behavior is greater than the movement amplitude of the second abnormal behavior.

[0093] In the specific implementation, based on the scikit-learn classification model, lightweight posture difference comparison is performed. Model actions can be specified, such as turning the head and reaching out. After the system recognizes the 3D skeleton, it will first perform a difference comparison and provide a set of difference similarities between the current skeleton and all model skeletons.

[0094] Based on the ST-GCN spatiotemporal graph convolutional network model, 3D skeletons are collected over a period of time to determine motion. Pre-trained actions are used, and the server identifies changes in the skeleton points of the same person over time. Using ST-GCN, the server determines the motion occurring during that time period. When a specific motion is detected, an alarm event is triggered, such as sending an alarm message to the server's message control center.

[0095] In this embodiment, whether the monitored target has abnormal behavior is determined by lightweight posture difference comparison and 3D skeleton comparison, thereby improving the speed of abnormal behavior detection.

[0096] In specific implementation, when abnormal behavior of the monitoring target is detected, an alarm message needs to be sent so that the management personnel can deal with the monitoring target with abnormal behavior in a timely manner.

[0097] Based on the above embodiment, after the step of determining whether the monitored target has abnormal behavior according to the second target image, the method further includes:

[0098] If the first abnormal behavior or the second abnormal behavior occurs, an alarm message is sent to a management personnel, where the alarm message includes location information of the monitoring target where the first abnormal behavior or the second abnormal behavior occurs.

[0099] It is understandable that after sending the alarm information to the management personnel, the monitoring personnel can also be reminded. The form of the reminder can be selected according to the actual situation. For example: in the online class application scenario, when abnormal behavior of students is detected (for example, leaving the location, picking up the mobile phone, etc.), a reminder message can be displayed on the student's display screen; in the posture competition application scenario, when abnormal behavior of the contestants is detected, the indicator light can be controlled to flash, the buzzer can be controlled to sound an alarm, etc.

[0100] Furthermore, in order to prevent misjudgment, an alarm message may be sent to a management staff member when the number of times a user's abnormal behavior is detected is greater than a threshold.

[0101] In the above embodiments, the abnormal behavior detection method is described in detail. This application also provides corresponding embodiments of the abnormal behavior detection device. It should be noted that this application describes the embodiments of the device from two perspectives: one is based on the functional module perspective, and the other is based on the hardware perspective.

[0102] Figure 2 This is a structural diagram of an abnormal behavior detection device provided in an embodiment of the present application, such as Figure 2 As shown, the device includes:

[0103] The first acquisition module 10 is used to acquire the image to be processed containing the monitored target taken by each camera;

[0104] A second acquisition module 11 is used to extract key points of the monitoring target in the image to be processed, and obtain a first target image based on the key points. The first target image is an image that only includes one monitoring target, and the key points are points corresponding to the body parts of the monitoring target;

[0105] a fusion module 12, configured to calculate the similarity of each first target image based on the key points, and fuse the first target images based on the similarity to obtain a second target image;

[0106] The judgment module 13 is configured to judge whether the monitored target has abnormal behavior according to the second target image.

[0107] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, and they will not be repeated here.

[0108] The present application provides an abnormal behavior detection device, including: obtaining an image to be processed that includes a monitoring target taken by each camera, so as to determine whether the monitoring target has abnormal behavior. Extracting the key points of the monitoring target in the image to be processed, and obtaining a first target image that only includes one monitoring target based on the key points, so as to facilitate the subsequent extraction and judgment of the behavior of the monitoring target. Calculating the similarity of the first target images taken by each camera based on the key points, and fusing each first target image based on the similarity to obtain a second target image, and judging whether the monitoring target has abnormal behavior based on the second target image. It can be seen that the abnormal behavior detection device provided by the present application, by first extracting the key points and obtaining the first target image based on the key points, does not need to prioritize the target matching work, thereby improving the image processing speed. Simultaneously, the images to be processed taken by multiple cameras are used to detect the monitoring target, thereby improving the accuracy of behavior detection.

[0109] Figure 3 This is a structural diagram of another abnormal behavior detection device provided in an embodiment of the present application, such as Figure 3 As shown, the abnormal behavior detection device includes: a memory 20 for storing computer programs;

[0110] The processor 21 is configured to implement the steps of the abnormal behavior detection method in the above embodiment when executing a computer program.

[0111] The server provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0112] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0113] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the abnormal behavior detection method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to the data mentioned in the above-mentioned abnormal behavior detection method.

[0114] In some embodiments, the abnormal behavior detection device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .

[0115] Those skilled in the art will understand that Figure 3 The structure shown in the figure does not constitute a limitation on the abnormal behavior detection device, and may include more or fewer components than shown in the figure.

[0116] The abnormal behavior detection device provided in the embodiment of the present application includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method:

[0117] Obtain the images to be processed that contain the monitored targets taken by each camera;

[0118] Extracting key points of the monitoring target in the image to be processed, and obtaining a first target image based on the key points, where the first target image is an image including only one monitoring target;

[0119] Calculating the similarity of each first target image according to the key points, and fusing the first target images according to the similarity to obtain a second target image;

[0120] Determine whether the monitored target has abnormal behavior based on the second target image.

[0121] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiment.

[0122] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0123] The above is a detailed introduction to the abnormal behavior detection method, device, and medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0124] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A method for detecting abnormal behavior, characterized in that: include: Obtain the images to be processed that contain the monitored targets taken by each camera; Extracting key points of the monitored target in the image to be processed, and acquiring a first target image based on the key points, wherein the first target image is an image including only one monitored target, and the key points are points corresponding to body parts of the monitored target, including points corresponding to the hands and face of the monitored target; performing image quality assessment on the first target images using a linear correlation coefficient and an assessment data set; if the image quality of the first target images meets a preset requirement, calculating similarity of the first target images based on the key points, and fusing the first target images based on the similarity to obtain a second target image; The process of fusing the first target images includes: performing matching and depth estimation on the coordinates of the key points in the first target images based on multi-view stereo vision, specifically calculating the spatial position, camera steering, camera resolution and curvature of each camera, obtaining an initial matrix of multiple cameras, and using the initial matrix to perform 3D reconstruction on the coordinates of the same target point in multiple cameras; determining whether the monitored target has abnormal behavior according to the second target image; Among them, judging whether the monitored target has abnormal behavior based on the second target image includes: obtaining the behavioral actions of the monitored target in each detection cycle based on the second target image; performing a lightweight posture difference comparison on the behavioral actions based on a classification model to determine whether there is a first abnormal behavior; if the first abnormal behavior does not exist, obtaining the detection skeleton model of the monitored target in the second target image; calculating the difference between the detection skeleton model and the standard skeleton model within a time period to determine whether there is a second abnormal behavior, and the movement amplitude of the first abnormal behavior is greater than the movement amplitude of the second abnormal behavior.

2. The abnormal behavior detection method according to claim 1, characterized in that: The extracting key points of the monitoring target in the image to be processed and acquiring a first target image according to the key points includes: Utilize the CNN network to extract the key points in the image and construct a detection skeleton; Acquire detection features of the detection skeleton and detection features of the key points; Combining the detection features of the detection skeleton and the classification loss function, matching the sample with the highest category confidence and position confidence; Obtaining a prediction frame of the first target image; Calculating a multi-task loss function using the detection features of the detection skeleton and the detection features of the key points, and performing a regression operation on the prediction box using the multi-task loss function to obtain the first target image; The calculation formula of the multi-task loss function is: ,in is the target classification loss function, is the target position regression loss function, is the target positioning loss function, is the feature point positioning loss function, is the weight of the target classification loss function, is the weight of the target position regression loss function, is the weight of the target positioning loss function, The weight of the loss function for locating the feature points.

3. The abnormal behavior detection method according to claim 2, characterized in that: Calculating the similarity of each of the first target images according to the key points includes: Calculating the correlation between the detection features of each key point and the response variable using the mutual information coefficient; The similarity of each of the first target images is calculated based on the key points whose correlation is higher than a correlation threshold.

4. The abnormal behavior detection method according to claim 1, characterized in that: After the step of determining whether the monitored target has abnormal behavior according to the second target image, the method further includes: If the first abnormal behavior or the second abnormal behavior occurs, an alarm message is sent to a management personnel, where the alarm message includes the location information of the monitoring target where the first abnormal behavior or the second abnormal behavior occurs.

5. The abnormal behavior detection method according to claim 1, characterized in that: The step of obtaining the images to be processed that contain the monitored target and are taken by each camera includes: The images to be processed that include the monitored target and are taken by each of the cameras at the same time are obtained.

6. An abnormal behavior detection device, characterized in that: include: The first acquisition module is used to acquire the image to be processed containing the monitored target taken by each camera; a second acquisition module, configured to extract key points of the monitored target in the image to be processed, and acquire a first target image based on the key points, wherein the first target image is an image including only one monitored target, and the key points are points corresponding to body parts of the monitored target, including points corresponding to the hands and face of the monitored target; a fusion module, configured to perform image quality evaluation on the first target images using a linear correlation coefficient and an evaluation data set; if the image quality of the first target images meets a preset requirement, calculate the similarity of each of the first target images based on the key points, and fuse the first target images based on the similarity to obtain a second target image; The fusion module is specifically configured to perform matching and depth estimation of the coordinates of the key points in each of the first target images based on multi-view stereo vision, specifically calculating the spatial position, camera steering, camera resolution, and curvature of each camera, obtaining an initial matrix of multiple cameras, and using the initial matrix to perform 3D reconstruction of the coordinates of the same target point across multiple cameras; a judgment module, configured to judge whether the monitored target has abnormal behavior according to the second target image; The judgment module is specifically configured to: obtain the behavior of the monitored target in each detection cycle according to the second target image; perform a lightweight posture difference comparison on the behavior based on the classification model to determine whether there is a first abnormal behavior; If the first abnormal behavior does not exist, obtain the detection skeleton model of the monitored target in the second target image; calculate the difference between the detection skeleton model and the standard skeleton model within a time period to determine whether there is a second abnormal behavior, and the movement amplitude of the first abnormal behavior is greater than the movement amplitude of the second abnormal behavior.

7. An abnormal behavior detection device, characterized in that: including a memory for storing a computer program; A processor, configured to implement the steps of the abnormal behavior detection method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the abnormal behavior detection method according to any one of claims 1 to 5 are implemented.

Images