Behavior recognition method and device based on depth recognition, computer equipment, readable storage medium and program product

Abstract

The invention relates to a behavior recognition method and device based on deep recognition, computer equipment, a readable storage medium and a program product. The method comprises the steps of recognizing the distance between each person based on a conference image through a behavior recognition model, outputting an initial abnormal behavior recognition result corresponding to each person, carrying out depth recognition on the conference image based on a depth recognition model, outputting depth information corresponding to the conference image, and carrying out abnormal behavior recognition on the conference image according to the initial abnormal behavior recognition result and the depth information. And determining an abnormal behavior identification result of the personnel. Compared with a traditional method of manually identifying whether the person has an abnormal behavior, the method carries out the preliminary behavior analysis of the person in the conference image through the behavior identification model, and employs the depth identification model to identify the depth information of the conference image. And the abnormal behavior recognition result of each person is determined by combining the preliminary behavior analysis result and the depth information, so that the technical effect of improving the accuracy of behavior recognition is achieved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a behavior recognition method, apparatus, computer equipment, computer-readable storage medium, and computer program product based on deep recognition. Background Art

[0002] In serious scenarios such as meetings, participants are required to maintain a serious attitude to maintain the normal progress of the meeting. Therefore, it is necessary to detect the behavior of meeting participants to promptly discover abnormal behavior of meeting participants. Currently, the method of analyzing the behavior of meeting participants is usually to manually detect abnormal behavior of meeting participants. However, manually detecting the behavior of meeting participants will reduce the detection accuracy. Summary of the invention

[0003] Based on this, it is necessary to provide a behavior recognition method, device, computer equipment, computer-readable storage medium and computer program product based on deep recognition that can improve detection accuracy in response to the above technical problems.

[0004] In a first aspect, the present application provides a behavior recognition method based on deep recognition, the method comprising:

[0005] Acquire a conference image to be analyzed; the conference image includes an image corresponding to each person to be analyzed;

[0006] Input the conference image into a trained behavior recognition model, and the behavior recognition model recognizes the distances between the persons based on the conference image, and outputs initial abnormal behavior recognition results corresponding to the persons in the conference image based on the distances;

[0007] Inputting the conference image into a trained depth recognition model, allowing the depth recognition model to perform depth recognition on the conference image and output depth information corresponding to the conference image;

[0008] According to each of the initial abnormal behavior recognition results and the depth information, an abnormal behavior recognition result corresponding to each of the persons in the conference image is determined.

[0009] In one embodiment, the inputting the conference image into a trained behavior recognition model, the behavior recognition model identifying the distances between the persons based on the conference image, and outputting initial abnormal behavior recognition results corresponding to the persons in the conference image based on the distances, comprises:

[0010] Input the conference image into a trained behavior recognition model, extract the human body features corresponding to the persons based on the conference image by the behavior recognition model, and determine the position areas of the persons in the conference image based on the human body features;

[0011] Acquire each distance between each of the location areas in the conference image;

[0012] Determine each abnormal behavior group based on at least two location areas to be identified whose distance is less than or equal to a preset distance threshold; wherein each abnormal behavior group represents that the initial abnormal behavior identification result of at least two persons to be identified corresponding to at least two location areas to be identified is that there is an abnormal behavior to be determined.

[0013] In one embodiment, the step of inputting the conference image into a trained depth recognition model, allowing the depth recognition model to perform depth recognition on the conference image, and outputting depth information corresponding to the conference image includes:

[0014] Inputting the conference image into a trained deep recognition model, and extracting color features, texture features, and structural features corresponding to the conference image by the deep recognition model;

[0015] According to the color feature, the texture feature and the structural feature, identifying each pixel depth information corresponding to each pixel point in the conference image;

[0016] Depth information corresponding to the conference image is output according to the depth information of each pixel.

[0017] In one embodiment, determining the abnormal behavior recognition result corresponding to each of the persons in the conference image according to each of the initial abnormal behavior recognition results and the depth information includes:

[0018] According to each of the initial abnormal behavior recognition results, each abnormal behavior group having pending abnormal behaviors in the conference image is determined; wherein each of the abnormal behavior groups includes at least two location areas to be identified corresponding to at least two persons to be identified;

[0019] For the position area to be identified corresponding to each person to be identified, determine the depth information of each pixel corresponding to each pixel point in the position area to be identified according to the depth information;

[0020] Determine, according to each pixel depth information, a maximum depth value corresponding to the position area to be identified as the area depth information corresponding to the position area to be identified;

[0021] For each abnormal behavior group, if there are at least two location areas with the same depth in the abnormal behavior group, it is determined that at least two persons to be identified corresponding to the at least two location areas with the same depth have abnormal behavior; wherein the difference between the depth information of at least two areas corresponding to the at least two location areas with the same depth is less than or equal to a preset depth threshold.

[0022] In one embodiment, obtaining the conference image to be analyzed includes:

[0023] Acquire the original conference image captured by the image acquisition device;

[0024] The image size is input according to a preset model, and the size of the original conference image is adjusted to obtain the conference image to be analyzed.

[0025] In one embodiment, before inputting the conference image into the trained deep recognition model, the method further includes:

[0026] Determine, according to each of the initial abnormal behavior recognition results, a location area corresponding to each of the persons in the conference image;

[0027] Determine the minimum personnel area in the conference image according to each of the position areas; the minimum personnel area represents the minimum area containing each of the personnel;

[0028] The conference image is cropped according to the minimum personnel area, and the cropped conference image is input into the trained depth recognition model.

[0029] In a second aspect, the present application provides a behavior recognition device based on deep recognition, the device comprising:

[0030] An acquisition module, used to acquire a conference image to be analyzed; the conference image includes an image corresponding to each person to be analyzed;

[0031] A first input module is used to input the conference image into a trained behavior recognition model, and the behavior recognition model recognizes the distances between the persons based on the conference image, and outputs initial abnormal behavior recognition results corresponding to the persons in the conference image based on the distances;

[0032] A second input module, used to input the conference image into a trained depth recognition model, so that the depth recognition model performs depth recognition on the conference image and outputs depth information corresponding to the conference image;

[0033] The recognition module is used to determine the abnormal behavior recognition results corresponding to each of the persons in the conference image according to each of the initial abnormal behavior recognition results and the depth information.

[0034] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0036] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0037] The above-mentioned behavior recognition method, device, computer equipment, computer-readable storage medium and computer program product based on deep recognition use a behavior recognition model to identify the distance between each person in the conference image, output the initial abnormal behavior recognition result corresponding to each person, perform deep recognition on the conference image based on the deep recognition model, output the depth information corresponding to the conference image, and determine the abnormal behavior recognition result of the person based on the initial abnormal behavior recognition result and the depth information. Compared with the traditional manual identification of whether a person has abnormal behavior, this solution uses a behavior recognition model to perform preliminary behavior analysis on the people in the conference image, uses a deep recognition model to identify the depth information of the conference image, and combines the results of the preliminary behavior analysis with the depth information to determine the abnormal behavior recognition result of each person, thereby achieving the technical effect of improving the accuracy of behavior recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0039] FIG1 is a schematic diagram of a flow chart of a behavior recognition method based on deep recognition in one embodiment;

[0040] FIG2 is a schematic diagram of a behavior recognition model recognition step in one embodiment;

[0041] FIG3 is a schematic diagram of a depth recognition step in one embodiment;

[0042] FIG4 is a schematic diagram of an abnormal behavior identification step in one embodiment;

[0043] FIG5 is a flow chart of a behavior recognition method based on depth recognition in another embodiment;

[0044] FIG6 is a structural block diagram of a behavior recognition device based on depth recognition in one embodiment;

[0045] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In one embodiment, as shown in FIG1 , a behavior recognition method based on deep recognition is provided. This embodiment uses the method applied to a terminal as an example. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server, including the following steps S202 to S208. Among them:

[0048] Step S202, obtaining a conference image to be analyzed; the conference image includes an image corresponding to each person to be analyzed.

[0049] The conference image may be an image captured by an image acquisition device. The image acquisition device may be set in a conference room or other conference location. The conference image may include images corresponding to each person attending the conference. The conference image may be a two-dimensional image. The terminal may pre-process the original conference image captured by the image acquisition device to obtain the conference image to be analyzed.

[0050] In one embodiment, obtaining a conference image to be analyzed includes: obtaining an original conference image captured by an image acquisition device; inputting an image size according to a preset model, resizing the original conference image, and obtaining the conference image to be analyzed.

[0051] In this embodiment, the image acquisition device set up in a conference room or other meeting place can be a camera, and the terminal can obtain the original conference image captured by the camera, and the model input image size can be preset in the terminal. Among them, the terminal can use the model to identify the conference image to determine whether each person therein has abnormal behavior. The model used to identify abnormal behavior has corresponding requirements for the input image, and the terminal can pre-set the preset model input image size based on the image input requirements corresponding to the behavior recognition model. The terminal can resize the above-collected original conference image according to the preset model input image size to obtain the conference image to be analyzed. Specifically, the terminal can crop the original conference image based on the preset model input image size, and then obtain the conference image to be analyzed that meets the above-mentioned preset model input image size.

[0052] Step S204, input the conference image into the trained behavior recognition model, and the behavior recognition model recognizes the distances between the persons based on the conference image, and outputs the initial abnormal behavior recognition results corresponding to the persons in the conference image based on the distances.

[0053] Among them, the behavior recognition model can be used to identify whether each person in the meeting image has abnormal behavior. Among them, each person needs to remain upright in the meeting, so the people in the meeting cannot get close to each other to whisper and other behaviors. The terminal can realize abnormal behavior recognition through target detection and algorithm cascade. For example, the terminal first detects the target through the basic target detection algorithm, and performs depth estimation on the detected target, removes the targets that are not in the same spatial position, and then introduces the monocular depth estimation algorithm. When outputting the post-detection processing, the problem of overlapping people can be effectively solved. Then the terminal can use the behavior recognition model to preliminarily identify the above abnormal behavior.

[0054] Among them, the terminal can pre-train the behavior recognition model to be trained. For example, the terminal can obtain a meeting image sample and a corresponding initial abnormal behavior recognition result sample, the terminal inputs the meeting image sample into the behavior recognition model, and compares the initial abnormal behavior recognition prediction result based on the output of the behavior recognition model with the corresponding initial abnormal behavior recognition result sample, thereby obtaining the corresponding first recognition result similarity. The terminal can adjust the model parameters of the behavior recognition model based on the comparison result of the above-mentioned first recognition result similarity and the preset first recognition result similarity threshold, until the preset training end condition is met, and a trained behavior recognition model is obtained. Among them, the preset training end condition includes but is not limited to within the preset training times, the above-mentioned comparison result is that the difference between the first recognition result similarity and the preset first recognition result similarity threshold is less than or equal to the preset similarity threshold, or the training times reach the above-mentioned preset training times.

[0055] Thus, the terminal can use the trained behavior recognition model to perform preliminary behavior recognition on the behavior of each person in the conference image. For example, the terminal can input the conference image into the trained behavior recognition model, and the behavior recognition model can recognize the distances between each person based on the conference image, and output the initial abnormal behavior recognition results corresponding to each person in the conference image based on each distance. Among them, each initial abnormal behavior recognition result represents a preliminary recognition result of whether there is abnormal behavior between the two above-mentioned persons. The above-mentioned preliminary abnormal behavior recognition result is a recognition result that does not take into account the spatial position of each person, that is, the depth information corresponding to each person in the image has not been considered. Therefore, the terminal needs to perform further depth recognition on the conference image before obtaining the final abnormal behavior recognition result corresponding to each person.

[0056] Step S206: input the conference image into the trained depth recognition model, and the depth recognition model performs depth recognition on the conference image, and outputs depth information corresponding to the conference image.

[0057] Among them, the deep recognition model can be a neural network model for identifying the depth information of the conference image. Among them, the terminal can pre-train the deep recognition model to be trained. For example, the terminal can obtain the conference image sample and the corresponding depth information sample, and the terminal inputs the conference image sample into the deep recognition model, and compares the depth information prediction result output by the deep recognition model with the corresponding depth information sample, thereby obtaining the corresponding second recognition result similarity. The terminal can adjust the model parameters of the deep recognition model based on the comparison result of the above-mentioned second recognition result similarity and the preset second recognition result similarity threshold, until the preset training end condition is met, and the trained deep recognition model is obtained. Among them, the preset training end condition includes but is not limited to within the preset training times, the above-mentioned comparison result is that the difference between the second recognition result similarity and the preset second recognition result similarity threshold is less than or equal to the preset similarity threshold, or the training times reach the above-mentioned preset training times.

[0058] Thus, the terminal can use the trained depth recognition model to identify the depth information corresponding to each person in the conference image. For example, the terminal inputs the conference image into the trained depth recognition model, and the depth recognition model performs depth recognition on the conference image and outputs the depth information corresponding to the conference image. Among them, the depth recognition model can obtain the depth information containing each depth value of each position in the conference image by identifying each pixel depth information of each pixel point in the conference image.

[0059] Step S208, determining the abnormal behavior recognition result corresponding to each person in the conference image according to each initial abnormal behavior recognition result and the depth information.

[0060] Among them, each initial abnormal behavior recognition result includes the recognition result of whether there is abnormal behavior between every two people in the conference image. Among them, abnormal behavior includes behaviors such as approaching and whispering. The terminal can determine the abnormal behavior recognition results corresponding to each person in the conference image in combination with each initial abnormal behavior recognition result and the above-mentioned depth information. For example, the terminal can determine the persons to be identified in the conference image who are suspected of having undetermined abnormal behaviors based on the initial abnormal behavior recognition results, and determine the position area of ​​each person to be identified in the conference image. The terminal determines whether the depth information of each position area is in the same depth range, so that the terminal can determine each person in the same depth range and the distance between the position areas is less than the preset distance threshold as a person with abnormal behavior, and then obtain the abnormal behavior recognition result.

[0061] In the above-mentioned behavior recognition method based on deep recognition, the distance between each person in the conference image is recognized by the behavior recognition model, and the initial abnormal behavior recognition result corresponding to each person is output. The conference image is deeply recognized based on the deep recognition model, and the depth information corresponding to the conference image is output. The abnormal behavior recognition result of the person is determined based on the initial abnormal behavior recognition result and the depth information. Compared with the traditional manual identification of whether a person has abnormal behavior, this solution uses the behavior recognition model to perform preliminary behavior analysis on the people in the conference image, uses the deep recognition model to identify the depth information of the conference image, and combines the results of the preliminary behavior analysis with the depth information to determine the abnormal behavior recognition result of each person, thereby achieving the technical effect of improving the accuracy of behavior recognition.

[0062] In one embodiment, a conference image is input into a trained behavior recognition model, and the behavior recognition model recognizes the distances between the various persons based on the conference image, and outputs the initial abnormal behavior recognition results corresponding to the various persons in the conference image based on the various distances, including: inputting the conference image into a trained behavior recognition model, and having the behavior recognition model extract the human body features corresponding to the various persons based on the conference image, and determining the various position areas of the various persons in the conference image based on the various human body features; obtaining the various distances between the various position areas in the conference image; determining various abnormal behavior groups based on at least two position areas to be identified whose distances are less than or equal to a preset distance threshold; wherein each abnormal behavior group characterizes that the initial abnormal behavior recognition results of at least two persons to be identified corresponding to at least two position areas to be identified are that there are pending abnormal behaviors.

[0063] In this embodiment, the terminal can identify abnormal behaviors between the above-mentioned personnel based on the behavior recognition model and the posture characteristics of the human body. For example, the terminal can input the conference image into the trained behavior recognition model, and the behavior recognition model can extract the human body features corresponding to each person based on the conference image, such as the features of the head and upper body posture, etc. The behavior recognition model determines the various position areas of each person in the conference image based on the various human body features. Among them, the behavior recognition model can determine the minimum rectangular detection frame corresponding to each person based on the human body features corresponding to each person, so that the behavior recognition model obtains the various position areas corresponding to each person based on each minimum rectangular detection frame.

[0064] The behavior recognition model can obtain the distances between the various position areas in the conference image. The above distances may be the distances between the centers of the various position areas. The behavior recognition model can obtain at least two position areas to be identified whose distances are less than or equal to a preset distance threshold, wherein there may be at least two positions whose distances are less than or equal to the preset distance threshold, which indicates that there may be abnormal behaviors such as private conversations between adjacent persons, so that the behavior recognition model can determine each abnormal behavior group based on the at least two position areas to be identified whose distances are less than or equal to the preset distance threshold. Each abnormal behavior group characterizes that the initial abnormal behavior recognition results of at least two persons to be identified corresponding to at least two position areas to be identified are that there are pending abnormal behaviors.

[0065] Among them, the various persons in the above-mentioned abnormal behavior groups may or may not be sitting in the same row. When the distance between people in the same row is too close, it will be determined that abnormal behavior exists. Therefore, the terminal can preliminarily identify the various persons to be identified who may have abnormal behavior through the behavior recognition model, and form abnormal behavior groups, and further use deep information to analyze each abnormal behavior group to determine the individuals with real abnormal behavior.

[0066] Specifically, as shown in FIG. 2, FIG. 2 is a schematic diagram of the identification step of the behavior recognition model in one embodiment. The above-mentioned behavior recognition model may be a yolov8 model. Among them, Yolov8 is a version of the YOLO series algorithm in the field of target detection. It excels in real-time object detection, segmentation and classification with its efficient and accurate target detection capabilities. Compared with the previous generation of algorithms, multiple optimizations and improvements have been made to further improve the detection accuracy and speed. The terminal can first crop the conference image to a preset size, input the trained yolov8 model for detection, and obtain the various personnel detection frames shown in FIG. 2, wherein the detection frame containing one person may be the location area to be identified corresponding to each person, and the detection frame containing at least two persons may be each abnormal behavior group. The behavior recognition model identifies each person who may have abnormal behaviors such as talking by comparing the distance and position between each location area, and forms an abnormal behavior group based on the location area of ​​each person.

[0067] Through this embodiment, the terminal can combine the behavior recognition model to identify the human body features corresponding to each person in the conference image, identify abnormal behaviors that may exist in the conference image, and form corresponding abnormal behavior groups. The terminal can then perform in-depth analysis based on the abnormal behavior groups to determine the individuals who actually have abnormal behaviors, thereby improving the accuracy of identifying abnormal behaviors in meetings.

[0068] In one embodiment, a conference image is input into a trained depth recognition model, the depth recognition model performs depth recognition on the conference image, and outputs depth information corresponding to the conference image, including: inputting the conference image into a trained depth recognition model, extracting color features, texture features, and structural features corresponding to the conference image by the depth recognition model; identifying each pixel depth information corresponding to each pixel point in the conference image based on the color features, texture features, and structural features; and outputting the depth information corresponding to the conference image based on the depth information of each pixel.

[0069] In this embodiment, the terminal can analyze the depth of each position in the conference image through a depth recognition model. The depth represents the distance of each position in the conference image from the image acquisition device. The farther away from the image acquisition device, the greater the depth. For example, the terminal can input the conference image into a trained depth recognition model, and the depth recognition model can extract the color features, texture features, and structural features corresponding to the conference image. Thus, the depth recognition model can identify the pixel depth information corresponding to each pixel point in the conference image based on the color features, texture features, and structural features. The pixel depth information represents the depth information of each pixel in the conference image. The depth recognition model can output the depth information corresponding to the conference image based on the depth information of each pixel. Different depths can be represented in the conference image using different grayscales, and the darker the color, the greater the depth.

[0070] Specifically, as shown in Figure 3, Figure 3 is a schematic diagram of the depth recognition step in an embodiment. The terminal can first crop the target area of ​​the conference image. Among them, due to the uncertainty of the angle of the monitoring shooting picture and the size of the photographed area, the upper and lower thresholds of the depth estimation will be affected to a certain extent. The depth recognition model can calculate the minimum target area including all personnel frames in the conference image, and crop the minimum target area. The above-mentioned depth recognition model can be a DepthanythingV2 depth estimation model. Among them, DepthanythingV2 is an algorithm model for monocular depth estimation, which can predict the depth information of the scene from a single image. Monocular depth estimation refers to estimating the distance and depth of each object in the scene using only an image obtained by one camera. After the terminal inputs the above-mentioned conference image into the DepthanythingV2 depth estimation model, the depth recognition model can crop the conference image, and then determine the depth information of each pixel in the conference image through a monocular depth estimation algorithm, such as the depth values ​​corresponding to the above-mentioned minimum target area.

[0071] Through this embodiment, the terminal can use the depth recognition model to identify the depth value in the conference image to form depth information, so that the terminal can combine the depth information and the initial abnormal behavior recognition results of the behavior recognition model to identify abnormal behavior of each person in the conference image, thereby improving the accuracy of identifying abnormal behavior in the meeting.

[0072] In one embodiment, based on each initial abnormal behavior recognition result and depth information, the abnormal behavior recognition result corresponding to each person in the conference image is determined, including: based on each initial abnormal behavior recognition result, each abnormal behavior group in the conference image with pending abnormal behaviors is determined; each abnormal behavior group includes at least two to-be-identified position areas corresponding to at least two to-be-identified persons; for each to-be-identified position area corresponding to the to-be-identified person, each pixel depth information corresponding to each pixel point in the to-be-identified position area is determined based on the depth information; based on each pixel depth information, the maximum depth value corresponding to the to-be-identified position area is determined as the regional depth information corresponding to the to-be-identified position area; for each abnormal behavior group, if there are at least two position areas with the same depth in the abnormal behavior group, it is determined that at least two to-be-identified persons corresponding to the at least two position areas with the same depth have abnormal behaviors; wherein the difference between at least two regional depth information corresponding to at least two position areas with the same depth is less than or equal to a preset depth threshold.

[0073] In this embodiment, the terminal can analyze whether each person in the conference image has abnormal behavior in combination with the initial abnormal behavior recognition result and the depth information. The initial abnormal behavior recognition result can be obtained based on the behavior recognition model, and the terminal can determine each abnormal behavior group in the conference image with pending abnormal behavior according to each initial abnormal behavior recognition result. Each abnormal behavior group includes at least two location areas to be identified corresponding to at least two persons to be identified.

[0074] For each position area to be identified corresponding to a person to be identified, the terminal can obtain the depth information corresponding to the conference image, so that the terminal can determine the pixel depth information corresponding to each pixel point in the position area to be identified based on the depth information, and determine the maximum depth value corresponding to the position area to be identified based on the pixel depth information as the regional depth information corresponding to the position area to be identified, that is, the terminal can fill the position area with the above-mentioned maximum depth value. For each abnormal behavior group, if the terminal detects that there are at least two position areas with the same depth in the abnormal behavior group, the terminal can determine that at least two persons to be identified corresponding to at least two position areas with the same depth have abnormal behavior. Among them, the difference between at least two regional depth information corresponding to at least two position areas with the same depth is less than or equal to the preset depth threshold, that is, when at least two position areas with the same depth are in the same depth range, it indicates that the corresponding at least two persons have abnormal behavior.

[0075] Specifically, as shown in FIG4, FIG4 is a schematic diagram of an abnormal behavior identification step in an embodiment. The terminal can obtain each abnormal behavior group based on the initial abnormal identification result, so that the terminal can obtain the personnel detection frame corresponding to each person in each abnormal behavior group, that is, the position area corresponding to each person, so that the terminal can take the depth of the maximum depth histogram for each position area as the depth of the position area, for example, the maximum depth value in each position area is taken as the overall depth of the position area. For each person in the abnormal behavior group, the terminal can determine whether these people are in the same depth range, for example, compare the depth difference between each position area corresponding to each person in the abnormal behavior with a preset depth threshold, determine each person belonging to the same depth range, and then correct the detection frame corresponding to the abnormal behavior group. The corrected detection frame is shown in FIG4, which includes two people in the same row, that is, the same depth range, thereby eliminating false detections in the front and back rows, and identifying each person with real abnormal behavior.

[0076] Through this embodiment, the terminal can combine the preliminary abnormal behavior recognition result and the depth information to detect whether each person in the conference image has abnormal behavior, thereby improving the accuracy of abnormal behavior recognition of conference personnel.

[0077] In one embodiment, before inputting the conference image into the trained deep recognition model, it also includes: determining the location area corresponding to each person in the conference image based on each initial abnormal behavior recognition result; determining the minimum person area in the conference image based on each location area; the maximum person area represents the minimum area containing each person; cropping the conference image according to the minimum person area, and inputting the cropped conference image into the above-mentioned trained deep recognition model.

[0078] In this embodiment, the above-mentioned depth recognition model can recognize the depth information in the conference image, and the terminal can crop the conference image in advance. For example, the terminal can determine the position area corresponding to each person in the conference image based on each initial abnormal behavior recognition result output by the behavior recognition model; and determine the minimum person area in the conference image based on each position area. The minimum person area represents the minimum area containing all people. Therefore, the terminal can crop the conference image according to the minimum person area and input the cropped conference image into the above-mentioned trained depth recognition model.

[0079] Specifically, the above-mentioned conference image can be captured by a camera. Due to the uncertainty of the shooting angle and the size of the photographed area, the upper and lower thresholds of the depth estimation will be affected to a certain extent. Therefore, the terminal can calculate the minimum target area of ​​the detection frame including all personnel, and crop the minimum target area, thereby inputting the cropped conference image into the trained depth recognition model.

[0080] Through this embodiment, the terminal can crop the conference image to obtain the smallest image containing all conference participants, and then perform depth recognition based on the cropped image based on the depth recognition model, thereby improving the accuracy of depth recognition of the conference image.

[0081] In an exemplary embodiment, as shown in FIG5 , FIG5 is a flow chart of a behavior recognition method based on deep recognition in another embodiment. In this embodiment, after the terminal obtains the input conference image, the behavior recognition model can be used to perform preliminary abnormal behavior recognition on the conference image. For example, the behavior recognition model detects each person contained in the conference image based on the target detection algorithm, and detects each person with abnormal behavior therein to form an abnormal behavior group. Among them, if the behavior recognition model does not detect a person, the detection ends; if a person is detected, the behavior recognition model can obtain the coordinates of each person in the conference image and form a corresponding location area.

[0082] The terminal can also use a monocular depth estimation algorithm in the depth recognition model to identify the depth information in the conference image, and use the depth information to correct the coordinates corresponding to each position area in each abnormal behavior group output by the behavior recognition model. Therefore, the terminal can end the detection when it detects that the person is not in the same depth range based on the person's position area and depth information; when it is detected that the person is in the same depth range, it can be determined that each person in the same depth range in the abnormal behavior group has abnormal behavior.

[0083] Through the above embodiments, the terminal performs preliminary behavior analysis on the personnel in the conference image through the behavior recognition model, uses the depth recognition model to identify the depth information of the conference image, and combines the results of the preliminary behavior analysis and the depth information to determine the abnormal behavior recognition results of each person, thereby achieving the technical effect of improving the accuracy of behavior recognition. In addition, by combining target detection with a depth estimation algorithm, the problem of the position of a two-dimensional image in three-dimensional space is effectively solved through depth estimation of the detected target, a more accurate target position relationship is obtained, and the interaction relationship between personnel can be judged more accurately. Compared with the key point posture detection algorithm, this model combination greatly reduces the complexity of data annotation and the consumption of computing resources.

[0084] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0085] Based on the same inventive concept, the embodiment of the present application also provides a behavior recognition device based on deep recognition for implementing the behavior recognition method based on deep recognition involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the behavior recognition device based on deep recognition provided below can refer to the limitations of the behavior recognition method based on deep recognition above, and will not be repeated here.

[0086] In an exemplary embodiment, as shown in FIG6 , a behavior recognition device based on depth recognition is provided, comprising: an acquisition module 500, a first input module 502, a second input module 504 and a recognition module 506, wherein:

[0087] The acquisition module 500 is used to acquire the conference image to be analyzed; the conference image includes the image corresponding to each person to be analyzed.

[0088] The first input module 502 is used to input the conference image into the trained behavior recognition model, and the behavior recognition model recognizes the distances between the persons based on the conference image, and outputs the initial abnormal behavior recognition results corresponding to the persons in the conference image based on the distances.

[0089] The second input module 504 is used to input the conference image into the trained depth recognition model, and the depth recognition model performs depth recognition on the conference image and outputs depth information corresponding to the conference image.

[0090] The recognition module 506 is used to determine the abnormal behavior recognition results corresponding to each person in the conference image according to each initial abnormal behavior recognition result and the depth information.

[0091] In one embodiment, the first input module 502 is used to input the conference image into a trained behavior recognition model, and the behavior recognition model extracts the human features corresponding to each person based on the conference image, and determines the position areas of each person in the conference image based on the human features; obtains the distances between the position areas in the conference image; and determines the abnormal behavior groups based on at least two position areas to be identified whose distances are less than or equal to a preset distance threshold; wherein each abnormal behavior group represents that the initial abnormal behavior recognition results of at least two persons to be identified corresponding to at least two position areas to be identified are that there are pending abnormal behaviors.

[0092] In one embodiment, the second input module 504 is used to input the conference image into a trained depth recognition model, and the depth recognition model extracts the color features, texture features and structural features corresponding to the conference image; identifies the pixel depth information corresponding to each pixel point in the conference image based on the color features, texture features and structural features; and outputs the depth information corresponding to the conference image based on the pixel depth information.

[0093] In one embodiment, the above-mentioned identification module 506 is used to determine each abnormal behavior group in the conference image where there are pending abnormal behaviors based on each initial abnormal behavior identification result; wherein each abnormal behavior group includes at least two to-be-identified position areas corresponding to at least two persons to be identified; for each to-be-identified position area corresponding to the person to be identified, determine each pixel depth information corresponding to each pixel point in the to-be-identified position area based on the depth information; determine the maximum depth value corresponding to the to-be-identified position area based on each pixel depth information as the regional depth information corresponding to the to-be-identified position area; for each abnormal behavior group, if there are at least two position areas with the same depth in the abnormal behavior group, it is determined that at least two to-be-identified persons corresponding to the at least two position areas with the same depth have abnormal behaviors; wherein the difference between at least two regional depth information corresponding to at least two position areas with the same depth is less than or equal to a preset depth threshold.

[0094] In one embodiment, the acquisition module 500 is used to acquire the original conference image acquired by the image acquisition device; input the image size according to a preset model, resize the original conference image, and obtain the conference image to be analyzed.

[0095] In one embodiment, the above-mentioned device also includes: a cropping module, which is used to determine the position area corresponding to each person in the conference image according to each initial abnormal behavior recognition result; determine the minimum person area in the conference image according to each position area; the minimum person area represents the minimum area containing each person; crop the conference image according to the minimum person area, and input the cropped conference image into the trained deep recognition model.

[0096] Each module in the above-mentioned behavior recognition device based on deep recognition can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0097] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a behavior recognition method based on depth recognition is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0098] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0099] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above-mentioned behavior recognition method based on deep recognition when executing the computer program.

[0100] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned behavior recognition method based on depth recognition is implemented.

[0101] In one embodiment, a computer program product is provided, including a computer program, which implements the above-mentioned behavior recognition method based on depth recognition when executed by a processor.

[0102] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0103] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0104] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0105] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A behavior recognition method based on deep recognition, characterized in that: The method comprises: Acquire a conference image to be analyzed; the conference image includes an image corresponding to each person to be analyzed; Input the conference image into a trained behavior recognition model, and the behavior recognition model recognizes the distances between the persons based on the conference image, and outputs initial abnormal behavior recognition results corresponding to the persons in the conference image based on the distances; Inputting the conference image into a trained depth recognition model, allowing the depth recognition model to perform depth recognition on the conference image and output depth information corresponding to the conference image; According to each of the initial abnormal behavior recognition results and the depth information, an abnormal behavior recognition result corresponding to each of the persons in the conference image is determined.

2. The method according to claim 1, characterized in that The step of inputting the conference image into a trained behavior recognition model, allowing the behavior recognition model to recognize distances between the persons based on the conference image, and outputting initial abnormal behavior recognition results corresponding to the persons in the conference image based on the distances includes: Input the conference image into a trained behavior recognition model, extract the human body features corresponding to the persons based on the conference image by the behavior recognition model, and determine the position areas of the persons in the conference image based on the human body features; Acquire each distance between each of the location areas in the conference image; Determine each abnormal behavior group based on at least two location areas to be identified whose distance is less than or equal to a preset distance threshold; wherein each abnormal behavior group represents that the initial abnormal behavior identification result of at least two persons to be identified corresponding to at least two location areas to be identified is that there is an abnormal behavior to be determined.

3. The method according to claim 1, characterized in that The step of inputting the conference image into a trained depth recognition model, performing depth recognition on the conference image by the depth recognition model, and outputting depth information corresponding to the conference image includes: Inputting the conference image into a trained deep recognition model, and extracting color features, texture features, and structural features corresponding to the conference image by the deep recognition model; According to the color feature, the texture feature and the structural feature, identifying each pixel depth information corresponding to each pixel point in the conference image; Depth information corresponding to the conference image is output according to the depth information of each pixel.

4. The method according to claim 1, characterized in that: The determining, according to each of the initial abnormal behavior recognition results and the depth information, an abnormal behavior recognition result corresponding to each of the persons in the conference image includes: According to each of the initial abnormal behavior recognition results, each abnormal behavior group having pending abnormal behaviors in the conference image is determined; wherein each of the abnormal behavior groups includes at least two location areas to be identified corresponding to at least two persons to be identified; For the position area to be identified corresponding to each person to be identified, determine the depth information of each pixel corresponding to each pixel point in the position area to be identified according to the depth information; Determine, according to each pixel depth information, a maximum depth value corresponding to the position area to be identified as the area depth information corresponding to the position area to be identified; For each abnormal behavior group, if there are at least two location areas with the same depth in the abnormal behavior group, it is determined that at least two persons to be identified corresponding to the at least two location areas with the same depth have abnormal behavior; wherein the difference between the depth information of at least two areas corresponding to the at least two location areas with the same depth is less than or equal to a preset depth threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The obtaining of the conference image to be analyzed includes: Acquire the original conference image captured by the image acquisition device; The image size is input according to a preset model, and the size of the original conference image is adjusted to obtain the conference image to be analyzed.

6. The method according to any one of claims 1 to 4, characterized in that: Before inputting the conference image into the trained deep recognition model, the method further includes: Determine, according to each of the initial abnormal behavior recognition results, a location area corresponding to each of the persons in the conference image; Determine the minimum personnel area in the conference image according to each of the position areas; the minimum personnel area represents the minimum area containing each of the personnel; The conference image is cropped according to the minimum personnel area, and the cropped conference image is input into the trained depth recognition model.

7. A behavior recognition device based on deep recognition, characterized in that: The device comprises: An acquisition module, used to acquire a conference image to be analyzed; the conference image includes an image corresponding to each person to be analyzed; A first input module is used to input the conference image into a trained behavior recognition model, and the behavior recognition model recognizes the distances between the persons based on the conference image, and outputs initial abnormal behavior recognition results corresponding to the persons in the conference image based on the distances; A second input module, used to input the conference image into a trained depth recognition model, so that the depth recognition model performs depth recognition on the conference image and outputs depth information corresponding to the conference image; The recognition module is used to determine the abnormal behavior recognition results corresponding to each of the persons in the conference image according to each of the initial abnormal behavior recognition results and the depth information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images