Group behavior recognition method, electronic device, and computer-readable storage medium

Abstract

The present application discloses a method for identifying group behavior, an electronic device, and a computer-readable storage medium. The generation method includes: obtaining a feature vector of text data used to describe group behavior; obtaining a group posture point sequence and a camera extrinsic parameter matrix based on image data from a predetermined monitoring angle; processing the feature vector of the text data and the camera extrinsic parameter matrix using a diffusion model to generate a group posture point sequence used as a template; and comparing the group posture point sequence with a group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence. This application solves the technical problem in related technologies of being unable to detect a specific group behavior specified by a user, achieving the goal of generating a group posture point sequence used as a template for a specific group behavior specified by a user, more accurately detecting group behavior, and significantly improving the effectiveness of group behavior recognition.

Description

Technical Field

[0001] The present application relates to the technical field of human behavior detection, and in particular to a group behavior recognition method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Currently, there is a lack of technical solutions for group behavior discrimination in abnormal behavior detection applications. Existing technologies can generally only judge abnormal behavior of a single person, and require specific categories of behavior data sets for training. They cannot meet the detection requirements of user-specified behavior categories. Such solutions have poor actual application experience, a narrow application scope, and have difficult-to-meet conditions such as the need for specific category training sets.

[0003] For the above problems, no effective solutions have been proposed yet. Summary of the Invention

[0004] The embodiments of the present application provide a group behavior recognition method, an electronic device, and a computer-readable storage medium to at least solve the technical problem in related technologies that it is impossible to detect a group behavior specified by a user.

[0005] According to one aspect of an embodiment of the present application, a group behavior recognition method is provided, comprising: receiving text data for describing group behavior, and obtaining a feature vector of the text data; collecting image data under a predetermined monitoring perspective, and obtaining a group posture point sequence and a camera extrinsic parameter matrix based on the image data; processing the feature vector of the text data and the camera extrinsic parameter matrix using a pre-trained diffusion model to generate a group posture point sequence used as a template, wherein the diffusion model is trained based on the feature vector of sample text data for describing group behavior, the camera extrinsic parameter matrix corresponding to the sample image data under the predetermined monitoring perspective, and the actual group posture point sequence contained in the sample image data; comparing the group posture point sequence with the group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence.

[0006] Optionally, obtaining a group posture point sequence based on the image data includes: detecting the human body position in the image data to obtain image data containing the human body position; performing posture point estimation on the image data containing the human body position to obtain a posture point set of the image data containing the human body position; associating the same human body in the image data containing the human body position in time series to obtain the motion path of each human body; and obtaining the group posture point sequence based on the image data containing the human body position, the posture point set of the image data containing the human body position, and the motion path of each human body, and performing clustering according to the human body position.

[0007] Optionally, before using a pre-trained diffusion model to process the feature vector of the text data and the camera extrinsic parameter matrix to generate a group posture point sequence used as a template, the method also includes: obtaining the feature vector of sample text data used to describe group behavior and the camera extrinsic parameter matrix corresponding to the sample image data under the predetermined monitoring perspective; inputting the feature vector of the sample text data and the camera extrinsic parameter matrix corresponding to the sample image data into the diffusion model to obtain a predicted group posture point sequence; obtaining the actual group posture point sequence contained in the sample image data; calculating the similarity between the predicted group posture point sequence and the actual group posture point sequence; determining a loss value corresponding to the similarity, and using the loss value to update the parameters of the diffusion model.

[0008] Optionally, the group posture point sequence is compared with the group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence, including: screening out a target group posture point sequence from the group posture point sequence used as a template; comparing the group posture point sequence with the target group posture point sequence to determine the group behavior corresponding to the group posture point sequence.

[0009] Optionally, a target group posture point sequence is screened out from the group posture point sequence used as a template, comprising: when there are multiple group posture point sequences used as templates, determining the colors of the human joint points corresponding to the multiple group posture point sequences used as templates, wherein different human joint points are identified by different colors; based on the colors of the human joint points, screening out at least one group posture point sequence from the multiple group posture point sequences used as templates as the target group posture point sequence.

[0010] Optionally, the group posture point sequence and the target group posture point sequence are compared to determine the group behavior corresponding to the group posture point sequence, including: calculating the matching degree between the group posture point sequence and the target group posture point sequence to obtain a target matching degree; when the target matching degree is greater than or equal to a preset matching degree threshold, determining that the group behavior corresponding to the group posture point sequence is the group behavior to be identified, wherein the group behavior to be identified is the group behavior described by the text data.

[0011] Optionally, the matching degree of the group posture point sequence and the target group posture point sequence is calculated to obtain the target matching degree, including: calculating the matching degree of the group posture point sequence and the target group posture point sequence based on the data frame order to obtain a matching degree sequence; and calculating the mean of the matching degrees in the matching degree sequence to obtain the target matching degree.

[0012] Optionally, after determining the group behavior corresponding to the group posture point sequence, the method further includes: displaying and alarming the group behavior corresponding to the group posture point sequence; and / or pushing the group behavior corresponding to the group posture point sequence to a target device.

[0013] According to another aspect of an embodiment of the present application, an electronic device is further provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the steps of any one of the above methods.

[0014] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided. Optionally, the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the steps of any one of the above methods.

[0015] In an embodiment of the present application, text data used to describe group behavior is received to obtain a feature vector of the text data; image data under a predetermined monitoring perspective is simultaneously collected, and a group posture point sequence and a camera extrinsic parameter matrix are obtained based on the image data; a pre-trained diffusion model is then used to process the feature vector of the text data and the camera extrinsic parameter matrix to generate a group posture point sequence used as a template; finally, the group posture point sequence and the group posture point sequence used as a template are compared to determine the group behavior corresponding to the group posture point sequence, thereby solving the technical problem in the related art that a certain group behavior specified by the user cannot be detected, and achieving the goal of generating a group posture point sequence used as a template for a certain group behavior specified by the user, more accurately detecting group behavior, and thus greatly improving the group behavior recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A flowchart of a group behavior recognition method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of a group posture point sequence generation method provided in an optional embodiment of the present application;

[0019] Figure 3 A schematic diagram of a group behavior recognition method provided in an optional embodiment of the present application;

[0020] Figure 4 A schematic diagram of a group behavior recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0022] It should be noted that the terms "first," "second," and the like in the specification, claims, and drawings of this application are used to distinguish between different objects, rather than to limit a particular order. Furthermore, the steps shown in the flowcharts of the accompanying drawings may be executed in a computer system, such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown.

[0023] According to one aspect of the embodiments of the present application, a group behavior recognition method is provided. Figure 1 The flowchart of the group behavior recognition method provided in the embodiment of the present application is as follows: Figure 1 As shown, the method includes the following steps:

[0024] Step S102, receiving text data for describing group behavior and obtaining a feature vector of the text data;

[0025] The above-mentioned group behavior refers to a group of people composed of two or more individuals who influence, interact and depend on each other in order to achieve a specific goal; for example, group behavior includes but is not limited to multi-person fighting, multi-person chatting and multi-person meetings.

[0026] The text data for describing group behavior can be directly obtained, or converted based on images, voice, etc. containing group behavior.

[0027] Optionally, a text encoder may be used to obtain a feature vector of the text data. The text encoder may adopt means such as text preprocessing, feature extraction, and feature selection to obtain the required feature vector of the text data.

[0028] The above-mentioned text preprocessing can convert text data into a processable form; text preprocessing includes but is not limited to removing stop words, stemming, spelling correction, etc.

[0029] The above-mentioned feature extraction can convert text data into feature vectors; feature extraction can be implemented in various ways, for example: text data can be represented as a word frequency vector; text data can be represented as a word importance vector, where the importance of a word is based on its frequency in the text dataset and the frequency of its appearance in other text datasets; text data can also be represented as a word vector, where the word vector is based on how it appears in the context.

[0030] The above feature selection can select the most relevant eigenvectors; feature selection can be achieved in many ways, for example: through the chi-square test, select the eigenvector most relevant to the target variable; through the mutual information method, select the eigenvector with the highest mutual information with the target variable; through regularization, select the eigenvector with the largest weight.

[0031] Step S104: collecting image data at a predetermined monitoring angle, and obtaining a group posture point sequence and a camera extrinsic parameter matrix based on the image data;

[0032] The above-mentioned image data can be collected from a video stream under a predetermined monitoring perspective, which includes multiple consecutive frames of images, and there is a temporal correlation between the multiple frames of images.

[0033] The group posture point sequence is a time series data recording the posture points of each individual in the group; wherein the posture points generally refer to key points of the human body, such as the position and direction of the head, arms, legs and other parts.

[0034] The above-mentioned camera extrinsic parameter matrix is ​​also called camera extrinsic parameter, which can be obtained based on a camera extrinsic parameter prediction branch. The camera extrinsic parameter prediction branch generates a camera extrinsic parameter matrix of predetermined image data based on the conversion between the world coordinate system and the camera coordinate system.

[0035] The camera extrinsic parameter matrix includes a rotation matrix and a translation vector, which are used to describe the position of the camera in three-dimensional space. The rotation matrix is ​​used to describe the rotation relationship between the camera coordinate system and the world coordinate system, and the translation vector is used to describe the position of the camera coordinate system in the world coordinate system.

[0036] It's important to note that the camera extrinsic matrix can be pre-set to ensure the camera captures the desired area and angle. For example, in a surveillance camera, the camera extrinsic matrix can be used to determine the camera's position and orientation to capture image data for a specific area. Furthermore, the camera extrinsic matrix can be used to map points in a 3D world coordinate system onto a 2D image plane, enabling tasks such as human position detection, human path detection, and human pose estimation.

[0037] Optionally, since the camera extrinsic parameter matrix corresponding to the image data under the same monitoring perspective is stable, any frame of image under a predetermined monitoring perspective can be selected to obtain the camera extrinsic parameter matrix based on the frame of image.

[0038] Step S106: Processing the feature vectors of the text data and the camera extrinsic parameter matrix using a pre-trained diffusion model to generate a group pose point sequence used as a template, wherein the diffusion model is trained based on the feature vectors of the sample text data used to describe group behavior, the camera extrinsic parameter matrix corresponding to the sample image data under a predetermined monitoring perspective, and the actual group pose point sequence contained in the sample image data;

[0039] The group posture point sequence used as a template is also a group posture point sequence template, which serves as a specified group behavior paradigm.

[0040] Step S108 : comparing the group posture point sequence with the group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence.

[0041] Through the above steps, text data used to describe group behavior can be received to obtain a feature vector of the text data; image data under a predetermined monitoring angle can be collected at the same time, and a group posture point sequence and a camera extrinsic parameter matrix can be obtained based on the image data; the feature vector of the text data and the camera extrinsic parameter matrix can then be processed using a pre-trained diffusion model to generate a group posture point sequence used as a template; finally, the group posture point sequence and the group posture point sequence used as a template are compared to determine the group behavior corresponding to the group posture point sequence, thereby solving the technical problem in related technologies that a certain group behavior specified by a user cannot be detected, achieving the goal of generating a group posture point sequence used as a template for a certain group behavior specified by a user, more accurately detecting group behavior, and thus greatly improving the group behavior recognition effect.

[0042] It should be noted that compared with the existing technology that uses image features for matching, which will be affected by weather, clothing, etc., and the actual effect is not good, this application uses a measurement method based on changes in group posture points to effectively solve the problem of differences in human body movements for the same behavior. Group posture point features have a more universal expression compared to image features and are not limited by interference such as clothing color.

[0043] As an optional embodiment, obtaining a group posture point sequence based on image data includes: detecting the human body position in the image data to obtain image data containing the human body position; performing posture point estimation on the image data containing the human body position to obtain a posture point set of the image data containing the human body position; associating the same human body in the image data containing the human body position in time series to obtain the motion path of each human body; and obtaining a group posture point sequence based on the image data containing the human body position, the posture point set of the image data containing the human body position, and the motion path of each human body, and performing clustering processing according to the human body position.

[0044] Optionally, human body position detection may be performed on image data using a deep learning algorithm, such as a human body detection algorithm based on a convolutional neural network, to obtain image data containing human body positions.

[0045] Optionally, pose point estimation is performed on image data containing human body positions. A deep learning algorithm, such as a pose estimation algorithm based on a convolutional neural network, can be used to obtain a pose point set of image data containing human body positions.

[0046] Optionally, the same human body in the image data containing the human body position is associated in time sequence, and a human body path detection algorithm, such as a human body path detection algorithm based on Kalman filtering, can be used to obtain the movement path of each human body.

[0047] Optionally, based on the image data containing the human body positions, the posture point set containing the image data containing the human body positions and the motion path of each human body, and clustering processing is performed according to the human body positions, a clustering algorithm, such as a clustering algorithm based on the K-Means algorithm, can be used to obtain a group posture point sequence.

[0048] In the embodiments of the present application, by obtaining detection results such as human position detection, human posture point estimation, and human path detection, and combining them with corresponding clustering algorithms, a group posture point sequence is accurately obtained from image data.

[0049] As an optional embodiment, before using a pre-trained diffusion model to process the feature vectors and camera extrinsic parameter matrix of text data to generate a group posture point sequence used as a template, the above method also includes: obtaining the feature vectors of sample text data used to describe group behavior and the camera extrinsic parameter matrix corresponding to sample image data under a predetermined monitoring perspective; inputting the feature vectors of the sample text data and the camera extrinsic parameter matrix corresponding to the sample image data into the diffusion model to obtain a predicted group posture point sequence; obtaining the actual group posture point sequence contained in the sample image data; calculating the similarity between the predicted group posture point sequence and the actual group posture point sequence; determining a loss value corresponding to the similarity, and using the loss value to update the parameters of the diffusion model.

[0050] The above diffusion model is used to generate data similar to the training data, corrupt the training data by continuously adding noise, and then learn to recover the data by reversing the noise process.

[0051] The training data includes, but is not limited to, feature vectors of sample text data describing group behavior, a camera extrinsic parameter matrix corresponding to sample image data from a predetermined monitoring perspective, and a sequence of actual group pose points contained in the sample image data. Furthermore, this application imposes no restrictions on the amount of training data, which can be set based on the needs of the application scenario.

[0052] Optionally, in the process of obtaining the feature vector of the sample text data used to describe the group behavior, the sample text data may be input into a text encoder to obtain the feature vector of the sample text data.

[0053] In the process of obtaining the camera extrinsic parameter matrix corresponding to the sample image data under the predetermined monitoring view angle, the sample image data can be input into the camera extrinsic parameter prediction branch to obtain the camera extrinsic parameter matrix corresponding to the sample image data.

[0054] It should be noted that the method of implementing camera extrinsic parameter prediction based on the camera extrinsic parameter prediction branch can effectively solve the problem of missing samples in group behavior detection under a predetermined monitoring perspective, thereby improving the reliability of model training.

[0055] Optionally, the eigenvector of the sample text data and the camera extrinsic parameter matrix corresponding to the sample image data can be input into the diffusion model to obtain a predicted group posture point sequence; then the similarity between the predicted group posture point sequence and the actual group posture point sequence is calculated to obtain the similarity between the predicted group posture point sequence and the actual group posture point sequence; then the similarity is used to calculate the loss value, and the parameters of the diffusion model are continuously updated by the loss value until the parameters of the diffusion model meet the preset convergence conditions.

[0056] Optionally, the similarity between the predicted group posture point sequence and the actual group posture point sequence is calculated, including: based on the data frame order, the predicted group posture point sequence and the actual group posture point sequence under the predetermined monitoring perspective are respectively calculated to obtain the similarity corresponding to each frame between the predicted group posture point sequence and the actual group posture point sequence; the similarity corresponding to each frame is calculated to obtain the average similarity; and the average similarity is used as the similarity between the predicted group posture point sequence and the actual group posture point sequence.

[0057] In an embodiment of the present application, a diffusion model is trained using feature vectors of sample text data describing group behavior, a camera extrinsic parameter matrix corresponding to sample image data from a predetermined monitoring perspective, and a sequence of actual group pose points contained in the sample image data, resulting in a pre-trained diffusion model. Simultaneously, through continuous training and updating of the massive amount of sample data and the actual group pose point sequences contained in the sample image data, the final diffusion model establishes a correlation between the data, the camera extrinsic parameters, and the pose point generation. During the diffusion model training process, the model's prediction accuracy can be improved by continuously optimizing its parameters.

[0058] As an optional embodiment, the group posture point sequence is compared with the group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence, including: screening out a target group posture point sequence from the group posture point sequence used as a template; comparing the group posture point sequence with the target group posture point sequence to determine the group behavior corresponding to the group posture point sequence.

[0059] In order to make the group posture point sequence used as a template better meet the actual application requirements and improve the accuracy of detecting group behavior, part of the group posture point sequence can be screened out from the group posture point sequence used as a template as the target group posture point sequence; then, the group posture point sequence is compared with the target group posture point sequence to determine the group behavior corresponding to the group posture point sequence.

[0060] The target group posture point sequence may be one or more group posture point sequences used as templates.

[0061] As an optional embodiment, a target group posture point sequence is screened out from a group posture point sequence used as a template, including: when there are multiple group posture point sequences used as templates, determining the colors of the human joint points corresponding to the multiple group posture point sequences used as templates, wherein different human joint points are identified by different colors; based on the colors of the human joint points, at least one group posture point sequence is screened out from the multiple group posture point sequences used as templates as the target group posture point sequence.

[0062] Optionally, if there are multiple group posture point sequences used as templates, different human joint points are assigned different colors for visualization, and the colors of the human joint points are used to screen templates to filter out group posture point sequence templates that do not meet application requirements, and at least one of the screened group posture point sequences is used as the target group posture point sequence.

[0063] In the embodiment of the present application, the above-mentioned method can improve the pertinence of the group posture point sequence template, thereby improving the subsequent group behavior recognition effect to a certain extent.

[0064] As an optional embodiment, the group posture point sequence and the target group posture point sequence are compared to determine the group behavior corresponding to the group posture point sequence, including: calculating the matching degree between the group posture point sequence and the target group posture point sequence to obtain a target matching degree; when the target matching degree is greater than or equal to a preset matching degree threshold, determining that the group behavior corresponding to the group posture point sequence is the group behavior to be identified, wherein the group behavior to be identified is the group behavior described by the text data.

[0065] Optionally, a matching degree is calculated between the group posture point sequence and the target group posture point sequence, which can be calculated using methods such as Euclidean distance and cosine similarity. After the target matching degree is calculated, if the target matching degree is greater than or equal to a preset matching degree threshold, the group behavior corresponding to the group posture point sequence is considered to be the group behavior to be identified.

[0066] Furthermore, a matching degree is calculated between the group posture point sequence and the target group posture point sequence to obtain a target matching degree, including: calculating a matching degree between the group posture point sequence and the target group posture point sequence based on the data frame order to obtain a matching degree sequence; and calculating the mean of the matching degrees in the matching degree sequence to obtain a target matching degree.

[0067] The above data frame sequence is the sequence of the data frames during transmission, and can be used to represent the temporal order of appearance of group posture points in each frame of image.

[0068] In the embodiment of the present application, the matching degree between the group posture point sequence and the target group posture point sequence is calculated through the data frame sequence, and the average of the matching degrees is obtained, thereby ensuring the accuracy and reliability of the matching degree.

[0069] As an optional embodiment, after determining the group behavior corresponding to the group posture point sequence, the above method also includes: displaying and alarming the group behavior corresponding to the group posture point sequence; and / or pushing the group behavior corresponding to the group posture point sequence to the target device.

[0070] Optionally, once group behavior is identified, an alarm device can be used to display the group behavior and send alerts to relevant personnel. For example, if group behavior is identified as a violent incident, an alert can be sent to security personnel so that timely action can be taken. Furthermore, group behavior can be pushed to target devices, such as smartphones or computers, so that users can be informed of the group behavior in a timely manner.

[0071] Figure 2A schematic diagram of a group posture point sequence generation method provided in an optional embodiment of the present application is shown as follows: Figure 2 As shown, during use, text is input (corresponding to the above-mentioned text data used to describe group behavior), and the trained text encoder will encode the input into a specific feature vector. At the same time, the image data under the predetermined monitoring angle is input into the camera extrinsic parameter prediction branch to predict the camera extrinsic parameter matrix of the image. Then, the feature vector of the text and the camera extrinsic parameter matrix are input into the diffusion generation model (corresponding to the above-mentioned diffusion model). After multi-step diffusion, the matching posture point result is output (corresponding to the above-mentioned group posture point sequence template or the group posture point sequence used as a template).

[0072] It should be noted that in the embodiments of the present application, there is no need to provide an offline trained abnormal behavior detection model. A group posture point sequence template (corresponding to the above-mentioned group posture point sequence used as a template) under a predetermined monitoring perspective is directly generated based on the text data used to describe group behavior and the camera external parameter prediction, so that the matched abnormal behavior in the video stream can be analyzed.

[0073] Figure 3 A schematic diagram of a group behavior recognition method provided in an optional embodiment of the present application is shown as follows: Figure 3 As shown in the figure, first, you need to input text data used to describe the group behavior to be identified, such as a fight between multiple people. Based on the text data used to describe the group behavior to be identified, multiple group posture point sequence templates are generated, and the posture points are visualized for users to filter and filter, and group posture point sequence templates that do not meet the requirements are filtered out; in addition, the accessed video stream is subjected to human position detection, human posture point estimation, human path detection, and group posture point sequence candidate generation. The generated candidate group posture point sequence is metrically matched with the predetermined group posture point sequence template. If the threshold requirements are met, an alarm display is performed.

[0074] Furthermore, human position detection detects human positions within the input video stream, human pose estimation estimates the pose points of the detected human images, and human path detection associates the same human in a temporal sequence, outputting an associated label for each human. Then, based on the human position detection, pose estimation, and path detection results from the image data, human positions are used to cluster groups, resulting in a group pose point sequence that reflects group behavior.

[0075] Furthermore, when measuring the matching, a metric algorithm (including but not limited to Euclidean clustering, Mahalanobis distance, cosine distance, etc.) is used to calculate the matching degree to obtain a matching degree sequence. If there are multiple matching degree sequences, the matching degree sequences are sorted from high to low according to their respective average matching degrees.

[0076] Furthermore, the relevant matching results are dynamically displayed and alarmed, and the relevant alarm information can be pushed to users through active push, which is convenient for users to confirm and manage.

[0077] According to another aspect of the embodiment of the present application, a group behavior recognition device is also provided. Figure 4 A schematic diagram of a group behavior recognition device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the group behavior recognition device includes: a first processing module 42, a second processing module 44, a third processing module 46 and a fourth processing module 48. The group behavior recognition device is described in detail below.

[0078] A first processing module 42 is configured to receive text data describing group behavior and obtain a feature vector of the text data;

[0079] A second processing module 44 is configured to collect image data from a predetermined monitoring perspective, and obtain a group posture point sequence and a camera extrinsic parameter matrix based on the image data;

[0080] a third processing module 46 for processing the feature vectors and camera extrinsic parameter matrix of the text data using a pre-trained diffusion model to generate a group pose point sequence used as a template, wherein the diffusion model is trained based on the feature vectors of the sample text data used to describe group behavior, the camera extrinsic parameter matrix corresponding to the sample image data under a predetermined monitoring perspective, and the actual group pose point sequence contained in the sample image data;

[0081] The fourth processing module 48 is configured to compare the group posture point sequence with a group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence.

[0082] In an embodiment of the present application, the group behavior recognition device receives text data used to describe group behavior and obtains a feature vector of the text data; simultaneously, it collects image data under a predetermined monitoring perspective, and obtains a group posture point sequence and a camera extrinsic parameter matrix based on the image data; then uses a pre-trained diffusion model to process the feature vector of the text data and the camera extrinsic parameter matrix to generate a group posture point sequence used as a template; finally, the group posture point sequence is compared with the group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence, thereby solving the technical problem in the related art that it is impossible to detect a certain group behavior specified by the user's needs, and achieving the goal of generating a group posture point sequence used as a template for a certain group behavior specified by the user's needs, more accurately detecting group behavior, and thus greatly improving the group behavior recognition effect.

[0083] It should be noted here that the above-mentioned first processing module 42, second processing module 44, third processing module 46 and fourth processing module 48 correspond to steps S102 to S108 in the method embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment.

[0084] Optionally, the above-mentioned first processing module 42 includes: a first processing unit, used to detect the human body position in the image data to obtain image data containing the human body position; a second processing unit, used to perform posture point estimation on the image data containing the human body position to obtain a posture point set of the image data containing the human body position; a third processing unit, used to associate the same human body in the image data containing the human body position in time sequence to obtain the motion path of each human body; a fourth processing unit, used to perform clustering processing based on the image data containing the human body position, the posture point set of the image data containing the human body position and the motion path of each human body, and according to the human body position, to obtain a group posture point sequence.

[0085] Optionally, the above-mentioned device also includes: a fifth processing module, wherein the fifth processing module includes: a fifth processing unit, which is used to process the feature vector and camera extrinsic parameter matrix of the text data using a pre-trained diffusion model to generate a group posture point sequence used as a template, and obtain the feature vector of the sample text data and the camera extrinsic parameter matrix corresponding to the sample image data under a predetermined monitoring perspective before generating the group posture point sequence used as a template; a sixth processing unit, which is used to input the feature vector of the sample text data and the camera extrinsic parameter matrix corresponding to the sample image data into the diffusion model to obtain a predicted group posture point sequence; a seventh processing unit, which is used to obtain the actual group posture point sequence contained in the sample image data; an eighth processing unit, which is used to calculate the similarity between the predicted group posture point sequence and the actual group posture point sequence; and a ninth processing unit, which is used to determine the loss value corresponding to the similarity and use the loss value to update the parameters of the diffusion model.

[0086] Optionally, the fourth processing module 48 includes: a tenth processing unit for filtering out a target group posture point sequence from the group posture point sequence used as a template; and an eleventh processing unit for comparing the group posture point sequence with the target group posture point sequence to determine the group behavior corresponding to the group posture point sequence.

[0087] Optionally, the above-mentioned tenth processing unit includes: a first processing sub-unit, for determining the colors of the human joint points corresponding to the multiple group posture point sequences used as templates when there are multiple group posture point sequences used as templates, wherein different human joint points are identified by different colors; a second processing sub-unit, for screening at least one group posture point sequence as a target group posture point sequence from the multiple group posture point sequences used as templates based on the colors of the human joint points.

[0088] Optionally, the above-mentioned eleventh processing unit includes: a third processing sub-unit, used to calculate the matching degree of the group posture point sequence with the target group posture point sequence to obtain the target matching degree; a fourth processing sub-unit, used to determine that the group behavior corresponding to the group posture point sequence is the group behavior to be identified when the target matching degree is greater than or equal to a preset matching degree threshold, wherein the group behavior to be identified is the group behavior described by the text data.

[0089] Optionally, the above-mentioned third processing subunit includes: a first calculation subunit, used to calculate the matching degree of the group posture point sequence and the target group posture point sequence based on the data frame order to obtain a matching degree sequence; a second calculation subunit, used to calculate the mean of the matching degrees in the matching degree sequence to obtain the target matching degree.

[0090] Optionally, the above-mentioned device also includes: a sixth processing module, which is used to display and alarm the group behavior corresponding to the group posture point sequence after determining the group behavior corresponding to the group posture point sequence; and / or push the group behavior corresponding to the group posture point sequence to the target device.

[0091] According to another aspect of an embodiment of the present application, an electronic device is also provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the steps of any one of the above methods.

[0092] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the steps of any of the above methods.

[0093] The computer-readable storage medium may be located in any computer terminal in a computer terminal group in a computer network, and / or located in any mobile terminal in a mobile terminal group, and the computer-readable storage medium includes a stored program.

[0094] According to another aspect of an embodiment of the present application, a computer program product is further provided, which, when executed on a data processing device, is suitable for executing a program initialized with any of the above method steps.

[0095] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the modules can be a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, modules or indirect coupling or communication connection of modules, which can be electrical or other forms.

[0097] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0098] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0099] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A group behavior recognition method, characterized in that: include: receiving text data for describing group behavior and obtaining a feature vector of the text data; Collecting image data under a predetermined monitoring perspective, and obtaining a group posture point sequence and a camera extrinsic parameter matrix based on the image data; Processing the feature vectors of the text data and the camera extrinsic parameter matrix using a pre-trained diffusion model to generate a group pose point sequence used as a template, wherein the diffusion model is trained based on the feature vectors of sample text data used to describe group behavior, the camera extrinsic parameter matrix corresponding to the sample image data under the predetermined monitoring perspective, and the actual group pose point sequence contained in the sample image data; The group posture point sequence is compared with the group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence.

2. The method according to claim 1, characterized in that Acquiring a group posture point sequence based on the image data, comprising: Detecting a human body position in the image data to obtain image data containing the human body position; Performing posture point estimation on the image data containing the position of the human body to obtain a posture point set of the image data containing the position of the human body; Associating the same human body in the image data containing the human body position in time sequence to obtain the motion path of each human body; Based on the image data containing the human body position, the posture point set of the image data containing the human body position and the motion path of each human body, clustering processing is performed according to the human body position to obtain the group posture point sequence.

3. The method according to claim 1, characterized in that Before using a pre-trained diffusion model to process the feature vector of the text data and the camera extrinsic parameter matrix to generate a group pose point sequence used as a template, the method further includes: Obtaining a feature vector of sample text data for describing group behavior and a camera extrinsic parameter matrix corresponding to sample image data under the predetermined monitoring perspective; Inputting the feature vector of the sample text data and the camera extrinsic parameter matrix corresponding to the sample image data into the diffusion model to obtain a predicted group posture point sequence; Acquire an actual group posture point sequence contained in the sample image data; Calculating the similarity between the predicted group posture point sequence and the actual group posture point sequence; A loss value corresponding to the similarity is determined, and parameters of the diffusion model are updated using the loss value.

4. The method according to claim 1, wherein Comparing the group posture point sequence with the group posture point sequence used as a template to determine the group behavior corresponding to the group posture point sequence includes: Filtering a target group posture point sequence from the group posture point sequence used as a template; The group posture point sequence is compared with the target group posture point sequence to determine the group behavior corresponding to the group posture point sequence.

5. The method according to claim 4, characterized in that Filtering a target group posture point sequence from the group posture point sequence used as a template includes: When there are multiple group posture point sequences used as templates, determining the colors of the human joint points corresponding to the multiple group posture point sequences used as templates, wherein different human joint points are marked with different colors; Based on the colors of the human body joints, at least one group posture point sequence is selected from the plurality of group posture point sequences used as templates as the target group posture point sequence.

6. The method according to claim 4, characterized in that Comparing the group posture point sequence with the target group posture point sequence to determine the group behavior corresponding to the group posture point sequence includes: Calculating the matching degree between the group posture point sequence and the target group posture point sequence to obtain a target matching degree; When the target matching degree is greater than or equal to a preset matching degree threshold, the group behavior corresponding to the group posture point sequence is determined to be the group behavior to be identified, wherein the group behavior to be identified is the group behavior described by the text data.

7. The method according to claim 6, characterized in that Calculating the matching degree between the group posture point sequence and the target group posture point sequence to obtain the target matching degree includes: Calculating the matching degree between the group posture point sequence and the target group posture point sequence based on the data frame order to obtain a matching degree sequence; The matching degrees in the matching degree sequence are averaged to obtain the target matching degree.

8. The method according to any one of claims 1 to 7, characterized in that After determining the group behavior corresponding to the group posture point sequence, the method further includes: The group behavior corresponding to the group posture point sequence is displayed and an alarm is given; and / or the group behavior corresponding to the group posture point sequence is pushed to a target device.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the steps of the method according to any one of claims 1 to 8.

Images