Behavior recognition model training method, behavior recognition method and related device

Abstract

The present application provides a behavior recognition model training method, a behavior recognition method and related devices, wherein the training method includes: obtaining at least one behavior image sample, a behavior label corresponding to at least one behavior image sample and at least one semantic label corresponding to at least one behavior image sample; inputting at least one behavior image sample into the behavior recognition model to obtain a first behavior prediction result and at least one preset target prediction result output by the behavior recognition model; training the behavior recognition model according to the behavior label, at least one semantic label, the first behavior prediction result and at least one preset target prediction result so that the first behavior prediction result approaches the behavior label and the at least one preset target prediction result approaches at least one semantic label; determining the behavior recognition model finally obtained by training as the target behavior recognition model. This technical solution can improve the recognition accuracy of the behavior recognition model.

Description

Technical Field

[0001] The present application relates to the field of behavior recognition, and in particular to a behavior recognition model training method, a behavior recognition method, and related devices. Background Art

[0002] In the field of computer vision processing, identifying the content of an image or video through a pre-trained visual recognition model to determine the behavior of living things (such as people, animals, etc.) in the image or video is a relatively common visual processing application scenario.

[0003] At present, researchers have proposed a behavior recognition model based on temporal segment networks (TSN) to identify the behavior of living things in videos. The TSN model can be composed of a spatial stream convolutional network and a temporal stream convolutional network. It sparsely samples short clips from the video for feature extraction and learns the temporal correlation characteristics of the image content, thereby improving the recognition accuracy. The behavior recognition model based on TSN is essentially a behavior classification model, which analyzes the image features to distinguish the image as a whole and thus determines the behavior category. For some images with similar image features, for example, when a pet is wearing clothes and running and jumping, the pet's clothes are similar to those of humans or the background of the pet is similar to that of humans. During recognition, it may be recognized as human behavior, resulting in behavior recognition errors and insufficient recognition accuracy. Summary of the invention

[0004] The present application provides a behavior recognition model training method, a behavior recognition method and related devices to solve the technical problem that the recognition accuracy of existing behavior recognition models is not accurate enough.

[0005] In a first aspect, a behavior recognition model training method is provided, comprising:

[0006] Acquire at least one behavior image sample, a behavior label corresponding to the at least one behavior image sample, and at least one semantic label corresponding to the at least one behavior image sample, wherein the behavior label is used to indicate the behavior type corresponding to the at least one behavior image sample, and a semantic label is used to indicate the possibility that a preset target exists in the at least one behavior image sample; the preset target is used to express the image semantics of the at least one behavior image sample and is related to the behavior type;

[0007] Inputting the at least one behavior image sample into a behavior recognition model in a current training cycle to obtain a first behavior prediction result and at least one preset target prediction result output by the behavior recognition model in the current training cycle;

[0008] Training the behavior recognition model in the current training cycle according to the behavior label, the at least one semantic label, the first behavior prediction result, and the at least one preset target prediction result, so that the first behavior prediction result is close to the behavior label and the at least one preset target prediction result is close to the at least one semantic label;

[0009] The behavior recognition model obtained by the last training cycle is determined as the target behavior recognition model.

[0010] In this technical solution, by obtaining at least one behavior image sample, a behavior label for indicating the behavior type corresponding to the behavior image sample, and a semantic label for indicating at least one preset target present in the at least one behavior image sample, at least one behavior image sample is input into a behavior recognition model, and a behavior prediction result and a preset target prediction result corresponding to the at least one behavior image sample are obtained through the behavior recognition model. Then, the behavior recognition model is trained according to the behavior label, the semantic label, the behavior prediction result, and the preset target prediction result, so that the behavior prediction result output by the behavior recognition model is close to the behavior label, and the preset target prediction result output by the behavior recognition model is close to the semantic label, and the behavior recognition model finally trained is determined as the target behavior recognition model. During the training process, the behavior recognition model is trained by combining the semantic labels of preset targets that express the image semantics of the behavior image samples and are related to the behavior types, so that the behavior prediction results output by the behavior recognition model are close to the behavior labels and the preset target prediction results output by the behavior recognition model are close to the semantic labels. The existence of semantic labels guides the learning of the behavior recognition model, so that in addition to learning the behavioral features in the image, the behavior recognition model also learns the semantic information in the image, and can then understand the image content from a semantic level, thereby improving the recognition accuracy of the behavior recognition model.

[0011] In combination with the first aspect, in a possible implementation, the behavior recognition model in the current training cycle is trained according to the behavior label, the at least one semantic label, the first behavior prediction result, and the at least one preset target prediction result, including: calculating a first loss according to the behavior label and the first behavior prediction result, the first loss is used to indicate the accuracy of the behavior recognition model in the current training cycle in identifying the behavior type; calculating a second loss according to the at least one semantic label and the at least one preset target prediction result, the second loss is used to indicate the accuracy of the behavior recognition model in the current training cycle in identifying the preset target; and training the behavior recognition model in the current training cycle according to the first loss and the second loss. By calculating the first loss indicating the accuracy of the behavior recognition model in identifying the behavior type and the second loss indicating the accuracy of the behavior recognition model in identifying the preset target, and training the model according to the two losses, the accuracy of the behavior recognition model in identifying the behavior type and the accuracy of identifying the preset target can be improved, thereby improving the overall accuracy of the behavior recognition model.

[0012] In combination with the first aspect, in a possible implementation, the second loss is calculated based on the at least one semantic label and the at least one preset target prediction result, including: determining the gap between the first semantic label and the preset target prediction result corresponding to the first semantic label to obtain the sub-loss corresponding to the first semantic label, the first semantic label being any one of the at least one semantic label; determining the weight corresponding to the first semantic label according to the image proportion of the preset target corresponding to the first semantic label in the at least one behavior image sample; and performing weighted summation of the sub-loss corresponding to each of the at least one semantic label according to the weight corresponding to each of the at least one semantic label to obtain the second loss. In the process of calculating the second loss indicating the accuracy of the behavior recognition model in identifying the preset target, by determining the loss weight corresponding to each preset target according to the proportion of various preset targets in the image, and obtaining the second loss by weighting, the behavior recognition model can learn the semantic information in the image with emphasis, thereby improving the accuracy of the behavior recognition model.

[0013] In combination with the first aspect, in a possible implementation, the behavior recognition model in the current training cycle is trained according to the first loss and the second loss, including: calculating the weight corresponding to the second loss according to the number of training cycles corresponding to the current training cycle and the number of semantic tags corresponding to the at least one semantic tag, wherein the weight corresponding to the second loss is negatively correlated with the number of training cycles; based on the weight corresponding to the second loss, weighted summing the first loss and the second loss to obtain the total loss of the behavior recognition model in the current training cycle; and training the behavior recognition model in the current cycle according to the total loss. During the training process, by updating the weight of the second loss indicating the accuracy of the behavior recognition model in identifying the preset target, the second loss gradually decreases with the increase in the number of training times, thereby enabling the behavior recognition model to be trained more reasonably, helping the behavior recognition model to learn better, and improving the accuracy of the behavior recognition model.

[0014] In combination with the first aspect, in a possible implementation, the obtaining of at least one semantic label corresponding to the at least one behavior image sample includes: inputting the at least one behavior image sample into a target recognition model to obtain a confidence score corresponding to each of the at least one preset targets recognized by the target recognition model; and determining the at least one semantic label according to the confidence score corresponding to each of the at least one preset targets. Obtaining the semantic label of the image sample through the target recognition model can make the semantic label more compatible with the image sample, thereby helping to improve the accuracy of the behavior recognition model.

[0015] In combination with the first aspect, in a possible implementation, after determining the at least one semantic label according to the confidence score corresponding to each of the at least one preset target, the method further includes: obtaining the position coordinates of a first preset target, the first preset target being any one of the at least one preset target; determining the image proportion of the first preset target in the at least one behavior image sample according to the position coordinates of the first preset target; and updating the at least one semantic label according to the image proportion of each preset target in the at least one behavior image sample. After obtaining the semantic label of the image sample, the semantic label is updated according to the proportion of the target indicated by the semantic label in the image sample, so that the semantic label can be made more accurate, which facilitates better learning of the behavior recognition model.

[0016] In combination with the first aspect, in a possible implementation, updating the at least one semantic label according to the image proportion of each preset target in the at least one behavior image sample includes: modifying the confidence score of the first preset target according to the image proportion of the first preset target in the at least one behavior image sample to update the semantic label corresponding to the first preset target. Updating the semantic label by modifying the confidence score of the preset target can quickly update the semantic label.

[0017] In a second aspect, a behavior recognition method is provided, comprising:

[0018] Obtain at least one behavioral image;

[0019] Inputting the at least one behavior image into a target behavior recognition model to obtain a second behavior prediction result output by the target behavior recognition model, wherein the target behavior recognition model is trained by the behavior recognition model training method of the first aspect above;

[0020] The behavior type corresponding to the at least one behavior image is determined according to the second behavior prediction result.

[0021] In this technical solution, in the process of training the target behavior recognition model, the preset target that expresses the image semantics of the behavior image samples and is related to the behavior type is combined to train the behavior recognition model, so that the behavior recognition model learns the semantic information in the image in addition to the image features of the image. Therefore, by identifying at least one behavior image through the target behavior recognition model, the behavior in the at least one behavior image can be more accurately identified.

[0022] In a third aspect, a behavior recognition model training device is provided, comprising:

[0023] A data acquisition module, used to acquire at least one behavior image sample, a behavior label corresponding to the at least one behavior image sample, and at least one semantic label corresponding to the at least one behavior image sample, wherein the behavior label is used to indicate the behavior type corresponding to the at least one behavior image sample, and a semantic label is used to indicate the possibility that a preset target exists in the at least one behavior image sample; the preset target is used to express the image semantics of the at least one behavior image sample and is related to the behavior type;

[0024] A result acquisition module, used for inputting the at least one behavior image sample into the behavior recognition model in the current training cycle to obtain a first behavior prediction result and at least one preset target prediction result output by the behavior recognition model in the current training cycle;

[0025] A training module, configured to train the behavior recognition model in the current training cycle according to the behavior label, the at least one semantic label, the first behavior prediction result, and the at least one preset target prediction result, so that the first behavior prediction result is close to the behavior label and the at least one preset target prediction result is close to the at least one semantic label;

[0026] The model determination module is used to determine the behavior recognition model obtained by the last training cycle as the target behavior recognition model.

[0027] In a fourth aspect, a behavior recognition device is provided, comprising:

[0028] An image acquisition module, used to acquire at least one behavior image;

[0029] A result output module, used for inputting the at least one behavior image into a target behavior recognition model to obtain a second behavior prediction result output by the target behavior recognition model, wherein the target behavior recognition model is trained by the behavior recognition model training method of the first aspect above;

[0030] A type determination module is used to determine the behavior type corresponding to the at least one behavior image according to the second behavior prediction result.

[0031] In a fifth aspect, a computer device is provided, comprising a memory and one or more processors, wherein the one or more processors are used to execute one or more computer programs stored in the memory, and when the one or more processors execute the one or more computer programs, the computer device implements the behavior recognition model training method of the first aspect or the behavior recognition method of the second aspect.

[0032] In a sixth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the behavior recognition model training method of the first aspect or the behavior recognition method of the second aspect.

[0033] The present application can achieve the following technical effects: during the training process, the behavior recognition model is trained by combining the semantic labels of the preset targets that express the image semantics of the behavior image samples and are related to the behavior types, so that the behavior prediction results output by the behavior recognition model are close to the behavior labels and the preset target prediction results output by the behavior recognition model are also close to the semantic labels. The existence of the semantic labels guides the learning of the behavior recognition model, so that in addition to learning the behavior features in the image, the behavior recognition model also learns the semantic information in the image, and can then understand the image content from a semantic level, thereby improving the recognition accuracy of the behavior recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of a behavior recognition model training method provided in an embodiment of the present application;

[0035] Figure 2A-2B It is a schematic diagram of the TSN model before and after improvement provided in the embodiment of the present application;

[0036] Figure 3 A flowchart of a behavior recognition method provided in an embodiment of the present application;

[0037] Figure 4 It is a structural schematic diagram of a behavior recognition model training device provided in an embodiment of the present application;

[0038] Figure 5 It is a structural diagram of a behavior recognition device provided in an embodiment of the present application.

[0039] Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0041] The technical solution of the present application can be applied to various scenarios that require behavior recognition or behavior monitoring. For example, it can be applied to a video platform to identify behaviors in videos to classify videos, or it can be applied to a video platform to identify behaviors in videos to monitor bad behaviors in videos, or it can be applied to video surveillance in public places to monitor illegal behaviors in public places, etc., not limited to the examples here.

[0042] The technical solution of the present application can be specifically applied to various types of computer devices, such as laptop computers, servers, and the like. The computer device can obtain a target behavior recognition model (i.e., a training process) by training the original behavior recognition model, wherein the behavior recognition model refers to a recognition model that distinguishes and recognizes the content in an image to determine the behavior of objects in the image. Furthermore, the computer device can perform behavior recognition (i.e., a recognition process) using the target recognition model obtained through training. The computer device in the training process and the computer device in the recognition process can be the same computer device or different computer devices, and this application does not impose any restrictions thereon.

[0043] The technical concept of the present application is as follows: in the process of training the behavior recognition model, in addition to using the behavior label indicating the behavior type corresponding to the image as a learning label, the semantic label indicating the semantic information in the image is also used as an auxiliary learning label to guide the behavior recognition model, so that the behavior model can not only learn the behavior features in the image, but also learn the semantic information contained in the image, so that the behavior recognition model has a stronger ability to understand the image, and thus the recognition accuracy of the behavior recognition model can be improved. Since the behavior recognition model has a stronger ability to understand the image, using the behavior recognition model for behavior recognition can achieve a more accurate recognition effect.

[0044] The technical solution of this application is described in detail below.

[0045] See also Figure 1 , Figure 1 A flow chart of a behavior recognition model training method provided in an embodiment of the present application, which method can be applied to a computing device, such as Figure 1 As shown, the method comprises the following steps:

[0046] S101, obtaining at least one behavior image sample, a behavior label corresponding to at least one behavior image sample, and at least one semantic label corresponding to at least one behavior image sample.

[0047] Among them, the behavior image sample refers to the behavior image used as a training sample. The behavior image presents a certain behavior in the form of an image. The behavior image can be, for example, an image presenting dancing, an image presenting a circus performance, and so on. In the case where the behavior recognition model recognizes the behavior type based on a behavior image, for example, the behavior recognition model is composed of an ordinary neural network, a behavior image can be used as a training sample; in the case where the behavior recognition model recognizes the behavior type based on a combination of multiple behavior images, for example, the behavior recognition model is a TSN model or an efficient convolutional network (ECO) model, multiple behavior images can be used as a training sample; wherein the multiple behavior images can be behavior images extracted from a video. Whether a training sample specifically includes one behavior image or multiple behavior images depends on the type and structure of the behavior recognition model, which is not limited in this application.

[0048] Among them, the behavior label is used to indicate the behavior type corresponding to at least one behavior image sample, and the behavior type refers to the type of behavior reflected by the content of the at least one behavior image sample. Assuming that in a behavior recognition scenario, the behaviors to be recognized are dancing, stage play, radio gymnastics, pet fighting, pet playing, and circus performances, then the behavior label can be used to indicate dancing, stage play, radio gymnastics, pet fighting, pet playing, or circus. For example, when the content of at least one behavior image sample reflects a pet fighting, the behavior label corresponding to the at least one behavior image sample is used to indicate a pet fighting.

[0049] The behavior label can be in the form of a unique hot encoding, and the number of bits of the behavior label is equal to the total number of behavior types. Still taking the six behaviors to be identified, namely dancing, stage play, radio gymnastics, pet fighting, pet playing and circus performance as an example, the behavior label (1,0,0,0,0,0) can be used to indicate dancing, the behavior label (0,1,0,0,0,0) can be used to indicate stage play, the behavior label (0,0,1,0,0,0) can be used to indicate radio gymnastics, the behavior label (0,0,0,1,0,0) can be used to indicate pet fighting, the behavior label (0,0,0,0,1,0) can be used to indicate pet playing, and the behavior label (0,0,0,0,0,1) can be used to indicate circus performance. Optionally, the behavior label can also be in other forms of expression, for example, it can be multi-hot encoding. This application does not limit the specific form of the behavior label.

[0050] Among them, the semantic label is used to indicate the possibility of the presence of a preset target in at least one behavior image sample, and one semantic label is used to indicate a preset target. The preset target is used to express the image semantics in the at least one image sample and is related to the behavior type, that is, a semantic label is used to indicate the possibility of a target related to the behavior type existing in the at least one image sample. The target refers to an object in the image, which can be an animal, a person, a flower, a tree, etc. The target related to the behavior type refers to a target that can distinguish the images corresponding to two behavior types with similar / possibly similar image features from the image semantics. For example, the behavior types are the six types of dancing, stage play, radio gymnastics, pet fighting, pet playing and circus performances mentioned above. Since dancing and pet fighting may have similar image backgrounds or image colors, the preset targets can be set to animals, people and other things. Animals and people can distinguish the two types of dancing and pet fighting from the image semantics. The number of semantic labels can be set based on classification requirements, which depends on the number of groups of images that need to be distinguished semantically.

[0051] In some possible scenarios, the semantic label can be expressed as a binary classification label, which can be used to indicate whether a preset target exists in at least one image sample, or to indicate the probability of the preset target existing in at least one image sample. Taking the preset targets as animals, people and other things as an example, the at least one image sample can correspond to three binary classification labels, which are respectively used to indicate whether there are animals in the at least one image sample, whether there are people in the at least one image sample, and whether there are other things in the at least one image sample; or, respectively, to indicate the probability of the existence of animals, people and other things in the at least one image sample. Optionally, the semantic label can also be expressed in other forms, for example, a semantic label can also be expressed in the form of a 1-bit code.

[0052] In a specific implementation, a training sample set can be formed by collecting behavior images corresponding to different behavior types in advance; then, a behavior label is pre-marked for each training sample according to the behavior type to which the training sample belongs, and a semantic label is pre-marked for each training sample according to the type of preset target contained in each training sample, so as to obtain a training sample set with training labels. During the training process, in each training cycle, one or more training samples can be obtained from the training sample set with training labels, thereby obtaining at least one behavior image sample, a behavior label corresponding to at least one behavior image sample, and at least one semantic label corresponding to at least one behavior image sample.

[0053] S102: Input at least one behavior image sample into a behavior recognition model in a current training cycle to obtain a first behavior prediction result and at least one preset target prediction result output by the behavior recognition model in the current training cycle.

[0054] In the embodiment of the present application, the behavior recognition model has the following two functions:

[0055] Function 1: Perform feature extraction and behavior recognition on at least one behavior image sample to obtain a first behavior prediction result. The first behavior prediction result is used to indicate the behavior type predicted by the behavior recognition model based on the at least one behavior image sample. The expression form of the first prediction result is consistent with the behavior label.

[0056] Among them, the first behavior prediction result may include the probability that the at least one behavior image sample belongs to each behavior type, and the behavior type corresponding to the maximum probability is the behavior type corresponding to the at least one behavior image sample identified by the behavior recognition model. Still taking the six behaviors to be identified, namely, dancing, stage play, radio gymnastics, pet fighting, pet playing, and circus performance as an example, the first behavior prediction result may include the probability that the content in the at least one behavior image sample belongs to dancing, the probability of belonging to stage play, the probability of belonging to radio gymnastics, the probability of belonging to pet fighting, the probability of belonging to pet playing, and the probability of belonging to circus performance. The behavior type corresponding to the maximum probability among these six probabilities is the behavior type identified by the behavior recognition model.

[0057] Function 2: Perform feature extraction and preset target recognition on at least one behavior image sample to obtain at least one preset target prediction result, where a preset target prediction result is used to indicate a preset target existing in the at least one behavior image sample predicted by the behavior recognition model based on the at least one behavior image sample. The number and expression form of the preset target prediction results are consistent with the semantic label.

[0058] Among them, a preset target prediction result may include the probability of the preset target existing in the at least one behavior image sample and the probability of the preset target not existing. Taking one of the preset targets as an animal as an example, a predicted target prediction result may include the probability of the preset target existing in the at least one behavior image sample and the probability of the preset target not existing.

[0059] Since the behavior recognition model generally only outputs behavior prediction results, that is, it has the above-mentioned function 1. In order to enable the behavior recognition model to output the preset target prediction results, the structure of the behavior recognition model needs to be improved so that the behavior recognition model has the above-mentioned function 2. The behavior recognition model of the present application is a behavior recognition model obtained by improving the structure of the behavior recognition model. In a feasible implementation, at least one preset target output layer can be additionally connected after the feature extraction network (network for feature extraction) of the behavior recognition model, and a preset target output layer is used to output a preset target prediction result, so that the behavior recognition model has the above-mentioned function 2, that is, the improved behavior recognition model adds at least one preset target output layer for outputting the preset target prediction result compared to the recognition model before the improvement. Among them, the preset target output layer can be a fully connected layer.

[0060] Specifically, taking the behavior recognition model as a TSN model without a temporal convolutional network as an example, see Figure 2A , Figure 2A It is a schematic diagram of the original structure of the TSN model without the temporal convolutional network. The feature extraction network of the TSN model without the temporal convolutional network includes multiple spatial convolutional networks X1 and feature layers X2; wherein, each spatial convolutional network X1 shares parameters, and a spatial convolutional network is used to extract features from a behavior image to obtain a first feature vector corresponding to the behavior image; each spatial convolutional network X1 is connected to the feature layer X2, and the feature layer X2 is used to average the first feature vectors corresponding to multiple behavior images to obtain a second feature vector. The feature layer X2 is connected to the behavior classification layer X3, and the behavior classification layer X3 is used to classify behaviors based on the second feature vector to obtain behavior types corresponding to multiple behavior images. The improved behavior recognition model in the present application can be as follows Figure 2B As shown, after the feature extraction network of TSN, there are n preset target output layers X4 connected, each preset target output layer X4 is connected to the feature layer X2, and one preset target output layer X4 is used to output a preset target prediction result; wherein n is the type of the preset target. For example, if the preset targets are animals, people, and other things, then n=3.

[0061] By connecting the preset target output layer after the feature extraction network of the behavior recognition model, only minor changes need to be made to the behavior recognition model to enable the behavior recognition model to output the preset target prediction results. Moreover, in the improved behavior recognition model, the preset target output layer for outputting the preset target prediction results and the behavior result output layer for outputting the behavior prediction results share the parameters in the feature extraction network, which can also help the behavior recognition model to better adjust and learn.

[0062] Optionally, the behavior recognition model may be improved by other methods of improving the behavior recognition model so that the improved behavior recognition model can output a preset target prediction result. The present application does not limit the method of improving the behavior recognition model.

[0063] S103, training the behavior recognition model in the current training cycle according to the behavior label corresponding to at least one behavior image sample, at least one semantic label corresponding to at least one behavior image sample, the first behavior prediction result and at least one preset target prediction result, so that the first behavior prediction result is close to the behavior label corresponding to at least one behavior image sample and the at least one preset target prediction result is close to at least one semantic label corresponding to at least one behavior image sample.

[0064] In the embodiment of the present application, training the behavior recognition model in the current training cycle refers to adjusting the parameters of the behavior recognition model in the current training cycle based on the error between the expected output of the behavior recognition model in the current training cycle and the actual output of the behavior recognition model in the current training cycle, so that the actual output of the behavior recognition model in the current training cycle approaches the expected output of the behavior recognition model in the current training cycle. Among them, the behavior label corresponding to the at least one behavior image sample and the at least one semantic label corresponding to the at least one behavior image sample reflect the expected output of the behavior recognition model in the current training cycle, and the first behavior prediction result and the at least one preset target prediction result reflect the actual output of the behavior recognition model in the current training cycle. Through training in multiple training cycles, the parameters of the behavior recognition model are adjusted multiple times, so that the actual output of the behavior recognition model can be infinitely close to the expected output of the behavior recognition model, and then converge. Among them, in each training cycle, the above steps S101 to S103 are executed.

[0065] In a feasible implementation, the behavior recognition model in the current training cycle may be trained through the following steps A1-A3.

[0066] A1. Calculate a first loss according to a behavior label corresponding to at least one behavior image sample and a first behavior prediction result.

[0067] Among them, the first loss is used to indicate the accuracy of the behavior recognition model in identifying the behavior type in the current training cycle, and reflects the gap between the first behavior prediction result and the behavior label corresponding to the at least one behavior image sample. The smaller the first loss, the smaller the gap between the first behavior prediction result and the behavior label corresponding to the at least one behavior image sample, the more behavior features learned by the behavior recognition model in the current training cycle, and the higher the accuracy of identifying the behavior type; the larger the first loss, the larger the gap between the first behavior prediction result and the behavior label corresponding to the at least one behavior image sample, the fewer behavior features learned by the behavior recognition model in the current training cycle, and the lower the accuracy of identifying the behavior type.

[0068] In a specific implementation, the first loss may be calculated based on the cross entropy. The calculation formula of the first loss may be as follows:

[0069]

[0070] Among them, L1 is the first loss, m is the total number of behavior types, and y 1i is the behavior label corresponding to at least one behavior image sample, y 2i is the first behavior prediction result, and i represents the behavior label or the i-th position in the first behavior prediction result.

[0071] Calculating the first loss through cross entropy can avoid gradient diffusion and enable the behavior recognition model to learn faster.

[0072] Optionally, the first loss may also be calculated based on an indicator for measuring the gap, such as the Euclidean distance, which is not limited in this application.

[0073] A2. Calculate a second loss based on at least one semantic label and at least one preset target prediction result corresponding to at least one behavior image sample.

[0074] Among them, the second loss is used to indicate the accuracy of the behavior recognition model in the current training cycle in identifying the preset target, reflecting the gap between the at least one preset target prediction result and the at least one behavior image sample. The smaller the second loss, the smaller the gap between the at least one preset target prediction result and the at least one semantic label corresponding to the at least one behavior image sample, the more semantic features learned by the behavior recognition model in the current training cycle, and the higher the accuracy of identifying the preset target; the larger the first loss, the larger the gap between the at least one preset target prediction result and the at least one semantic label corresponding to the at least one behavior image sample, the fewer semantic features learned by the behavior recognition model in the current training cycle, and the lower the accuracy of identifying the preset target.

[0075] In a specific implementation, the second loss may be calculated based on the cross entropy. The calculation formula of the second loss may be as follows:

[0076]

[0077] Among them, L2 is the second loss, n is the type of preset target, y 3j is the jth semantic label corresponding to at least one action image sample, y 4j Predict the result for the jth preset target.

[0078] Calculating the second loss through cross entropy can avoid gradient diffusion and enable the behavior recognition model to learn faster.

[0079] Optionally, the second loss may be calculated by other loss calculation methods, which are not limited in this application.

[0080] A3. Train the behavior recognition model in the current training cycle according to the first loss and the second loss.

[0081] Here, the total loss of the behavior recognition model in the current training cycle can be calculated based on the first loss and the second loss. The total loss is used to measure the total error between the expected output of the behavior recognition model in the current training cycle and the actual output of the behavior recognition model in the current training cycle. The behavior recognition model in the current training cycle is trained based on the total loss. Specifically, the parameters of the behavior recognition model in the current training cycle can be adjusted based on the total loss and the stochastic gradient descent method based on the back propagation method.

[0082] In some possible cases, the first loss and the second loss can be directly summed to obtain the total loss of the behavior recognition model in the current training cycle. In this case, the loss of the behavior recognition model in the current training cycle is calculated as: L3 = L1 + L2. Among them, L3 is the total loss, L1 is the first loss, and L2 is the second loss.

[0083] In some other possible situations, in order to enable the behavior recognition model to learn better, the behavior recognition model can be made to perform different learning in different training cycles.

[0084] Specifically, in a feasible implementation, the behavior recognition model can be made to perform different learning in different training cycles by adjusting the weights of the first loss and / or the second loss in different training cycles. The above step A3 may include the following steps A31-A33.

[0085] A31. Calculate a weight corresponding to the second loss according to the number of training cycles corresponding to the current training cycle and the number of semantic tags corresponding to at least one semantic tag.

[0086] A32, based on the weight corresponding to the second loss, weighted sum the first loss and the second loss to obtain the total loss of the behavior recognition model in the current training cycle.

[0087] A33, training the behavior recognition model in the current training cycle according to the total loss of the behavior recognition model in the current training cycle.

[0088] The weight corresponding to the second loss is negatively correlated with the number of training cycles. Specifically, the weight corresponding to the second loss can be calculated as follows:

[0089]

[0090] Among them, K is the weight corresponding to the second loss, C is the experimental constant that can be obtained based on experience, n is the type of preset target, and t is the number of training cycles.

[0091] In this case, if the weight corresponding to the first loss remains unchanged in each training cycle, the total loss is calculated as: L3 = L1 + K * L2; if the weight corresponding to the first loss changes in each training cycle, the total loss is calculated as L3 = S * L1 + K * L2. Among them, L3 is the total loss, L1 is the first loss, L2 is the second loss, and S is the weight of the first loss in the current training cycle.

[0092] By determining the weight corresponding to the second loss according to the current training cycle, and making the weight corresponding to the second loss negatively correlated with the number of training cycles, the influence of the second loss on the learning of the behavior recognition model is gradually reduced, so that the behavior model can be learned with focus during the training process, thereby improving the learning effect of the behavior recognition model, thereby improving the accuracy of the behavior recognition model.

[0093] S104, determining the behavior recognition model obtained by the last training cycle as the target behavior recognition model.

[0094] Among them, when the total loss of the behavior recognition model in the current training cycle is less than the preset loss threshold or the number of training cycles corresponding to the current training cycle reaches the preset number of cycles, it is determined that the behavior recognition model in the current training cycle converges and the current training cycle is the last training cycle.

[0095] In the above technical scheme, by obtaining at least one behavior image sample, a behavior label for indicating the behavior type corresponding to the behavior image sample, and a semantic label for indicating at least one preset target present in the at least one behavior image sample, at least one behavior image sample is input into a behavior recognition model, and a behavior prediction result and a preset target prediction result corresponding to the at least one behavior image sample are obtained through the behavior recognition model. Then, the behavior recognition model is trained according to the behavior label, the semantic label, the behavior prediction result, and the preset target prediction result, so that the behavior prediction result output by the behavior recognition model is close to the behavior label, and the preset target prediction result output by the behavior recognition model is close to the semantic label, and the behavior recognition model finally trained is determined as the target behavior recognition model. During the training process, the behavior recognition model is trained by combining the semantic labels of preset targets that express the image semantics of the behavior image samples and are related to the behavior types, so that the behavior prediction results output by the behavior recognition model are close to the behavior labels and the preset target prediction results output by the behavior recognition model are close to the semantic labels. The existence of semantic labels guides the learning of the behavior recognition model, so that in addition to learning the behavioral features in the image, the behavior recognition model also learns the semantic information in the image, and can then understand the image content from a semantic level, thereby improving the recognition accuracy of the behavior recognition model.

[0096] Optionally, in some possible situations, the semantic label of at least one behavior image sample may also be determined by using a model to identify semantic categories in the behavior image samples.

[0097] The specific implementation of obtaining at least one semantic label corresponding to at least one behavior image sample may include the following steps B1-B2:

[0098] B1. Input at least one behavior image sample into a target recognition model to obtain a confidence score corresponding to at least one preset target output by the target recognition model.

[0099] Here, the target recognition model is a pre-trained model for identifying preset targets. The target recognition model can recognize behavioral images and output a confidence score for the presence of preset targets in the behavioral images. The confidence score is used to reflect the possibility of the presence of preset targets in the behavioral images. The higher the confidence score, the higher the possibility of the presence of preset targets in the behavioral images. The higher the confidence score, the lower the possibility of the presence of preset targets in the behavioral images. For each behavioral image sample, the target recognition model will output the confidence score of each preset target in the behavioral image sample. When the number of behavioral image samples is multiple, the target recognition model will output multiple confidence scores for each preset target, and the number of confidence scores for each preset target is equal to the number of behavioral image samples.

[0100] The target recognition model includes but is not limited to a single shot multibox detector (SSD) model, a YOLO model or a fast R-CNN model, etc. Specifically, the target recognition model may be a yolov5 model, wherein the yolo v5 model may be pre-trained based on the coco dataset, and the trained yolo v5 model may recognize 80 types of targets, which may meet a variety of training requirements.

[0101] B2. Determine at least one semantic label according to the confidence score corresponding to at least one preset target.

[0102] Specifically, when the number of behavioral image samples is one, the confidence scores corresponding to at least one preset target output by the target recognition model can be directly combined to obtain at least one semantic label. For example, the preset targets are animals, people, and other things. The confidence scores of animals output by the target recognition model are 0.93, 0.01, and 0.2, respectively; the semantic label is (0.93, 0.01, 0.2), which can also be split into 3 binary classification labels, namely, the binary classification labels corresponding to animals (0.93, 0.07), the binary classification labels corresponding to people (0.01, 0.99), and the binary classification labels corresponding to other things (0.2, 0.8).

[0103] When there are multiple behavioral image samples, the target recognition model will output multiple confidence scores for any preset target. The multiple confidence scores of any preset target can be averaged to obtain the semantic label corresponding to the preset target. For example, if the preset target is an animal and the number of behavioral image samples is 25, then after the 25 behavioral image samples are input into the target recognition model, the target recognition model will output 25 confidence scores corresponding to the animal. The 25 confidence scores corresponding to the animal can be averaged to obtain the semantic label corresponding to the animal. By processing the multiple confidence scores corresponding to each preset target in the same way, multiple semantic labels can be obtained.

[0104] The semantic labels of image samples are obtained through the target recognition model, so that the semantic labels can change accordingly with the changes in the size of the behavior image samples input to the behavior recognition model in the current training week. This not only automatically annotates the semantic labels of the behavior image samples, but also makes the semantic labels more matched with the behavior image samples of the behavior recognition model in the current training cycle, thereby helping to improve the accuracy of the behavior recognition model.

[0105] Optionally, in some possible situations, the learning intensity of the behavior recognition model for the preset target may be adjusted according to the image proportion of the preset target in the behavior image samples.

[0106] Among them, the learning intensity of the behavior recognition model for the preset target can be adjusted in the following ways.

[0107] The first method is: in the process of calculating the loss of the behavior recognition model, different weights are added to the preset targets with different image proportions according to the image proportion of the preset targets in the behavior image samples, so as to adjust the learning intensity of the behavior recognition model for the preset targets.

[0108] Specifically, the above step A2 may include the following steps A21-A23.

[0109] A21, determining a gap between the first semantic label and a preset target prediction result corresponding to the first semantic label to obtain a sub-loss corresponding to the first semantic label.

[0110] Here, the first semantic label is any one of the at least one semantic label corresponding to the at least one behavior image sample. The sub-loss corresponding to the first semantic label can be calculated based on the cross entropy, wherein the calculation formula of the sub-loss corresponding to the first semantic label can be as follows:

[0111] g j =-(y 3j logy 4j -(1-y 3j )log(1-y 4j ))

[0112] Among them, y 3j is the jth semantic label (i.e., the first semantic label), y 4j The j-th preset target prediction result g corresponding to the j-th semantic label j is the sub-loss corresponding to the j-th semantic label.

[0113] A22, determining a weight corresponding to the first semantic tag according to a preset target corresponding to the first semantic tag and an image proportion in at least one behavior image sample.

[0114] Specifically, the image proportion of the preset target corresponding to the first semantic label in at least one behavior image sample may be determined as the weight corresponding to the first semantic label.

[0115] A23, according to the weight corresponding to the at least one semantic tag, weighted sum the sub-losses corresponding to the at least one semantic tag to obtain a second loss.

[0116] Specifically, the calculation formula of the second loss is as follows:

[0117]

[0118] Among them, L2 is the second loss, q j is the weight corresponding to the jth semantic label, g j is the sub-loss corresponding to the j-th semantic label.

[0119] In the process of calculating the second loss indicating the accuracy of the behavior recognition model in identifying preset targets, the weight of each preset target is determined according to the proportion of various preset targets in the image, and the second loss is obtained by weighting. This enables the behavior recognition model to learn the semantic information in the image with emphasis, thereby improving the accuracy of the behavior recognition model.

[0120] The second method: in the process of setting semantic labels, the semantic labels of the preset targets are modified according to the image proportion of the preset targets in the behavior image samples.

[0121] In the case where the semantic labels are set for direct labeling, in a feasible implementation, after the semantic labels are set, the image proportion of each preset target in the behavioral image sample can be used as the coefficient of the semantic label corresponding to each preset target, and multiplied by the semantic label corresponding to each preset target to modify the semantic label of each preset target.

[0122] For example, the preset targets are animals, people, and other things. There are animals and people in the behavior image samples. The semantic labels corresponding to the behavior image samples are (1, 1, 0). Among them, the proportion of animals in the behavior image samples is 0.3, and the proportion of people in the behavior image samples is 0.5. The modified semantic labels are (0.3, 0.5, 0).

[0123] In the case where the semantic label is set by the target recognition model, the semantic label of each preset target can be modified through the following steps C1-C3.

[0124] C1. Obtain the position coordinates of the first preset target.

[0125] Here, the first preset target is any preset target identified by the target recognition model. After at least one behavior image sample is input into the target recognition model, the target recognition model can output the position coordinates of the preset target identified by the target recognition model in addition to the confidence score of the preset target, wherein the position coordinates are used to indicate the position of the preset target in at least one behavior image sample. Specifically, the position coordinates of the preset target can be expressed as (x, y, w, h), wherein (x, y) refers to the fixed-point coordinates of the upper left corner of the preset target, and w and h refer to the width and height of the preset target.

[0126] C2. Determine the image proportion of the first preset target in at least one behavior image sample according to the position coordinates of the first preset target.

[0127] Specifically, the area of ​​the first preset target can be calculated according to the width and height of the first preset target, and then the ratio of the area of ​​the first preset target to the area in the behavior image sample is determined as the image proportion of the first preset target in at least one behavior image sample.

[0128] Among them, in the case where there are multiple behavioral image samples, the image proportion of the first preset target in each behavioral image sample can be determined separately to obtain multiple image proportions, and then the average of the multiple image proportions can be determined as the image proportion corresponding to the first preset target. For example, if the first preset target is an animal and the number of behavioral image samples is 25, then the image proportion of the animal in each behavioral image sample can be determined to obtain 25 image proportions, and the average of the 25 image proportions can be used as the image proportion of the animal. Optionally, after obtaining multiple image proportions, the largest image proportion among the multiple image proportions can also be used as the image proportion corresponding to the first preset target.

[0129] C3. Update at least one semantic label according to the image proportion of each preset target in at least one behavior image sample.

[0130] In a feasible implementation, the confidence score of the first preset target can be modified according to the image proportion of the first preset target in at least one image sample to update the semantic label corresponding to the first preset target, wherein the semantic label corresponding to the first preset target refers to the semantic label used to indicate the first preset target. Specifically, the image proportion of the first preset target in at least one image sample can be used as a coefficient of the semantic label corresponding to the first preset target, and the image proportion of the first preset target in at least one image sample can be multiplied by the semantic label corresponding to the first preset target to update the semantic label corresponding to the first preset target.

[0131] Optionally, the value of the semantic label corresponding to the first preset target may be adjusted according to the image proportion of the first preset target in at least one image sample to update the semantic label corresponding to the first preset target. Specifically, if the image proportion of the first preset target in at least one image sample is greater than the preset image proportion, it means that the first preset target is larger, and the value of the semantic label corresponding to the first preset target may be adjusted to a first value, such as 1; if the image proportion of the first preset target in at least one image sample is not greater than the preset image proportion, it means that the first preset target is smaller, and the value of the semantic label corresponding to the first preset target may be adjusted to a second value, such as 0.

[0132] It should be understood that by updating each semantic label by updating the semantic label corresponding to the first preset target, all semantic labels can be updated. The method of modifying the confidence score of the first preset target is not limited to the above method of using the image proportion of the first preset target in at least one image sample as the coefficient of the semantic label corresponding to the first preset target; the method of updating and adjusting the semantic label is not limited to the above two methods.

[0133] After obtaining the semantic label of the image sample, the semantic label is updated according to the proportion of the target indicated by the semantic label in the image sample, so that the semantic label can be made more accurate and the behavior recognition model can be better learned.

[0134] It should be noted that adjusting the learning intensity of the behavior recognition model for the preset target according to the image proportion of the preset target in the behavior image sample is not limited to the above method, and this application does not limit it.

[0135] After training the target behavior recognition model, you can use it to perform behavior recognition. Figure 3 , Figure 3 A flow chart of a behavior recognition method provided in an embodiment of the present application, which method can be applied to a computer device; Figure 3 As shown, the method comprises the following steps:

[0136] S201, obtaining at least one behavior image.

[0137] Here, the concept of behavior images can be referred to the description of the aforementioned step S101, which will not be repeated here. The number of behavior images depends on the specific design of the behavior recognition model.

[0138] S202: Input at least one behavior image into a target behavior recognition model to obtain a second behavior prediction result output by the target behavior recognition model.

[0139] Here, the target behavior recognition model outputs the second behavior prediction result in a similar manner to the specific implementation manner in which the behavior recognition model outputs the first behavior prediction result in the aforementioned step S102. The second prediction result is used to indicate the behavior type predicted by the target behavior recognition model, wherein the second prediction result may include the probability that the at least one behavior image belongs to each behavior type.

[0140] S203: Determine a behavior type corresponding to at least one behavior image according to the second behavior prediction result.

[0141] Specifically, the behavior type corresponding to the maximum probability may be determined as the behavior type corresponding to at least one behavior image.

[0142] In this technical solution, in the process of training the target behavior recognition model, the preset target that expresses the image semantics of the behavior image samples and is related to the behavior type is combined to train the behavior recognition model, so that the behavior recognition model learns the semantic information in the image in addition to the image features of the image. Therefore, by identifying at least one behavior image through the target behavior recognition model, the behavior in the at least one behavior image can be more accurately identified.

[0143] The method of the present application is introduced above. In order to better implement the method of the present application, the device of the present application is introduced next.

[0144] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a behavior recognition model training device provided in an embodiment of the present application. The behavior recognition model training device may be a computer device or a part of a computer device. Figure 4 As shown, the behavior recognition model training device 30 includes:

[0145] The data acquisition module 301 is used to acquire at least one behavior image sample, a behavior label corresponding to the at least one behavior image sample, and at least one semantic label corresponding to the at least one behavior image sample, wherein the behavior label is used to indicate the behavior type corresponding to the at least one behavior image sample, and a semantic label is used to indicate the possibility that a preset target exists in the at least one behavior image sample; the preset target is used to express the image semantics of the at least one behavior image sample and is related to the behavior type;

[0146] A result acquisition module 302 is used to input the at least one behavior image sample into the behavior recognition model in the current training cycle to obtain a first behavior prediction result and at least one preset target prediction result output by the behavior recognition model in the current training cycle;

[0147] A training module 303 is used to train the behavior recognition model in the current training cycle according to the behavior label, the at least one semantic label, the first behavior prediction result and the at least one preset target prediction result, so that the first behavior prediction result is close to the behavior label and the at least one preset target prediction result is close to the at least one semantic label;

[0148] The model determination module 304 is used to determine the behavior recognition model obtained by the last training cycle as the target behavior recognition model.

[0149] In one possible design, the above-mentioned training module 303 is specifically used to: calculate a first loss based on the behavior label and the first behavior prediction result, and the first loss is used to indicate the accuracy of the behavior recognition model in the current training cycle in identifying the behavior type; calculate a second loss based on the at least one semantic label and the at least one preset target prediction result, and the second loss is used to indicate the accuracy of the behavior recognition model in the current training cycle in identifying the preset target; train the behavior recognition model in the current training cycle based on the first loss and the second loss.

[0150] In one possible design, the training module 303 is specifically used to: determine the gap between a first semantic label and a preset target prediction result corresponding to the first semantic label to obtain a sub-loss corresponding to the first semantic label, where the first semantic label is any one of the at least one semantic label; determine a weight corresponding to the first semantic label based on the image proportion of the preset target corresponding to the first semantic label in the at least one behavioral image sample; and perform weighted summation of the sub-losses corresponding to each of the at least one semantic label based on the weights corresponding to each of the at least one semantic label to obtain the second loss.

[0151] In one possible design, the above-mentioned training module 303 is specifically used to: calculate the weight corresponding to the second loss according to the number of training cycles corresponding to the current training cycle and the number of semantic tags corresponding to the at least one semantic tag, wherein the weight corresponding to the second loss is negatively correlated with the number of training cycles; based on the weight corresponding to the second loss, perform weighted summation of the first loss and the second loss to obtain the total loss of the behavior recognition model in the current training cycle; and train the behavior recognition model in the current cycle according to the total loss.

[0152] In one possible design, the data acquisition module 301 is specifically used to: input the at least one behavior image sample into the target recognition model to obtain the confidence score corresponding to each of the at least one preset targets identified by the target recognition model; and determine the at least one semantic label based on the confidence score corresponding to each of the at least one preset targets.

[0153] In a possible design, the data acquisition module 301 is also used to: obtain the position coordinates of a first preset target, where the first preset target is any one of the at least one preset target; determine the image proportion of the first preset target in the at least one behavior image sample according to the position coordinates of the first preset target; and update the at least one semantic label according to the image proportion of each preset target in the at least one behavior image sample.

[0154] In a possible design, the data acquisition module 301 updates the at least one semantic label according to the image proportion of each preset target in the at least one behavioral image sample, including: modifying the confidence score of the first preset target according to the image proportion of the first preset target in the at least one behavioral image sample to update the semantic label corresponding to the first preset target.

[0155] It should be noted that Figure 4 For details not mentioned in the corresponding embodiments, please refer to the aforementioned Figure 1 The description of the method embodiment will not be repeated here.

[0156] The above-mentioned device, by obtaining at least one behavior image sample, a behavior label for indicating the behavior type corresponding to the behavior image sample and a semantic label for indicating at least one preset target present in the at least one behavior image sample, inputs at least one behavior image sample into a behavior recognition model, obtains the behavior prediction result and the preset target prediction result corresponding to the at least one behavior image sample through the behavior recognition model, and then trains the behavior recognition model according to the behavior label, the semantic label, the behavior prediction result and the preset target prediction result, so that the behavior prediction result output by the behavior recognition model is close to the behavior label, and the preset target prediction result output by the behavior recognition model is close to the semantic label, and the behavior recognition model finally trained is determined as the target behavior recognition model. During the training process, the behavior recognition model is trained by combining the semantic labels of preset targets that express the image semantics of the behavior image samples and are related to the behavior types, so that the behavior prediction results output by the behavior recognition model are close to the behavior labels and the preset target prediction results output by the behavior recognition model are close to the semantic labels. The existence of semantic labels guides the learning of the behavior recognition model, so that in addition to learning the behavioral features in the image, the behavior recognition model also learns the semantic information in the image, and can then understand the image content from a semantic level, thereby improving the recognition accuracy of the behavior recognition model.

[0157] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a behavior recognition device provided in an embodiment of the present application. The behavior recognition device may be a computer device or a part of a computer device. Figure 5 As shown, the behavior recognition device 40 includes:

[0158] An image acquisition module 401 is used to acquire at least one behavior image;

[0159] A result output module 402 is used to input the at least one behavior image into a target behavior recognition model to obtain a second behavior prediction result output by the target behavior recognition model, wherein the target behavior recognition model is obtained by training through the above method embodiment;

[0160] The type determination module 403 is used to determine the behavior type corresponding to the at least one behavior image according to the second behavior prediction result.

[0161] It should be noted that Figure 5 For details not mentioned in the corresponding embodiments, please refer to the aforementioned Figure 3 The description of the method embodiment will not be repeated here.

[0162] The above-mentioned device combines the preset targets that express the image semantics of the behavior image samples and are related to the behavior type to train the behavior recognition model during the process of training the target behavior recognition model, so that the behavior recognition model learns the semantic information in the image in addition to the image features of the image. Therefore, by identifying at least one behavior image through the target behavior recognition model, the behavior in the at least one behavior image can be more accurately identified.

[0163] See also Figure 6 , Figure 6 1 is a schematic diagram of a computer device provided in an embodiment of the present application, wherein the computer device 50 includes a processor 501 and a memory 502. The processor 501 is connected to the memory 502, for example, the processor 501 may be connected to the memory 502 via a bus.

[0164] The processor 301 is configured to support the computer device 50 to perform the corresponding functions in the method in the above method embodiment. The processor 501 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0165] The memory 502 is used to store program codes, etc. The memory 302 may include a volatile memory (VM), such as a random access memory (RAM); the memory 402 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 502 may also include a combination of the above-mentioned types of memories.

[0166] In some possible situations, the processor 501 may call the program code to perform the following operations:

[0167] Acquire at least one behavior image sample, a behavior label corresponding to the at least one behavior image sample, and at least one semantic label corresponding to the at least one behavior image sample, wherein the behavior label is used to indicate the behavior type corresponding to the at least one behavior image sample, and a semantic label is used to indicate the possibility that a preset target exists in the at least one behavior image sample; the preset target is used to express the image semantics of the at least one behavior image sample and is related to the behavior type;

[0168] Inputting the at least one behavior image sample into a behavior recognition model in a current training cycle to obtain a first behavior prediction result and at least one preset target prediction result output by the behavior recognition model in the current training cycle;

[0169] Training the behavior recognition model in the current training cycle according to the behavior label, the at least one semantic label, the first behavior prediction result, and the at least one preset target prediction result, so that the first behavior prediction result is close to the behavior label and the at least one preset target prediction result is close to the at least one semantic label;

[0170] The behavior recognition model obtained by the last training cycle is determined as the target behavior recognition model.

[0171] In some other possible situations, the processor 501 may call the program code to perform the following operations:

[0172] Obtain at least one behavioral image;

[0173] Inputting the at least one behavior image into a target behavior recognition model to obtain a second behavior prediction result output by the target behavior recognition model, wherein the target behavior recognition model is trained by the above method embodiment;

[0174] The behavior type corresponding to the at least one behavior image is determined according to the second behavior prediction result.

[0175] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method described in the above embodiment.

[0176] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0177] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A behavior recognition model training method, characterized in that: include: Acquire at least one behavior image sample, a behavior label corresponding to the at least one behavior image sample, and at least one semantic label corresponding to the at least one behavior image sample, wherein the behavior label is used to indicate the behavior type corresponding to the at least one behavior image sample, and a semantic label is used to indicate the possibility that a preset target exists in the at least one behavior image sample; the preset target is used to express the image semantics of the at least one behavior image sample and is related to the behavior type; Inputting the at least one behavior image sample into a behavior recognition model in a current training cycle to obtain a first behavior prediction result and at least one preset target prediction result output by the behavior recognition model in the current training cycle; Training the behavior recognition model in the current training cycle according to the behavior label, the at least one semantic label, the first behavior prediction result, and the at least one preset target prediction result, so that the first behavior prediction result is close to the behavior label and the at least one preset target prediction result is close to the at least one semantic label; The behavior recognition model obtained by the last training cycle is determined as the target behavior recognition model. The training of the behavior recognition model in the current training cycle according to the behavior label, the at least one semantic label, the first behavior prediction result and the at least one preset target prediction result includes: calculating a first loss according to the behavior label and the first behavior prediction result, wherein the first loss is used to indicate the accuracy of the behavior recognition model in the current training cycle in identifying the behavior type; Calculating a second loss according to the at least one semantic tag and the at least one preset target prediction result, wherein the second loss is used to indicate the accuracy of the behavior recognition model in the current training cycle in identifying the preset target; The behavior recognition model in the current training cycle is trained according to the first loss and the second loss. The calculating the second loss according to the at least one semantic label and the at least one preset target prediction result includes: determining a gap between a first semantic label and a preset target prediction result corresponding to the first semantic label to obtain a sub-loss corresponding to the first semantic label, wherein the first semantic label is any one of the at least one semantic label; Determining a weight corresponding to the first semantic tag according to a preset target corresponding to the first semantic tag and an image proportion in the at least one behavior image sample; According to the weight corresponding to each of the at least one semantic tags, weighted summation is performed on the sub-losses corresponding to each of the at least one semantic tags to obtain the second loss, The training of the behavior recognition model in the current training cycle according to the first loss and the second loss includes: calculating a weight corresponding to the second loss according to the number of training cycles corresponding to the current training cycle and the number of semantic tags corresponding to the at least one semantic tag, wherein the weight corresponding to the second loss is negatively correlated with the number of training cycles; Based on the weight corresponding to the second loss, performing a weighted summation on the first loss and the second loss to obtain a total loss of the behavior recognition model in the current training cycle; The behavior recognition model in the current cycle is trained according to the total loss.

Images