An image behavior recognition model training method and an image behavior recognition method

Abstract

The present application relates to the technical field of computer vision, and particularly provides an image behavior recognition model training method and an image behavior recognition method, aiming to solve the problem of long time consumption and unstable effect of the existing image behavior recognition method. To this end, the image behavior recognition model training method comprises: obtaining an initial recognition model, the initial recognition model comprising a classification head and a segmentation head, obtaining initial training model parameters and a training sample data set, training the initial recognition model based on the initial training model parameters and the training sample data set, removing the segmentation head from the trained initial recognition model to obtain an image behavior recognition model. Through the above implementation, the model can effectively focus on the overall posture information of the person and the associated information of each different part of the human body, reduce the model parameter amount, thereby reducing the inference time consumption, and improving the accuracy and efficiency of image behavior recognition.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to an image behavior recognition model training method and an image behavior recognition method. Background Technology

[0002] Existing behavior recognition methods aim to identify the actions / behaviors performed by people in videos or images through algorithms. These methods have important applications in human-computer interaction, health monitoring, and security management, such as gesture recognition, fall detection, and fight detection. Compared to video behavior recognition, which can fully acquire spatial and temporal motion information over a period of time, image behavior recognition only takes a single image as input, providing only spatial information at a single moment. Therefore, relying solely on image classification techniques makes it difficult to achieve high recognition accuracy in image behavior recognition methods.

[0003] Current image action recognition methods typically use human keypoint information or local bounding boxes as aids to enable the model to effectively focus on human posture information. However, human keypoint algorithms are time-consuming and perform poorly for occluded or special human postures, such as sitting or lying down. Local bounding boxes also usually need to be obtained through human keypoint algorithms or detection algorithms, which similarly suffer from time consumption and inconsistent results. These problems make image action recognition methods unsatisfactory in terms of both accuracy and speed.

[0004] Accordingly, a new technical solution is needed in this field to solve the above problems. Summary of the Invention

[0005] To overcome the above-mentioned shortcomings, this invention is proposed to provide an image behavior recognition model training method and an image behavior recognition method that solves or at least partially solves the technical problems of long time consumption and unstable performance of existing image behavior recognition methods.

[0006] In a first aspect, a method for training an image behavior recognition model is provided, the method comprising:

[0007] Obtain an initial recognition model, which includes a classification head and a segmentation head;

[0008] Obtain the initial training model parameters and training sample dataset;

[0009] The initial recognition model is trained based on the initial training model parameters and the training sample dataset;

[0010] The image behavior recognition model is obtained by removing the segmentation head from the trained initial recognition model.

[0011] In one technical solution of the above image behavior recognition model training method, obtaining the initial training model parameters includes:

[0012] Obtain the human body analysis training dataset;

[0013] The first human body analysis model is trained based on the human body analysis training dataset to obtain the parameters of the first human body analysis model;

[0014] The parameters of the first human body analysis model are used as the parameters of the initial training model.

[0015] In one technical solution of the above image behavior recognition model training method, the initial recognition model includes a human body parsing sub-model and an image behavior recognition sub-model, wherein the human body parsing sub-model includes the segmentation head, and the image behavior recognition model includes the classification head; training the initial recognition model includes:

[0016] The human body parsing sub-model and the image behavior recognition sub-model are trained based on the initial training model parameters and the training sample dataset;

[0017] When both the human body parsing sub-model and the image behavior recognition sub-model converge to a preset error value, the initial recognition model training is completed.

[0018] In one technical solution of the above image behavior recognition model training method, the preset error value includes a preset classification error value and a preset segmentation error value. The step of completing the initial recognition model training when both the human body parsing sub-model and the image behavior recognition sub-model converge to the preset error value includes:

[0019] Based on the classification head, predict the scores of different categories of behavior and obtain the classification error value;

[0020] The segmentation map is predicted based on the segmentation head, and the segmentation error value is obtained;

[0021] The initial recognition model training is completed when the classification error value converges to the preset classification error value and the segmentation error value converges to the preset segmentation error value.

[0022] In one technical solution of the above image behavior recognition model training method, obtaining the initial recognition model includes:

[0023] The initial recognition model is obtained by adding the classification head to the first human body parsing model, wherein the first human body parsing model is equipped with the segmentation head;

[0024] Alternatively, the initial recognition model can be obtained by adding the classification head to the second human body parsing model, wherein the second human body parsing model and the first human body parsing model have the same structure.

[0025] In a second aspect, an image behavior recognition model training device is provided, the device comprising:

[0026] The first acquisition module is configured to acquire an initial recognition model, which includes a classification head and a segmentation head;

[0027] The second acquisition module is configured to acquire the initial training model parameters and the training sample dataset;

[0028] A training module is configured to train the initial recognition model based on the initial training model parameters and the training sample dataset;

[0029] The removal module is configured to remove the segmentation head from the trained initial recognition model to obtain the image behavior recognition model.

[0030] In a third aspect, an image behavior recognition method is provided, the method comprising:

[0031] Acquire the image to be identified;

[0032] The image to be identified is input into the image behavior recognition model to obtain the behavior recognition result;

[0033] The image behavior recognition model is trained based on the image behavior recognition model training method described in any one of the above-mentioned technical solutions for image behavior recognition model training methods.

[0034] In a fourth aspect, an image behavior recognition device is provided, the device comprising:

[0035] The third acquisition module is configured to acquire the image to be recognized;

[0036] The recognition module is configured to input the image to be recognized into an image behavior recognition model to obtain a behavior recognition result, wherein the image behavior recognition model is trained based on the image behavior recognition model training method described in any of the above-mentioned technical solutions of the image behavior recognition model training method.

[0037] In a fifth aspect, an electronic device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to perform the image behavior recognition model training method described in any one of the above-described technical solutions of the image behavior recognition model training method or the image behavior recognition method described in any one of the above-described technical solutions of the image behavior recognition method.

[0038] In a sixth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored, the program codes being adapted to be loaded and run by a processor to perform the image behavior recognition model training method described in any of the above-described technical solutions of the image behavior recognition model training method or the image behavior recognition method described in any of the above-described technical solutions of the image behavior recognition method.

[0039] The present invention comprises one or more of the following technical solutions: Beneficial effects:

[0040] In implementing the technical solution of this invention, an initial recognition model is first obtained, which includes a classification head and a segmentation head. Then, initial training model parameters and a training sample dataset are obtained. The initial recognition model is trained based on these parameters and the training sample dataset. Finally, the segmentation head is removed from the trained initial recognition model to obtain an image behavior recognition model. Through this implementation, behavior classification can be learned based on the classification head of the initial recognition model, and different parts of the human body can be segmented based on the segmentation head. This allows the model to effectively focus on the overall posture information of the person and the correlation information between different parts of the human body. Removing the segmentation head from the trained initial recognition model reduces the number of model parameters, resulting in an end-to-end image behavior recognition model. This reduces inference time and improves the accuracy and efficiency of image behavior recognition. Attached Figure Description

[0041] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Wherein:

[0042] Figure 1 This is a schematic diagram of the main steps of an image behavior recognition model training method according to an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the main steps for obtaining initial training model parameters according to an embodiment of the present invention;

[0044] Figure 3This is a schematic diagram of the main steps for training an initial recognition model according to an embodiment of the present invention.

[0045] Figure 4 This is a schematic diagram of the main steps for training an initial recognition model according to another embodiment of the present invention;

[0046] Figure 5 This is a main structural block diagram of an image behavior recognition model training device according to an embodiment of the present invention;

[0047] Figure 6 This is a schematic flowchart of the main steps of an image behavior recognition method according to an embodiment of the present invention;

[0048] Figure 7 This is a main structural block diagram of an image behavior recognition device according to an embodiment of the present invention;

[0049] Figure 8 This is a schematic diagram of the main structure of an electronic device according to an embodiment of the present invention.

[0050] List of reference numerals in the attached diagram:

[0051] 501: First acquisition module; 502: Second acquisition module; 503: Training module; 504: Removal module; 701: Third acquisition module; 702: Recognition module; 801: Processor; 802: Storage device. Detailed Implementation

[0052] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0053] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0054] Here we will first explain some of the terms involved in this invention.

[0055] Image segmentation: refers to the techniques and processes of dividing an image into several regions according to different categories and extracting targets of interest.

[0056] Loss function: Each sample, after being processed by the model, yields a predicted value. The difference between the predicted value and the true value is called the loss. A loss function is a type of function that calculates the difference between the predicted value and the true value.

[0057] Classification head: The part of the model used to calculate the score for each category.

[0058] Segmentation head: The part of the model used to predict and generate segmentation maps.

[0059] Human body segmentation model: an image segmentation model designed to segment different parts of the human body in an image, such as the head, arms, and torso.

[0060] As described in the background section, existing behavior recognition methods aim to identify what actions / behaviors a person is performing in a video or image through algorithms. These methods have important applications in fields such as human-computer interaction, health monitoring, and security management, including gesture recognition, fall detection, and fight detection. Compared to video behavior recognition, which can fully acquire spatial and temporal motion information over a period of time, image behavior recognition only takes a single image as input, meaning it only has spatial information at a single moment. Therefore, applying image classification techniques alone is insufficient to achieve high recognition accuracy in image behavior recognition methods.

[0061] Current image action recognition methods typically use human keypoint information or local bounding boxes as aids to enable the model to effectively focus on human posture information. However, human keypoint algorithms are time-consuming and perform poorly for occluded or special human postures, such as sitting or lying down. Local bounding boxes also usually need to be obtained through human keypoint algorithms or detection algorithms, which similarly suffer from time consumption and inconsistent results. These problems make image action recognition methods unsatisfactory in terms of both accuracy and speed.

[0062] To address the above problems, this invention provides an image behavior recognition model training method and an image behavior recognition method.

[0063] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of an image behavior recognition model training method according to an embodiment of the present invention. Figure 1 As shown, the image behavior recognition model training method in this embodiment of the invention mainly includes the following steps S101 to S104.

[0064] Step S101: Obtain the initial recognition model.

[0065] The initial recognition model includes a classification head and a segmentation head.

[0066] Step S102: Obtain the initial training model parameters and training sample dataset.

[0067] Step S103: Train the initial recognition model based on the initial training model parameters and the training sample dataset.

[0068] Step S104: Remove the segmentation head from the trained initial recognition model to obtain the image behavior recognition model.

[0069] Based on the method described in steps S101 to S104 above, behavior classification can be learned based on the classification head of the initial recognition model, and different parts of the human body can be segmented based on the segmentation head. This allows the model to effectively focus on the overall posture information of the human body and the correlation information of different parts of the human body. Removing the segmentation head from the trained initial recognition model can reduce the number of model parameters, resulting in an end-to-end image behavior recognition model. This reduces inference time and improves the accuracy and efficiency of image behavior recognition.

[0070] The following provides a further explanation of steps S101 to S104.

[0071] In some embodiments of step S101 above, obtaining the initial recognition model includes:

[0072] An initial recognition model is obtained by adding a classification head to the first human body parsing model, wherein the first human body parsing model is equipped with a segmentation head.

[0073] Specifically, the first human body analysis model is an image segmentation model designed to segment different parts of the human body in an image, such as the head, arms, and torso, and therefore includes a segmentation head.

[0074] The first human body analysis model can be FCN, DeepLab, UperNet, or other image segmentation models; no specific restrictions are imposed here.

[0075] Among them, the FCN (Fully Convolutional Networks) model is a fully convolutional network, belonging to deep learning image segmentation models. Its core idea is: to use a fully convolutional network without fully connected layers to take an image of arbitrary size as input; to use skip connections to combine image features of different scales, making the model more accurate and robust; and to use deconvolution to upsample the extracted low-resolution image, outputting a segmented image with the same resolution as the original image, preserving the spatial information of the original image, thus enabling prediction of every pixel in the original image.

[0076] DeepLab is a deep learning-based image semantic segmentation model. Its main principle is to use a Convolutional Neural Network (CNN) to identify different objects or scene elements in an image and classify these elements into different categories. DeepLab models include DeepLabv1, DeepLabv2, DeepLabv3, and DeepLabv3+.

[0077] UperNet is a unified perception parsing network, a feature extractor based on Feature Pyramid Networks (FPN). It has a hierarchical structure, can learn from heterogeneous data in multiple image datasets, can unify and combine visual knowledge, and can effectively predict hierarchical outputs.

[0078] In practical applications, those skilled in the art can choose FCN, DeepLab, UPerNet or other image segmentation models as the first human body parsing model according to the specific scenario, and no limitation is made here.

[0079] Furthermore, adding a classification head to the first human body parsing model with a segmentation head can yield an initial recognition model.

[0080] In some other embodiments of step S101 above, obtaining the initial recognition model further includes:

[0081] An initial recognition model is obtained by adding a classification head to the second human body parsing model. The second human body parsing model has the same structure as the first human body parsing model. For the sake of convenience and brevity, the specific working process and related explanations of the second human body parsing model can be referred to the content described in the above embodiment of the first human body parsing model, and will not be repeated here.

[0082] The above is a further explanation of step S101. The following is a further explanation of step S102.

[0083] In some embodiments of step S102 above, see Appendix Figure 2 , Figure 2 This is a schematic diagram illustrating the main steps of obtaining initial training model parameters according to an embodiment of the present invention. Figure 2 As shown, step S102 mainly includes the following steps S1021 to S1023.

[0084] Step S1021: Obtain the human body analysis training dataset.

[0085] The human body analysis training dataset can be obtained from open-source datasets, and there is no specific requirement here.

[0086] Step S1022: Train the first human body parsing model based on the human body parsing training dataset to obtain the parameters of the first human body parsing model.

[0087] In some implementations, if the first human body parsing model uses the FCN model, then the obtained parameters of the first human body parsing model are the convolution kernel parameters of the FCN model.

[0088] Among them, the convolution kernel parameter is a parameter form in deep learning. By using the convolution kernel parameter, features in the image can be extracted, thereby better understanding the image data.

[0089] In some implementations, if the first human body parsing model uses other image segmentation models such as DeepLab or UperNet, the parameters of the first human body parsing model obtained are the model's BN parameters, weight parameters, etc.

[0090] Among them, the BN parameters are the parameters of the batch normalization (BN) layer in the model, mainly including γ and β. γ and β are learnable parameters used to control the scaling and offset of the normalized data.

[0091] Weight parameters are the variables corresponding to each feature in the model. These variables are used to adjust the model's prediction results, and each variable has a specific value representing the importance of that feature in the model's prediction results.

[0092] It should be noted that the above examples of the first human body parsing model parameters are only illustrative. In practical applications, those skilled in the art can obtain the first human body parsing model parameters based on the specific image segmentation model used, and no limitation is made here.

[0093] Step S1023: Use the parameters of the first human body parsing model as the initial training model parameters.

[0094] After obtaining the parameters of the first human body parsing model, the parameters of the first human body parsing model can be used as the initial training model parameters to train the initial recognition model.

[0095] In some implementations, the training sample dataset obtained in step S102 is a behavior recognition dataset, which can be obtained from open source datasets and is not limited here.

[0096] The above is a further explanation of step S102. The following is a further explanation of step S103.

[0097] In some embodiments of step S103 above, the initial recognition model includes a human body parsing sub-model and an image behavior recognition sub-model. The human body parsing sub-model includes a segmentation head, and the image behavior recognition sub-model includes a classification head.

[0098] Specifically, the initial recognition model can be obtained by adding a classification head to the first human body parsing model after the first human body parsing model outputs its parameters, or it can be obtained directly by adding a classification head to the second human body parsing model. The second human body parsing model has the same structure as the first human body parsing model and both are equipped with a segmentation head.

[0099] Furthermore, in some embodiments, see Appendix Figure 3 , Figure 3 This is a schematic diagram illustrating the main steps of training an initial recognition model according to an embodiment of the present invention. Figure 3 As shown, step S103 mainly includes the following steps S1031 to S1032.

[0100] Step S1031: Train the human body parsing sub-model and the image behavior recognition sub-model based on the initial training model parameters and the training sample dataset.

[0101] In some implementations, the human body parsing sub-model and the image behavior recognition sub-model in the initial recognition model can be trained simultaneously based on the initial training model parameters and the training sample dataset.

[0102] In other implementations, the human body parsing sub-model and the image behavior recognition sub-model in the initial recognition model can be trained in a preset order based on the initial training model parameters and the training sample dataset, without limitation here.

[0103] Step S1032: When both the human body parsing sub-model and the image behavior recognition sub-model converge to the preset error value, the initial recognition model training is completed.

[0104] The preset error values ​​include preset classification error values ​​and preset segmentation error values.

[0105] Specifically, see the appendix. Figure 4 , Figure 4 This is a schematic diagram illustrating the main steps of training an initial recognition model according to another embodiment of the present invention. Figure 4 As shown, it mainly includes the following steps S401 to S403.

[0106] Step S401: Predict the scores of different categories of behavior based on the classification head and obtain the classification error value.

[0107] In some implementations, the classification head is a simple fully connected network used to predict scores for different categories of behavior.

[0108] During the training phase, different images can be pre-labeled with corresponding behavior category labels. The classification error value is calculated using the labels and scores, specifically through loss functions such as cross-entropy loss and Focal loss. During the inference phase, the behavior with the highest score is the final behavior recognition result.

[0109] Step S402: Predict the segmentation map based on the segmentation head and obtain the segmentation error value.

[0110] In some implementations, the segmentation head is a simple fully connected network used to predict the segmentation map.

[0111] During the training phase, different parts of the human body, such as the head, arms, and torso, can be pre-labeled. The segmentation error value can be calculated using the labels and the segmentation map. Specifically, the segmentation error value can be calculated using loss functions such as cross-entropy loss and Diceloss.

[0112] Step S403: When the classification error value converges to the preset classification error value and the segmentation error value converges to the preset segmentation error value, the initial recognition model training is completed.

[0113] Specifically, the image behavior recognition sub-model can be iteratively optimized using the classification error value, and the human body parsing sub-model can be iteratively optimized using the segmentation error value. When both the classification error value and the segmentation error value converge to the preset error value, the initial recognition model training is completed.

[0114] The above is a further explanation of step S103. The following is a further explanation of step S104.

[0115] In some embodiments of step S104 above, the segmentation head can be removed from the trained initial recognition model to obtain an image behavior recognition model.

[0116] The initial recognition model includes a classification head and a segmentation head. During training, the classification head can learn behavior classification, and the segmentation head can segment different parts of the human body, enabling the model to effectively focus on the overall posture information of the human body and the correlation information of different parts of the human body.

[0117] Removing the segmentation head from the trained initial recognition model can reduce the number of model parameters, resulting in an end-to-end image behavior recognition model, reducing inference time, and improving the accuracy and efficiency of image behavior recognition.

[0118] The above is a further explanation of step S104.

[0119] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0120] Furthermore, the present invention also provides an image behavior recognition model training device.

[0121] See appendix Figure 5 , Figure 5 This is a main structural block diagram of an image behavior recognition model training device according to an embodiment of the present invention. Figure 5 As shown, the image behavior recognition model training device in this embodiment of the invention mainly includes a first acquisition module 501, a second acquisition module 502, a training module 503, and a removal module 504. In some embodiments, the first acquisition module 501 can be configured to acquire an initial recognition model, wherein the initial recognition model includes a classification head and a segmentation head. The second acquisition module 502 can be configured to acquire initial training model parameters and a training sample dataset. The training module 503 can be configured to train the initial recognition model based on the initial training model parameters and the training sample dataset. The removal module 504 can be configured to remove the segmentation head from the trained initial recognition model to obtain an image behavior recognition model. In one embodiment, a description of the specific implementation function can be found in steps S101 to S104.

[0122] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.

[0123] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0124] The above-mentioned image behavior recognition model training device is used for execution Figures 1 to 4 The image behavior recognition model training method embodiments shown are similar in technical principle, technical problem solved and technical effect produced. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the image behavior recognition model training device can be referred to the content described in the embodiments of the image behavior recognition model training method, and will not be repeated here.

[0125] Furthermore, after obtaining the image behavior recognition model, the present invention also provides an image behavior recognition method.

[0126] See appendix Figure 6 , Figure 6 This is a schematic flowchart illustrating the main steps of an image behavior recognition method according to an embodiment of the present invention. Figure 6 As shown, the image behavior recognition method in this embodiment of the invention mainly includes the following steps S601 to S602.

[0127] Step S601: Obtain the image to be identified.

[0128] Step S602: Input the image to be recognized into the image behavior recognition model to obtain the behavior recognition result.

[0129] The image behavior recognition model is trained based on the image behavior recognition model training method described in the above-mentioned image behavior recognition model training method embodiment.

[0130] Furthermore, the present invention also provides an image behavior recognition device.

[0131] See appendix Figure 7 , Figure 7 This is a main structural block diagram of a behavior recognition device according to an embodiment of the present invention. Figure 7As shown, the image behavior recognition device in this embodiment of the invention mainly includes a third acquisition module 701 and a recognition module 702. In some embodiments, the third acquisition module 701 can be configured to acquire an image to be recognized. The recognition module 702 can be configured to input the image to be recognized into an image behavior recognition model to obtain a behavior recognition result, wherein the image behavior recognition model is trained based on the image behavior recognition model training method described in the above-described image behavior recognition model training method embodiment.

[0132] In one implementation, a description of the specific functions can be found in steps S601 to S602.

[0133] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.

[0134] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0135] The aforementioned image behavior recognition device is used to perform Figure 6 The image behavior recognition method embodiments shown are similar in technical principle, the technical problems they solve, and the technical effects they produce. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the image behavior recognition device can be referred to the content described in the embodiments of the image behavior recognition method, and will not be repeated here.

[0136] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0137] Furthermore, the present invention also provides an electronic device. (See appendix.) Figure 8 , Figure 8 This is a schematic diagram of the main structure of an electronic device according to an embodiment of the present invention. Figure 8 As shown, the electronic device in this embodiment of the invention mainly includes a processor 801 and a storage device 802. The storage device 802 can be configured to store a program for executing the image behavior recognition model training method or the image behavior recognition method of the above-described method embodiments. The processor 801 can be configured to execute the program in the storage device 802, which includes, but is not limited to, a program for executing the image behavior recognition model training method or the image behavior recognition method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of the present invention.

[0138] In some possible embodiments of the present invention, the electronic device may include multiple processors 801 and multiple storage devices 802. The program executing the image behavior recognition model training method or image behavior recognition method of the above method embodiments can be divided into multiple subroutines. Each subroutine can be loaded and run by a processor 801 to execute different steps of the image behavior recognition model training method or image behavior recognition method of the above method embodiments. Specifically, each subroutine can be stored in different storage devices 802, and each processor 801 can be configured to execute programs in one or more storage devices 802 to jointly implement the image behavior recognition model training method or image behavior recognition method of the above method embodiments. That is, each processor 801 executes different steps of the image behavior recognition model training method or image behavior recognition method of the above method embodiments to jointly implement the image behavior recognition model training method or image behavior recognition method of the above method embodiments.

[0139] The aforementioned multiple processors 801 can be processors deployed on the same device. For example, the aforementioned electronic device can be a high-performance device composed of multiple processors, and the aforementioned multiple processors 801 can be processors configured on that high-performance device. Alternatively, the aforementioned multiple processors 801 can also be processors deployed on different devices. For example, the aforementioned electronic device can be a server cluster, and the aforementioned multiple processors 801 can be processors on different servers within the server cluster.

[0140] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for executing the image behavior recognition model training method or image behavior recognition method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described image behavior recognition model training method or image behavior recognition method. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0141] It should be noted that the relevant user personal information involved in the various embodiments of this application is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and is personal information that users actively provide or generate during the use of the product / service, as well as personal information obtained with user authorization.

[0142] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0143] The data acquisition and collection actions involved in the embodiments of this disclosure are all performed after authorization by the user or object, or after full authorization by all parties.

[0144] The technical solution of the present invention has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for training an image behavior recognition model, characterized in that, The method includes: Obtain an initial recognition model, which includes a classification head and a segmentation head; Obtaining initial training model parameters and training sample dataset, wherein obtaining initial training model parameters includes: obtaining human body analysis training dataset; training a first human body analysis model based on the human body analysis training dataset to obtain the first human body analysis model parameters; and using the first human body analysis model parameters as the initial training model parameters. The initial recognition model is trained based on the initial training model parameters and the training sample dataset; The segmentation head is removed from the trained initial recognition model to obtain the image behavior recognition model; The initial recognition model includes a human body parsing sub-model and an image behavior recognition sub-model. The human body parsing sub-model includes the segmentation head, and the image behavior recognition sub-model includes the classification head. During the training of the initial recognition model, behavior classification is learned through the classification head, and different parts of the human body are segmented through the segmentation head, so that the initial recognition model focuses on the overall posture information of the human body and the correlation information of different parts of the human body. The training of the initial recognition model includes: training the human body parsing sub-model and the image behavior recognition sub-model based on the initial training model parameters and the training sample dataset; when both the human body parsing sub-model and the image behavior recognition sub-model converge to a preset error value, the initial recognition model training is completed, and the preset error value includes a preset classification error value and a preset segmentation error value.

2. The image behavior recognition model training method according to claim 1, characterized in that, The step of completing the initial recognition model training when both the human body parsing sub-model and the image behavior recognition sub-model converge to a preset error value includes: Based on the classification head, predict the scores of different categories of behavior and obtain the classification error value; The segmentation map is predicted based on the segmentation head, and the segmentation error value is obtained; The initial recognition model training is completed when the classification error value converges to the preset classification error value and the segmentation error value converges to the preset segmentation error value.

3. The image behavior recognition model training method according to claim 1, characterized in that, The process of obtaining the initial recognition model includes: After obtaining the parameters of the first human body parsing model, the initial recognition model is obtained by adding the classification head based on the first human body parsing model, wherein the first human body parsing model is equipped with the segmentation head; Alternatively, the initial recognition model can be obtained by adding the classification head to the second human body parsing model, wherein the second human body parsing model and the first human body parsing model have the same structure.

4. An image behavior recognition model training device, characterized in that, The device includes: The first acquisition module is configured to acquire an initial recognition model, which includes a classification head and a segmentation head; The second acquisition module is configured to acquire initial training model parameters and training sample dataset, wherein acquiring the initial training model parameters includes: acquiring a human body analysis training dataset; training a first human body analysis model based on the human body analysis training dataset to obtain the first human body analysis model parameters; and using the first human body analysis model parameters as the initial training model parameters. A training module is configured to train the initial recognition model based on the initial training model parameters and the training sample dataset; The removal module is configured to remove the segmentation head from the trained initial recognition model to obtain the image behavior recognition model. The initial recognition model includes a human body parsing sub-model and an image behavior recognition sub-model. The human body parsing sub-model includes the segmentation head, and the image behavior recognition sub-model includes the classification head. During the training of the initial recognition model, behavior classification is learned through the classification head, and different parts of the human body are segmented through the segmentation head, so that the initial recognition model focuses on the overall posture information of the human body and the correlation information of different parts of the human body. The training of the initial recognition model includes: training the human body parsing sub-model and the image behavior recognition sub-model based on the initial training model parameters and the training sample dataset; when both the human body parsing sub-model and the image behavior recognition sub-model converge to a preset error value, the initial recognition model training is completed, and the preset error value includes a preset classification error value and a preset segmentation error value.

5. An image behavior recognition method, characterized in that, The method includes: Acquire the image to be identified; The image to be identified is input into the image behavior recognition model to obtain the behavior recognition result; The image behavior recognition model is trained based on the image behavior recognition model training method according to any one of claims 1 to 3.

6. An image behavior recognition device, characterized in that, The device includes: The third acquisition module is configured to acquire the image to be recognized; The recognition module is configured to input the image to be recognized into an image behavior recognition model to obtain a behavior recognition result, wherein the image behavior recognition model is trained based on the image behavior recognition model training method according to any one of claims 1 to 3.

7. An electronic device comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the image behavior recognition model training method of any one of claims 1 to 3 or the image behavior recognition method of claim 5.

8. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the image behavior recognition model training method of any one of claims 1 to 3 or the image behavior recognition method of claim 5.

Images