A training method and device for target recognition model
By training the initial recognition model with multiple network layer models, the problem of high complexity of existing human movement recognition algorithms is solved, and more efficient calculations and more accurate recognition results are achieved.
Patent Information
- Application Number
- CN202011280520.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-16
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-11-16
AI Technical Summary
The existing deep learning algorithm for human movement recognition is based on RGB data. The excessive number of network layers leads to high algorithm complexity, large calculation volume and low calculation efficiency.
By inputting the sample image into the initial recognition model, a training probability sequence is generated, and multiple recognition models with a large number of network layers are trained to the initial recognition model with a small number of network layers until the loss value is not greater than the preset threshold, and the training is obtained The target recognition model is reduced, and the computational efficiency is improved.
The complexity of the model is reduced, the computational amount is reduced, the computational efficiency is improved, and the recognition accuracy is improved through the weighted average of multimodal images.
Smart Images

Figure CN112464760B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of deep learning technology, and in particular to a training method and device for a target recognition model. Background Art
[0002] With the rapid development of image and video capture equipment manufacturing technology, video storage and analysis technology, people can quickly and easily obtain more diverse and complex video images. Based on the image sequence recorded by the video capture device, recognizing the actions in the video has become an important branch of computer vision and pattern recognition technology.
[0003] Currently, most deep learning algorithms for human motion recognition are based on RGB data. The deep neural network used in RGB data has too many layers, resulting in high algorithm complexity, large amount of calculation, and low calculation efficiency. Summary of the invention
[0004] The purpose of the embodiment of the present application is to provide a method and device for training a target recognition model to solve the problem of complex recognition network algorithms. The specific technical solution is as follows:
[0005] In a first aspect, a method for training a target recognition model is provided, the method comprising:
[0006] Inputting the sample image into the initial recognition model to obtain a training probability sequence in the initial recognition model, wherein the training probability sequence is a sequence containing probabilities of all associated actions of the sample object in the sample image;
[0007] The initial recognition model is trained by using the sample probability sequence, the training probability sequence and the loss function until the loss value between the sample probability sequence and the training probability sequence is no greater than a preset threshold, thereby obtaining a trained target recognition model, wherein part of the sample probability sequence comes from the first recognition model, and the number of network layers of the first recognition model is not less than the number of network layers of the initial recognition model.
[0008] Optionally, after obtaining the trained target recognition model, the method further includes:
[0009] Inputting the target image into the target recognition model to obtain a target probability sequence in the target recognition model;
[0010] The weighted average result of multiple target training probabilities is taken as the final probability sequence;
[0011] The action category corresponding to the maximum probability value in the final probability sequence is selected as the target action category of the target object in the target image.
[0012] Optionally, the loss function includes a first loss function and a second loss function, the sample probability sequence includes a first probability sequence and a second probability sequence, the first probability sequence is a sequence containing correct action probabilities in a trained first recognition model, and the second probability sequence is a sequence containing label action probabilities, and the initial recognition model is trained by the sample probability sequence, the training probability sequence and the loss function until the loss value of the sample probability sequence and the training probability sequence is not greater than a preset threshold, and the trained target recognition model is obtained, including:
[0013] The initial recognition model is trained by using the first probability sequence and the training probability sequence until a first loss value between the first probability sequence and the training probability sequence is not greater than a first threshold, and the initial recognition model is trained by using the second probability sequence and the training probability sequence until a second loss value between the second probability sequence and the training probability sequence is not greater than a second threshold;
[0014] Determining a target loss function including a target loss value, wherein the target loss value is obtained by weighted summing the first loss value and the second loss value;
[0015] The trained initial recognition model is used as the target recognition model.
[0016] Optionally, before training the initial recognition model using the sample probability sequence, the training probability sequence and the loss function, the method further includes:
[0017] Inputting the sample image into a trained first recognition model to obtain a second probability sequence in the first recognition model, wherein the second probability sequence is a sequence containing probabilities of actions currently performed by the sample object in the sample image;
[0018] Obtaining a second probability sequence in all first recognition models;
[0019] The weighted average result of all the second probability sequences is used as the first probability sequence.
[0020] Optionally, before inputting the sample image into the trained first recognition model, the method further includes:
[0021] Acquire the sample image, wherein the sample image includes a depth image and a coded frame image, a motion vector image, and a residual frame image extracted from a compressed video;
[0022] The second recognition model is trained using the sample images to obtain the trained first recognition model.
[0023] Optionally, the training the second recognition model by using the sample image to obtain the trained first recognition model includes:
[0024] Inputting the sample image into the second recognition model to obtain a recognition result output by the second recognition model, wherein the recognition result is used to indicate the category to which the sample object action in the sample image belongs;
[0025] When the recognition result is inconsistent with the labeling result of the sample image, the model parameters of the second recognition model are adjusted to obtain the first recognition model, wherein the recognition result output by the first recognition model is consistent with the labeling result, and the labeling result is used to indicate the category to which the sample object action in the sample image belongs.
[0026] In a second aspect, a training device for a target recognition model is provided, the device comprising:
[0027] A first input module, used to input the sample image into the initial recognition model to obtain a training probability sequence in the initial recognition model, wherein the training probability sequence is a sequence containing probabilities of all associated actions of the sample object in the sample image;
[0028] A training module is used to train the initial recognition model through a sample probability sequence, the training probability sequence and a loss function until the loss value between the sample probability sequence and the training probability sequence is no greater than a preset threshold, thereby obtaining a trained target recognition model, wherein part of the sample probability sequence comes from a first recognition model, and the number of network layers of the first recognition model is not less than the number of network layers of the initial recognition model.
[0029] Optionally, the device further comprises:
[0030] A second input module, used for inputting a target image into the target recognition model to obtain a target probability sequence in the target recognition model;
[0031] As a module, it is used to take the weighted average result of multiple target training probabilities as the final probability sequence;
[0032] A selection module is used to select the action category corresponding to the maximum probability value in the final probability sequence as the target action category of the target object in the target image.
[0033] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0034] Memory, used to store computer programs;
[0035] The processor is used to implement any of the method steps described above when executing a program stored in the memory.
[0036] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the method steps described is implemented.
[0037] Beneficial effects of the embodiments of the present application:
[0038] The embodiment of the present application provides a method for training a target recognition model, the method comprising: a server inputs a sample image into an initial recognition model to obtain a training probability sequence in the initial recognition model, and then trains the initial recognition model through the sample probability sequence, the training probability sequence, and the loss function until the loss value of the sample probability sequence and the training probability sequence is no greater than a preset threshold, thereby obtaining a trained target recognition model, wherein part of the sample probability sequence comes from a first recognition model, and the number of network layers of the first recognition model is not less than the number of network layers of the initial recognition model. The present application trains an initial recognition model with a smaller number of network layers through a first recognition model with a larger number of network layers, thereby reducing model complexity, reducing the amount of algorithm calculation, and improving model training efficiency.
[0039] Of course, implementing any product or method of the present application does not necessarily require achieving all of the above advantages at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0041] Figure 1 A flow chart of a method for training a target recognition model provided in an embodiment of the present application;
[0042] Figure 2 A flow chart of a method for determining a target action category provided in an embodiment of the present application;
[0043] Figure 3 A schematic diagram of target recognition model training provided in an embodiment of the present application;
[0044] Figure 4 A schematic diagram of the fusion of the teacher model and the student model provided in the embodiment of the present application;
[0045] Figure 5 A flowchart of target recognition model training provided in an embodiment of the present application;
[0046] Figure 6 A schematic diagram of the structure of a device for training a target recognition model provided in an embodiment of the present application;
[0047] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] An embodiment of the present application provides a method for training a target recognition model, which can be applied to a server to train a target recognition model with a smaller number of network layers.
[0050] The following will describe in detail a method for training a target recognition model provided by an embodiment of the present application in combination with a specific implementation method. Figure 1 As shown, the specific steps are as follows:
[0051] Step 101: Input the sample image into the initial recognition model to obtain a training probability sequence in the initial recognition model.
[0052] The training probability sequence is a sequence containing the probabilities of all associated actions of the sample objects in the sample images.
[0053] The server obtains a sample image containing a sample object, wherein the sample object in the sample image is performing a certain action. The server needs to establish a target recognition model for identifying the action category of the object in the image, wherein the action category includes running, jumping, standing, raising hands, squatting, etc.
[0054] The server inputs the sample image into the initial recognition model for training to obtain the action category output by the initial recognition model. During the training process, a training probability sequence is generated in the initial recognition model. The training probability sequence contains multiple probability values, each probability value corresponds to an associated action that may occur in the sample object, and the probability value with the largest value corresponds to the action category output by the initial recognition model.
[0055] Step 102: Train the initial recognition model through the sample probability sequence, the training probability sequence and the loss function until the loss value of the sample probability sequence and the training probability sequence is no greater than a preset threshold, thereby obtaining a trained target recognition model.
[0056] Among them, part of the sample probability sequence comes from the first recognition model, and the number of network layers of the first recognition model is not less than the number of network layers of the initial recognition model.
[0057] The server obtains a sample probability sequence, which contains the probability of the correct category to which the object action in the image belongs. The sample probability sequence has two sources, one source is the first recognition model, and the other source is the label action category. The server trains the loss function of the initial recognition model through the sample probability sequence, the training probability sequence and the loss function until the loss value of the sample probability sequence and the training probability sequence is no greater than a preset threshold, thereby obtaining a trained target recognition model.
[0058] In the present application, the number of network layers of the first recognition model is not less than that of the initial recognition model. The first recognition model with a larger number of network layers is used to train the initial recognition model with a smaller number of network layers to obtain a target recognition model. The target recognition model contains the accurate probability sequence of the first recognition model. The target recognition model has fewer network layers, which can reduce the complexity of the model algorithm, reduce the amount of calculation, and improve calculation efficiency.
[0059] In addition, the probability sequence of the first recognition model has a high accuracy. Using this probability sequence as a transfer training initial recognition model is conducive to the initial recognition model to obtain the trained probability sequence more accurately and quickly. Transferring data from the first recognition model with more network layers to the training recognition model with fewer network layers is a knowledge distillation technology. The knowledge distillation in this application can use an RGB-D video action recognition algorithm.
[0060] As an optional implementation, Figure 2 As shown, after obtaining the trained target recognition model, the method further includes:
[0061] Step 201: Input the target image into the target recognition model to obtain the target probability sequence in the target recognition model.
[0062] After the server obtains the trained target recognition model, it inputs the target image into the target recognition model to obtain a target probability sequence in the target recognition model, wherein the target probability sequence includes the probability corresponding to the target action.
[0063] Step 202: taking the weighted average result of multiple target training probabilities as the final probability sequence.
[0064] The present application includes multiple target recognition models, and different target recognition models are trained by different initial recognition models. The present application may include multiple initial recognition models, and each initial recognition model corresponds to a different modality of the same image corresponding to an input sample image, which results in a different target probability sequence generated for each target recognition model.
[0065] Different modal images of the same image include coded frame image (I-frame image), motion vector image (P-frame motion vector image), residual frame image (P-frame motion vector image) and depth image (P-frame motion vector).
[0066] RGB uses compressed images and videos, which are easy to store and transmit. Images and videos can be divided into I frames (coded frames), P frames (predicted frames) and zero or B frames (bidirectional frames) by codecs. The server extracts I frame images, motion vector images and residual frame images in P frames from compressed videos, and extracts depth images from depth data.
[0067] The server calculates the weighted average result of multiple target probability sequences, and then uses the weighted average result as the final probability sequence. In this way, the final probability sequence integrates the probabilities of images of different modalities, making the final probability sequence very accurate.
[0068] Step 203: Select the action category corresponding to the maximum probability value in the final probability sequence as the target action category of the target object in the target image.
[0069] The final probability sequence includes multiple probability values, the maximum probability value corresponds to the target action category of the target object in the target image, and the server selects the maximum probability value as the target action category of the target object.
[0070] In the present application, in the process of determining the target action category of the target object, the target probability sequences of multiple target recognition models are fused to obtain the final probability sequence, that is, multiple target probability sequences are obtained using multimodal images. Multimodal images can make the action category corresponding to the final probability sequence more accurate.
[0071] As an optional implementation, the loss function includes a first loss function and a second loss function, the sample probability sequence includes a first probability sequence and a second probability sequence, the first probability sequence is a sequence containing the probability of correct actions in the trained first recognition model, and the second probability sequence is a sequence containing the probability of label actions. The initial recognition model is trained by the sample probability sequence, the training probability sequence and the loss function until the loss value of the sample probability sequence and the training probability sequence is not greater than a preset threshold, and the trained target recognition model includes:
[0072] The initial recognition model is trained by the first probability sequence and the training probability sequence until the first loss value of the first probability sequence and the training probability sequence is not greater than the first threshold, and the initial recognition model is trained by the second probability sequence and the training probability sequence until the second loss value of the second probability sequence and the training probability sequence is not greater than the second threshold; a target loss function including a target loss value is determined, wherein the target loss value is obtained by the weighted sum of the first loss value and the second loss value; and the trained initial recognition model is used as the target recognition model.
[0073] The sample probability sequence includes a first probability sequence and a second probability sequence. The first probability sequence is obtained through the trained first recognition model. The sample image of the first recognition model is the same as the sample image of the initial recognition model, which makes it easier to train the initial recognition model through the first recognition model, making the training result more accurate. The first probability sequence contains the probability corresponding to the correct action category of the sample object in the sample image, and the probability corresponding to the correct action category has the largest value. The second probability sequence contains the probability corresponding to the label action category in the sample image, and the probability has the largest value in the second probability sequence.
[0074] The server trains the first loss function of the initial recognition model through the training probability sequence in the initial recognition model and the first probability sequence in the first recognition model until the first loss value of the first probability sequence and the training probability sequence is no greater than the first threshold, in which case the training of the first loss function is completed.
[0075] The server trains the second loss function of the initial recognition model through the second probability sequence and the training probability sequence in the initial recognition model until the second loss value of the second probability sequence and the training probability sequence is no greater than the second threshold, in which case the training of the second loss function is completed.
[0076] After the first loss function and the second loss function are trained, the server performs a weighted summation of the first loss value and the second loss value to obtain a target loss value, and the server uses the trained initial recognition model as the target recognition model.
[0077] In the present application, the first probability sequence is equivalent to the soft probability of the first recognition model, the training probability sequence is equivalent to the soft probability of the initial recognition model, the first loss function is equivalent to the first cross entropy, and the server generates the first cross entropy through the soft probability of the first recognition model and the soft probability of the initial recognition model; the second probability sequence is equivalent to the soft probability of the label action category, the second loss function is equivalent to the second cross entropy, and the server generates the second cross entropy through the soft probability of the label action category and the soft probability of the initial recognition model.
[0078] In the present application, during the training process of the target recognition model, the first probability sequence of the trained first recognition model and the second probability sequence corresponding to the label action category are integrated, and the probability training loss function corresponding to the correct action of the sample object is used to make the obtained loss function more accurate and the output result of the target recognition model more accurate.
[0079] As an optional implementation, before training the initial recognition model through the sample probability sequence, the training probability sequence and the loss function, the method also includes: inputting the sample image into the trained first recognition model to obtain a second probability sequence in the first recognition model, wherein the second probability sequence is a sequence of probabilities of the actions currently performed by the sample object in the sample image; obtaining the second probability sequences in all first recognition models; and taking the weighted average result of all second probability sequences as the first probability sequence.
[0080] Before the server trains the initial recognition model through the first probability sequence, it needs to obtain the first probability sequence of the first recognition model. The server obtains the sample image, and then inputs the sample image into the trained first recognition model to obtain the second probability sequence in the first recognition model. Since the first recognition model is similar to the initial recognition model in the training process, it is also trained by using sample images of different modalities. The second probability sequences obtained by different first recognition models are different. In order to integrate the various image modalities, the server calculates the weighted average result of all second probability sequences and uses the weighted average result as the first probability sequence.
[0081] In the present application, the service fuses multiple second probability sequences. Since different second probability sequences are obtained from sample images of different modalities, the fusion of second probability sequences can fuse different modalities of the image, compensate for the unstable characteristics of lighting, facial features or image texture, and improve recognition accuracy.
[0082] As an optional implementation, before inputting the sample image into the trained first recognition model, the method also includes: obtaining a sample image, wherein the sample image includes a depth image and a coded frame image, a motion vector image, and a residual frame image extracted from a compressed video; training the second recognition model through the sample image to obtain a trained first recognition model.
[0083] The sample images used for training the first recognition model are the same as those of the initial recognition model. The sample images include depth images obtained based on depth data, and coded frame images, motion vector images, and residual frame images extracted from compressed videos. After the server obtains the sample images, it trains the second recognition model through the sample images to obtain the trained first recognition model.
[0084] As an optional implementation, the second recognition model is trained by using sample images to obtain the trained first recognition model, including:
[0085] The server obtains a sample image and a labeling result of the sample image, wherein the labeling result is used to indicate the category to which the sample object action in the sample image belongs; the server inputs the sample image into a second recognition model to obtain a recognition result output by the second recognition model, wherein the recognition result is used to indicate the category to which the sample object action in the sample image belongs; when the labeling result is inconsistent with the recognition result, the model parameters of the second recognition model are adjusted until the recognition result output by the first recognition model is consistent with the labeling result, thereby obtaining the first recognition model.
[0086] Figure 3 The figure is a schematic diagram of the target recognition model training. The teacher model is the first recognition model with more network layers, and the student model is the initial training model with fewer network layers. The I-frame image, P-frame motion vector, P-frame residual and Depth image are used as input images to train the teacher model. As can be seen from the figure, the four input images of the teacher model are different modes of the same image. The network layers of the teacher model corresponding to the I-frame image and the Depth image are 152 layers, and the network layers of the teacher model corresponding to the P-frame motion vector and the P-frame residual are 18 layers. The I-frame image is used as the input image to train the student model. The network layers of any student model are 18 layers, which reduces the model complexity, reduces the number of network parameters, and improves the computational efficiency.
[0087] Figure 4 This is a schematic diagram of the fusion of the teacher model and the student model. It can be seen that the teacher model and the student model are in a many-to-many relationship. Each teacher model forms multiple cross entropies with its own soft probability and the soft probabilities of multiple student models. Each student model is trained by the soft probabilities of multiple teacher models. The teacher model and the student model use the same sample images.
[0088] Figure 5This is a flowchart for training the target recognition model. The four teacher models use different inputs to obtain four Gsoftmax (second probability sequence), and the four Gsoftmax are added and averaged to obtain the Soft Label (first probability sequence). The student model uses ifame as the sample image to obtain two identical Gsoftmaxes, and then one of the Gsoftmax and the Soft Label generates LossL1 (first loss value), and the other Gsoftmax and the Hard Label generate LossL2 (second loss value). LossL1 and LossL2 are added and calculated to obtain Total Loss (target loss value). The student model uses other input images (such as P-frame motion vector, P-frame motion vector or P-frame motionvector) as sample images, and the training process is the same.
[0089] Based on the same technical concept, the present application also provides a training device for a target recognition model, such as Figure 6 As shown, the device comprises:
[0090] The first input module 601 is used to input the sample image into the initial recognition model to obtain a training probability sequence in the initial recognition model, wherein the training probability sequence is a sequence containing the probabilities of all associated actions of the sample object in the sample image;
[0091] The first training module 602 is used to train the initial recognition model through sample probability sequences, training probability sequences and loss functions until the loss value between the sample probability sequence and the training probability sequence is no greater than a preset threshold, thereby obtaining a trained target recognition model, wherein part of the sample probability sequence comes from the first recognition model, and the number of network layers of the first recognition model is not less than the number of network layers of the initial recognition model.
[0092] Optionally, the device further comprises:
[0093] The second input module is used to input the target image into the target recognition model to obtain the target probability sequence in the target recognition model;
[0094] The first is a module for taking the weighted average result of multiple target training probabilities as the final probability sequence;
[0095] The selection module is used to select the action category corresponding to the maximum probability value in the final probability sequence as the target action category of the target object in the target image.
[0096] Optionally, the loss function includes a first loss function and a second loss function, the sample probability sequence includes a first probability sequence and a second probability sequence, the first probability sequence is a sequence containing correct action probabilities in the trained first recognition model, and the second probability sequence is a sequence containing label action probabilities, and the first training module 602 includes:
[0097] A training unit, used to train the initial recognition model through the first probability sequence and the training probability sequence until a first loss value between the first probability sequence and the training probability sequence is not greater than a first threshold, and to train the initial recognition model through the second probability sequence and the training probability sequence until a second loss value between the second probability sequence and the training probability sequence is not greater than a second threshold;
[0098] A determination unit, configured to determine a target loss function including a target loss value, wherein the target loss value is obtained by weighted summing of the first loss value and the second loss value;
[0099] As a unit, it is used to use the trained initial recognition model as the target recognition model.
[0100] Optionally, the device further comprises:
[0101] A third input module is used to input the sample image into the trained first recognition model to obtain a second probability sequence in the first recognition model, wherein the second probability sequence is a sequence containing probabilities of actions currently performed by the sample object in the sample image;
[0102] A first acquisition module, used to acquire the second probability sequence in all first recognition models;
[0103] The second module is used to take the weighted average result of all the second probability sequences as the first probability sequence.
[0104] Optionally, the device further comprises:
[0105] A second acquisition module is used to acquire a sample image, wherein the sample image includes a depth image and a coded frame image, a motion vector image and a residual frame image extracted from a compressed video;
[0106] The second training module is used to train the second recognition model through sample images to obtain a trained first recognition model.
[0107] Optionally, the second training module includes:
[0108] An input unit, used to input the sample image into the second recognition model to obtain a recognition result output by the second recognition model, wherein the recognition result is used to indicate the category to which the sample object action in the sample image belongs;
[0109] An adjustment unit is used to adjust the model parameters of the second recognition model to obtain the first recognition model when the recognition result is inconsistent with the annotation result of the sample image, wherein the recognition result output by the first recognition model is consistent with the annotation result, and the annotation result is used to indicate the category to which the sample object action in the sample image belongs.
[0110] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.
[0111] Memory 703, used for storing computer programs;
[0112] The processor 701 is used to implement the above steps when executing the program stored in the memory 703.
[0113] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0114] The communication interface is used for communication between the above electronic device and other devices.
[0115] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0116] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0117] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.
[0118] In another embodiment of the present invention, a computer program product including instructions is provided, which enables the computer to execute any one of the methods in the above embodiments when the computer program product is executed on the computer.
[0119] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0120] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0121] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A method for training a target recognition model, characterized in that: The method comprises: Inputting the sample image into the initial recognition model to obtain a training probability sequence in the initial recognition model, wherein the training probability sequence is a sequence containing probabilities of all associated actions of the sample object in the sample image; The initial recognition model is trained by using the sample probability sequence, the training probability sequence and the loss function until the loss value between the sample probability sequence and the training probability sequence is not greater than a preset threshold, thereby obtaining a trained target recognition model, wherein part of the sample probability sequence comes from the first recognition model, and the number of network layers of the first recognition model is not less than the number of network layers of the initial recognition model; Among them, the loss function includes a first loss function and a second loss function, the sample probability sequence includes a first probability sequence and a second probability sequence, the first probability sequence is a sequence containing the probability of correct actions in the trained first recognition model, and the second probability sequence is a sequence containing the probability of label actions, and the initial recognition model is trained by the sample probability sequence, the training probability sequence and the loss function until the loss value of the sample probability sequence and the training probability sequence is not greater than a preset threshold, and the trained target recognition model includes: The initial recognition model is trained by using the first probability sequence and the training probability sequence until a first loss value between the first probability sequence and the training probability sequence is not greater than a first threshold, and the initial recognition model is trained by using the second probability sequence and the training probability sequence until a second loss value between the second probability sequence and the training probability sequence is not greater than a second threshold; Determining a target loss function including a target loss value, wherein the target loss value is obtained by weighted summing the first loss value and the second loss value; The trained initial recognition model is used as the target recognition model.
2. The method according to claim 1, characterized in that After obtaining the trained target recognition model, the method further includes: Inputting the target image into the target recognition model to obtain a target probability sequence in the target recognition model; The weighted average result of multiple target training probabilities is taken as the final probability sequence; The action category corresponding to the maximum probability value in the final probability sequence is selected as the target action category of the target object in the target image.
3. The method according to claim 1, characterized in that Before training the initial recognition model using the sample probability sequence, the training probability sequence and the loss function, the method further includes: Inputting the sample image into a trained first recognition model to obtain a second probability sequence in the first recognition model, wherein the second probability sequence is a sequence containing probabilities of actions currently performed by the sample object in the sample image; Obtaining a second probability sequence in all first recognition models; The weighted average result of all the second probability sequences is used as the first probability sequence.
4. The method according to claim 3, characterized in that Before inputting the sample image into the trained first recognition model, the method further includes: Acquire the sample image, wherein the sample image includes a depth image and a coded frame image, a motion vector image, and a residual frame image extracted from a compressed video; The second recognition model is trained using the sample images to obtain the trained first recognition model.
5. The method according to claim 4, characterized in that The training of the second recognition model by the sample image to obtain the trained first recognition model comprises: Inputting the sample image into the second recognition model to obtain a recognition result output by the second recognition model, wherein the recognition result is used to indicate the category to which the sample object action in the sample image belongs; When the recognition result is inconsistent with the labeling result of the sample image, the model parameters of the second recognition model are adjusted to obtain the first recognition model, wherein the recognition result output by the first recognition model is consistent with the labeling result, and the labeling result is used to indicate the category to which the sample object action in the sample image belongs.
6. A training device for a target recognition model, characterized in that: The device comprises: A first input module, used to input the sample image into the initial recognition model to obtain a training probability sequence in the initial recognition model, wherein the training probability sequence is a sequence containing probabilities of all associated actions of the sample object in the sample image; A training module, used to train the initial recognition model through a sample probability sequence, the training probability sequence and a loss function, until the loss value between the sample probability sequence and the training probability sequence is not greater than a preset threshold, thereby obtaining a trained target recognition model, wherein part of the sample probability sequence comes from a first recognition model, and the number of network layers of the first recognition model is not less than the number of network layers of the initial recognition model; The loss function includes a first loss function and a second loss function, the sample probability sequence includes a first probability sequence and a second probability sequence, the first probability sequence is a sequence containing the probability of correct actions in the trained first recognition model, and the second probability sequence is a sequence containing the probability of label actions, and the training module is used for: The initial recognition model is trained by using the first probability sequence and the training probability sequence until a first loss value between the first probability sequence and the training probability sequence is not greater than a first threshold, and the initial recognition model is trained by using the second probability sequence and the training probability sequence until a second loss value between the second probability sequence and the training probability sequence is not greater than a second threshold; Determining a target loss function including a target loss value, wherein the target loss value is obtained by weighted summing the first loss value and the second loss value; The trained initial recognition model is used as the target recognition model.
7. The device according to claim 6, characterized in that The device also includes: A second input module, used for inputting a target image into the target recognition model to obtain a target probability sequence in the target recognition model; As a module, it is used to take the weighted average result of multiple target training probabilities as the final probability sequence; A selection module is used to select the action category corresponding to the maximum probability value in the final probability sequence as the target action category of the target object in the target image.
8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 5 when executing a program stored in a memory.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.
