Human body recognition model training method and device
By constructing a dual-stream effective attention network and a multi-stage effective attention network, combined with a residual neural network, the problem of low recognition accuracy of human body recognition models under occlusion conditions is solved, and efficient recognition of occluded images is achieved.
Patent Information
- Application Number
- CN202211644144.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-20
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2042-12-20
AI Technical Summary
Existing human body recognition models have low accuracy in occluded situations and have failed to effectively improve model structure to cope with occluded images.
A two-stream effective attention network and a multi-stage effective attention network are constructed, with a residual neural network as the backbone network, connecting the two-stream effective attention network and the multi-stage effective attention network. Features are output through the multiple stages of the residual neural network, and interactive computation and salient feature extraction are performed using the two-stream effective attention network and the multi-stage effective attention network. Training is carried out in combination with a loss function.
It improves the accuracy of human body recognition model in image recognition under occlusion conditions and enhances the model's ability to recognize occluded images.
Smart Images
Figure CN115953803B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of machine learning technology, and in particular to a training method and apparatus for a human body recognition model. Background Technology
[0002] In practical human recognition, the images to be recognized are often occluded. However, the training of human recognition models does not adequately consider this occlusion, leading to low accuracy in recognizing occluded images after training. For example, current technologies only use occluded images during training without improving the model structure based on the characteristics of occluded images, nor do they propose training methods for occluded images and improved model structures.
[0003] In realizing the present invention, the inventors discovered at least the following technical problems in the related technology: the human body recognition model trained based on traditional model training methods has low accuracy in recognizing images with occlusion. Summary of the Invention
[0004] In view of this, the present disclosure provides a training method, apparatus, electronic device, and computer-readable storage medium for a human body recognition model, in order to solve the problem in the prior art that the accuracy of human body recognition models trained based on traditional model training methods is low when recognizing images with occlusion.
[0005] A first aspect of this disclosure provides a method for training a human body recognition model, comprising: constructing a two-stream effective attention network and a multi-stage effective attention network, using a residual neural network as the backbone network, connecting the two-stream effective attention network and the multi-stage effective attention network to obtain a human body recognition model; acquiring a human body training dataset, inputting training samples from the human body training dataset into the human body recognition model, wherein the training samples are labeled with a first preset number of effective regions and a second preset number of invalid regions; outputting multiple stage features through the multiple stage networks of the residual neural network; performing two-stream interactive calculations on each stage feature using the two-stream effective attention network to obtain interactive features corresponding to each stage feature; processing each stage feature and the interactive features corresponding to that stage feature using the multi-stage effective attention network to obtain salient features corresponding to each stage feature; and training the human body recognition model using a loss function based on the salient features corresponding to each stage feature.
[0006] A second aspect of this disclosure provides a training apparatus for a human body recognition model, comprising: a construction module configured to construct a two-stream effective attention network and a multi-stage effective attention network, using a residual neural network as the backbone network, connecting the two-stream effective attention network and the multi-stage effective attention network to obtain a human body recognition model; an acquisition module configured to acquire a human body training dataset, inputting training samples from the human body training dataset into the human body recognition model, wherein the training samples are labeled with a first preset number of effective regions and a second preset number of invalid regions; a first processing module configured to output multiple stage features through the multiple stage networks of the residual neural network; a second processing module configured to perform two-stream interactive calculations on each stage feature using the two-stream effective attention network to obtain interactive features corresponding to each stage feature; a third processing module configured to process each stage feature and the interactive features corresponding to that stage feature using the multi-stage effective attention network to obtain salient features corresponding to each stage feature; and a training module configured to train the human body recognition model based on the salient features corresponding to each stage feature using a loss function.
[0007] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.
[0008] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.
[0009] The beneficial effects of this embodiment compared to the prior art are as follows: This embodiment constructs a dual-stream effective attention network and a multi-stage effective attention network, using a residual neural network as the backbone network, connecting the dual-stream effective attention network and the multi-stage effective attention network to obtain a human recognition model; it acquires a human training dataset, inputs training samples from the human training dataset into the human recognition model, wherein the training samples are labeled with a first preset number of effective regions and a second preset number of invalid regions; it outputs multiple stage features through multiple stage networks of the residual neural network; it uses the dual-stream effective attention network to perform dual-stream interactive calculations on each stage feature to obtain the interactive features corresponding to each stage feature; it uses the multi-stage effective attention network to process each stage feature and the interactive features corresponding to that stage feature to obtain the salient features corresponding to each stage feature; based on the salient features corresponding to each stage feature, it uses a loss function to complete the training of the human recognition model. Therefore, by adopting the above technical means, the problem of low accuracy in recognizing images with occlusion in the human recognition model trained based on traditional model training methods in the prior art can be solved, thereby improving the accuracy of the model in recognizing images with occlusion. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure;
[0012] Figure 2 This is a flowchart illustrating a training method for a human body recognition model provided in an embodiment of this disclosure;
[0013] Figure 3 This is a schematic diagram of the structure of a training device for a human body recognition model provided in an embodiment of this disclosure;
[0014] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0015] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.
[0016] The following will describe in detail, with reference to the accompanying drawings, a training method and apparatus for a human body recognition model according to an embodiment of the present disclosure.
[0017] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure. The application scenario may include terminal devices 101, 102, and 103, server 104, and network 105.
[0018] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays that support communication with server 104, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. Terminal devices 101, 102, and 103 can be implemented as multiple software programs or software modules, or as a single software program or software module; this disclosure does not impose any limitations on this. Furthermore, various applications can be installed on terminal devices 101, 102, and 103, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.
[0019] Server 104 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 104 can be a single server, a server cluster consisting of several servers, or a cloud computing service center. This embodiment of the disclosure does not impose any limitations on these aspects.
[0020] It should be noted that server 104 can be either hardware or software. When server 104 is hardware, it can be various electronic devices that provide various services to terminal devices 101, 102, and 103. When server 104 is software, it can be multiple software programs or software modules that provide various services to terminal devices 101, 102, and 103, or it can be a single software program or software module that provides various services to terminal devices 101, 102, and 103. This disclosure does not limit the scope of the embodiments.
[0021] Network 105 can be a wired network using coaxial cable, twisted pair, and fiber optic connection, or it can be a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), Infrared, etc. This disclosure does not limit the scope of the network.
[0022] Users can establish a communication connection with server 104 via network 105 through terminal devices 101, 102, and 103 to receive or send information, etc. It should be noted that the specific types, quantities, and combinations of terminal devices 101, 102, and 103, server 104, and network 105 can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any limitations on this.
[0023] Figure 2 This is a flowchart illustrating a training method for a human body recognition model provided in an embodiment of this disclosure. Figure 2 The training method for human body recognition models can be derived from... Figure 1 The computer or server, or the software on the computer or server, executes the command. For example... Figure 2 As shown, the training method for this human recognition model includes:
[0024] S201, construct a dual-stream effective attention network and a multi-stage effective attention network, use a residual neural network as the backbone network, connect the dual-stream effective attention network and the multi-stage effective attention network to obtain a human recognition model;
[0025] S202, Obtain the human training dataset, input the training samples in the human training dataset into the human recognition model, wherein the training samples are marked with a first preset number of valid regions and a second preset number of invalid regions.
[0026] S203 outputs multiple stage features through multiple stages of the residual neural network;
[0027] S204, using a dual-stream effective attention network to perform dual-stream interaction calculation on the features of each stage, to obtain the interaction features corresponding to each stage feature;
[0028] S205, using a multi-stage effective attention network to process the features of each stage and the corresponding interaction features of that stage feature, to obtain the salient features corresponding to each stage feature;
[0029] S206, based on the salient features corresponding to each stage, uses a loss function to complete the training of the human recognition model.
[0030] The residual neural network sequentially comprises a zero-stage network, a first-stage network, a second-stage network, a third-stage network, and a fourth-stage network. For example, a ResNet50 residual neural network includes a zero-stage network (Stage 0), a first-stage network (Stage 1), a second-stage network (Stage 2), a third-stage network (Stage 3), and a fourth-stage network (Stage 4). In this embodiment, the multiple stages of the residual neural network refer to the second-stage network, the third-stage network, and the fourth-stage network.
[0031] Using a residual neural network as the backbone network, and connecting a two-stream effective attention network and a multi-stage effective attention network, a human recognition model is obtained. This model can be achieved by sequentially connecting the two-stream effective attention network and the multi-stage effective attention network after the second-stage network, the third-stage network, and the fourth-stage network. The output of each multi-stage effective attention network is input into the next stage network of the corresponding stage network. For example, the output of the multi-stage effective attention network following the second-stage network is input into the third-stage network. Since there are no subsequent stage networks in the fourth-stage network, the output of the multi-stage effective attention network following it does not need to be input into other stage networks.
[0032] It should be noted that the human training dataset includes multiple training samples, each of which has corresponding multiple stage features, interaction features, and salient features.
[0033] A well-trained human body recognition model can be used for various tasks such as human body recognition, human body weight recognition, human body retrieval, human body comparison, human body tracking, and human body filing.
[0034] According to the technical solution provided in this disclosure, a dual-stream effective attention network and a multi-stage effective attention network are constructed. A residual neural network is used as the backbone network, connecting the dual-stream effective attention network and the multi-stage effective attention network to obtain a human recognition model. A human training dataset is obtained, and training samples from the human training dataset are input into the human recognition model. The training samples are labeled with a first preset number of effective regions and a second preset number of invalid regions. Multiple stage features are output through the multiple stage networks of the residual neural network. The dual-stream effective attention network is used to perform dual-stream interactive calculations on each stage feature to obtain the interactive features corresponding to each stage feature. The multi-stage effective attention network is used to process each stage feature and its corresponding interactive features to obtain the salient features corresponding to each stage feature. Based on the salient features corresponding to each stage feature, a loss function is used to train the human recognition model. Therefore, by adopting the above technical means, the problem of low accuracy in recognizing occluded images by human recognition models trained using traditional model training methods in the prior art can be solved, thereby improving the accuracy of the model in recognizing occluded images.
[0035] Multiple stage features are output through multiple stage networks of the residual neural network, including: outputting first stage features, second stage features, and third stage features through the second stage network, third stage network, and fourth stage network of the residual neural network, respectively; wherein the residual neural network includes a zero-stage network, a first stage network, a second stage network, a third stage network, and a fourth stage network.
[0036] A dual-stream effective attention network is used to perform dual-stream interactive computation on the features of each stage, obtaining the interactive features corresponding to each stage feature. This includes: for each stage feature: performing deformable convolution computation, batch normalization, and first activation processing sequentially to obtain the first feature corresponding to that stage feature; performing dilated convolution computation, batch normalization, and second activation processing sequentially to obtain the second feature corresponding to that stage feature; performing deformable convolution computation, batch normalization, and first activation processing sequentially on the first feature to obtain the third feature corresponding to that stage feature; and performing feature stacking and dilated convolution computation sequentially on the first and second features. The process involves several steps: first, batch normalization, and second activation, to obtain the fourth feature corresponding to this stage. Then, the third and fourth features are sequentially processed by feature stacking, deformable convolution, batch normalization, and first activation to obtain the fifth feature corresponding to this stage. Finally, the fifth feature is sequentially processed by average pooling and fully connected layers to obtain the sixth feature corresponding to this stage. The fourth feature is sequentially processed by deformable convolution, batch normalization, and first activation to obtain the seventh feature corresponding to this stage. Finally, the sixth and seventh features are sequentially processed by feature stacking, convolution, and third activation to obtain the interactive feature corresponding to this stage.
[0037] The first activation step can be performed using the ReLU activation function, the second activation step can be performed using the Mish activation function, and the third activation step can be performed using the sigmoid activation function. Feature stacking, also known as feature concatenation, involves combining two or more features together to obtain a single feature. Fully connected layer processing involves passing the feature through a fully connected network layer.
[0038] As mentioned above, two-stream interactive computing refers to processing and interacting the first and second features corresponding to each stage feature multiple times.
[0039] A multi-stage effective attention network is used to process the features of each stage and the corresponding interaction features to obtain the salient features corresponding to each stage feature. This includes: performing feature stacking, deformable convolution calculation, batch normalization and first activation processing on each stage feature and the corresponding stage feature in sequence to obtain the salient features corresponding to each stage feature.
[0040] Based on the salient features corresponding to each stage, the human body recognition model is trained using a loss function, including: training the human body recognition model to identify the effective and ineffective regions of the image using a first and second loss function based on the salient features corresponding to each stage; and training the human body recognition model to identify the target objects in the image using a third loss function based on the salient features corresponding to each stage. The training of the effective regions of the human body recognition model and the training of the target objects in the image are performed simultaneously.
[0041] Based on the salient features corresponding to each stage feature, the training of the human body recognition model for the effective and invalid regions of the image is completed using a first loss function and a second loss function. This includes: determining the first part of the features corresponding to the labeled effective regions on the training samples based on the salient features corresponding to each stage feature; determining the second part of the features corresponding to the labeled invalid regions on the training samples based on the salient features corresponding to each stage feature; training the human body recognition model for the effective regions of the image using the first loss function based on the first part of the features corresponding to each stage feature; and training the human body recognition model for the invalid regions of the image using the second loss function based on the second part of the features corresponding to each stage feature. The training of the human body recognition model for the effective regions of the image and the training of the human body recognition model for the invalid regions of the image are performed simultaneously.
[0042] In this disclosure, all features can be understood as feature maps. A salient feature is a feature map, and the first part of the feature corresponding to a salient feature is the data of multiple points on that feature map, with the number of points equal to a first preset number. The first part of the feature corresponding to each stage feature is the first part of the salient feature corresponding to that stage feature. The same applies to the second part of the feature.
[0043] In one optional embodiment, the method includes: training the human recognition model to identify effective regions of images based on the probability that the first part of the features corresponding to each stage of the human recognition model is a feature corresponding to an effective region in the training sample, using a first loss function; and training the human recognition model to identify invalid regions of images based on the probability that the second part of the features corresponding to each stage of the human recognition model is a feature corresponding to an invalid region in the training sample, using a second loss function.
[0044] The determination of the first and second parts of features corresponding to each stage is completed based on the annotation information of the training samples; the human recognition model also determines the probability that the first part of the feature corresponding to each stage is a feature corresponding to a valid region in the training samples. The same applies to the second part of features.
[0045] The first loss function, loss1, is as follows:
[0046] loss1 = -Σlog(p i )
[0047] At this time p i The probability that the first part of the feature corresponding to the i-th stage feature is the feature corresponding to the effective region in the training sample.
[0048] loss2=-Σlog(1-p i )
[0049] At this time p i The probability that the second part of the feature corresponding to the i-th stage feature is the feature corresponding to the effective region in the training sample.
[0050] The third loss function is the loss function of the residual neural network itself. This third loss function is used by the human recognition model to identify target objects in images. It should be noted that, in addition to being labeled with a first preset number of valid regions and a second preset number of invalid regions, the training samples are also labeled with the identifiers of the target objects within the training samples.
[0051] It's important to note that training the human body recognition model to identify valid and invalid regions in an image is related to training it to identify target objects within that image. Training the human body recognition model to identify target objects in an image is essentially training the model to identify target objects based on valid and invalid regions.
[0052] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.
[0053] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.
[0054] Figure 3 This is a schematic diagram of a training device for a human body recognition model provided in an embodiment of this disclosure. Figure 3 As shown, the training device for this human recognition model includes:
[0055] Module 301 is configured to construct a dual-stream effective attention network and a multi-stage effective attention network, with a residual neural network as the backbone network, connecting the dual-stream effective attention network and the multi-stage effective attention network to obtain a human recognition model;
[0056] The acquisition module 302 is configured to acquire a human training dataset and input training samples from the human training dataset into a human recognition model, wherein the training samples are labeled with a first preset number of valid regions and a second preset number of invalid regions.
[0057] The first processing module 303 is configured to output multiple stage features through multiple stage networks of the residual neural network;
[0058] The second processing module 304 is configured to use a dual-stream effective attention network to perform dual-stream interaction calculation on the features of each stage to obtain the interaction features corresponding to each stage feature.
[0059] The third processing module 305 is configured to use a multi-stage effective attention network to process the features of each stage and the interaction features corresponding to the features of that stage, so as to obtain the salient features corresponding to each stage feature.
[0060] Training module 306 is configured to train the human recognition model using a loss function based on the salient features corresponding to each stage feature.
[0061] The residual neural network sequentially comprises a zero-stage network, a first-stage network, a second-stage network, a third-stage network, and a fourth-stage network. For example, a ResNet50 residual neural network includes a zero-stage network (Stage 0), a first-stage network (Stage 1), a second-stage network (Stage 2), a third-stage network (Stage 3), and a fourth-stage network (Stage 4). In this embodiment, the multiple stages of the residual neural network refer to the second-stage network, the third-stage network, and the fourth-stage network.
[0062] Using a residual neural network as the backbone network, and connecting a two-stream effective attention network and a multi-stage effective attention network, a human recognition model is obtained. This model can be achieved by sequentially connecting the two-stream effective attention network and the multi-stage effective attention network after the second-stage network, the third-stage network, and the fourth-stage network. The output of each multi-stage effective attention network is input into the next stage network of the corresponding stage network. For example, the output of the multi-stage effective attention network following the second-stage network is input into the third-stage network. Since there are no subsequent stage networks in the fourth-stage network, the output of the multi-stage effective attention network following it does not need to be input into other stage networks.
[0063] It should be noted that the human training dataset includes multiple training samples, each of which has corresponding multiple stage features, interaction features, and salient features.
[0064] A well-trained human body recognition model can be used for various tasks such as human body recognition, human body weight recognition, human body retrieval, human body comparison, human body tracking, and human body filing.
[0065] According to the technical solution provided in this disclosure, a dual-stream effective attention network and a multi-stage effective attention network are constructed. A residual neural network is used as the backbone network, connecting the dual-stream effective attention network and the multi-stage effective attention network to obtain a human recognition model. A human training dataset is obtained, and training samples from the human training dataset are input into the human recognition model. The training samples are labeled with a first preset number of effective regions and a second preset number of invalid regions. Multiple stage features are output through the multiple stage networks of the residual neural network. The dual-stream effective attention network is used to perform dual-stream interactive calculations on each stage feature to obtain the interactive features corresponding to each stage feature. The multi-stage effective attention network is used to process each stage feature and its corresponding interactive features to obtain the salient features corresponding to each stage feature. Based on the salient features corresponding to each stage feature, a loss function is used to train the human recognition model. Therefore, by adopting the above technical means, the problem of low accuracy in recognizing occluded images by human recognition models trained using traditional model training methods in the prior art can be solved, thereby improving the accuracy of the model in recognizing occluded images.
[0066] Optionally, the first processing module 303 is further configured to output first-stage features, second-stage features, and third-stage features through the second-stage network, third-stage network, and fourth-stage network of the residual neural network, respectively; wherein the residual neural network includes a zero-stage network, a first-stage network, a second-stage network, a third-stage network, and a fourth-stage network.
[0067] Optionally, the second processing module 304 is further configured to, for each stage feature: sequentially perform deformable convolution calculation, batch normalization processing, and first activation processing on the stage feature to obtain a first feature corresponding to the stage feature; sequentially perform dilated convolution calculation, batch normalization processing, and second activation processing on the stage feature to obtain a second feature corresponding to the stage feature; sequentially perform deformable convolution calculation, batch normalization processing, and first activation processing on the first feature to obtain a third feature corresponding to the stage feature; and sequentially perform feature stacking processing, dilated convolution calculation, batch normalization processing, and second activation processing on the first feature and the second feature. The process involves several steps: first, processing the third and fourth features to obtain the fourth feature corresponding to this stage; second, performing feature stacking, deformable convolution calculation, batch normalization, and first activation on the third and fourth features sequentially to obtain the fifth feature corresponding to this stage; third, performing average pooling and fully connected layer processing on the fifth feature sequentially to obtain the sixth feature corresponding to this stage; fourth, performing deformable convolution calculation, batch normalization, and first activation on the fourth feature sequentially to obtain the seventh feature corresponding to this stage; and finally, performing feature stacking, convolution calculation, and third activation on the sixth and seventh features sequentially to obtain the interaction feature corresponding to this stage.
[0068] The first activation step can be performed using the ReLU activation function, the second activation step can be performed using the Mish activation function, and the third activation step can be performed using the sigmoid activation function. Feature stacking, also known as feature concatenation, involves combining two or more features together to obtain a single feature. Fully connected layer processing involves passing the feature through a fully connected network layer.
[0069] As mentioned above, two-stream interactive computing refers to processing and interacting the first and second features corresponding to each stage feature multiple times.
[0070] Optionally, the third processing module 305 is further configured to perform feature stacking, deformable convolution calculation, batch normalization and first activation processing on each stage feature and the stage feature in sequence to obtain the salient features corresponding to each stage feature.
[0071] Optionally, the training module 306 is further configured to train the effective and invalid regions of the human body recognition model in the image based on the salient features corresponding to each stage feature, using a first loss function and a second loss function; and to train the target object in the image based on the salient features corresponding to each stage feature, using a third loss function; wherein the training of the effective region of the image and the training of the target object in the image are performed simultaneously.
[0072] Optionally, the training module 306 is further configured to: determine a first part of the features corresponding to the marked valid regions on the training samples based on the salient features corresponding to the features at each stage; determine a second part of the features corresponding to the marked invalid regions on the training samples based on the salient features corresponding to the features at each stage; train the human recognition model to recognize the valid regions of the image using a first loss function based on the first part of the features at each stage; and train the human recognition model to recognize the invalid regions of the image using a second loss function based on the second part of the features at each stage; wherein the training of the human recognition model to recognize the valid regions of the image and the training of the human recognition model to recognize the invalid regions of the image are performed simultaneously.
[0073] In this disclosure, all features can be understood as feature maps. A salient feature is a feature map, and the first part of the feature corresponding to a salient feature is the data of multiple points on that feature map, with the number of points equal to a first preset number. The first part of the feature corresponding to each stage feature is the first part of the salient feature corresponding to that stage feature. The same applies to the second part of the feature.
[0074] Optionally, the training module 306 is further configured to train the human recognition model to identify the effective region of the image based on the probability that the first part of the feature corresponding to each stage of the human recognition model is the feature corresponding to the effective region in the training sample, using a first loss function; and to train the human recognition model to identify the invalid region of the image based on the probability that the second part of the feature corresponding to each stage of the human recognition model is the feature corresponding to the invalid region in the training sample, using a second loss function.
[0075] The determination of the first and second parts of features corresponding to each stage is completed based on the annotation information of the training samples; the human recognition model also determines the probability that the first part of the feature corresponding to each stage is a feature corresponding to a valid region in the training samples. The same applies to the second part of features.
[0076] The first loss function, loss1, is as follows:
[0077] loss1 = -Σlog(p i )
[0078] At this time p i The probability that the first part of the feature corresponding to the i-th stage feature is the feature corresponding to the effective region in the training sample.
[0079] loss2=-∑log(1-p i )
[0080] At this time p i The probability that the second part of the feature corresponding to the i-th stage feature is the feature corresponding to the effective region in the training sample.
[0081] The third loss function is the loss function of the residual neural network itself. This third loss function is used by the human recognition model to identify target objects in images. It should be noted that, in addition to being labeled with a first preset number of valid regions and a second preset number of invalid regions, the training samples are also labeled with the identifiers of the target objects within the training samples.
[0082] It's important to note that training the human body recognition model to identify valid and invalid regions in an image is related to training it to identify target objects within that image. Training the human body recognition model to identify target objects in an image is essentially training the model to identify target objects based on valid and invalid regions.
[0083] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.
[0084] Figure 4This is a schematic diagram of the electronic device 4 provided in an embodiment of this disclosure. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.
[0085] Electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or different components.
[0086] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0087] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM of the electronic device 4. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 4. The memory 402 can also include both internal and external storage units of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.
[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0089] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.
[0090] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.
Claims
1. A training method for a human body recognition model, characterized in that, include: A dual-stream effective attention network and a multi-stage effective attention network are constructed, with a residual neural network as the backbone network, connecting the dual-stream effective attention network and the multi-stage effective attention network to obtain a human recognition model; Obtain a human training dataset, and input the training samples in the human training dataset into the human recognition model, wherein the training samples are marked with a first preset number of valid regions and a second preset number of invalid regions. Multiple stage features are output through multiple stages of the residual neural network, respectively; The dual-stream effective attention network is used to perform dual-stream interaction calculation on the features of each stage to obtain the interaction features corresponding to each stage feature. The multi-stage effective attention network is used to process the features of each stage and the corresponding interaction features of that stage feature to obtain the salient features corresponding to each stage feature. Based on the salient features corresponding to each stage, the human body recognition model is trained using a loss function. The step of using the dual-stream effective attention network to perform dual-stream interaction calculations on the features of each stage to obtain the interaction features corresponding to each stage feature includes: For each stage characteristic: The first feature corresponding to the stage feature is obtained by sequentially performing deformable convolution calculation, batch normalization processing and first activation processing on the feature of the stage. The dilated convolution calculation, batch normalization, and second activation processing are performed sequentially on the feature of this stage to obtain the second feature corresponding to the feature of this stage. The first feature is sequentially subjected to the deformable convolution calculation, the batch normalization process, and the first activation process to obtain the third feature corresponding to the feature at this stage. The first feature and the second feature are sequentially subjected to feature stacking, dilated convolution calculation, batch normalization, and the second activation to obtain the fourth feature corresponding to the feature at this stage. The third feature and the fourth feature are sequentially subjected to the feature stacking process, the deformable convolution calculation, the batch normalization process, and the first activation process to obtain the fifth feature corresponding to the feature of this stage. The fifth feature is subjected to average pooling and fully connected layer processing in sequence to obtain the sixth feature corresponding to this stage feature; The fourth feature is sequentially subjected to the deformable convolution calculation, the batch normalization process, and the first activation process to obtain the seventh feature corresponding to the feature at this stage. The sixth and seventh features are sequentially subjected to feature stacking, convolution calculation, and third activation processing to obtain the interactive features corresponding to the features at this stage.
2. The method according to claim 1, characterized in that, The process of outputting multiple stage features through multiple stages of the residual neural network includes: The first-stage features, second-stage features, and third-stage features are respectively output through the second-stage network, third-stage network, and fourth-stage network of the residual neural network; The residual neural network includes a zero-stage network, a first-stage network, a second-stage network, a third-stage network, and a fourth-stage network.
3. The method according to claim 1, characterized in that, The process of using the multi-stage effective attention network to process the features of each stage and the corresponding interaction features to obtain the salient features corresponding to each stage feature includes: For each stage feature, feature stacking, deformable convolution calculation, batch normalization, and first activation are performed sequentially to obtain the salient features corresponding to each stage feature.
4. The method according to claim 1, characterized in that, The training of the human recognition model based on the salient features corresponding to each stage features, using a loss function, includes: Based on the salient features corresponding to each stage, the first loss function and the second loss function are used to train the human body recognition model to identify the effective and invalid regions of the image. Based on the salient features corresponding to each stage, the third loss function is used to train the human body recognition model to identify target objects in the image. The training of the human body recognition model to identify the effective and invalid regions of the image and the training of the human body recognition model to identify the target object in the image are performed simultaneously.
5. The method according to claim 4, characterized in that, The training of the human recognition model to identify effective and invalid regions of an image, based on the salient features corresponding to each stage feature and utilizing a first loss function and a second loss function, includes: Determine the first part of the features corresponding to the effective regions labeled on the training samples based on the salient features corresponding to the features of each stage feature; The second part of the features corresponding to the invalid regions labeled on the training samples is determined based on the salient features corresponding to the features of each stage. Based on the first part of features corresponding to each stage feature, the first loss function is used to complete the training of the effective region of the human body recognition model to recognize the image. Based on the second part of the features corresponding to each stage, the second loss function is used to complete the training of the human body recognition model to identify invalid regions of the image. The training of the human body recognition model for the effective regions of the image and the training of the human body recognition model for the invalid regions of the image are performed simultaneously.
6. The method according to claim 5, characterized in that, include: Based on the probability that the first part of the features corresponding to the features of each stage identified by the human body recognition model is the feature corresponding to the effective region in the training sample, the first loss function is used to complete the training of the effective region of the image identified by the human body recognition model. Based on the probability that the second part of the feature corresponding to each stage feature identified by the human body recognition model is the feature corresponding to the invalid region in the training sample, the second loss function is used to complete the training of the human body recognition model to identify invalid regions in the image.
7. A training device for a human body recognition model, characterized in that, include: The construction module is configured to construct a dual-stream effective attention network and a multi-stage effective attention network, with a residual neural network as the backbone network, connecting the dual-stream effective attention network and the multi-stage effective attention network to obtain a human recognition model; The acquisition module is configured to acquire a human training dataset and input training samples from the human training dataset into the human recognition model, wherein the training samples are labeled with a first preset number of valid regions and a second preset number of invalid regions. The first processing module is configured to output multiple stage features through multiple stage networks of the residual neural network, respectively. The second processing module is configured to use the dual-stream effective attention network to perform dual-stream interaction calculation on the features of each stage to obtain the interaction features corresponding to each stage feature. The third processing module is configured to use the multi-stage effective attention network to process each stage feature and the corresponding interaction feature to obtain the salient feature corresponding to each stage feature. The training module is configured to train the human recognition model using a loss function based on the salient features corresponding to each stage feature. The second processing module is specifically configured as follows: For each stage feature: deformable convolution calculation, batch normalization processing, and first activation processing are sequentially performed on the stage feature to obtain the first feature corresponding to that stage feature; dilated convolution calculation, batch normalization processing, and second activation processing are sequentially performed on the stage feature to obtain the second feature corresponding to that stage feature; deformable convolution calculation, batch normalization processing, and first activation processing are sequentially performed on the first feature to obtain the third feature corresponding to that stage feature; feature stacking processing, dilated convolution calculation, batch normalization processing, and second activation processing are sequentially performed on the first feature and the second feature to obtain the third feature. The fourth feature corresponding to the stage feature is obtained by sequentially performing the feature stacking process, the deformable convolution calculation, the batch normalization process, and the first activation process on the third feature and the fourth feature. The fifth feature is then sequentially processed by average pooling and fully connected layer processing to obtain the sixth feature corresponding to the stage feature. The fourth feature is then sequentially processed by the deformable convolution calculation, the batch normalization process, and the first activation process to obtain the seventh feature corresponding to the stage feature. The sixth feature and the seventh feature are then sequentially processed by the feature stacking process, convolution calculation, and the third activation process to obtain the interaction feature corresponding to the stage feature.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Examination abnormal behavior identification method based on time displacement and attention mechanism
CN112507920A
Pedestrian re-identification method and device
CN115147870A