A training method, device, electronic device and storage medium for recognition model

Abstract

The present application relates to the field of image recognition technology, and in particular to a method, device, electronic device and storage medium for training a recognition model, which are used to improve the accuracy of wearing recognition of a target object. The method includes: performing cyclic iterative training on the recognition model in training according to the target training sample set to obtain a trained recognition model for determining whether the target object is wearing the target object; in one training, extracting features of the target sample image in the target training sample to obtain at least two sample object features, and obtaining the predicted wearing category of the target sample image and the predicted scene category of each sample object feature based on the sample object features; adjusting the parameters of the recognition model based on the difference between the predicted wearing category and the labeled wearing category, and the difference between each predicted scene category and the labeled scene category. Since the present application determines whether the target object is wearing the target object through the trained recognition model, the recognition accuracy is improved.

Description

Background Art

[0002] With the popularization of intelligent technology, the demand for compliance of human wear in various industries has gradually increased. Take the wearing of safety helmets as an example. At the construction site, in order to ensure the personal safety of workers and prevent workers from being injured by falling objects, they are required to wear safety helmets in strict accordance with regulations. By standardizing the clothing and wearing behavior of relevant personnel, production and operation accidents caused by illegal wearing can be effectively reduced.

[0003] In the related art, the following methods are mainly used to detect whether a person in a video is wearing a target object: a temporal hidden network is introduced, the convolutional features of the current frame video image are fused with the convolutional features of the previous frame video image, and the current frame video image is detected based on the fused convolutional features to determine whether the human body is wearing the target object. However, the above method introduces a temporal hidden network, resulting in a slow detection speed, and due to the cluttered background of the work site, the above method has poor generalization when deployed in different work sites, and thus the recognition accuracy of whether the human body is wearing the target object is low. Therefore, how to improve the accuracy of identifying whether the target object is wearing the target object is a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device and storage medium for training a recognition model to improve the accuracy of wearable recognition of a target object.

[0005] The present invention provides a method for training a recognition model, including:

[0006] According to the target training sample set, a recognition model in training is subjected to cyclic iterative training to obtain a trained recognition model; the trained recognition model is used to identify whether a target object in an image to be identified wears a target object; each target training sample includes: a target sample image, and a labeled wearing category representing whether a sample object in the target sample image wears a target object, and a labeled scene category representing a sampling scene of the target sample image; wherein the following operations are performed in one cyclic iterative training:

[0007] Using the recognition model in the training, multi-dimensional feature extraction is performed on the target sample image in the selected target training sample to obtain sample object features of at least two different dimensions, and based on the sample object features of at least two different dimensions, a predicted wearing category corresponding to the target sample image and a predicted scene category corresponding to each sample object feature are obtained;

[0008] Based on the difference between the predicted wearing category and the annotated wearing category, and the difference between each predicted scene category and the annotated scene category, parameters of the recognition model in training are adjusted.

[0009] An embodiment of the present application provides a training device for a recognition model, comprising:

[0010] The training unit is used to perform cyclic iterative training on the recognition model in training according to the target training sample set to obtain a trained recognition model; each target training sample includes: a target sample image, and a labeled wearing category representing whether the sample object in the target sample image wears the target object, and a labeled scene category representing the sampling scene of the target sample image; the training unit performs the following operations in one cyclic iterative training:

[0011] Using the recognition model in the training, multi-dimensional feature extraction is performed on the target sample image in the selected target training sample to obtain sample object features of at least two different dimensions, and based on the sample object features of at least two different dimensions, a predicted wearing category corresponding to the target sample image and a predicted scene category corresponding to each sample object feature are obtained;

[0012] Based on the difference between the predicted wearing category and the annotated wearing category, and the difference between each predicted scene category and the annotated scene category, parameters of the recognition model in training are adjusted.

[0013] Optionally, the sample object feature includes a first sample object feature, a second sample object feature and a third sample object feature;

[0014] The training unit is specifically used for:

[0015] Performing global and local feature extraction on the target sample image to obtain sample image features;

[0016] Performing feature mapping on the sample image features to obtain first sample object features of the image dimension;

[0017] Performing feature mapping on the first sample object features to obtain second sample object features of instance dimension;

[0018] Perform wear feature extraction on the second sample object feature to obtain a third sample object feature in pixel dimension.

[0019] Optionally, the training unit is specifically used for:

[0020] Perform target detection on the mapped features to obtain the predicted boundary information of the detected target;

[0021] Based on the predicted boundary information, mapping the to-be-mapped features to the sample object features;

[0022] Among them, if the feature to be mapped is the sample image feature, the detection target is the sample object, and the sample object feature is the first sample object feature; if the feature to be mapped is the first sample object feature, the detection target is the target part of the sample object related to the target object being worn, and the sample object feature is the second sample object feature.

[0023] Optionally, the recognition model includes a domain classification network; and the training unit is specifically used for:

[0024] The features of each sample object are respectively input into the corresponding domain classification network to obtain the predicted scene category corresponding to each sample object feature.

[0025] Optionally, the recognition model includes a wear classification network; and the training unit is specifically used for:

[0026] The third sample object feature is input into the wear classification network to obtain the predicted wear category corresponding to the target sample image.

[0027] Optionally, each target training sample further includes: annotated boundary information of the sample object in the target sample image and annotated boundary information of the target part; performing target detection on the features to be mapped to obtain predicted boundary information of the detected target, and the training unit is specifically used for:

[0028] Based on at least one positive sample unit in the feature to be mapped, predicting a boundary of the detection target to obtain at least one candidate boundary information of the detection target;

[0029] Based on the confidence of the at least one candidate boundary information, the predicted boundary information of the detection target is determined, and the confidence of the at least one candidate boundary information is determined based on the intersection-and-union ratio between each candidate boundary information and the corresponding labeled boundary information.

[0030] Optionally, the predicted boundary information includes: predicted center position coordinates of the detection target and predicted object frame size; the training unit is specifically used for:

[0031] For each positive sample unit, perform the following operations:

[0032] For a positive sample unit, determining the center position information of the detection target and the position offset information corresponding to the center position information;

[0033] Adjusting the center position information based on the position offset information to obtain the predicted center position coordinates of the detection target;

[0034] Based on the predicted center position coordinates, the predicted object frame size of the detection target is obtained.

[0035] Optionally, the training unit is specifically used for:

[0036] Based on the marked boundary information, determining at least one candidate sample unit;

[0037] Determine the loss value of each candidate sample unit based on the difference between the candidate boundary information of each candidate sample unit and the corresponding marked boundary information;

[0038] The loss values ​​are sorted according to their sizes, and the candidate sample units corresponding to the loss values ​​within the first preset order range of the sorting results are taken as the positive sample units.

[0039] Optionally, the training unit is specifically used for:

[0040] Determine the intersection-and-union ratio between the predicted boundary information and the labeled boundary information of each candidate sample unit respectively;

[0041] The intersection-and-union ratios are sorted according to their sizes, and the first preset order range is determined based on the intersection-and-union ratios of the sorted results within the second preset order range.

[0042] Optionally, the training unit is specifically used for:

[0043] Acquire an initial training sample set, wherein each initial training sample in the initial training sample set includes an initial sample image, and the sample images in the initial training sample set are obtained by sampling different sampling scenes;

[0044] Based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, the initial training sample set is expanded to obtain the target training sample set.

[0045] Optionally, the training unit is further used for:

[0046] Cutting and pasting each initial sample image in the initial training sample set to obtain an intermediate training sample set;

[0047] The method of expanding the initial training sample set based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method to obtain the target training sample set includes:

[0048] Based on the at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, data enhancement is performed on the intermediate sample images in the intermediate training sample set to obtain the target training sample set.

[0049] Optionally, the training unit is specifically used for:

[0050] An initial enhancement parameter set is obtained, and the initial enhancement parameter set is iteratively updated to obtain each target enhancement parameter, wherein the initial enhancement parameter set includes initial enhancement parameters corresponding to each data enhancement mode, and the following operations are performed in one iterative update process:

[0051] Screening out initial enhancement parameters that meet preset conditions, and for the initial enhancement parameters that meet the conditions, randomly exchanging values ​​of at least two initial enhancement parameters to obtain each of the exchanged initial enhancement parameters;

[0052] Randomly mutating the values ​​of the exchanged initial enhancement parameters to obtain the mutated initial enhancement parameters, and updating the number of iterations;

[0053] If the updated number of iterations meets the iteration stop condition, the mutated initial enhancement parameters are used as the target enhancement parameters.

[0054] Optionally, the device further includes a determining unit, configured to:

[0055] Based on the trained recognition model, feature extraction is performed on the image to be recognized containing the target object to obtain an image feature map;

[0056] Performing target detection based on the image feature map to obtain a foreground feature map containing the target object;

[0057] Performing target detection based on the foreground feature map to obtain a part feature map including a target part of the target object that is related to the target object being worn;

[0058] Based on the part feature map, it is determined whether the target subject is wearing a target object.

[0059] An embodiment of the present application provides a target object recognition device, comprising:

[0060] An extraction unit, used for extracting global and local features of an image to be identified containing a target object based on a trained recognition model to obtain an image feature map;

[0061] A first detection unit, configured to perform object detection based on the image feature map to obtain a foreground feature map containing the target object;

[0062] A second detection unit is used to detect a target part based on the foreground feature map to obtain a part feature map including the target part, where the target part is a part of the target object that is related to the target object being worn;

[0063] A determination unit is used to determine whether the target subject wears the target object based on the part feature map.

[0064] An electronic device provided in an embodiment of the present application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the above-mentioned recognition model training methods.

[0065] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the above-mentioned recognition model training methods.

[0066] An embodiment of the present application provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of any one of the above-mentioned recognition model training methods.

[0067] The beneficial effects of this application are as follows:

[0068] The training method, device, electronic device and storage medium of the recognition model provided by the embodiments of the present application can be used to determine whether the target object in the image to be recognized is wearing the target object by performing cyclic iterative training on the recognition model in training according to the target training sample set; wherein, the following operations are performed in one cyclic iterative training: multi-dimensional feature extraction is performed on the target sample image in the selected target training sample using the recognition model in training to obtain sample object features of at least two different dimensions, and the obtained sample object features are representative through the hierarchical transmission of features of different dimensions; based on the sample object features of at least two different dimensions, the predicted wearing category corresponding to the target sample image and the predicted scene category corresponding to each sample object feature are obtained; based on the difference between the predicted wearing category and the labeled wearing category, and the difference between each predicted scene category and the labeled scene category, the parameters of the recognition model in training are adjusted, thereby effectively improving the domain adaptation capability and generalization of the recognition model, and being able to adapt to the target object wearing recognition of the target object in the image to be recognized in different scenes, thereby improving the recognition accuracy.

[0069] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0071] Figure 1 An optional schematic diagram of an application scenario in an embodiment of the present application;

[0072] Figure 2 This is a flowchart of an implementation of a method for training a recognition model in an embodiment of the present application;

[0073] Figure 3A is a flowchart of an implementation method of a target object recognition method in an embodiment of the present application;

[0074] Figure 3B A schematic diagram of the structure of a three-stage domain adaptive helmet detection network in an embodiment of the present application;

[0075] Figure 4 A detection flow diagram of a target detection method in an embodiment of the present application;

[0076] Figure 5 is a schematic diagram of a candidate sample unit in an embodiment of the present application;

[0077] Figure 6 A schematic diagram of iterative update of a data enhancement parameter set in an embodiment of the present application;

[0078] Figure 7 A flowchart of a data enhancement strategy in an embodiment of the present application is shown;

[0079] Figure 8 A schematic diagram of a flow chart of a genetic evolution algorithm in an embodiment of the present application;

[0080] Fig. 9 A schematic diagram of a process of detecting a helmet in an embodiment of the present application;

[0081] Fig.10 A structural diagram of a training device for a recognition model in an embodiment of the present application;

[0082] Fig.11 A schematic diagram of the structure of a target object recognition device in an embodiment of the present application;

[0083] Fig.12 A schematic diagram of a hardware structure of an electronic device to which an embodiment of the present application is applied;

[0084] Fig.13 The present invention is a schematic diagram of the hardware structure of another electronic device to which the embodiments of the present application are applied. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0086] The following is an introduction to some concepts involved in the embodiments of the present application.

[0087] Target training sample set: It is obtained after data enhancement of the initial training sample set and is used to train the recognition model in training. According to the different sampling scenes of the target sample images in the target training sample set, the target training sample set can be divided into the source domain dataset and the target domain dataset.

[0088] Domain adaptation: In machine learning, when the data distribution in the source domain dataset is different from that in the target domain dataset, but the tasks of the two are the same, this special transfer learning is domain adaptation. In the embodiment of the present application, based on the idea of ​​domain adaptation, the recognition model is trained iteratively using the target training sample set. The recognition model can recognize whether the target object in the to-be-recognized image of different sampling scenes is wearing the target object, and has strong generalization ability and high recognition accuracy.

[0089] Sample image features: refers to the features obtained by extracting global and local features of the sample image. The sample image features do not distinguish between the foreground and background in the image, and what is obtained is the feature representation of the entire image.

[0090] Sample object feature: refers to a feature obtained by further extracting features based on sample image features, wherein the first sample object feature refers to the feature of the sample object in the sample image, the second sample object feature refers to the feature of the target part of the sample object, and the third sample object feature refers to the feature obtained by extracting the second sample object feature.

[0091] Genetic evolutionary algorithm: It is a computational model of biological evolution process that simulates the natural selection and genetic mechanism of Darwin's theory of biological evolution. It is a method of searching for the optimal solution by simulating the natural evolution process. All individuals in a population are taken as objects, and randomization technology is used to guide the efficient search of an encoded parameter space. Among them, selection, crossover and mutation constitute the basic operations of the genetic evolutionary algorithm. In the embodiment of the present application, the initial enhancement parameter set is iteratively updated by the genetic evolutionary algorithm, the best enhancement parameter subset is searched, and the enhancement parameters in the best enhancement parameter subset are used as the target enhancement parameters.

[0092] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0093] like Figure 1 As shown, it is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario diagram includes two terminal devices 110 and a server 120.

[0094] In the embodiment of the present application, the terminal device 110 includes but is not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals and other devices; the terminal device may be installed with a client related to target object wear recognition, which may be software (such as a browser, video software, etc.), or a web page, a small program, etc. The server 120 is a background server corresponding to the software or web page, small program, etc., or a server specifically used for target object wear recognition, which is not specifically limited in the present application. The server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0095] It should be noted that the training method of the recognition model in the embodiment of the present application can be executed by an electronic device, and the electronic device can be a server 120 or a terminal device 110, that is, the method can be executed by the server 120 or the terminal device 110 alone, or can be executed by the server 120 and the terminal device 110 together. For example, when the method is executed by the server 120 and the terminal device 110 together, the server 120 performs cyclic iterative training on the recognition model in training according to the target training sample set, obtains the trained recognition model, and sends the trained recognition model to the terminal device 1110, so that the terminal device 110 can identify whether the target object in the image to be identified is wearing the target object through the recognition model.

[0096] In an optional implementation, the terminal device 110 and the server 120 may communicate with each other via a communication network.

[0097] In an optional implementation, the communication network is a wired network or a wireless network.

[0098] It should be noted that Figure 1 The figure is only an example. In fact, the number of terminal devices and servers is not limited and is not specifically limited in the embodiments of the present application.

[0099] In an embodiment of the present application, when there are multiple servers, the multiple servers can form a blockchain, and the servers are nodes on the blockchain; as in the training method of the recognition model disclosed in the embodiment of the present application, the target training sample data set involved can be saved on the blockchain.

[0100] In addition, the embodiments of the present application can be applied to various scenarios, including not only image recognition scenarios, but also including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving and other scenarios.

[0101] The following describes the training method of the recognition model provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation methods of the present application are not limited in this regard.

[0102] See also Figure 2 As shown, it is an implementation flow chart of a method for training a recognition model provided in an embodiment of the present application. Taking the execution subject as a server as an example, the specific implementation process of the method includes the following steps S21-S22:

[0103] S21: the server performs cyclic iterative training on the recognition model being trained according to the target training sample set to obtain a trained recognition model;

[0104] Among them, the trained recognition model is used to identify whether the target object in the image to be identified is wearing the target object. Specifically, the target object can be a pair of gloves. For example, the power industry requires the wearing of insulating gloves during operation, and the firefighting industry requires the wearing of firefighting gloves during operation. The target object can also be a safety helmet. At the construction site, in order to ensure the personal safety of the workers and prevent falling objects from injuring the workers, the work site is required to wear a safety helmet in strict accordance with the regulations. The target object can also be a helmet. In order to reduce the losses caused by traffic accidents, it is stipulated that electric vehicle drivers must wear helmets. It should be noted that the above only takes gloves, safety helmets and helmets as examples to illustrate the target objects in this application. In fact, it can also be other wearable objects, and this application does not make specific limitations here.

[0105] Each target training sample in the target training sample set in step S21 includes: a target sample image, a labeled wearing category representing whether the sample object in the target sample image wears the target object, and a labeled scene category representing the sampling scene of the target sample image. The labeled wearing category refers to whether the sample object wears the target object or does not wear the target object, and the labeled scene category is used to distinguish target sample images captured and sampled in different sampling scenes.

[0106] Specifically, taking the target object as a safety helmet as an example, the background of the image to be identified is the construction site, and the construction site often has a cluttered background. If the target training sample set obtained by sampling only at the construction site A is used to train the recognition model, the generalization of the recognition model will be poor, and it will be difficult to cope with complex background changes. When the recognition model is applied to other construction site scenes for safety helmet wearing recognition, the recognition accuracy rate will decrease. Therefore, the target sample images in this application are obtained by sampling different work sites, and the recognition model trained based on the target training sample set can adapt to the target object wearing recognition of the target object in the image to be identified in different scenes, effectively improving the recognition accuracy rate.

[0107] The following operations are performed in one cycle of training iterations:

[0108] S211: The server uses the recognition model in training to perform multi-dimensional feature extraction on the target sample image in the selected target training sample to obtain sample object features of at least two different dimensions, and based on the sample object features of at least two different dimensions, obtains the predicted wearing category corresponding to the target sample image and the predicted scene category corresponding to each sample object feature;

[0109] Among them, the predicted wearing category is used to characterize whether the target object is wearing the target object, and the predicted scene category is used to characterize the sampling scene category of the target sample image corresponding to the sample object feature.

[0110] In an optional implementation, the recognition model includes a domain classification network; the predicted scene category corresponding to the sample object feature is obtained by:

[0111] The features of each sample object are input into the corresponding domain classification network respectively to obtain the predicted scene category corresponding to each sample object feature.

[0112] Among them, the predicted scene category corresponding to the sample object feature is the predicted scene category corresponding to the target sample image corresponding to the sample object feature. During the model training process, the training goal is to make the domain classification network unable to distinguish the scene category corresponding to the sample object feature, indicating that the extracted sample object features are public features, rather than features in a certain type of training data. Therefore, the recognition model can adapt to the target object wearing recognition of the target object in the image to be recognized in different scenes, has adaptive ability, and effectively improves the speed and generalization of helmet detection.

[0113] In an optional embodiment, the recognition model includes a wear classification network; the predicted wear category corresponding to the target sample image is obtained by:

[0114] The third sample object feature is input into the wear classification network to obtain the predicted wear category corresponding to the target sample image.

[0115] Specifically, the third sample object feature is input into the wear classification network to perform pixel-level feature alignment to obtain the predicted wear category corresponding to the target sample object, that is, to identify whether the target object is wearing a helmet.

[0116] S212: The server adjusts parameters of the recognition model in training based on the difference between the predicted wear category and the annotated wear category, and the difference between each predicted scene category and the annotated scene category.

[0117] Specifically, a loss function can be constructed based on the difference between the predicted wearing category and the labeled wearing category, and the difference between each predicted scene category and the labeled scene category, and parameters of the recognition model in training can be adjusted based on the loss function.

[0118] S22: The server determines whether the target object in the image to be identified is wearing the target object based on the trained recognition model.

[0119] Among them, after the recognition model is trained, it can be used to determine whether the target object in the image to be recognized is wearing the target object. The recognition process is:

[0120] In an embodiment of the present application, by performing cyclic iterative training on the recognition model in training according to the target training sample set, the trained recognition model obtained can be used to determine whether the target object in the image to be identified is wearing the target object; wherein, in one cyclic iterative training, the following operations are performed: using the recognition model in training to perform multi-dimensional feature extraction on the target sample image in the selected target training sample, to obtain sample object features of at least two different dimensions, through the hierarchical transmission of features of different dimensions, the obtained sample object features are representative, based on the sample object features of at least two different dimensions, the predicted wearing category corresponding to the target sample image and the predicted scene category corresponding to each sample object feature are obtained; based on the difference between the predicted wearing category and the labeled wearing category, and the difference between each predicted scene category and the labeled scene category, the parameters of the recognition model in training are adjusted, which effectively improves the domain adaptation ability and generalization of the recognition model, and can adapt to the target object wearing recognition of the target object in the image to be identified in different scenes, thereby improving the recognition accuracy.

[0121] In an optional implementation, the sample object feature includes a first sample object feature, a second sample object feature and a third sample object feature; step S211 may be implemented as the following steps 1-4:

[0122] Step 1: Extract global and local features of the target sample image to obtain sample image features;

[0123] Step 2: Perform feature mapping on the sample image features to obtain the first sample object features of the image dimension;

[0124] Step 3: Perform feature mapping on the first sample object feature to obtain the second sample object feature of the instance dimension;

[0125] Step 4: Perform wear feature extraction on the second sample object feature to obtain the third sample object feature in pixel dimension.

[0126] Among them, global and local feature extraction can be performed on the target sample image based on the full convolutional network, and the obtained sample image features are obtained by extracting features from the entire image; then, target detection is performed on the obtained sample image features in step 2, and taking the detection target as the human foreground as an example, the detected human body frame is mapped to the sample image features to obtain the first sample object feature; then, in step 3, target detection is performed on the first sample object feature, and the detection target is the human head, and the detected head frame is mapped to the first sample object feature to obtain the second sample object feature; finally, in step 4, wearing feature extraction can be performed on the second sample object feature based on the convolutional network to obtain the third sample object feature.

[0127] In the embodiment of the present application, a three-stage domain adaptive detection network is used to decompose the helmet detection task, realize the hierarchical transfer and adaptability of features, and effectively improve the speed and generalization of helmet detection.

[0128] See also Figure 3A As shown, it is an implementation flow chart of a target object recognition method provided in an embodiment of the present application. Taking the execution subject as a server as an example, the specific implementation process of the method includes the following steps S301-S304:

[0129] S301: The server extracts global and local features of the image to be identified containing the target object based on the trained recognition model to obtain an image feature map;

[0130] The image feature map and the sample image features are obtained by extracting global and local features from the image. Specifically, before extracting features from the image to be identified, the image to be identified can be processed by data normalization and then sent to the recognition model to obtain an output result.

[0131] S302: The server performs object detection based on the image feature map to obtain a foreground feature map containing the target object;

[0132] S303: The server performs target part detection based on the foreground feature map to obtain a part feature map including the target part;

[0133] Among them, the target part is the part of the target object that is related to the target object being worn. For example, if the target object is a glove, the target part is the hand; if the target part is a helmet, the target part is the head. By performing object detection based on the image feature map and obtaining the object boundary information, feature mapping is performed on the image feature map based on the object boundary information to obtain the first object feature of the image dimension (i.e., the foreground feature map); by performing target part detection based on the foreground feature map and obtaining the target part boundary information, feature mapping is performed on the foreground feature map based on the target part boundary information to obtain the second object feature of the instance dimension (i.e., the part feature map). The method of obtaining the object boundary information and the target part boundary information can refer to the embodiment of obtaining the predicted boundary information of the detection target during the model training process below, which will not be repeated here.

[0134] S304: The server determines whether the target object is wearing the target object based on the part feature map.

[0135] Specifically, convolution feature extraction may be performed on the part feature map to obtain a third object feature in a pixel dimension, and whether the target object is wearing the target object may be determined based on the third object feature.

[0136] In the embodiment of the present application, whether the target object in the image to be identified is wearing the target object is determined based on the trained recognition model. By transferring the features in a hierarchical manner, the recognition task of the target object is decomposed. The target object wearing recognition can be performed on the target object in the image to be identified in different scenes, and the recognition accuracy is high.

[0137] The following takes the example of detecting the wearing of safety helmets by workers at a construction site to introduce the training method of the recognition model in this application, and the target object is a safety helmet.

[0138] See also Figure 3B , which is a structural diagram of a three-stage domain adaptive helmet detection network (i.e., recognition model) in an embodiment of the present application. The three-stage domain adaptive helmet detection network is mainly composed of four parts: a feature extraction network, a first-stage detector, a second-stage detector, and a three-stage classifier. Among them, the feature extraction network uses a fully convolutional network to extract global and local features of the input image to obtain an image feature map. The first-stage detector performs human foreground detection on the extracted image feature map, and maps the detected human body frame to the image feature to obtain a foreground feature map. The foreground feature map is sent to the second-stage detector for human body part detection. The detected human head frame is mapped to the foreground feature map to obtain a part feature map, and the part feature map is then sent to the three-stage classifier to determine the category of the helmet (i.e., wearing a helmet or not wearing a helmet). Before the three-stage classifier makes a category determination, feature extraction can be performed through the convolution layer. In order to achieve domain invariance of network features and improve the anti-interference ability of the three-stage domain adaptive helmet detection network to changes in environmental background, personnel foreground and helmet type, a domain classifier is added in each of the three stages to align features from multiple dimensions such as image level, instance level and pixel level.

[0139] It should be noted that the domain classifier is set to improve the recognition model's ability to resist interference with environmental background, personnel foreground, and changes in helmet types. After the recognition model is trained, there is no need to add a domain classifier during the testing phase.

[0140] In the embodiment of the present application, accurate prediction of the helmet is achieved through hierarchical feature transfer, and three groups of domain classifiers are introduced into the network to effectively improve the domain adaptation ability of the model.

[0141] In an optional implementation, the sample object features are obtained through the following steps S31-S32:

[0142] S31: Performing target detection on the mapped features to obtain predicted boundary information of the detected target;

[0143] S32: Mapping the features to be mapped into sample object features based on the predicted boundary information;

[0144] Specifically, if the feature to be mapped is a sample image feature, the detection target is a sample object (i.e., a human foreground), the predicted boundary information of the detection target is a human frame, and the sample object feature is a first sample object feature; if the feature to be mapped is a first sample object feature, the detection target is a target part of the sample object that is related to the target object (i.e., a human head), the predicted boundary information of the detection target is a human head frame, and the sample object feature is a second sample object feature. The predicted boundary information can be represented by the center point position coordinates, width, and height of the detection target.

[0145] In an optional implementation, each target training sample in the target training sample set further includes: annotated boundary information of the sample object in the target sample image and annotated boundary information of the target part; step S31 can be implemented as the following steps S41-S42:

[0146] S41: Based on at least one positive sample unit in the feature to be mapped, predict the boundary of the detection target to obtain at least one candidate boundary information of the detection target;

[0147] S42: Determine predicted boundary information of the detection target based on the confidence of at least one candidate boundary information, wherein the confidence of at least one candidate boundary information is determined based on an intersection-over-union ratio between each candidate boundary information and corresponding annotated boundary information.

[0148] Specifically, the positive sample unit with mapping features is responsible for predicting the boundaries of the detection target. If the feature to be mapped is a sample image feature, the detection target is the human body in the target sample image. If the feature to be mapped is a first sample object feature, the detection target is the human head. Taking the feature to be mapped as a sample image feature as an example, each positive sample unit predicts candidate boundary information of a human body. For example, positive sample unit 1 corresponds to candidate boundary information 1, positive sample unit 2 corresponds to candidate boundary information 2, and positive sample unit 3 corresponds to candidate boundary information 3. The intersection and union ratios between candidate boundary information 1-3 and the labeled boundary information of the human body are calculated respectively, and the confidence of candidate boundary information 1 is 0.1, the confidence of candidate boundary information 2 is 0.4, and the confidence of candidate boundary information 3 is 0.2. Then candidate boundary information 2 is used as the predicted boundary information of the detection target.

[0149] In an optional implementation, the predicted boundary information includes: predicted center position coordinates of the detection target and predicted object frame size; step S41 may be implemented as follows:

[0150] First, for a positive sample unit, determine the center position information of the detection target and the position offset information corresponding to the center position information; then, adjust the center position information based on the position offset information to obtain the predicted center position coordinates of the detection target; finally, based on the predicted center position coordinates, obtain the predicted object frame size of the detection target.

[0151] Specifically, the predicted object frame size can be represented by the width and height of the detected object. In the embodiment of the present application, an anchor-free frame detection head is used to perform object detection, and the detection task is divided into three branches: a coarse positioning branch, a fine positioning branch, and a size branch. Figure 4 , which is a detection flow diagram of a target detection method in an embodiment of the present application, wherein the coarse positioning branch coarsely locates the detection target through the response threshold on the heat map to obtain the center position information, Figure 4 The position of the shaded part in (1) is the center position information. The fine positioning branch predicts the position offset information corresponding to the center position information. The predicted center point position coordinates of the detected target are obtained by combining the center position information and the position offset information of the fine positioning branch. Finally, the size branch is responsible for directly predicting the width and height of the detected target to obtain the predicted object box size.

[0152] In an embodiment of the present application, a detection head is dynamically allocated without an anchor frame, and the traditional single detection head is disassembled into three detection branches. The decoupled detection head is used to accelerate the convergence of the model and improve the recognition accuracy.

[0153] In an optional implementation, the positive sample unit is obtained by:

[0154] First, based on the annotated boundary information, at least one candidate sample unit is determined; based on the difference between the candidate boundary information of each candidate sample unit and the corresponding annotated boundary information, the loss value of each candidate sample unit is determined; then, the loss values ​​are sorted according to size, and the candidate sample unit corresponding to the loss value within the first preset order range of the sorting result is taken as the positive sample unit.

[0155] Specifically, before each model training, the positive sample units need to be determined. Figure 5 , which is a schematic diagram of a candidate sample unit in an embodiment of the present application. First, based on the real annotation box (annotation boundary information), a rough candidate sample unit is determined, that is, Figure 5 Then, the loss value between the predicted box (predicted boundary information) of the candidate sample unit and the true annotation box is calculated, and the first d boxes with the smallest loss value are taken. k (ie, within the first preset order range) candidate sample units are used as positive sample units, and the remaining sample units are used as negative sample units.

[0156] In an embodiment of the present application, by introducing a dynamic positive and negative sample allocation strategy and determining the positive sample unit before each model training, the convergence speed of the model training can be effectively improved.

[0157] In an optional implementation, the first preset order range is determined in the following manner:

[0158] First, the intersection-and-union ratio between the predicted boundary information and the marked boundary information of each candidate sample unit is determined respectively; then, each intersection-and-union ratio is sorted according to size, and the first preset order range is determined based on the intersection-and-union ratio of the sorted result within the second preset order range.

[0159] Specifically, the intersection-and-union ratio of the predicted box and the true labeled box of the candidate sample unit is calculated, and each intersection-and-union ratio is sorted. Then, the intersection-and-union ratios of the top k (i.e., the second preset order range) are summed and rounded up to obtain d k .

[0160] In an embodiment of the present application, the detection head is disassembled into coarse positioning, fine positioning and size prediction tasks by decoupling, and the positive and negative sample units are dynamically adjusted to effectively improve the convergence speed of model training.

[0161] In an optional implementation, the target training sample set is obtained by following the steps S51-S52:

[0162] S51: Obtain an initial training sample set;

[0163] S52: Based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, the initial training sample set is expanded to obtain a target training sample set.

[0164] Among them, each initial training sample in the initial training sample set includes an initial sample image. The sample images in the initial training sample set are obtained by sampling different sampling scenes. The sample images in the initial training sample set can be divided into an initial source domain data set and an initial target domain data set according to different sampling scenes. Data enhancement methods include color transformation, translation, rotation, flipping, scaling, shearing, perspective transformation, image stitching and mosaic, etc. Each data enhancement method corresponds to two enhancement parameters of probability and amplitude. Different combinations of data enhancement methods are adjusted through different enhancement parameters to expand the initial training sample set. For example, the initial training sample set can be expanded according to the data enhancement method and the corresponding target enhancement parameter in the following table to obtain the target training sample set.

[0165]

[0166] In an optional implementation manner, at least one target enhancement parameter corresponding to each data enhancement method is obtained in the following manner:

[0167] Obtaining an initial enhancement parameter set, iteratively updating the initial enhancement parameter set, and obtaining each target enhancement parameter;

[0168] The initial enhancement parameter set includes initial enhancement parameters corresponding to each data enhancement method. The initial enhancement parameter set may be obtained by random initialization. For example, all initial enhancement parameters are set to 0.5.

[0169] In an optional implementation, the following steps are performed during an iterative update process:

[0170] Firstly, the initial enhancement parameters that meet the preset conditions are screened out. For the initial enhancement parameters that meet the conditions, the values ​​of at least two initial enhancement parameters are randomly exchanged to obtain the exchanged initial enhancement parameters. Then, the values ​​of the exchanged initial enhancement parameters are randomly mutated to obtain the mutated initial enhancement parameters, and the number of iterations is updated. If the updated number of iterations meets the iteration stop condition, the mutated initial enhancement parameters are used as the target enhancement parameters.

[0171] Specifically, the best data enhancement subset (the best data enhancement subset includes the target enhancement parameters) can be quickly searched out by the genetic evolution algorithm. First, the fitness (cost function) of each initial enhancement parameter is calculated, and the initial enhancement parameters that meet the preset conditions are screened out by the fitness, wherein the fitness of the initial enhancement parameters can be determined in the following manner: the initial training sample set is expanded by the initial enhancement parameters, the recognition model is iteratively trained based on the obtained target training sample set, and the fitness of the initial enhancement parameters is determined according to the recognition accuracy of the trained recognition model for the image to be recognized; then, for the initial enhancement parameters that meet the conditions, the values ​​of at least two initial enhancement parameters are randomly exchanged. Since only the initial enhancement parameters in the initial enhancement parameter set are exchanged, the initial enhancement parameter set is still the value obtained by initial initialization after multiple iterative updates. For example, all probabilities are initialized to 0.5 and the amplitude is 0.3. After N iterative updates, the probability and amplitude can only be 0.3 or 0.5. Therefore, the initial enhancement parameters need to be mutated so that new values ​​can appear in the initial enhancement parameter set. The number of iterations is increased by 1 after each iteration update. For example, the iteration stop condition is set to when the number of iterations reaches 1000. Then, when the number of iterations reaches 1000, the initial enhancement parameters in the current initial enhancement parameter set are used as the target enhancement parameters.

[0172] In the embodiments of the present application, various data enhancement methods are introduced, and the enhancement parameters are adjusted based on the genetic evolution algorithm to screen out the optimal data enhancement subset, thereby ensuring the universality of the training data and effectively improving the detection accuracy of the helmet.

[0173] See also Figure 6 , which is a schematic diagram of iterative update of a data enhancement parameter set in an embodiment of the present application. First, the data enhancement parameter set is initialized to obtain enhancement parameter subset 1. Then, in the first iteration: the probability of color transformation is 0.5 and the amplitude of flipping is 0.1, the probability of translation is 0.2 and the amplitude is 0.6, the probability of shearing is 0.2 and the probability of mosaic is 0.9, the probability of mutated flipping is 1, the amplitude of mutated flipping is 0.5, and the amplitude of mutated scaling is 0.7, and enhancement parameter subset 2 is obtained; in the second iteration: the probability of translation is 0.6 and the amplitude is 0.2, the amplitude of rotation is 0.4 and the probability of flipping is 1, the probability of mutated color transformation is 0.8, and enhancement parameter subset 3 is obtained. It should be noted that in Figure 6 The iterative update process of the data enhancement parameter set in the present application is described by taking a two-iteration process as an example. In practice, the number of iterations may be greater.

[0174] See also Figure 7 , which is a flow chart of a data enhancement strategy in an embodiment of the present application, mainly including the following two steps:

[0175] Step S71: Generate a source domain dataset and a target domain dataset;

[0176] Step S72: Perform data enhancement on the source domain dataset and the target domain dataset.

[0177] Among them, the enhancement strategy adopted in step S72 mainly includes a pre-enhancement unit and a super-parameter evolution unit. Among them, the pre-enhancement unit mainly cuts and pastes the images in the data set to preliminarily expand the data set; the super-parameter evolution unit divides the data enhancement parameters into two parameters: probability and amplitude, so as to adjust the combination of color transformation, translation, rotation, flipping, scaling, shearing, perspective transformation, picture splicing and mosaic data enhancement, and then quickly searches for the best enhancement parameter subset through the genetic evolution algorithm to improve the generalization of the data. Among them, the basic steps of the genetic evolution algorithm are as follows:

[0178] First, the enhancement parameter set is initialized, and then the fitness (i.e., cost function) of each enhancement parameter is calculated to determine whether a single enhancement parameter meets the preset conditions, that is, to determine whether a single enhancement parameter is retained or discarded in the iteration. If it is discarded, this enhancement parameter can be set to 0 in one iteration, and then a new enhancement parameter combination is generated through crossover and mutation, and it is determined whether the stopping condition is reached. If not, the iteration is continued. When the stopping condition is reached, the optimal enhancement parameter subset can be output.

[0179] See also Figure 8 , which is a flow chart of a genetic evolution algorithm in an embodiment of the present application, comprising the following steps:

[0180] S801: Initialize an enhanced parameter set;

[0181] S802: Calculate the fitness of each enhancement parameter;

[0182] S803: Determine enhancement parameters that meet preset conditions;

[0183] S804: Randomly exchange values ​​of at least two enhancement parameters;

[0184] S805: Randomly mutate the value of at least one enhancement parameter;

[0185] S806: Whether the iteration stop condition is met, if yes, execute step S807, otherwise return to step S802;

[0186] S807: Output the best enhancement parameter subset.

[0187] In the embodiment of the present application, a data enhancement pre-module of a genetic evolutionary algorithm is introduced to enrich the training data set and screen out the optimal enhancement subset, thereby solving the problem of single data in traditional algorithms.

[0188] In an optional implementation, before step S51, the following steps may also be performed:

[0189] Cut out and paste each initial sample image in the initial training sample set to obtain an intermediate training sample set;

[0190] Then step S52 can be implemented as follows:

[0191] Based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, data enhancement is performed on the intermediate sample images in the intermediate training sample set to obtain a target training sample set.

[0192] In the embodiment of the present application, in order to solve the problem of insufficient generalization ability and poor detection accuracy of the safety helmet detection model at the construction site, a data enhancement pre-module based on a genetic evolutionary algorithm is proposed to screen out the optimal data enhancement parameter subset, and a three-stage domain adaptive safety helmet detection network is used to achieve accurate prediction of safety helmets and domain adaptation of the model. In addition, a detection head with dynamic allocation without anchor frames is used to effectively improve the convergence speed of model training, and finally a domain adaptive safety helmet detection solution for construction site workers is realized.

[0193] See also Fig. 9, which is a flowchart of a helmet detection method in an embodiment of the present application, and is divided into the following two parts according to whether it is in the model training stage:

[0194] The model training phase includes the following steps S91-S94:

[0195] S91: construct an initial training sample set including a source domain dataset and a target domain dataset;

[0196] S92: Expand the initial training sample set into the target training sample set through data enhancement;

[0197] S93: Input the target training sample set into the three-stage domain adaptive helmet detection network;

[0198] S94: output recognition result;

[0199] The model testing phase includes the following steps S94-S96:

[0200] S95: data normalization processing;

[0201] S96: Input the normalized image data into the three-stage domain adaptive helmet detection network;

[0202] S94: Output the recognition result.

[0203] In an embodiment of the present application, based on the problem of workers wearing safety helmets on construction sites, a detection method based on domain adaptation is proposed. Through a genetically evolved data enhancement module, a three-stage domain adaptive detection network, and decoupled detection, the problems of the existing three-stage domain adaptive safety helmet detection network, such as single data, low precision, and poor generalization, are overcome.

[0204] Based on the same inventive concept, the present application also provides a training device for a recognition model. Fig.10 As shown, it is a schematic diagram of the structure of a training device 1000 for a recognition model, which may include:

[0205] The training unit 1001 is used to perform cyclic iterative training on the recognition model in training according to the target training sample set to obtain a trained recognition model; each target training sample in the target training sample set includes: a target sample image, and a labeled wearing category representing whether the sample object in the target sample image wears the target object, and a labeled scene category representing the sampling scene of the target sample image; wherein the training unit 1001 performs the following operations in one cyclic iterative training:

[0206] Using the recognition model in training to perform multi-dimensional feature extraction on the target sample image in the selected target training sample, obtaining sample object features of at least two different dimensions, and based on the sample object features of at least two different dimensions, obtaining the predicted wearing category corresponding to the target sample image and the predicted scene category corresponding to each sample object feature;

[0207] Based on the differences between the predicted wear categories and the labeled wear categories, and the differences between each predicted scene category and the labeled scene category, the parameters of the recognition model in training are adjusted.

[0208] Optionally, the sample object feature includes a first sample object feature, a second sample object feature and a third sample object feature;

[0209] The training unit 1001 is specifically used for:

[0210] Perform global and local feature extraction on the target sample image to obtain sample image features;

[0211] Performing feature mapping on the sample image features to obtain first sample object features of the image dimension;

[0212] Performing feature mapping on the first sample object feature to obtain a second sample object feature of the instance dimension;

[0213] Perform wear feature extraction on the second sample object feature to obtain a third sample object feature in pixel dimension.

[0214] Optionally, the training unit 1001 is specifically used for:

[0215] Perform target detection on the mapped features to obtain the predicted boundary information of the detected target;

[0216] Based on the predicted boundary information, the features to be mapped are mapped into sample object features;

[0217] Among them, if the feature to be mapped is a sample image feature, the detection target is the sample object, and the sample object feature is the first sample object feature; if the feature to be mapped is the first sample object feature, the detection target is the target part of the sample object related to the target object's wear, and the sample object feature is the second sample object feature.

[0218] Optionally, the recognition model includes a domain classification network 1002; the training unit 1001 is specifically used for:

[0219] The features of each sample object are input into the corresponding domain classification network respectively to obtain the predicted scene category corresponding to each sample object feature.

[0220] Optionally, the recognition model includes a wear classification network 1003; the training unit 1001 is specifically used for:

[0221] The third sample object feature is input into the wear classification network to obtain the predicted wear category corresponding to the target sample image.

[0222] Optionally, each target training sample in the target training sample set further includes: annotated boundary information of the sample object in the target sample image and annotated boundary information of the target part; performing target detection on the features to be mapped to obtain predicted boundary information of the detected target, and the training unit 1001 is specifically used for:

[0223] Based on at least one positive sample unit in the feature to be mapped, predicting the boundary of the detection target to obtain at least one candidate boundary information of the detection target;

[0224] Based on the confidence of at least one candidate boundary information, the predicted boundary information of the detection target is determined, and the confidence of at least one candidate boundary information is determined based on the intersection-and-union ratio between each candidate boundary information and the corresponding labeled boundary information.

[0225] Optionally, the predicted boundary information includes: predicted center position coordinates of the detection target and predicted object frame size; the training unit 1001 is specifically used for:

[0226] For each positive sample unit, perform the following operations:

[0227] For a positive sample unit, determine the center position information of the detection target and the position offset information corresponding to the center position information;

[0228] Adjust the center position information based on the position offset information to obtain the predicted center position coordinates of the detection target;

[0229] Based on the predicted center position coordinates, the predicted object box size of the detected target is obtained.

[0230] Optionally, the training unit 1001 is specifically used for:

[0231] Based on the marked boundary information, determining at least one candidate sample unit;

[0232] Determine the loss value of each candidate sample unit based on the difference between the candidate boundary information of each candidate sample unit and the corresponding labeled boundary information;

[0233] The loss values ​​are sorted according to their sizes, and the candidate sample units corresponding to the loss values ​​within the first preset order range of the sorting results are taken as positive sample units.

[0234] Optionally, the training unit 1001 is specifically used for:

[0235] Determine the intersection-and-union ratio between the predicted boundary information and the labeled boundary information of each candidate sample unit respectively;

[0236] The intersection-and-union ratios are sorted according to their sizes, and the first preset order range is determined based on the intersection-and-union ratios of the sorted results within the second preset order range.

[0237] Optionally, the training unit 1001 is specifically used for:

[0238] Acquire an initial training sample set, each initial training sample in the initial training sample set includes an initial sample image, and the sample images in the initial training sample set are obtained by sampling different sampling scenes;

[0239] Based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, the initial training sample set is expanded to obtain a target training sample set.

[0240] Optionally, the training unit 1001 is further used for:

[0241] Cut out and paste each initial sample image in the initial training sample set to obtain an intermediate training sample set;

[0242] Based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, the initial training sample set is expanded to obtain a target training sample set, including:

[0243] Based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, data enhancement is performed on the intermediate sample images in the intermediate training sample set to obtain a target training sample set.

[0244] Optionally, the training unit 1001 is specifically used for:

[0245] An initial enhancement parameter set is obtained, and the initial enhancement parameter set is iteratively updated to obtain each target enhancement parameter. The initial enhancement parameter set includes initial enhancement parameters corresponding to each data enhancement method. The following operations are performed during an iterative update process:

[0246] Screening out initial enhancement parameters that meet preset conditions, and for the initial enhancement parameters that meet the conditions, randomly exchanging values ​​of at least two initial enhancement parameters to obtain each of the exchanged initial enhancement parameters;

[0247] The values ​​of each initial enhancement parameter after random mutation exchange are obtained, and the number of iterations is updated;

[0248] If the updated number of iterations meets the iteration stop condition, the mutated initial enhancement parameters are used as the target enhancement parameters.

[0249] Optionally, the device further includes a determining unit 1004, configured to:

[0250] Based on the trained recognition model, feature extraction is performed on the image to be recognized containing the target object to obtain an image feature map;

[0251] Perform target detection based on the image feature map to obtain a foreground feature map containing the target object;

[0252] Performing target detection based on the foreground feature map to obtain a part feature map including a target part in the target object that is related to the wear of the target object;

[0253] Based on the part feature map, it is determined whether the target object is wearing the target object.

[0254] Based on the same inventive concept, the present application also provides a device for identifying a target object. Fig.11 As shown, it is a schematic diagram of the structure of a target object recognition device 1100, which may include:

[0255] The extraction unit 1101 is used to extract global and local features of the image to be identified containing the target object based on the trained recognition model to obtain an image feature map;

[0256] A first detection unit 1102 is used to perform object detection based on the image feature map to obtain a foreground feature map containing the target object;

[0257] The second detection unit 1103 is used to detect the target part based on the foreground feature map, and obtain a part feature map containing the target part, where the target part is a part of the target object that is related to the wear of the target object;

[0258] The determination unit 1104 is configured to determine whether the target subject is wearing the target object based on the part feature map.

[0259] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0260] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0261] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. In one embodiment, the electronic device may be a server, such as Figure 1 In this embodiment, the structure of the electronic device can be as follows: Fig.12 As shown, it includes a memory 1201 , a communication module 1203 and one or more processors 1202 .

[0262] The memory 1201 is used to store computer programs executed by the processor 1202. The memory 1201 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0263] The memory 1201 may be a volatile memory, such as a random-access memory (RAM); the memory 1201 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1201 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1201 may be a combination of the above memories.

[0264] The processor 1202 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1202 is used to implement the above-mentioned recognition model training method when calling the computer program stored in the memory 1201 .

[0265] The communication module 1203 is used to communicate with terminal devices and other servers.

[0266] The specific connection medium between the memory 1201, the communication module 1203 and the processor 1202 is not limited in the embodiment of the present application. Fig.12 In the embodiment, the memory 1201 and the processor 1202 are connected via a bus 1204. The bus 1204 is connected to the processor 1202 via a bus 1204. Fig.12 The connections between the other components are only for illustration and are not intended to be limiting. The bus 1204 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.12 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0267] The memory 1201 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the training method of the recognition model of the embodiment of the present application. The processor 1202 is used to execute the above-mentioned training method of the recognition model, such as Figure 2 shown.

[0268] In another embodiment, the electronic device may also be other electronic devices, such as Figure 1 The terminal device 110 shown in FIG. 1 is a terminal device 110 shown in FIG. 1 . In this embodiment, the structure of the electronic device can be as follows: Fig.13 As shown, it includes: a communication component 1310, a memory 1320, a display unit 1330, a camera 1340, a sensor 1350, an audio circuit 1360, a Bluetooth module 1370, a processor 1380 and other components.

[0269] The communication component 1310 is used to communicate with the server. In some embodiments, a circuit wireless fidelity (Wireless Fidelity, WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The electronic device can help the user to send and receive information through the WiFi module.

[0270] The memory 1320 can be used to store software programs and data. The processor 1380 executes various functions and data processing of the terminal device 110 by running the software programs or data stored in the memory 1320. The memory 1320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 1320 stores an operating system that enables the terminal device 110 to run. In the present application, the memory 1320 can store an operating system and various application programs, and can also store a computer program for executing the training method of the recognition model of the embodiment of the present application.

[0271] The display unit 1330 may also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 1330 may include a display screen 1332 disposed on the front of the terminal device 110. The display screen 1332 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 1330 may be used to display a training user interface of a recognition model in an embodiment of the present application, etc.

[0272] The display unit 1330 can also be used to receive input digital or character information and generate signal input related to user settings and function control of the terminal device 110. Specifically, the display unit 1330 may include a touch screen 1331 arranged on the front of the terminal device 110, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0273] The touch screen 1331 may be covered on the display screen 1332, or the touch screen 1331 and the display screen 1332 may be integrated to realize the input and output functions of the terminal device 110, and the integrated touch screen may be referred to as a touch display screen. In the present application, the display unit 1330 may display the application and the corresponding operation steps.

[0274] The camera 1340 can be used to capture still images, and users can post comments on the images taken by the camera 1340 through the application. The camera 1340 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the processor 1380 to convert it into a digital image signal.

[0275] The terminal device may further include at least one sensor 1350, such as an acceleration sensor 1351, a distance sensor 1352, a fingerprint sensor 1353, and a temperature sensor 1354. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0276] The audio circuit 1360, the speaker 1361, and the microphone 1362 can provide an audio interface between the user and the terminal device 110. The audio circuit 1360 can transmit the electrical signal converted from the received audio data to the speaker 1361, which is converted into a sound signal for output. The terminal device 110 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1362 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1360 and converted into audio data, and then the audio data is output to the communication component 1310 to be sent to, for example, another terminal device 110, or the audio data is output to the memory 1320 for further processing.

[0277] The Bluetooth module 1370 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 1370 to exchange data.

[0278] The processor 1380 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 1320 and calling data stored in the memory 1320. In some embodiments, the processor 1380 may include one or more processing units; the processor 1380 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 1380. In the present application, the processor 1380 can run the operating system, application programs, user interface display and touch response, as well as the training method of the recognition model of the embodiment of the present application. In addition, the processor 1380 is coupled to the display unit 1330.

[0279] In some possible implementations, various aspects of the recognition model training method provided in the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to perform the steps in the recognition model training method according to various exemplary embodiments of the present application described above in this specification. For example, the electronic device may perform the following steps: Figure 2 Follow the steps shown in .

[0280] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0281] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, apparatus, or device.

[0282] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0283] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0284] Computer programs for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer program may be executed entirely on the user electronic device, partially on the user device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In the case of a remote electronic device, the remote electronic device may be connected to the user electronic device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0285] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0286] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain a computer-usable computer program.

[0287] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program commands. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0288] These computer program commands may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the commands stored in the computer readable memory produce an article of manufacture comprising a command device, the command device implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0289] These computer program commands can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the commands executed on the computer or other programmable device provide the instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0290] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0291] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for training a recognition model, characterized in that: The method includes: According to the target training sample set, a recognition model in training is subjected to cyclic iterative training to obtain a trained recognition model; wherein the target training sample set is obtained by expanding the initial training sample set based on at least one data enhancement method and at least one corresponding target enhancement parameter, and the at least one target enhancement parameter is obtained by updating the initial enhancement parameter set through a genetic evolution algorithm; the trained recognition model is used to identify whether the target object in the image to be identified is wearing the target object; each target training sample includes: a target sample image, and a labeled wearing category representing whether the sample object in the target sample image is wearing the target object, and a labeled scene category representing the sampling scene of the target sample image, and the detection network of the recognition model includes a coarse positioning branch, a fine positioning branch, and a size branch; the following operations are performed in one cyclic iterative training: Using the recognition model in the training, multi-dimensional feature extraction is performed on the target sample image in the selected target training sample to obtain sample object features of at least two different dimensions, and based on the sample object features of at least two different dimensions, a predicted wearing category corresponding to the target sample image and a predicted scene category corresponding to each sample object feature are obtained; Based on the difference between the predicted wearing category and the annotated wearing category, and the difference between each predicted scene category and the annotated scene category, adjusting parameters of the recognition model in training; The sample object features include a first sample object feature, a second sample object feature and a third sample object feature; the method of using the recognition model in the training to perform multi-dimensional feature extraction on the target sample image in the selected target training sample to obtain sample object features of at least two different dimensions includes: Perform global and local feature extraction on the target sample image to obtain sample image features; perform feature mapping on the sample image features to obtain first sample object features in image dimension; perform feature mapping on the first sample object features to obtain second sample object features in instance dimension; perform wearing feature extraction on the second sample object features to obtain third sample object features in pixel dimension.

2. The method according to claim 1, characterized in that The sample object features are obtained by: Perform target detection on the mapped features to obtain the predicted boundary information of the detected target; Based on the predicted boundary information, mapping the to-be-mapped features to the sample object features; Among them, if the feature to be mapped is the sample image feature, the detection target is the sample object, and the sample object feature is the first sample object feature; if the feature to be mapped is the first sample object feature, the detection target is the target part of the sample object related to the target object being worn, and the sample object feature is the second sample object feature.

3. The method according to claim 1, characterized in that The recognition model includes a domain classification network; The predicted scene category corresponding to each sample object feature is obtained by: The features of each sample object are respectively input into the corresponding domain classification network to obtain the predicted scene category corresponding to each sample object feature.

4. The method according to claim 1, characterized in that The recognition model includes a wear classification network; the step of obtaining the predicted wear category corresponding to the target sample image based on the sample object features of the at least two different dimensions includes: The third sample object feature is input into the wear classification network to obtain the predicted wear category corresponding to the target sample image.

5. The method according to claim 2, characterized in that Each target training sample also includes: the labeled boundary information of the sample object in the target sample image and the labeled boundary information of the target part; the target detection is performed on the mapped features to obtain the predicted boundary information of the detected target, including: Based on at least one positive sample unit in the feature to be mapped, predicting a boundary of the detection target to obtain at least one candidate boundary information of the detection target; Based on the confidence of the at least one candidate boundary information, the predicted boundary information of the detection target is determined, and the confidence of the at least one candidate boundary information is determined based on the intersection-and-union ratio between each candidate boundary information and the corresponding labeled boundary information.

6. The method according to claim 5, characterized in that The predicted boundary information includes: predicted center position coordinates of the detection target and predicted object frame size; the step of performing boundary prediction on the detection target based on at least one positive sample unit in the feature to be mapped to obtain at least one predicted boundary information of the detection target includes: For each positive sample unit, perform the following operations: For a positive sample unit, determining the center position information of the detection target and the position offset information corresponding to the center position information; Adjusting the center position information based on the position offset information to obtain the predicted center position coordinates of the detection target; Based on the predicted center position coordinates, the predicted object frame size of the detection target is obtained.

7. The method according to claim 5, characterized in that The positive sample unit is obtained by: Based on the marked boundary information, determining at least one candidate sample unit; Determine the loss value of each candidate sample unit based on the difference between the candidate boundary information of each candidate sample unit and the corresponding marked boundary information; The loss values ​​are sorted according to their sizes, and the candidate sample units corresponding to the loss values ​​within the first preset order range of the sorting results are taken as the positive sample units.

8. The method according to claim 7, characterized in that The first preset order range is determined by: Determine the intersection-and-union ratio between the predicted boundary information and the labeled boundary information of each candidate sample unit respectively; The intersection-and-union ratios are sorted according to their sizes, and the first preset order range is determined based on the intersection-and-union ratios of the sorted results within the second preset order range.

9. The method according to any one of claims 1 to 8, characterized in that: The target training sample set is obtained by the following method, including: Acquire an initial training sample set, wherein each initial training sample in the initial training sample set includes an initial sample image, and the sample images in the initial training sample set are obtained by sampling different sampling scenes; Based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, the initial training sample set is expanded to obtain the target training sample set.

10. The method according to claim 9, characterized in that Before the initial training sample set is expanded based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method to obtain the target training sample set, the method further includes: Cutting and pasting each initial sample image in the initial training sample set to obtain an intermediate training sample set; The method of expanding the initial training sample set based on at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method to obtain the target training sample set includes: Based on the at least one data enhancement method and at least one target enhancement parameter corresponding to each data enhancement method, data enhancement is performed on the intermediate sample images in the intermediate training sample set to obtain the target training sample set.

11. The method according to claim 10, characterized in that At least one target enhancement parameter corresponding to each of the data enhancement modes is obtained by: An initial enhancement parameter set is obtained, and the initial enhancement parameter set is iteratively updated to obtain each target enhancement parameter, wherein the initial enhancement parameter set includes initial enhancement parameters corresponding to each data enhancement mode, and the following operations are performed in one iterative update process: Screening out initial enhancement parameters that meet preset conditions, and for the initial enhancement parameters that meet the preset conditions, randomly exchanging values ​​of at least two initial enhancement parameters to obtain each of the exchanged initial enhancement parameters; Randomly mutating the values ​​of the exchanged initial enhancement parameters to obtain the mutated initial enhancement parameters, and updating the number of iterations; If the updated number of iterations meets the iteration stop condition, the mutated initial enhancement parameters are used as the target enhancement parameters.

12. An object recognition method, characterized in that: The method includes: Based on a trained recognition model, extract global and local features of an image to be recognized containing a target object to obtain an image feature map, wherein the recognition model is obtained by training based on the training method for the recognition model described in any one of claims 1 to 11; Performing object detection based on the image feature map to obtain a foreground feature map containing the target object; Performing target part detection based on the foreground feature map to obtain a part feature map including the target part, wherein the target part is a part of the target object that is related to the wear of the target object; Based on the part feature map, it is determined whether the target subject is wearing a target object.

13. A training device for a recognition model, characterized in that: include: A training unit is used to perform cyclic iterative training on a recognition model in training according to a target training sample set to obtain a trained recognition model; wherein the target training sample set is obtained by expanding an initial training sample set based on at least one data enhancement method and at least one corresponding target enhancement parameter, and the at least one target enhancement parameter is obtained by updating the initial enhancement parameter set through a genetic evolution algorithm; the trained recognition model is used to identify whether a target object in an image to be identified wears a target object; each target training sample includes: a target sample image, and a labeled wearing category representing whether a sample object in the target sample image wears a target object, and a labeled scene category representing a sampling scene of the target sample image, and the detection network of the recognition model includes a coarse positioning branch, a fine positioning branch, and a size branch; the training unit performs the following operations in one cyclic iterative training: Using the recognition model in the training, multi-dimensional feature extraction is performed on the target sample image in the selected target training sample to obtain sample object features of at least two different dimensions, and based on the sample object features of at least two different dimensions, a predicted wearing category corresponding to the target sample image and a predicted scene category corresponding to each sample object feature are obtained; Based on the difference between the predicted wearing category and the annotated wearing category, and the difference between each predicted scene category and the annotated scene category, adjusting parameters of the recognition model in training; The sample object features include a first sample object feature, a second sample object feature and a third sample object feature; the method of using the recognition model in the training to perform multi-dimensional feature extraction on the target sample image in the selected target training sample to obtain sample object features of at least two different dimensions includes: Perform global and local feature extraction on the target sample image to obtain sample image features; perform feature mapping on the sample image features to obtain first sample object features in image dimension; perform feature mapping on the first sample object features to obtain second sample object features in instance dimension; perform wearing feature extraction on the second sample object features to obtain third sample object features in pixel dimension.

14. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any method described in claim 1 to 11 or 12.

15. A computer-readable storage medium, characterized in that: It includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any method described in claim 1 to 11 or 12.

16. A computer program product, characterized in that It includes a computer program, which is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of any method described in claim 1 to 11 or 12.

Images