Network training, unlocking, object tracking method, device, equipment and storage medium

Abstract

The present disclosure provides a network training, unlocking, and object tracking method, apparatus, device, and storage medium. The network training method includes: acquiring multiple sample infrared images; inputting the multiple sample infrared images into the target network to be trained, and obtaining the first image features corresponding to each sample infrared image; training the target network to be trained based on the second image features and the first image features corresponding to each sample infrared image until a trained target network is obtained. The sample infrared images include derived infrared images generated based on real infrared images, and / or real infrared images; the second image features are obtained by extracting features from the sample infrared images using a trained feature extraction network. The present disclosure uses derived infrared images to expand the number of sample infrared images when the number of real infrared images is insufficient, thereby improving the training effect of the target network.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a network training, unlocking, object tracking method, apparatus, device, and storage medium. Background Art

[0002] Face recognition is a critical issue in computer vision and security. It can be used in scenarios such as unlocking a phone, passing through gates, and clocking in at access control stations. Infrared face recognition involves comparing infrared facial images.

[0003] The network used for infrared face matching takes infrared face images as input and extracts the feature vector corresponding to each infrared face image. It then uses the distance between the feature vectors corresponding to different infrared face images to determine whether the infrared face images belong to the same person.

[0004] The infrared images used in training the above network are relatively rare in life, and it is difficult to obtain a large number of infrared images. Insufficient training samples can easily lead to poor performance of the trained network. Summary of the Invention

[0005] The embodiments of the present disclosure at least provide a network training, unlocking, object tracking method, apparatus, device and storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides a network training method, comprising:

[0007] Acquire multiple sample infrared images; wherein the sample infrared images include derived infrared images generated based on real infrared images, and / or real infrared images;

[0008] Inputting the plurality of sample infrared images into a target network to be trained, and obtaining a first image feature corresponding to each sample infrared image;

[0009] Based on the second image features and the first image features corresponding to each sample infrared image, the target network to be trained is trained until a trained target network is obtained; wherein the second image features are obtained by extracting features from the sample infrared images using the trained feature extraction network.

[0010] By adding derived infrared images to sample infrared images, the disclosed embodiments expand the number of sample infrared images when the number of real infrared images is insufficient, thereby improving the training accuracy of the target network. Furthermore, because the trained feature extraction network has high feature extraction accuracy, using the second image features obtained by extracting features from sample infrared images using the trained feature extraction network to train the target network can effectively improve training accuracy.

[0011] In an optional embodiment, when the sample infrared image includes the derived infrared image, acquiring a plurality of sample infrared images includes:

[0012] generating an initial derivative infrared image corresponding to each of the real infrared images;

[0013] For each initial derived infrared image, extracting a third image feature corresponding to the initial derived infrared image, and determining image accuracy information corresponding to the initial derived infrared image based on the extracted third image feature;

[0014] Based on the image accuracy information, a plurality of derived infrared images are selected from the generated plurality of initial derived images.

[0015] The disclosed embodiment screens the initial derived infrared image according to the image accuracy information corresponding to the initial derived infrared image to obtain a derived infrared image with image accuracy that meets the requirements, thereby improving the quality of the sample infrared image and thus improving the training effect of the target network.

[0016] In an optional implementation, determining the image accuracy information corresponding to the initial derived infrared image based on the extracted third image feature includes:

[0017] determining a feature vector corresponding to the initial derived infrared image based on a third image feature of the initial derived infrared image;

[0018] Image precision information corresponding to the initial derived infrared image is determined based on the modulus of the feature vector corresponding to the initial derived infrared image.

[0019] The disclosed embodiment can more accurately reflect the image accuracy of the initial derived infrared image through the modulus length of the feature vector corresponding to the initial derived infrared image. By using the modulus length to filter the initial derived infrared image, the quality of the filtered derived infrared image can be improved, and then the training effect of the target network can be improved.

[0020] In an optional embodiment, generating an initial derived infrared image corresponding to each of the real infrared images includes:

[0021] Generate an initial derivative infrared image corresponding to each of the real infrared images using the trained image generation neural network;

[0022] The image generation neural network is trained by the following steps:

[0023] Inputting multiple acquired real infrared images into the image generation neural network to be trained respectively, and obtaining the predicted derivative infrared image corresponding to each real infrared image;

[0024] Based on multiple groups of image pairs, the image generation neural network to be trained is trained until a training cutoff condition is met, thereby obtaining a trained image generation neural network; wherein each group of image pairs includes a real infrared image and a predicted derivative infrared image corresponding to the real infrared image.

[0025] The embodiment of the present disclosure utilizes a trained image generation neural network to generate an initial derived infrared image, which can ensure improved accuracy and efficiency of the initial derived infrared image.

[0026] In an optional embodiment, the training cutoff condition includes that the number of iterations of training the image generation neural network to be trained based on the multiple groups of image pairs is greater than a preset number.

[0027] The embodiment of the present disclosure uses the number of training iterations greater than a preset number as the training cutoff condition for the image generation neural network, which can effectively ensure the accuracy of the derived infrared image generated by the trained image generation neural network.

[0028] In an optional embodiment, the training of the target network to be trained based on the second image feature and the first image feature corresponding to each sample infrared image includes:

[0029] determining first similarity information between the second image feature and the corresponding first image feature based on the second image feature corresponding to each sample infrared image and the first image feature;

[0030] The target network to be trained is trained based on the first similarity information corresponding to each sample infrared image.

[0031] The embodiment of the present disclosure trains the target network with the goal of maximizing the similarity corresponding to the first similarity information, which can improve the training effect of the target network.

[0032] In an optional implementation, the training of the target network to be trained based on the first similarity information corresponding to each sample infrared image includes:

[0033] generating a first training loss based on the first similarity information corresponding to each sample infrared image;

[0034] Determining a first gradient corresponding to the first training loss;

[0035] The target network to be trained is trained based on the first gradient.

[0036] The embodiment of the present disclosure uses the first gradient that can accurately reflect the first training loss to train the target network, which can ensure the training effect of the target network and reduce the computational complexity of network training.

[0037] In an optional embodiment, the plurality of sample infrared images include at least one sample infrared image corresponding to each of the plurality of first preset tasks; the first training loss includes a first sub-training loss corresponding to each of the plurality of first preset tasks; and the first gradient includes a first sub-gradient corresponding to each of the plurality of first preset tasks.

[0038] The training of the target network to be trained based on the first gradient includes:

[0039] Determining first gradient means corresponding to a plurality of first preset tasks based on the first sub-gradient corresponding to each first preset task;

[0040] The target network to be trained is trained based on the first gradient mean.

[0041] The disclosed embodiment trains the target network based on sample infrared images corresponding to different preset tasks, which can improve the adaptability of the trained target network to various application scenarios and improve the detection accuracy of the target network for tasks in various scenarios.

[0042] In an optional embodiment, the sample infrared image includes a real infrared image;

[0043] The training of the target network to be trained based on the second image feature and the first image feature corresponding to each sample infrared image includes:

[0044] Obtain the standard classification information corresponding to each real infrared image;

[0045] Determining predicted classification information corresponding to each real infrared image based on a first image feature corresponding to each real infrared image;

[0046] The target network to be trained is trained based on the standard classification information, the predicted classification information, the second image feature, and the first image feature.

[0047] When the embodiment of the present disclosure uses real infrared images to train the target network, on the one hand, the classification ability of the target network is trained based on standard classification information and predicted classification information; on the other hand, based on the second image features and the first image features, the trained target network can not only accurately determine the image features of the image, but also accurately classify the image, making the trained target network function more complete.

[0048] In an optional implementation, the training of the target network to be trained based on the standard classification information, the predicted classification information, the second image feature, and the first image feature includes:

[0049] determining, based on a second image feature corresponding to each real infrared image and the first image feature, second similarity information between the second image feature and the corresponding first image feature;

[0050] Determining prediction accuracy information corresponding to each real infrared image based on the standard classification information corresponding to each real infrared image and the predicted classification information;

[0051] determining a second training loss based on the second similarity information and the prediction accuracy information corresponding to each real infrared image;

[0052] The target network to be trained is trained based on the determined second training loss.

[0053] The disclosed embodiment utilizes the second similarity information and the prediction accuracy information to more accurately determine the second training loss, thereby ensuring the processing effect of the trained target neural network on image feature extraction and classification detection.

[0054] In an optional implementation, the training of the target network to be trained based on the second training loss includes:

[0055] Determining a second gradient based on the second training loss;

[0056] The target network to be trained is trained based on the determined second gradient.

[0057] The embodiment of the present disclosure utilizes the second gradient that can accurately reflect the second training loss to train the target network, thereby ensuring the training effect of the target network and reducing the computational complexity of network training.

[0058] In an optional embodiment, the sample infrared image further includes a derivative infrared image;

[0059] The step of training the target network to be trained based on the determined second gradient includes:

[0060] determining, based on a second image feature corresponding to each derived infrared image and the first image feature, third similarity information between the second image feature and the corresponding first image feature;

[0061] determining a third training loss based on the third similarity information corresponding to each derived infrared image;

[0062] Determining a third gradient based on the third training loss;

[0063] The target network to be trained is trained based on the determined second gradient and the third gradient.

[0064] The disclosed embodiment uses the second gradient corresponding to the real infrared image and the third gradient corresponding to the derived infrared image to train the target network, which expands the number of sample infrared images and enables the trained target network to have higher detection accuracy.

[0065] In an optional implementation, the training of the target network to be trained based on the determined second gradient and the third gradient includes:

[0066] determining a second gradient mean based on the determined second gradient and the third gradient;

[0067] The target network to be trained is trained based on the second gradient mean.

[0068] The embodiment of the present disclosure uses the mean of the second gradient and the third gradient to train the target network, which can not only ensure the training accuracy but also reduce the computational complexity of the training.

[0069] In an optional embodiment, the real infrared image includes at least one real infrared image corresponding to each of the plurality of second preset tasks; the second training loss includes a second sub-training loss corresponding to each of the plurality of second preset tasks; and the second gradient includes a second sub-gradient corresponding to each of the plurality of second preset tasks.

[0070] The determining of a second gradient mean based on the determined second gradient and the third gradient includes:

[0071] A second gradient mean is determined based on the second sub-gradient corresponding to each second preset task and the third gradient.

[0072] When the embodiment of the present disclosure uses real infrared images and derived infrared images of different preset tasks to train the target network, it can improve the adaptability of the trained target network to different application scenarios and obtain detection results with higher accuracy.

[0073] In a second aspect, an embodiment of the present disclosure further provides an unlocking method, including:

[0074] Obtaining a first infrared face image captured by the device to be unlocked;

[0075] Performing feature recognition on the first infrared facial image using a trained target network to obtain first facial image features corresponding to the first infrared facial image; wherein the target network is a network trained according to the steps of the first aspect above, or any possible implementation of the first aspect;

[0076] When the first facial image feature matches the image feature corresponding to a pre-stored standard facial image, the device to be unlocked is controlled to perform an unlocking operation.

[0077] In a third aspect, the present disclosure further provides an object tracking method, including:

[0078] acquiring a second infrared face image for object tracking;

[0079] Performing feature recognition on the second infrared facial image using the trained target network to obtain second facial image features corresponding to the second infrared facial image; wherein the target network is a network trained according to the steps of the first aspect or any possible implementation of the first aspect;

[0080] In a case where the second facial image feature matches the image feature corresponding to the object to be tracked, the object corresponding to the second infrared facial image is determined to be the object to be tracked.

[0081] In a fourth aspect, an embodiment of the present disclosure further provides a network training device, comprising:

[0082] A first acquisition module is configured to acquire a plurality of sample infrared images; wherein the sample infrared images include derived infrared images generated based on real infrared images and / or real infrared images;

[0083] An input module, configured to input the plurality of sample infrared images into a target network to be trained, and obtain a first image feature corresponding to each sample infrared image;

[0084] The first training module is used to train the target network to be trained based on the second image feature and the first image feature corresponding to each sample infrared image until a trained target network is obtained; wherein the second image feature is obtained by extracting features of the sample infrared image using the trained feature extraction network.

[0085] In a fifth aspect, an embodiment of the present disclosure further provides an unlocking device, comprising:

[0086] A second acquisition module is used to acquire a first infrared face image captured by the device to be unlocked;

[0087] a first recognition module, configured to perform feature recognition on the first infrared facial image using a trained target network to obtain first facial image features corresponding to the first infrared facial image; wherein the target network is a network trained according to the method described in the first aspect or any possible implementation method of the first aspect;

[0088] The control module is configured to control the device to be unlocked to perform an unlocking operation when the first facial image feature matches the image feature corresponding to a pre-stored standard facial image.

[0089] In a sixth aspect, an embodiment of the present disclosure further provides an object tracking device, comprising:

[0090] a third acquisition module, configured to acquire a second infrared face image for object tracking;

[0091] a second recognition module, configured to perform feature recognition on the second infrared facial image using a trained target network to obtain second facial image features corresponding to the second infrared facial image; wherein the target network is a network trained according to the method described in the first aspect or any possible implementation method of the first aspect;

[0092] The second determining module is configured to determine that the object corresponding to the second infrared facial image is the object to be tracked if the second facial image feature matches the image feature corresponding to the object to be tracked.

[0093] In the seventh aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are performed.

[0094] In an eighth aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps in the implementation of the above-mentioned second aspect are performed.

[0095] In the ninth aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps in the implementation of the above-mentioned third aspect are performed.

[0096] In the tenth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0097] In an eleventh aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the implementation of the second aspect are executed.

[0098] In a twelfth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the implementation of the third aspect are executed.

[0099] For a description of the effects of the above-mentioned network training, unlocking, object tracking devices, equipment and storage media, please refer to the description of the above-mentioned network training, unlocking and object tracking methods, which will not be repeated here.

[0100] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0102] Figure 1 A flowchart of a network training method provided by an embodiment of the present disclosure is shown;

[0103] Figure 2 A schematic diagram of a process for generating a derivative infrared image provided by an embodiment of the present disclosure is shown;

[0104] Figure 3 A schematic diagram of a network training device provided by an embodiment of the present disclosure is shown;

[0105] Figure 4 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0106] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0107] The network used for infrared face matching takes infrared face images as input and extracts the feature vector corresponding to each infrared face image. It then uses the distance between the feature vectors corresponding to different infrared face images to determine whether the infrared face images belong to the same person.

[0108] The infrared images used in training the above network are relatively rare in life, and it is difficult to obtain a large number of infrared images. Insufficient training samples can easily lead to poor performance of the trained network.

[0109] Based on this, the present disclosure provides a network training method, comprising: obtaining multiple sample infrared images; wherein the sample infrared images include derived infrared images generated based on real infrared images, and / or real infrared images; inputting the multiple sample infrared images into a target network to be trained, and obtaining first image features corresponding to each sample infrared image; training the target network to be trained based on the second image features and the first image features corresponding to each sample infrared image until a trained target network is obtained; wherein the second image features are obtained by extracting features from the sample infrared images using a trained feature extraction network. The embodiment of the present disclosure expands the number of sample infrared images by adding derived infrared images to the sample infrared images when the number of real infrared images is insufficient, thereby improving the training accuracy of the target network. In addition, since the trained feature extraction network has a high feature extraction accuracy, training the target network using the second image features obtained by extracting features from the sample infrared images using the trained feature extraction network can effectively improve the training accuracy.

[0110] The defects in the above solutions are the results obtained by the inventors after practice and careful research. Therefore, the process of discovering the above problems and the solutions proposed by this disclosure for the above problems below should be the contributions made by the inventors to this disclosure during the disclosure process.

[0111] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0112] To facilitate understanding of this embodiment, a network training method disclosed in an embodiment of the present disclosure is first introduced in detail. The executor of the network training method provided in an embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a server or other processing device. In some possible implementation methods, the network training method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0113] The network training method provided by the embodiment of the present disclosure is described below by taking the execution subject as a server as an example.

[0114] See also Figure 1 FIG. 1 is a flow chart of a network training method provided by an embodiment of the present disclosure, wherein the method includes S101 to S103, wherein:

[0115] S101: Acquire multiple sample infrared images; wherein the sample infrared images include derived infrared images generated based on real infrared images, and / or real infrared images.

[0116] Sample infrared images are infrared images containing human faces used to train the target network. The target network is a network that can at least recognize facial features and identify the identity of the face in infrared facial images. The trained target network can be applied to scenarios requiring infrared facial image recognition, such as unlocking a phone, passing through a gate, and clocking in at access control stations.

[0117] The real face image refers to an infrared image of a face in the real world, that is, the face image contained in the real face image is a face image of a person in the real world.

[0118] In the disclosed embodiment, a real face image may be obtained by photographing a person in the real world, or an uploaded real face image may be obtained from the Internet, such as an identity document image, a surveillance image, etc.

[0119] A derived infrared image refers to an infrared image derived from a real facial image (the process of generating a derived infrared image based on a real facial image will be introduced later), that is, the facial image contained in the derived infrared image is a derived facial image of a person, and the facial image contained in the derived infrared image may be similar to the facial image of a person existing in the real world (this may include but is not limited to the facial image contained in the obtained real infrared image), but the face contained in the derived infrared image does not necessarily exist in the real world.

[0120] During the specific implementation process, the sample infrared images used to train the target network may include only real infrared images, only derived infrared images, or both real infrared images and derived infrared images. During the training process, if only real infrared images are used as sample infrared images, considering that real infrared images in real life are relatively rare and difficult to collect, the insufficient number of sample infrared images may result in a low recognition accuracy rate of the trained target network. If only derived infrared images are used as sample infrared images, considering the difference in authenticity between derived infrared images and real infrared images, the recognition accuracy rate of the trained target network may also be low. Therefore, in the embodiment of the present disclosure, it is preferred that both real infrared images and derived infrared images be used as sample infrared images. On the one hand, the number of sample infrared images can be increased, and on the other hand, the types of sample infrared images can be increased, thereby improving the features extracted or the recognition accuracy rate of the trained target network.

[0121] S102: Input the plurality of sample infrared images into a target network to be trained, and obtain a first image feature corresponding to each sample infrared image.

[0122] In the disclosed embodiment, the first image feature refers to a facial feature in a sample infrared image. Here, multiple sample infrared images can be simultaneously input into the target network to be trained. Alternatively, a preset number of sample infrared images from the multiple sample infrared images can be simultaneously input into the target network to be trained. Alternatively, multiple sample infrared images can be sequentially input into the target network to be trained. The number of sample infrared images input into the target network to be trained depends on the target network's ability to process sample infrared images simultaneously.

[0123] S103: Based on the second image feature and the first image feature corresponding to each sample infrared image, the target network to be trained is trained until a trained target network is obtained; wherein the second image feature is obtained by extracting features of the sample infrared image using the trained feature extraction network.

[0124] The feature extraction network refers to a network that can extract facial features from the sample infrared image. The feature extraction network here can be trained at least before step S103 is executed, and the second image features can also be obtained by extracting features from the sample infrared image using the feature extraction network at least before step S103 is executed.

[0125] Since the feature extraction network has been trained, the ability of the feature extraction network to extract image features is better than that of the target network to be trained. The embodiment of the present disclosure trains the target network based on the second image features and the first image features corresponding to each sample infrared image. The ultimate goal is to make the ability of the target network to extract image features close to or reach the ability of the feature extraction network to extract image features.

[0126] In the early stages of training the target network, there will definitely be differences in the image features extracted by the target network and the feature extraction network for the same sample infrared image, and the differences may be relatively large. Therefore, the target network can be trained based on the difference between the first image feature extracted by the target network and the second image feature extracted by the feature extraction network. After multiple rounds of training the target network, the difference between the first image feature extracted by the target network and the second image feature extracted by the feature extraction network can become smaller and smaller, that is, the target network's ability to extract image features can become closer and closer to that of the feature extraction network.

[0127] In one possible implementation, a trained target network can be obtained by training the target network based on the similarity between the second image feature and the first image feature. Specifically, first, based on the second image feature and the first image feature corresponding to each sample infrared image, first similarity information between the second image feature and the corresponding first image feature can be determined; then, based on the first similarity information corresponding to each sample infrared image, the target network to be trained is trained.

[0128] When calculating the first similarity information between the second image feature and the corresponding first image feature, the second image feature vector corresponding to the second image feature and the first image feature vector corresponding to the first image feature can be used to calculate the cosine similarity between the second image feature vector and the first image feature vector, and the cosine similarity is used as the first similarity information to train the target network to be trained.

[0129] As the number of training times increases, the first similarity information between the second image feature and the corresponding first image feature will become larger and larger, that is, the gap between the second image feature and the corresponding first image feature will become smaller and smaller, until the number of training times reaches the preset number of times or the first similarity information reaches the preset similarity value, the training is completed, and a trained target network is obtained.

[0130] In a specific implementation, in the process of training the target network based on the first similarity information corresponding to each sample infrared image, a first training loss can be first generated based on the first similarity information corresponding to each sample infrared image; then a first gradient corresponding to the first training loss is determined; and finally, based on the first gradient, the target network to be trained is trained.

[0131] The gradient can represent the directional derivative of the loss function at that point in the direction in which it reaches its maximum value. Here, the first training loss can be the first loss function corresponding to the target network. The first gradient represents the value and direction used to optimize the target network. As the number of training times increases, the first gradient decreases, and the value of the independent variable corresponding to the minimum value of the first loss function can be calculated. Therefore, by training the target network based on the first gradient, an optimized target network can be obtained.

[0132] The preset task refers to the task of processing multiple sample infrared images by the target network. Considering that the tasks that the target network performs on multiple sample infrared images may be different, for example, the target network may perform image segmentation on a portion of the sample infrared images, the target network may perform classification on another portion of the sample infrared images, and the target network may perform feature extraction on another portion of the sample infrared images; or considering that when too many sample infrared images are input into the target network at one time, the target network cannot process them normally, therefore, in the embodiment of the present disclosure, multiple sample infrared images can be grouped according to the preset tasks to obtain a sample infrared image group corresponding to each first preset task. The number of sample infrared images contained in the sample infrared image group corresponding to each first preset task may not be specifically limited here, but the sample infrared image group corresponding to each first preset task should contain at least one sample infrared image.

[0133] For at least one sample infrared image corresponding to each first preset task, a first sub-training loss can be generated based on the first similarity information corresponding to each sample infrared image, and then a first sub-gradient corresponding to each first sub-training loss can be determined. That is, the multiple sample infrared images include at least one sample infrared image corresponding to each of the multiple first preset tasks; the first training loss includes the first sub-training loss corresponding to each of the multiple first preset tasks; and the first gradient includes the first sub-gradient corresponding to each of the multiple first preset tasks.

[0134] Therefore, in another possible implementation, when executing the process of training the target network based on the first gradient, first, based on the first sub-gradient corresponding to each first preset task, the first gradient mean corresponding to multiple first preset tasks can be determined; and the target network is trained based on the first gradient mean.

[0135] The first gradient mean refers to the average value obtained by weighted summation of the first sub-gradients corresponding to multiple first preset tasks.

[0136] The above-mentioned process of training the target network is the main process of training the target network based on the sample infrared images. The sample infrared images here may include multiple derived infrared images and / or multiple real infrared images. However, if the sample infrared images only include real infrared images, there are some differences in the training of the target network and the above-mentioned training process. Please refer to the following for details.

[0137] Considering that a real infrared image corresponds to a human face existing in the real world, the real infrared image will correspond to real label information. Therefore, in order to train the target network to classify the input infrared image, when the sample infrared image includes a real infrared image, in a possible implementation, step S103 may include the following steps:

[0138] Step 11: Obtain the standard classification information corresponding to each real infrared image.

[0139] The standard classification information may refer to real classification information, which may be manually calibrated before the method is executed, or obtained by classification using a trained classification network. The standard classification information may be based on facial features in real infrared images, specifically classification based on gender, age, skin color, etc.; or it may be based on identity identifiers corresponding to faces in real infrared images.

[0140] Step 12: Based on the first image feature corresponding to each real infrared image, determine the predicted classification information corresponding to each real infrared image.

[0141] Here, after the real infrared image is input into the target network, the feature extraction layer of the target network first extracts the first image feature corresponding to the real infrared image, and then the classification layer of the target network determines the predicted classification information corresponding to each real infrared image based on the first image feature.

[0142] Step 13: Based on the standard classification information, the predicted classification information, the second image features and the first image features, the target network to be trained is trained.

[0143] Based on the standard classification information and the predicted classification information, the classification capability of the target network can be trained, so that the trained target network can classify the input infrared image. Based on the second image features and the first image features, the feature extraction capability of the target network can be trained, so that the trained target network can extract features from the input infrared image and obtain accurate image features.

[0144] In a specific embodiment, when training the target network based on the standard classification information, the predicted classification information, the second image feature, and the first image feature, the gradient for the target network can be calculated based on the standard classification information, the predicted classification information, the second image feature, and the first image feature, and then the target network to be trained is trained based on the second gradient. Specifically, the following steps can be performed:

[0145] Step 21: Based on the second image feature and the first image feature corresponding to each real infrared image, determine second similarity information between the second image feature and the corresponding first image feature.

[0146] Here, the second image features corresponding to each real infrared image are extracted by the target network, and the first image features corresponding to each real infrared image are extracted by the feature extraction network. For each real infrared image, the second image features of the real infrared image are extracted using the target network, and the first image features of the real infrared image are extracted using the feature extraction network.

[0147] Then, for each real infrared image, the cosine similarity between the second image feature vector and the first image feature vector is calculated using the second image feature vector corresponding to the second image feature and the first image feature vector corresponding to the first image feature, and the cosine similarity is used as the second similarity information.

[0148] Step 22: Based on the standard classification information and the predicted classification information corresponding to each real infrared image, determine the prediction accuracy information corresponding to each real infrared image.

[0149] The standard classification information corresponding to each real infrared image is obtained in the aforementioned step 11, and the predicted classification information corresponding to each real infrared image is obtained by target network classification.

[0150] For each real infrared image, based on the standard classification information and the predicted classification information, the prediction probability of the real infrared image on each standard classification information can be determined, that is, the prediction accuracy information corresponding to each real infrared image can be determined.

[0151] Step 23: Determine a second training loss based on the second similarity information and prediction accuracy information corresponding to each real infrared image.

[0152] In actual calculations, the second similarity information and the prediction accuracy information can be normalized first. Since the normalized second similarity information and the prediction accuracy information have a linear relationship, the second training loss can be determined by weighting the normalized second similarity information and the prediction accuracy information using a weight coefficient.

[0153] Step 24: Based on the determined second training loss, the target network to be trained is trained.

[0154] Here, a second gradient can be determined based on the determined second training loss, and then the target network to be trained is trained based on the determined second gradient.

[0155] As previously stated, in the disclosed embodiments, it is preferred to use multiple derived infrared images and multiple real infrared images simultaneously as sample infrared images. Having detailed the process of training the target network when the sample infrared images only include real infrared images, the following details the process of training the target network when the sample infrared images also include derived infrared images. Therefore, in one possible implementation, when the sample infrared images also include derived infrared images, training the target network to be trained based on the determined second gradient includes the following steps:

[0156] Step 31: Based on the second image feature and the first image feature corresponding to each derived infrared image, determine third similarity information between the second image feature and the corresponding first image feature.

[0157] Here, the second image features corresponding to each derived infrared image are extracted by the target network, and the first image features corresponding to each derived infrared image are extracted by the feature extraction network. For each derived infrared image, the second image features of the derived infrared image are extracted using the target network, and the first image features of the derived infrared image are extracted using the feature extraction network.

[0158] Then, for each derived infrared image, the cosine similarity between the second image feature vector and the first image feature vector is calculated using the second image feature vector corresponding to the second image feature and the first image feature vector corresponding to the first image feature, and the cosine similarity is used as the third similarity information.

[0159] Step 32: Determine a third training loss based on the third similarity information corresponding to each derived infrared image.

[0160] The third similarity information here can also be normalized to obtain the third training loss.

[0161] Step 33: Based on the third training loss, determine a third gradient.

[0162] Step 34: Based on the determined second gradient and third gradient, the target network is trained.

[0163] It is known that the second gradient is the gradient corresponding to the real infrared image, and the third gradient is the gradient corresponding to the derived infrared image. Here, the second gradient and the third gradient can be weighted summed to obtain the second gradient mean. Finally, the target network is trained based on the second gradient mean.

[0164] In one possible implementation, considering that the real infrared images may have different sources and therefore different processing tasks, the real infrared images may be divided into multiple real infrared images corresponding to second preset tasks based on the different preset tasks. The number of real infrared images included in each real infrared image group corresponding to the second preset task is not specifically limited, but each real infrared image group corresponding to the second preset task must include at least one real infrared image.

[0165] For at least one real infrared image corresponding to each second preset task, a second sub-training loss can be generated based on the second similarity information corresponding to each real infrared image, and then a second sub-gradient corresponding to each second sub-training loss can be determined. That is, the plurality of real infrared images include at least one real infrared image corresponding to each second preset task in the plurality of second preset tasks; the second training loss includes a second sub-training loss corresponding to each second preset task in the plurality of second preset tasks; and the second gradient includes a second sub-gradient corresponding to each second preset task in the plurality of second preset tasks.

[0166] Therefore, in another possible implementation, when determining the second gradient mean based on the determined second and third gradients, the second gradient mean can be determined based on the second sub-gradient and the third gradient corresponding to each second preset task. Specifically, the second sub-gradient and the third gradient corresponding to each second preset task are summed, and the mean of each gradient is determined, which is the second gradient mean.

[0167] The following describes the process of generating derivative infrared images based on real face images. Figure 2 The process diagram for generating a derivative infrared image shown includes the following steps:

[0168] S201: Generate an initial derivative infrared image corresponding to each of the real infrared images.

[0169] To improve the accuracy of target network training, when acquiring real infrared images, real infrared images of the target style can be acquired. The style of the real infrared image can include the contrast of the real infrared image, the facial expression in the image, the angle, and the depth information in the camera space. The acquired real infrared image can be a real infrared image of the same target style.

[0170] After obtaining multiple real infrared images, facial feature points in the real infrared images can be aligned to obtain the aligned faces and crop them to achieve normalization processing of the real infrared images.

[0171] Then, for each real infrared image, an initial derived infrared image corresponding to the real infrared image is obtained.

[0172] S202: For each initial derived infrared image, extract a third image feature corresponding to the initial derived infrared image, and determine image accuracy information corresponding to the initial derived infrared image based on the extracted third image feature.

[0173] In a specific implementation, the image accuracy information may be image resolution information. Higher image accuracy information indicates better quality of the initial derived infrared image. Here, a feature vector corresponding to each initial derived infrared image may be determined based on the third image feature of each initial derived infrared image. Then, based on the modulus of the feature vector corresponding to each initial derived infrared image, the image accuracy information corresponding to each initial derived infrared image may be determined.

[0174] S203: Based on the image accuracy information, select multiple derived infrared images from the generated multiple initial derived images.

[0175] Here, you can sort multiple initial derived images by image accuracy and select multiple derived infrared images with image accuracy greater than the preset image accuracy. Alternatively, you can select a preset number of derived infrared images from multiple initial derived images based on image accuracy.

[0176] In one possible implementation, a trained image generation neural network may be used to generate an initial derived infrared image corresponding to each real infrared image.

[0177] The image generation neural network can be obtained by training together with the image discrimination neural network.

[0178] Specifically, the step of training the image generation neural network includes: inputting multiple acquired real infrared images into the image generation neural network to be trained, respectively, to obtain a predicted derivative infrared image corresponding to each real infrared image; and training the image generation neural network to be trained based on multiple image pairs until a training cutoff condition is met, thereby obtaining a trained image generation neural network. Each image pair includes a real infrared image and a predicted derivative infrared image corresponding to the real infrared image. The training cutoff condition can be that the number of training iterations of the image generation neural network to be trained based on the multiple image pairs exceeds a preset number. Specifically, when the preset number of training iterations is reached, the training of the image generation neural network ends, resulting in a trained image generation neural network.

[0179] In a specific implementation, the image generation neural network can be a generative adversarial network, i.e., it includes a generator network and a discriminator network. Therefore, the process of training the image generation neural network can be as follows: in each training iteration, the generator network randomly inputs some noise and generates a predicted derivative infrared image based on the input noise. The generated predicted derivative infrared image and multiple real infrared images are then used to supervise the discriminator network's judgment of the predicted derivative infrared image. Here, the discriminator network determines whether each infrared image is a real infrared image or a predicted derivative infrared image. Based on the judgment results of the real infrared images and the predicted derivative infrared images, the parameters of the generator network and the discriminator network to be trained are tuned to train the discriminator network's ability to distinguish images from authenticity. Based on the judgment results of the discriminator network on the predicted derivative infrared images, the parameters of the generator network are also tuned. After multiple rounds of training of the untrained generator network and the discriminator network for a predetermined number of times, a trained image generation neural network is obtained.

[0180] When using the trained image generation neural network to generate initial derivative infrared images corresponding to each real infrared image based on multiple real infrared images, the generation network in the image generation neural network is actually used to generate the initial derivative infrared image corresponding to each real infrared image.

[0181] The disclosed embodiment also provides an unlocking method that can be applied to scenarios where mobile phones, gates, etc. are unlocked based on infrared facial images.

[0182] First, a first infrared face image captured by the device to be unlocked is obtained.

[0183] The device to be unlocked here can be a mobile phone, a face recognition access control machine or other device with infrared face image acquisition function.

[0184] Then, the trained target network is used to perform feature recognition on the first infrared face image to obtain the first face image feature corresponding to the first infrared face image; wherein, the target network is a network trained according to the network training method provided in the embodiment of the present disclosure.

[0185] Finally, when the first facial image feature matches the image feature corresponding to the pre-stored standard facial image, the device to be unlocked is controlled to perform an unlocking operation.

[0186] The training process of the target network is as described above and will not be repeated here. A first similarity threshold can be set. If the similarity between the first facial image features and the image features corresponding to the pre-stored standard facial image is greater than the first similarity threshold, the first facial image features can be considered to match the image features corresponding to the pre-stored standard facial image. In this case, feature recognition is successful, and the device to be unlocked can be controlled to perform an unlocking operation.

[0187] The disclosed embodiments further provide an object tracking method, which can be applied to a scenario of tracking a target person (eg, a suspect) that may appear in a public place.

[0188] First, a second infrared face image for object tracking is acquired.

[0189] Here, the second infrared facial image may be captured by one or more infrared image acquisition devices set at different locations (such as shopping malls, intersections, railway stations, subway security checkpoints, etc.). The second infrared facial image includes the captured facial image.

[0190] Then, the trained target network is used to perform feature recognition on the second infrared facial image to obtain second facial image features corresponding to the second infrared facial image; wherein, the target network is a network trained according to the network training method provided in the embodiment of the present disclosure.

[0191] Finally, when the second facial image feature matches the image feature corresponding to the object to be tracked, the object corresponding to the second infrared facial image is determined to be the object to be tracked.

[0192] The training process of the target network is as described above and will not be repeated here. Here, a second similarity threshold can be set. If the similarity between the second facial image features and the image features corresponding to the object to be tracked is greater than the second similarity threshold, it can be considered that the second facial image features match the image features corresponding to the object to be tracked. In this case, feature recognition passes, and the object corresponding to the second infrared facial image can be determined to be the object to be tracked.

[0193] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0194] Based on the same inventive concept, the embodiment of the present disclosure also provides a network training device corresponding to the network training method. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned network training method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0195] Reference Figure 3 FIG. 1 is a schematic diagram of the architecture of a network training device provided by an embodiment of the present disclosure, wherein the device includes: a first acquisition module 301, an input module 302, and a first training module 303; wherein,

[0196] A first acquisition module 301 is configured to acquire a plurality of sample infrared images; wherein the sample infrared images include derived infrared images generated based on real infrared images, and / or real infrared images;

[0197] An input module 302 is configured to input the plurality of sample infrared images into a target network to be trained, and obtain a first image feature corresponding to each sample infrared image;

[0198] The first training module 303 is used to train the target network to be trained based on the second image features and the first image features corresponding to each sample infrared image until a trained target network is obtained; wherein the second image features are obtained by extracting features from the sample infrared images using the trained feature extraction network.

[0199] In a possible implementation, the method further includes: a generating module, a first determining module, and a selecting module;

[0200] The generating module is configured to generate an initial derived infrared image corresponding to each of the real infrared images;

[0201] a first determining module configured to extract, for each initial derived infrared image, a third image feature corresponding to each initial derived infrared image, and determine, based on the extracted third image features, image accuracy information corresponding to each initial derived infrared image;

[0202] A selection module is used to select multiple derived infrared images from the generated multiple initial derived images based on the image accuracy information.

[0203] In a possible implementation manner, the first determining module is specifically configured to: determine a feature vector corresponding to the initial derived infrared image based on a third image feature of the initial derived infrared image;

[0204] Image precision information corresponding to the initial derived infrared image is determined based on the modulus of the feature vector corresponding to the initial derived infrared image.

[0205] In a possible implementation manner, the generating module is specifically configured to:

[0206] Generate an initial derivative infrared image corresponding to each of the real infrared images using the trained image generation neural network;

[0207] The device further comprises:

[0208] An input module is used to input the acquired multiple real infrared images into the image generation neural network to be trained, and obtain a predicted derivative infrared image corresponding to each real infrared image;

[0209] The second training module is used to train the image generation neural network to be trained based on multiple groups of image pairs until a training cutoff condition is met, thereby obtaining a trained image generation neural network; wherein each group of image pairs includes a real infrared image and a predicted derivative infrared image corresponding to the real infrared image.

[0210] In a possible implementation, the training cutoff condition includes that the number of iterations of training the image generation neural network to be trained based on the multiple groups of image pairs is greater than a preset number.

[0211] In a possible implementation, the first training module 303 is specifically configured to:

[0212] determining first similarity information between the second image feature and the corresponding first image feature based on the second image feature corresponding to each sample infrared image and the first image feature;

[0213] The target network to be trained is trained based on the first similarity information corresponding to each sample infrared image.

[0214] In a possible implementation, the first training module 303 is specifically configured to:

[0215] generating a first training loss based on the first similarity information corresponding to each sample infrared image;

[0216] Determining a first gradient corresponding to the first training loss;

[0217] The target network to be trained is trained based on the first gradient.

[0218] In one possible implementation, the plurality of sample infrared images include at least one sample infrared image corresponding to each of a plurality of first preset tasks; the first training loss includes a first sub-training loss corresponding to each of the plurality of first preset tasks; and the first gradient includes a first sub-gradient corresponding to each of the plurality of first preset tasks.

[0219] The first training module 303 is specifically configured to:

[0220] Determining first gradient means corresponding to a plurality of first preset tasks based on the first sub-gradient corresponding to each first preset task;

[0221] The target network to be trained is trained based on the first gradient mean.

[0222] In one possible implementation, the sample infrared image includes a real infrared image; and the first training module 303 is specifically configured to:

[0223] Obtain the standard classification information corresponding to each real infrared image;

[0224] Determining predicted classification information corresponding to each real infrared image based on a first image feature corresponding to each real infrared image;

[0225] The target network to be trained is trained based on the standard classification information, the predicted classification information, the second image feature, and the first image feature.

[0226] In a possible implementation, the first training module 303 is specifically configured to:

[0227] determining, based on a second image feature corresponding to each real infrared image and the first image feature, second similarity information between the second image feature and the corresponding first image feature;

[0228] Determining prediction accuracy information corresponding to each real infrared image based on the standard classification information corresponding to each real infrared image and the predicted classification information;

[0229] determining a second training loss based on the second similarity information and the prediction accuracy information corresponding to each real infrared image;

[0230] The target network to be trained is trained based on the determined second training loss.

[0231] In a possible implementation, the first training module 303 is specifically configured to:

[0232] Determining a second gradient based on the second training loss;

[0233] The target network to be trained is trained based on the determined second gradient.

[0234] In one possible implementation, the sample infrared image further includes a derived infrared image;

[0235] The first training module 303 is specifically configured to:

[0236] determining, based on a second image feature corresponding to each derived infrared image and the first image feature, third similarity information between the second image feature and the corresponding first image feature;

[0237] determining a third training loss based on the third similarity information corresponding to each derived infrared image;

[0238] Determining a third gradient based on the third training loss;

[0239] The target network to be trained is trained based on the determined second gradient and the third gradient.

[0240] In a possible implementation, the first training module 303 is specifically configured to:

[0241] determining a second gradient mean based on the determined second gradient and the third gradient;

[0242] The target network to be trained is trained based on the second gradient mean.

[0243] In one possible implementation, the real infrared image includes at least one real infrared image corresponding to each of the plurality of second preset tasks; the second training loss includes a second sub-training loss corresponding to each of the plurality of second preset tasks; and the second gradient includes a second sub-gradient corresponding to each of the plurality of second preset tasks.

[0244] The training module 303 is specifically configured to determine a second gradient mean based on the second sub-gradient corresponding to each second preset task and the third gradient.

[0245] The embodiment of the present disclosure also provides an unlocking device corresponding to the unlocking method, the unlocking device includes: a second acquisition module, a first identification module and a control module; wherein,

[0246] A second acquisition module is used to acquire a first infrared face image captured by the device to be unlocked;

[0247] a first recognition module, configured to perform feature recognition on the first infrared facial image using a trained target network to obtain first facial image features corresponding to the first infrared facial image; wherein the target network is a network trained according to the above-mentioned network training method;

[0248] The control module is configured to control the device to be unlocked to perform an unlocking operation when the first facial image feature matches the image feature corresponding to a pre-stored standard facial image.

[0249] Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned unlocking method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0250] The present disclosure also provides an object tracking device corresponding to the object tracking method, the object tracking device comprising: a third acquisition module, a second recognition module and a second determination module; wherein,

[0251] a third acquisition module, configured to acquire a second infrared face image for object tracking;

[0252] a second recognition module, configured to perform feature recognition on the second infrared facial image using a trained target network to obtain second facial image features corresponding to the second infrared facial image; wherein the target network is a network trained according to the above-mentioned network training method;

[0253] The second determining module is configured to determine that the object corresponding to the second infrared facial image is the object to be tracked if the second facial image feature matches the image feature corresponding to the object to be tracked.

[0254] Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned object tracking method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0255] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0256] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 44 is a schematic diagram of the structure of a computer device 400 provided in an embodiment of the present disclosure, including a processor 401, a memory 402, and a bus 403. The memory 402 is used to store execution instructions and includes a memory 4021 and an external memory 4022. The memory 4021 is also referred to as internal memory and is used to temporarily store operation data in the processor 401 and data exchanged with an external memory 4022 such as a hard disk. The processor 401 exchanges data with the external memory 4022 through the memory 4021. When the computer device 400 is running, the processor 401 communicates with the memory 402 via the bus 403, so that the processor 401 executes the following instructions:

[0257] Acquire multiple sample infrared images; wherein the sample infrared images include derived infrared images generated based on real infrared images, and / or real infrared images;

[0258] Inputting the plurality of sample infrared images into a target network to be trained, and obtaining a first image feature corresponding to each sample infrared image;

[0259] Based on the second image features and the first image features corresponding to each sample infrared image, the target network to be trained is trained until a trained target network is obtained; wherein the second image features are obtained by extracting features from the sample infrared images using the trained feature extraction network.

[0260] Based on the same technical concept, the embodiments of the present disclosure also provide another computer device, including a processor, a memory, and a bus. The memory is used to store execution instructions and includes internal memory and external memory. The internal memory here is also called internal memory, which is used to temporarily store the calculation data in the processor and the data exchanged with external memory such as a hard disk. The processor exchanges data with the external memory through the internal memory. When the computer device is running, the processor and the memory communicate through the bus, so that the processor executes the following instructions:

[0261] Obtaining a first infrared face image captured by the device to be unlocked;

[0262] Performing feature recognition on the first infrared facial image using a trained target network to obtain first facial image features corresponding to the first infrared facial image; wherein the target network is a network trained according to the above-mentioned network training method;

[0263] When the first facial image feature matches the image feature corresponding to a pre-stored standard facial image, the device to be unlocked is controlled to perform an unlocking operation.

[0264] Based on the same technical concept, the embodiments of the present disclosure also provide another computer device, including a processor, a memory, and a bus. The memory is used to store execution instructions and includes internal memory and external memory. The internal memory here is also called internal memory, which is used to temporarily store the calculation data in the processor and the data exchanged with external memory such as a hard disk. The processor exchanges data with the external memory through the internal memory. When the computer device is running, the processor and the memory communicate through the bus, so that the processor executes the following instructions:

[0265] acquiring a second infrared face image for object tracking;

[0266] Performing feature recognition on the second infrared facial image using the trained target network to obtain second facial image features corresponding to the second infrared facial image; wherein the target network is a network trained according to the above-mentioned network training method;

[0267] In a case where the second facial image feature matches the image feature corresponding to the object to be tracked, the object corresponding to the second infrared facial image is determined to be the object to be tracked.

[0268] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the network training method described in the above method embodiments, the steps of the unlocking method described in the above method embodiments, or the steps of the object tracking method described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0269] An embodiment of the present disclosure also provides a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the network training method described in the above method embodiment, or execute the steps of the unlocking method described in the above method embodiment, or execute the steps of the object tracking method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0270] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0271] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0272] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0273] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0274] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0275] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A network training method, characterized in that: include: Acquire multiple sample infrared images; wherein the sample infrared images include derived infrared images generated based on real infrared images, and real infrared images; Inputting the plurality of sample infrared images into a target network to be trained, and obtaining a first image feature corresponding to each sample infrared image; Training the target network to be trained based on the second image feature and the first image feature corresponding to each sample infrared image until a trained target network is obtained; wherein the second image feature is obtained by extracting features from the sample infrared image using the trained feature extraction network; Acquire multiple derived infrared images generated from real infrared images, including: generating an initial derivative infrared image corresponding to each of the real infrared images; For each initial derived infrared image, extracting a third image feature corresponding to the initial derived infrared image, and determining a feature vector corresponding to the initial derived infrared image based on the third image feature of the initial derived infrared image; and determining image accuracy information corresponding to the initial derived infrared image based on a modulus length of the feature vector corresponding to the initial derived infrared image; Based on the image accuracy information, a plurality of derived infrared images are selected from the generated plurality of initial derived infrared images.

2. The method according to claim 1, characterized in that Generating an initial derived infrared image corresponding to each real infrared image includes: Generate an initial derivative infrared image corresponding to each of the real infrared images using the trained image generation neural network; The image generation neural network is trained by the following steps: Inputting multiple acquired real infrared images into the image generation neural network to be trained respectively, and obtaining the predicted derivative infrared image corresponding to each real infrared image; Based on multiple groups of image pairs, the image generation neural network to be trained is trained until a training cutoff condition is met, thereby obtaining a trained image generation neural network; wherein each group of image pairs includes a real infrared image and a predicted derivative infrared image corresponding to the real infrared image.

3. The method according to claim 2, characterized in that The training cutoff condition includes that the number of iterations of training the image generation neural network to be trained based on the multiple groups of image pairs is greater than a preset number.

4. The method according to any one of claims 1 to 3, characterized in that The training of the target network to be trained based on the second image feature and the first image feature corresponding to each sample infrared image includes: determining first similarity information between the second image feature and the corresponding first image feature based on the second image feature corresponding to each sample infrared image and the first image feature; The target network to be trained is trained based on the first similarity information corresponding to each sample infrared image.

5. The method according to claim 4, characterized in that The training of the target network to be trained based on the first similarity information corresponding to each sample infrared image includes: generating a first training loss based on the first similarity information corresponding to each sample infrared image; Determining a first gradient corresponding to the first training loss; The target network to be trained is trained based on the first gradient.

6. The method according to claim 5, characterized in that The plurality of sample infrared images include at least one sample infrared image corresponding to each of the plurality of first preset tasks; the first training loss includes a first sub-training loss corresponding to each of the plurality of first preset tasks; the first gradient includes a first sub-gradient corresponding to each of the plurality of first preset tasks; The training of the target network to be trained based on the first gradient includes: Determining first gradient means corresponding to a plurality of first preset tasks based on the first sub-gradient corresponding to each first preset task; The target network to be trained is trained based on the first gradient mean.

7. The method according to claim 1, characterized in that The sample infrared image includes a real infrared image; The training of the target network to be trained based on the second image feature and the first image feature corresponding to each sample infrared image includes: Obtain the standard classification information corresponding to each real infrared image; Determining predicted classification information corresponding to each real infrared image based on a first image feature corresponding to each real infrared image; The target network to be trained is trained based on the standard classification information, the predicted classification information, the second image feature, and the first image feature.

8. The method according to claim 7, characterized in that The training of the target network to be trained based on the standard classification information, the predicted classification information, the second image feature, and the first image feature includes: determining, based on a second image feature corresponding to each real infrared image and the first image feature, second similarity information between the second image feature and the corresponding first image feature; Determining prediction accuracy information corresponding to each real infrared image based on the standard classification information corresponding to each real infrared image and the predicted classification information; determining a second training loss based on the second similarity information and the prediction accuracy information corresponding to each real infrared image; The target network to be trained is trained based on the determined second training loss.

9. The method according to claim 8, characterized in that The training of the target network to be trained based on the second training loss includes: Determining a second gradient based on the second training loss; The target network to be trained is trained based on the determined second gradient.

10. The method according to claim 9, characterized in that The sample infrared image also includes a derived infrared image; The step of training the target network to be trained based on the determined second gradient includes: determining, based on a second image feature corresponding to each derived infrared image and the first image feature, third similarity information between the second image feature and the corresponding first image feature; determining a third training loss based on the third similarity information corresponding to each derived infrared image; Determining a third gradient based on the third training loss; The target network to be trained is trained based on the determined second gradient and the third gradient.

11. The method according to claim 10, characterized in that The step of training the target network to be trained based on the determined second gradient and the third gradient includes: determining a second gradient mean based on the determined second gradient and the third gradient; The target network to be trained is trained based on the second gradient mean.

12. The method according to claim 11, characterized in that The real infrared image includes at least one real infrared image corresponding to each of the plurality of second preset tasks; the second training loss includes a second sub-training loss corresponding to each of the plurality of second preset tasks; The second gradient includes a second sub-gradient corresponding to each second preset task in the plurality of second preset tasks; The determining of a second gradient mean based on the determined second gradient and the third gradient includes: A second gradient mean is determined based on the second sub-gradient corresponding to each second preset task and the third gradient.

13. An unlocking method, characterized in that: include: Obtaining a first infrared face image captured by the device to be unlocked; Performing feature recognition on the first infrared facial image using a trained target network to obtain first facial image features corresponding to the first infrared facial image; wherein the target network is a network trained according to the method of any one of claims 1 to 12; When the first facial image feature matches the image feature corresponding to a pre-stored standard facial image, the device to be unlocked is controlled to perform an unlocking operation.

14. An object tracking method, characterized in that: include: acquiring a second infrared face image for object tracking; Performing feature recognition on the second infrared facial image using the trained target network to obtain second facial image features corresponding to the second infrared facial image; wherein the target network is a network trained according to the method of any one of claims 1 to 12; In a case where the second facial image feature matches the image feature corresponding to the object to be tracked, the object corresponding to the second infrared facial image is determined to be the object to be tracked.

15. A network training device, characterized in that: include: A first acquisition module is configured to acquire a plurality of sample infrared images; wherein the sample infrared images include derived infrared images generated based on a plurality of real infrared images, and real infrared images; acquiring a plurality of derived infrared images generated based on the real infrared images comprises: generating an initial derived infrared image corresponding to each of the real infrared images; extracting, for each initial derived infrared image, a third image feature corresponding to the initial derived infrared image, and determining, based on the third image feature of the initial derived infrared image, a feature vector corresponding to the initial derived infrared image; determining, based on the modulus of the feature vector corresponding to the initial derived infrared image, image accuracy information corresponding to the initial derived infrared image; and selecting, based on the image accuracy information, a plurality of derived infrared images from the generated plurality of initial derived infrared images; An input module, configured to input the plurality of sample infrared images into a target network to be trained, and obtain a first image feature corresponding to each sample infrared image; The first training module is used to train the target network to be trained based on the second image feature and the first image feature corresponding to each sample infrared image until a trained target network is obtained; wherein the second image feature is obtained by extracting features of the sample infrared image using the trained feature extraction network.

16. A computer device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the network training method according to any one of claims 1 to 12, the steps of the unlocking method according to claim 13, or the steps of the object tracking method according to claim 14 are performed.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the network training method according to any one of claims 1 to 12, or the steps of the unlocking method according to claim 13, or the steps of the object tracking method according to claim 14.

Images