Image recognition model training method, device, equipment and medium

By entering sample images into the image recognition model and calculating the loss value for training, the problem of low accuracy of the image recognition model is solved, and the significance and distinction of image features are improved.

CN114359547BActive Publication Date: 2024-12-03CHINA ORDNANCE SCI INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111666239.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-31
Publication Date
2024-12-03
Estimated Expiration
2041-12-31

AI Technical Summary

Technical Problem

In the prior art, the accuracy of the image recognition model is not high, which affects the significance and distinction of the image features extracted by the image recognition model.

Method used

By sequentially inputting each sample image pair in the sample image set into the original image recognition model, the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair are output, and The second feature vector corresponding to the target image sample in each sample image pair is output, the loss value is determined based on the pixel value and the feature vector, and the training is performed to improve the model accuracy.

Benefits of technology

The accuracy of the image recognition model is improved, the significance and distinction of object features in the image are enhanced, and the characteristics of the same object are similar, while the characteristics of the regions of the non-objects are not similar.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114359547B_ABST
    Figure CN114359547B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide an image recognition model training method, apparatus, device and medium. In the present application, the original image recognition model outputs a first estimated image and a second estimated image corresponding to a reference image sample in each sample image pair, and determines a first sub-loss value according to the pixel value of the corresponding pixel point in the first estimated image, and determines a second loss value according to the pixel value of the corresponding pixel point in the second estimated image, and performs pixel-level contrast learning. The first feature vector corresponding to the reference image sample in each sample image pair and the second feature vector corresponding to the target image sample in each sample image pair are output, and a third sub-loss value is determined according to the first feature vector and the second feature vector, and image-level contrast learning is performed. Since pixel-level contrast and image-level contrast learning are performed, the accuracy of the image recognition model and the significance or distinguishability of the object features in the image extracted by the image recognition model are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an image recognition model training method, device, equipment and medium. Background Art

[0002] With the rapid development of deep learning, accurately determining the feature representation of objects that can represent "visual association" plays a vital role in fields such as video object tracking, key point tracking and optical flow estimation. Among them, "visual association" is characterized by the fact that the features of regions representing the same object on different images are similar and have a certain significance, while the features of regions representing different objects on different images are dissimilar and have a certain degree of distinguishability.

[0003] In the related art, in order to obtain the feature expression form of an object in an image, an image recognition model is pre-trained, and the target image is input into the pre-trained image recognition model to obtain the feature expression form of the object in the target image output by the image recognition model. Generally, only one method of pixel-level contrast learning or image-level contrast learning is used to train the image recognition model. If only image-level contrast learning is used, the effect on fine-grained tracking tasks is poor. If only pixel-level contrast learning is used, there is a lack of high-level discrimination restrictions on association learning, resulting in the fact that the features corresponding to the areas of the same object output in the image recognition model are not highly correlated, and the features corresponding to the areas of different objects output are not highly non-correlated. In other words, the differences in the feature expressions corresponding to the areas of the same object output in the image recognition model are too large, and the differences in the feature expressions corresponding to the areas of different objects output by the image recognition model are too small. The accuracy of the image recognition model is not high, which affects the saliency or distinguishability of the object features in the image extracted by the image recognition model. Summary of the invention

[0004] The present application provides an image recognition model training method, device, equipment and medium to solve the problem that the image recognition model in the prior art has low accuracy, which affects the significance or distinctiveness of image features extracted by the image recognition model.

[0005] The present application provides an image recognition model training method, the method comprising:

[0006] Input each sample image pair in the sample image set into the original image recognition model in sequence, output the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair, and output the second feature vector corresponding to the target image sample in each sample image pair; wherein the reference image sample and the target image sample in the same sample image pair belong to the same video; and the reference image samples in different sample image pairs belong to different videos;

[0007] Determine a first sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image; determine a second sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image; determine a third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair; determine a target loss value according to the first sub-loss value, the second sub-loss value and the third sub-loss value;

[0008] The original image recognition model is trained according to the target loss value.

[0009] Further, determining each sample image pair in the sample image set includes:

[0010] For each pre-saved video, a reference image and a target image belonging to the video are obtained; the reference image and the target image are input into a pre-trained foreground region acquisition model, and a first foreground image corresponding to the reference image and a second foreground image corresponding to the target image are output; for each foreground image in the first foreground image and the second foreground image, a sub-foreground image corresponding to an object contained in the foreground image is determined;

[0011] The sub-foreground image of the first foreground image is determined as a reference image sample in a sample image pair, and the sub-foreground image of the second foreground image is determined as a target image sample in the sample image pair.

[0012] Furthermore, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting a first estimated image corresponding to the reference image sample in each sample image pair includes:

[0013] The original image recognition model performs a convolution operation on the reference image sample and the target image sample in each input sample image pair to determine the feature vector corresponding to each pixel; for each sample image pair, for each first pixel in the reference image sample in the sample image pair, according to the second pixel corresponding to the first pixel in the target image sample in the sample image pair, and a preset range, determine each third pixel within the set range including the second pixel; construct a first similarity matrix based on the similarity between the feature vectors of the first pixel and each third pixel; determine the first weight value corresponding to each third pixel based on the first similarity matrix and a preset normalization function, and determine the first estimated pixel value of the first pixel based on the pixel value of each third pixel and the corresponding first weight value; determine the first estimated image corresponding to the reference image sample based on the first estimated pixel value of each first pixel in the reference image sample.

[0014] Furthermore, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting a second estimated image corresponding to the reference image sample in each sample image pair includes:

[0015] For each sample image pair, for each first pixel point in the reference image sample in the sample image pair, determine each fifth pixel point within the set range including the fourth pixel point based on the target image sample in the sample image pair, a preset number of fourth pixel points corresponding to the target image samples in other sample image pairs other than the sample image pair, and a preset range; construct a second similarity matrix based on the similarity between the feature vectors of the first pixel point and each fifth pixel point; determine the second weight value corresponding to each fifth pixel point based on the second similarity matrix and the preset normalization function, and determine the second estimated pixel value of the first pixel point based on the pixel value of each fifth pixel point and the corresponding second weight value; determine the second estimated image corresponding to the reference image sample based on the second estimated pixel value of each first pixel point in the reference image sample.

[0016] Furthermore, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting the first feature vector corresponding to the reference image sample in each sample image pair includes:

[0017] The original image recognition model samples the reference image sample in each input sample image pair to obtain a target feature block; and performs pooling processing on the target feature block to obtain a first feature vector corresponding to the reference image sample.

[0018] Furthermore, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting the second feature vector corresponding to the target image sample in each sample image pair includes:

[0019] The original image recognition model determines, for each sixth pixel point included in the target feature block, each seventh pixel point within a preset range that includes the position in the target sample image according to the position of the sixth pixel point in the reference image sample; constructs a third similarity matrix according to the similarity between the feature vector of the sixth pixel point and the feature vector corresponding to each seventh pixel point; determines a third weight corresponding to each seventh pixel point according to the third similarity matrix and the preset normalization function; determines an estimated position corresponding to the sixth pixel point in the target image sample according to the third weight corresponding to each seventh pixel point and the third position information corresponding to each seventh pixel point;

[0020] Determine an average estimated position according to the estimated position corresponding to each sixth pixel point, and determine the corresponding estimated feature block on the target image sample according to the average estimated position and the target feature block size;

[0021] The estimated feature map blocks are pooled to obtain a second feature vector corresponding to the target image sample.

[0022] Further, determining the first sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image includes:

[0023] For each pixel in the reference image sample, determine a first difference between a pixel value of a corresponding pixel in the first estimated image and the pixel value of the pixel; determine whether the first difference is less than a preset difference threshold, and if so, determine a first product between the square of the first difference and a preset value as the loss value corresponding to the pixel; if not, determine a second difference between the absolute value of the first difference and the preset value as the loss value corresponding to the pixel;

[0024] A first sub-loss value is determined according to a first average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

[0025] Further, determining the second sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image includes:

[0026] For each pixel in the reference image sample, determine a third difference between the pixel value of the corresponding pixel in the second estimated image and the pixel value of the pixel; determine whether the third difference is less than a preset difference threshold, and if so, determine the second product between the square of the third difference and a preset value as the loss value corresponding to the pixel; if not, determine the fourth difference between the absolute value of the third difference and the preset value as the loss value corresponding to the pixel;

[0027] A second sub-loss value is determined according to a second average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

[0028] Further, determining the third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair includes:

[0029] For each sample image pair, determine a first dot product of a first eigenvector and a second eigenvector corresponding to the sample image pair; and determine a second dot product of the first eigenvector corresponding to the sample image pair and second eigenvectors corresponding to other sample image pairs except the sample image pair, and determine a sum value corresponding to the second dot product;

[0030] A third sub-loss value is determined according to a ratio of the first dot product to the sum value.

[0031] The present application provides an image recognition model training device, the device comprising:

[0032] A processing module, used for sequentially inputting each sample image pair in the sample image set into the original image recognition model, outputting a first estimated image, a second estimated image and a first feature vector corresponding to the reference image sample in each sample image pair, and outputting a second feature vector corresponding to the target image sample in each sample image pair; wherein the reference image sample and the target image sample in the same sample image pair belong to the same video; and the reference image samples in different sample image pairs belong to different videos;

[0033] A determination module, configured to determine a first sub-loss value according to each reference image sample and a pixel value of a corresponding pixel point in the corresponding first estimated image; determine a second sub-loss value according to each reference image sample and a pixel value of a corresponding pixel point in the corresponding second estimated image; determine a third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair; determine a target loss value according to the first sub-loss value, the second sub-loss value, and the third sub-loss value;

[0034] A training module is used to train the original image recognition model according to the target loss value.

[0035] Furthermore, the determination module is also used to obtain a reference image and a target image belonging to each pre-saved video; input the reference image and the target image into a pre-trained foreground area acquisition model, and output a first foreground image corresponding to the reference image and a second foreground image corresponding to the target image; for each foreground image in the first foreground image and the second foreground image, determine a sub-foreground image corresponding to an object contained in the foreground image; determine the sub-foreground image of the first foreground image as a reference image sample in a sample image pair, and determine the sub-foreground image of the second foreground image as a target image sample in the sample image pair.

[0036] Furthermore, the processing module is specifically used for the original image recognition model to perform convolution operations on the reference image sample and the target image sample in each input sample image pair to determine the feature vector corresponding to each pixel; for each sample image pair, for each first pixel in the reference image sample in the sample image pair, according to the second pixel corresponding to the first pixel in the target image sample in the sample image pair, and a preset range, determine each third pixel within the set range including the second pixel; construct a first similarity matrix based on the similarity between the feature vectors of the first pixel and each third pixel; determine the first weight value corresponding to each third pixel based on the first similarity matrix and a preset normalization function, and determine the first estimated pixel value of the first pixel based on the pixel value of each third pixel and the corresponding first weight value; determine the first estimated image corresponding to the reference image sample based on the first estimated pixel value of each first pixel in the reference image sample.

[0037] Furthermore, the processing module is specifically used to determine, for each sample image pair and for each first pixel in the reference image sample in the sample image pair, each fifth pixel within the set range including the fourth pixel based on the target image sample in the sample image pair, a preset number of fourth pixels corresponding to the target image samples in other sample image pairs other than the sample image pair, and a preset range; construct a second similarity matrix based on the similarity between the feature vectors of the first pixel and each fifth pixel; determine the second weight value corresponding to each fifth pixel based on the second similarity matrix and the preset normalization function, and determine the second estimated pixel value of the first pixel based on the pixel value of each fifth pixel and the corresponding second weight value; and determine the second estimated image corresponding to the reference image sample based on the second estimated pixel value of each first pixel in the reference image sample.

[0038] Furthermore, the processing module is specifically used for the original image recognition model to sample the reference image sample in each sample image pair input to obtain a target feature block; and to perform pooling processing on the target feature block to obtain a first feature vector corresponding to the reference image sample.

[0039] Furthermore, the processing module is specifically used for the original image recognition model to determine, for each sixth pixel point contained in the target feature block, according to the position of the sixth pixel point in the reference image sample, each seventh pixel point within a preset range containing the position in the target sample image; construct a third similarity matrix according to the similarity between the feature vector of the sixth pixel point and the feature vector corresponding to each seventh pixel point; determine the third weight corresponding to each seventh pixel point according to the third similarity matrix and the preset normalization function; determine the estimated position corresponding to the sixth pixel point in the target image sample according to the third weight corresponding to each seventh pixel point and the third position information corresponding to each seventh pixel point; determine the average estimated position according to the estimated position corresponding to each sixth pixel point, and determine the corresponding estimated feature block on the target image sample according to the average estimated position and the size of the target feature block; and perform a pooling operation on the estimated feature block to obtain a second feature vector corresponding to the target image sample.

[0040] Furthermore, the determination module is specifically used to determine, for each pixel point in the reference image sample, a first difference between the pixel value of the corresponding pixel point in the first estimated image and the pixel value of the pixel point; determine whether the first difference is less than a preset difference threshold, and if so, determine the first product between the square of the first difference and a preset numerical value as the loss value corresponding to the pixel point; if not, determine the second difference between the absolute value of the first difference and the preset numerical value as the loss value corresponding to the pixel point; and determine a first sub-loss value based on a first average loss value of the loss values ​​corresponding to each pixel point in the reference image sample.

[0041] Furthermore, the determination module is specifically used to determine, for each pixel point in the reference image sample, a third difference between the pixel value of the corresponding pixel point in the second estimated image and the pixel value of the pixel point; determine whether the third difference is less than a preset difference threshold, and if so, determine the second product between the square of the third difference and the preset numerical value as the loss value corresponding to the pixel point; if not, determine the fourth difference between the absolute value of the third difference and the preset numerical value as the loss value corresponding to the pixel point; and determine the second sub-loss value based on the second average loss value of the loss value corresponding to each pixel point in the reference image sample.

[0042] Furthermore, the determination module is specifically used to determine, for each sample image pair, a first dot product of a first eigenvector and a second eigenvector corresponding to the sample image pair; and determine a second dot product of a first eigenvector corresponding to the sample image pair and second eigenvectors corresponding to other sample image pairs except the sample image pair, and determine a sum value corresponding to the second dot product; and determine a third sub-loss value based on a ratio of the first dot product to the sum value.

[0043] The present application provides an electronic device, which includes a processor, and the processor is used to implement the steps of any of the above-mentioned image recognition model training methods when executing a computer program stored in a memory.

[0044] The present application provides a computer-readable storage medium storing a computer program executable by a terminal. When the program runs on the terminal, the terminal executes the steps of any one of the above-mentioned image recognition model training methods.

[0045] In the present application, each sample image pair in the sample image set is sequentially input into the original image recognition model, and the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair are output, and the second feature vector corresponding to the target image sample in each sample image pair is output, wherein the reference image sample and the target image sample in the same sample image pair belong to the same video, and the reference image samples in different sample image pairs belong to different videos, a first sub-loss value is determined according to the pixel values ​​of the corresponding pixels in each reference image sample and the corresponding first estimated image, a second sub-loss value is determined according to the pixel values ​​of the corresponding pixels in each reference image sample and the corresponding second estimated image, a third sub-loss value is determined according to the first feature vector and the second feature vector corresponding to each sample image pair, a target loss value is determined according to the first sub-loss value, the second sub-loss value and the third sub-loss value, and the original image recognition model is trained according to the target loss value. Since in the embodiment of the present application, the original image recognition model outputs the first estimated image and the second estimated image corresponding to the reference image sample in each sample image pair, and determines the first sub-loss value according to the pixel value of the corresponding pixel point in the first estimated image, and determines the second loss value according to the pixel value of the corresponding pixel point in the second estimated image, and performs pixel-level comparative learning, the original image recognition model also outputs the first eigenvector corresponding to the reference image sample in each sample image pair and the second eigenvector corresponding to the target image sample in each sample image pair, and determines the third sub-loss value according to the first eigenvector and the second eigenvector, and performs image-level comparative learning. Since both pixel-level comparative learning and image-level comparative learning are performed, the accuracy of the image recognition model is improved, and the significance or distinguishability of the object features in the image extracted by the image recognition model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A schematic diagram of a process of an image recognition model training method provided in some embodiments of the present application;

[0048] Figure 2 A schematic diagram of the structure of an image recognition model training device provided in some embodiments of the present application;

[0049] Figure 3 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0051] In the present application, each sample image pair in the sample image set is sequentially input into the original image recognition model, and the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair are output, and the second feature vector corresponding to the target image sample in each sample image pair is output, wherein the reference image sample and the target image sample in the same sample image pair belong to the same video, and the reference image samples in different sample image pairs belong to different videos, a first sub-loss value is determined according to the pixel values ​​of the corresponding pixels in each reference image sample and the corresponding first estimated image, a second sub-loss value is determined according to the pixel values ​​of the corresponding pixels in each reference image sample and the corresponding second estimated image, a third sub-loss value is determined according to the first feature vector and the second feature vector corresponding to each sample image pair, a target loss value is determined according to the first sub-loss value, the second sub-loss value and the third sub-loss value, and the original image recognition model is trained according to the target loss value.

[0052] In order to improve the accuracy of the image recognition model and improve the significance or distinctiveness of the object features in the image extracted by the image recognition model, the embodiments of the present application provide an image recognition model training method, device, equipment and medium.

[0053] Embodiment 1:

[0054] Figure 1 A schematic diagram of a process of an image recognition model training method provided in some embodiments of the present application, the process comprising the following steps:

[0055] S101: Input each sample image pair in the sample image set into the original image recognition model in turn, output the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair, and output the second feature vector corresponding to the target image sample in each sample image pair; wherein the reference image sample and the target image sample in the same sample image pair belong to the same video; and the reference image samples in different sample image pairs belong to different videos.

[0056] An embodiment of the present application provides an image recognition model training method applied to an electronic device, which may be a smart terminal, a PC, a server, or the like.

[0057] In the embodiment of the present application, in order to obtain a high-precision image recognition model, a sample image set is pre-configured, and the image recognition model is trained according to each sample image pair included in the sample image set, wherein the sample image pair includes a reference image sample and a target image sample.

[0058] In order to train the original image recognition model, in the present application, after obtaining the sample image set, each sample image pair in the sample image set is sequentially input into the image face recognition model, and for each sample image pair, the original image recognition model outputs the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in the sample image pair, and the second feature vector corresponding to the target image sample in the sample image pair, wherein the first estimated image is the first estimated image corresponding to the reference image sample obtained by performing pixel-level contrast learning on the reference image sample and the target image sample in the sample image pair, wherein the second estimated image is the second estimated image corresponding to the reference image sample obtained by performing pixel-level contrast learning on the reference image sample in the sample image pair, the target image sample in the sample image pair and the target sample images in other sample image pairs except the sample image. The first feature vector is a feature vector estimated to characterize the reference image sample, and the second feature vector is a feature vector estimated to characterize the target image sample by performing image-level contrast learning on the reference image sample and the target sample image in the sample image pair.

[0059] S102: Determine a first sub-loss value based on each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image; determine a second sub-loss value based on each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image; determine a third sub-loss value based on the first eigenvector and the second eigenvector corresponding to each sample image pair; determine a target loss value based on the first sub-loss value, the second sub-loss value and the third sub-loss value.

[0060] In the present application, the original image recognition model can be trained after obtaining the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair contained in the sample image set, as well as the second feature vector corresponding to the target image sample in each sample image pair.

[0061] Since the criterion for whether the original image recognition model has been trained is whether the loss value corresponding to each sample image pair of the original image recognition model currently reaches the convergence condition, if so, it means that the currently trained original image recognition model has a high recognition ability and a high recognition accuracy, so it can be determined that the original image recognition model training is completed.

[0062] In the present application, in order to complete the training of the original image recognition model, after determining the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair contained in the sample image set, and the second feature vector corresponding to the target image sample in each sample image pair, the target loss value can be determined according to the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair contained in the sample image set, and the second feature vector corresponding to the target image sample in each sample image pair.

[0063] In the present application, a first sub-loss value may be determined based on each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image, wherein the first sub-loss value is a part of the target loss value.

[0064] Specifically, in the present application, for each pixel point in each reference image sample, the difference between the pixel value of the pixel point and the pixel value of the corresponding pixel point in the first estimated image can be determined, and the difference can be determined as the first loss value corresponding to the pixel point. The sum of the first loss values ​​corresponding to each pixel point in each reference image sample can be determined as the second loss value corresponding to each reference image sample, and the sum of the second loss values ​​corresponding to each sample image in the sample image set can be determined as the first sub-loss value.

[0065] In the present application, a second sub-loss value may be determined based on each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image, wherein the second sub-loss value is a part of the target loss value.

[0066] In the present application, for each pixel point in each reference image sample, the difference between the pixel value of the pixel point and the pixel value of the corresponding pixel point in the second estimated image can be determined, and the difference can be determined as the third loss value corresponding to the pixel point. The sum of the third loss values ​​corresponding to each pixel point in each reference image sample can be determined as the fourth loss value corresponding to each reference image sample, and the second sub-loss value can be determined as the sum of the fourth loss values ​​corresponding to each sample image in the sample image set.

[0067] In the present application, a third sub-loss value may be determined according to the first eigenvector and the second eigenvector corresponding to each sample image pair, wherein the third sub-loss value is a part of the target loss value.

[0068] In the present application, in order to determine the target loss value, in the present application, the target loss value is determined according to the first sub-loss value, the second sub-loss value and the third sub-loss value. Specifically, the sum of weights can be determined according to the first sub-loss value and the preset first weight, the second sub-loss value and the preset second weight, and the third sub-loss value and the preset third weight, and the sum of weights can be determined as the target loss value. The largest loss value among the first sub-loss value, the second sub-loss value and the third sub-loss value can also be determined as the target loss value.

[0069] S103: Training the original image recognition model according to the target loss value.

[0070] In the present application, after determining the target loss value, the original image recognition model is trained based on the target loss value, wherein the smaller the target loss value is, the better the training result of the trained image recognition model is, the higher the accuracy of image recognition is, the more similar the features corresponding to the regions of the same object are, and the more dissimilar the features corresponding to the regions of different objects are.

[0071] Since in the embodiment of the present application, the original image recognition model outputs the first estimated image and the second estimated image corresponding to the reference image sample in each sample image pair, and determines the first sub-loss value according to the pixel value of the corresponding pixel point in the first estimated image, and determines the second loss value according to the pixel value of the corresponding pixel point in the second estimated image, and performs pixel-level comparative learning, the original image recognition model also outputs the first eigenvector corresponding to the reference image sample in each sample image pair and the second eigenvector corresponding to the target image sample in each sample image pair, and determines the third sub-loss value according to the first eigenvector and the second eigenvector, and performs image-level comparative learning. Since both pixel-level comparative learning and image-level comparative learning are performed, the accuracy of the image recognition model is improved, and the significance or distinguishability of the object features in the image extracted by the image recognition model is improved.

[0072] Embodiment 2:

[0073] In order to accurately determine each sample image pair, based on the above embodiment, in the embodiment of the present application, determining each sample image pair in the sample image set includes:

[0074] For each pre-saved video, a reference image and a target image belonging to the video are obtained; the reference image and the target image are input into a pre-trained foreground region acquisition model, and a first foreground image corresponding to the reference image and a second foreground image corresponding to the target image are output; for each foreground image in the first foreground image and the second foreground image, a sub-foreground image corresponding to an object contained in the foreground image is determined;

[0075] The sub-foreground image of the first foreground image is determined as a reference image sample in a sample image pair, and the sub-foreground image of the second foreground image is determined as a target image sample in the sample image pair.

[0076] In order to facilitate the subsequent training of the original image recognition model based on the sample image pairs in the sample image set, in the present application, sample image pairs that are helpful for the subsequent training of the original recognition model are obtained in advance. In the present application, in order to obtain sample image pairs, multiple videos are pre-saved in the electronic device, and for each pre-saved video, a reference image and a target image belonging to the video are obtained. Among them, for any two videos among the multiple videos, they are different videos, and each video determines a reference image and a target image. Specifically, for each video, the selection of the reference image and the target image belonging to the video is not limited.

[0077] If the reference image and the target image belonging to the video are directly determined as the sample image pair, the background area of ​​the reference image and the target image belonging to the video may cause certain interference to the subsequent training of the original image recognition model. Therefore, in the present application, in order to obtain a high-precision image recognition model, after obtaining the reference image and the target image belonging to each pre-saved video, the first foreground image corresponding to the reference image of the video and the second foreground image corresponding to the target image can be obtained.

[0078] In order to obtain the first foreground image corresponding to the reference image of each pre-saved video, and the second foreground image corresponding to the target image, in the present application, a foreground area acquisition model can be pre-trained, and the reference image sample and the target image sample can be input into the pre-trained foreground area acquisition model, and the first foreground image corresponding to the reference image sample and the second foreground image corresponding to the target image sample are output.

[0079] In order to obtain reference image samples and target image samples of the same size, in the present application, a sub-foreground image corresponding to an object contained in the foreground image can be obtained for each foreground image in the first foreground image and the second foreground image; wherein the sub-foreground image corresponding to the reference image sample is the same size as the sub-foreground image of the target image sample.

[0080] In the present application, in order to determine the sub-foreground image corresponding to the object contained in each foreground image in the first foreground image and the second foreground image, the maximum connected component of the foreground image is calculated respectively, and the sub-foreground image corresponding to the object contained in the foreground image is determined based on the center of the bounding box surrounding the maximum connected component and a preset range.

[0081] The process of calculating the maximum connected component of the foreground image is a prior art and will not be described in detail here.

[0082] The sub-foreground image of the first foreground image is determined as a reference image sample in the sample image pair, and the sub-foreground image of the second foreground image is determined as a target image sample in the sample image pair.

[0083] Embodiment 3:

[0084] In order to determine the first estimated image corresponding to the reference image sample in each sample image pair, based on the above embodiments, in the embodiment of the present application, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting the first estimated image corresponding to the reference image sample in each sample image pair includes:

[0085] The original image recognition model performs a convolution operation on the reference image sample and the target image sample in each input sample image pair to determine the feature vector corresponding to each pixel; for each sample image pair, for each first pixel in the reference image sample in the sample image pair, according to the second pixel corresponding to the first pixel in the target image sample in the sample image pair, and a preset range, determine each third pixel within the set range including the second pixel; construct a first similarity matrix based on the similarity between the feature vectors of the first pixel and each third pixel; determine the first weight value corresponding to each third pixel based on the first similarity matrix and a preset normalization function, and determine the first estimated pixel value of the first pixel based on the pixel value of each third pixel and the corresponding first weight value; determine the first estimated image corresponding to the reference image sample based on the first estimated pixel value of each first pixel in the reference image sample.

[0086] In the present application, in order to determine the first estimated image corresponding to the reference image sample in each sample image pair, the original image recognition model determines the feature vector corresponding to each pixel point for the reference image sample and the target image sample in each input sample image pair. Specifically, for each image sample in the reference image sample and the target image sample in each sample image pair, a convolution operation is performed on the image sample to determine the feature vector corresponding to each pixel point in the image sample.

[0087] For each sample image pair, for each first pixel point in the reference image sample in the sample image pair, determine each third pixel point within the set range including the second pixel point based on the second pixel point corresponding to the first pixel point in the target image sample and a preset range, wherein the preset range can be a range corresponding to a square frame with a preset length as the side length and the second pixel point corresponding to the first pixel point in the target image sample as the center, or a range corresponding to a circular frame with a preset length as the radius and the second pixel point corresponding to the first pixel point in the target image sample as the center. Specifically, the preset range can be set according to requirements.

[0088] After determining each third pixel point within the set range including the second pixel point, a first similarity matrix is ​​constructed according to the similarity between the feature vectors of the first pixel point and each third pixel point, wherein the similarity between the feature vectors can be determined according to the Euclidean distance, and each element in the first similarity matrix is ​​the similarity between the feature vectors of the first pixel point and each third pixel point. For example, if there are 9 third pixel points within the set range including the second pixel point, then the number of elements included in the first similarity matrix is ​​also 9.

[0089] The process of determining the Euclidean distance is a prior art and will not be described in detail here.

[0090] In order to determine the first estimated image, the first estimated pixel value corresponding to each pixel in the first estimated image can be determined first. Specifically, the importance of each third pixel to the first estimated pixel value of the corresponding pixel in the first estimated image is different. For example, for each first pixel in the reference image sample, the pixel value of the third pixel with greater similarity to the first pixel has a greater influence on the first estimated pixel value of the pixel corresponding to the first pixel in the first estimated image, and the pixel value of the third pixel with less similarity to the first pixel has a smaller influence on the first estimated pixel value of the pixel corresponding to the first pixel in the first estimated image.

[0091] In the present application, in order to accurately determine the first estimated image, the importance of each third pixel to the estimated pixel corresponding to the pixel in the first estimated image can be determined first. Specifically, the first weight value corresponding to each third pixel can be determined according to the preset first similarity matrix and the preset normalization function, and the first estimated pixel value of the first pixel can be determined according to the pixel value of each third pixel and the corresponding first weight value. Specifically, the weight sum can be determined according to the pixel value of each third pixel and the corresponding first weight value, and the weight sum can be determined as the first estimated pixel value of the first pixel.

[0092] After determining the first estimated pixel value of each first pixel point in the reference image sample, a first estimated image corresponding to the reference image sample is determined according to the first estimated pixel value of each first pixel point in the reference image sample, wherein the first estimated image has the same image size as the reference image sample.

[0093] Embodiment 4:

[0094] In order to accurately determine the second estimated image, based on the above embodiments, in the embodiment of the present application, each sample image pair in the sample image set is sequentially input into the original image recognition model, and the second estimated image corresponding to the reference image sample in each sample image pair is output, including:

[0095] For each sample image pair, for each first pixel point in the reference image sample in the sample image pair, determine each fifth pixel point within the set range including the fourth pixel point based on the target image sample in the sample image pair, a preset number of fourth pixel points corresponding to the target image samples in other sample image pairs other than the sample image pair, and a preset range; construct a second similarity matrix based on the similarity between the feature vectors of the first pixel point and each fifth pixel point; determine the second weight value corresponding to each fifth pixel point based on the second similarity matrix and the preset normalization function, and determine the second estimated pixel value of the first pixel point based on the pixel value of each fifth pixel point and the corresponding second weight value; determine the second estimated image corresponding to the reference image sample based on the second estimated pixel value of each first pixel point in the reference image sample.

[0096] In the present application, in order to improve the robustness of the image recognition model, noise and interference can also be added to train the image recognition model. Since different sample image pairs belong to different videos and there are differences between images in different videos, for each sample image pair, when determining the second estimated image corresponding to the reference image sample, the interference of the target image samples in other sample image pairs can be increased.

[0097] Specifically, for each sample image pair, for each first pixel point in the reference image sample in the sample image pair, according to the fourth pixel points corresponding to the first pixel point in the target image sample, a preset number of target image samples in other sample image pairs other than the sample image pair, and a preset range, determine each fifth pixel point within the set range including the fourth pixel point, wherein the preset number can be set as required, and can be 2, 3, 5, etc. The preset range can be the range corresponding to a square frame with a preset length as the side length and the fourth pixel point corresponding to the first pixel point in the target image sample as the center, or the range corresponding to a circular frame with a preset length as the radius and the fourth pixel point corresponding to the first pixel point in the target image sample as the center. Specifically, the preset range can be set as required. In the present application, the preset ranges corresponding to different target image samples are the same.

[0098] After determining each fifth pixel point within the set range including the fourth pixel point, a second similarity matrix is ​​constructed according to the similarity between the feature vectors of the first pixel point and each fourth pixel point.

[0099] In the present application, since the corresponding similarity will be determined between the first pixel point and the target image sample in the sample image pair and each fifth pixel point corresponding to each target image sample in a preset number of target image samples in other sample image pairs other than the sample image pair, for the sake of convenience of description, the similarity corresponding to the first pixel point and each fifth pixel point in a target image sample is referred to as a group of similarities, and since the target image sample in the sample image pair and the target image samples in a preset number of other sample image pairs other than the sample image pair include multiple target image samples, multiple groups of similarities can be determined.

[0100] In the present application, an average similarity matrix can be determined based on the similarity matrices corresponding to multiple groups of similarities, and the average similarity matrix can be determined as the second similarity matrix. The weights and matrices of the similarity matrices corresponding to multiple groups of similarities can also be determined based on the similarity matrices corresponding to multiple groups of similarities and preset similarity matrix weights, and the weights and matrices can be determined as the second similarity matrix.

[0101] In the present application, the similarity matrices corresponding to the multiple groups of similarities may also be spliced ​​to obtain a second similarity matrix, and the specific splicing method is not limited here.

[0102] In the present application, in order to accurately determine the second estimated image, the importance of each fifth pixel to the estimated pixel corresponding to the pixel in the first estimated image can be determined first. Specifically, the second weight value corresponding to each fifth pixel can be determined according to the preset second similarity matrix and the preset normalization function, and the second estimated pixel value of the first pixel can be determined according to the pixel value of each fifth pixel and the corresponding second weight value. Specifically, the weight sum can be determined according to the pixel value of each fifth pixel and the corresponding second weight value, and the weight sum can be determined as the second estimated pixel value of the first pixel.

[0103] After determining the second estimated pixel value of each first pixel point in the reference image sample, a second estimated image corresponding to the reference image sample is determined according to the second estimated pixel value of each first pixel point in the reference image sample, wherein the second estimated image has the same image size as the reference image sample.

[0104] Embodiment 5:

[0105] In order to accurately determine the first feature vector corresponding to the reference image sample, on the basis of the above embodiments, in the embodiment of the present application, each sample image pair in the sample image set is sequentially input into the original image recognition model, and the first feature vector corresponding to the reference image sample in each sample image pair is output, including:

[0106] The original image recognition model samples the reference image sample in each input sample image pair to obtain a target feature block; and performs pooling processing on the target feature block to obtain a first feature vector corresponding to the reference image sample.

[0107] In the present application, in order to determine the first feature vector corresponding to the reference image sample, the original image recognition model samples the reference image sample to obtain a target feature block, and pools the target feature block to obtain the first feature vector corresponding to the reference image sample.

[0108] Embodiment 6:

[0109] In order to accurately determine the second feature vector corresponding to the target image sample, based on the above embodiments, in the embodiment of the present application, each sample image pair in the sample image set is sequentially input into the original image recognition model, and the second feature vector corresponding to the target image sample in each sample image pair is output, including:

[0110] The original image recognition model determines, for each sixth pixel point included in the target feature block, each sixth pixel point within a preset range that includes the position in the target sample image according to the position of the sixth pixel point in the reference image sample; constructs a third similarity matrix according to the similarity between the feature vector of the sixth pixel point and the feature vector corresponding to each seventh pixel point; determines a third weight corresponding to each seventh pixel point according to the third similarity matrix and the preset normalization function; determines an estimated position corresponding to the seventh pixel point in the target image sample according to the third weight corresponding to each seventh pixel point and the third position information corresponding to each seventh pixel point;

[0111] Determine an average estimated position according to the estimated position corresponding to each sixth pixel point, and determine the corresponding estimated feature block on the target image sample according to the average estimated position and the target feature block size;

[0112] The estimated feature map blocks are pooled to obtain a second feature vector corresponding to the target image sample.

[0113] The original image recognition model determines, for each sixth pixel point contained in the target feature block, every seventh pixel point within a preset range containing the position in the target sample image according to the position of the sixth pixel point in the reference image sample, wherein the preset range can be a range corresponding to a square frame with a preset length as the side length and the position corresponding to the sixth pixel point in the target image sample as the center, or a range corresponding to a circular frame with a preset length as the radius and the position corresponding to the sixth pixel point in the target image sample as the center. Specifically, the preset range can be set according to requirements.

[0114] After determining each seventh pixel point within the set range including the sixth pixel point, a third similarity matrix is ​​constructed according to the similarity between the feature vectors of the sixth pixel point and each seventh pixel point, wherein each element in the third similarity matrix is ​​the similarity between the feature vectors of the sixth pixel point and each seventh pixel point, for example, if there are 25 seventh pixel points within the set range including the sixth pixel point, then the number of elements included in the third similarity matrix is ​​also 25.

[0115] Since the importance of each seventh pixel point to the estimated position of the corresponding pixel point in the estimated feature block is different, for example, the position of the seventh pixel point with greater similarity to the sixth pixel point has a greater influence on the estimated position of the pixel point corresponding to the sixth pixel point in the estimated feature block, and the position of the seventh pixel point with less similarity to the sixth pixel point has a smaller influence on the estimated position of the pixel point corresponding to the sixth pixel point in the estimated feature block.

[0116] In the present application, in order to accurately determine the estimated feature block, the importance of each seventh pixel point to the estimated position of the corresponding pixel point in the estimated feature block can be determined first. Specifically, the third weight value corresponding to each seventh pixel point can be determined according to the preset third similarity matrix and the preset normalization function, and the estimated position corresponding to the sixth pixel point in the target image sample can be determined according to the position of each seventh pixel point and the corresponding third weight value. Specifically, the weight sum is determined according to the position of each seventh pixel point and the corresponding third weight value, and the weight sum is determined as the estimated position corresponding to the sixth pixel point in the target image sample.

[0117] In the present application, after determining the estimated position corresponding to each sixth pixel point, the average estimated position can also be determined based on the estimated position corresponding to each sixth pixel point, and the corresponding estimated feature block on the target image sample can be determined based on the average estimated position and the target feature block size. Specifically, with the average estimated position as the center, an estimated feature block with the same size as the target feature block is determined.

[0118] After the estimated feature block is determined, in order to determine the second feature vector corresponding to the target image sample, the estimated feature block is pooled to obtain the second feature vector corresponding to the target image sample.

[0119] Embodiment 7:

[0120] In order to accurately determine the first sub-loss value, based on the above embodiments, in the embodiment of the present application, the first sub-loss value is determined according to the pixel value of the corresponding pixel point in each reference image sample and the corresponding first estimated image, including:

[0121] For each pixel in the reference image sample, determine a first difference between a pixel value of a corresponding pixel in the first estimated image and the pixel value of the pixel; determine whether the first difference is less than a preset difference threshold, and if so, determine a first product between the square of the first difference and a preset value as the loss value corresponding to the pixel; if not, determine a second difference between the absolute value of the first difference and the preset value as the loss value corresponding to the pixel;

[0122] A first sub-loss value is determined according to a first average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

[0123] Since the smaller the difference between the corresponding pixel points in the reference image sample and the corresponding first estimated image is, the more accurate the current image recognition model is, and the smaller the first sub-loss value is.

[0124] Therefore, in order to determine the first sub-loss value, in the present application, for each pixel point in the reference image sample, the first difference between the pixel value of the corresponding pixel point in the first estimated image and the pixel value of the pixel point can be determined, and a difference threshold is also pre-set in the electronic device. The first difference and the preset difference threshold are compared, and the loss value corresponding to the pixel point is determined based on the comparison result.

[0125] If it is determined that the first difference is less than the preset difference threshold, the first product between the square of the first difference and the preset value is determined as the loss value corresponding to the pixel, wherein the preset value may be 0.6, 0.5, etc. Specifically, the preset value may be set as required. In the present application, the preset value is 0.5. If it is determined that the first difference is not less than the preset difference threshold, the second difference between the absolute value of the first difference and the preset value is determined as the loss value corresponding to the pixel.

[0126] After determining the loss value corresponding to each pixel in each reference image sample, the sum of the loss values ​​corresponding to each pixel in each reference image sample can be determined as the first sub-loss value. In the present application, the first average loss value of the loss values ​​corresponding to each pixel in each reference image sample can also be determined as the first sub-loss value.

[0127] Specifically, the first sub-loss value can be determined according to the following formula:

[0128]

[0129]

[0130] Among them, L1 is the first sub-loss value, is the loss value corresponding to the i-th pixel in the k-th reference image sample, is the pixel value of the pixel corresponding to the i-th pixel in the k-th reference image sample on the first estimated image, I i,k is the pixel value corresponding to the i-th pixel in the k-th reference image sample, N is the total number of pixels contained in the reference image sample, which is also the total number of pixels contained in the first estimated image, and M is the total number of sample image pairs contained in the sample image set.

[0131] Embodiment 8:

[0132] In order to accurately determine the second sub-loss value, based on the above embodiments, in the embodiment of the present application, the second sub-loss value is determined according to the pixel value of the corresponding pixel point in each reference image sample and the corresponding second estimated image, including:

[0133] For each pixel in the reference image sample, determine a third difference between the pixel value of the corresponding pixel in the second estimated image and the pixel value of the pixel; determine whether the third difference is less than a preset difference threshold, and if so, determine the second product between the square of the third difference and a preset value as the loss value corresponding to the pixel; if not, determine the fourth difference between the absolute value of the third difference and the preset value as the loss value corresponding to the pixel;

[0134] A second sub-loss value is determined according to a second average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

[0135] Since the smaller the difference between the corresponding pixels in the reference image sample and the corresponding second estimated image is, the more accurate the current image recognition model is, and the smaller the second sub-loss value is.

[0136] Therefore, in order to determine the second sub-loss value, in the present application, for each pixel in the reference image sample, the third difference between the pixel value of the corresponding pixel in the second estimated image and the pixel value of the pixel can be determined, and the third difference and the preset difference threshold can be compared, and the loss value corresponding to the pixel can be determined based on the comparison result.

[0137] If it is determined that the third difference is less than the preset difference threshold, the second product between the square of the third difference and the preset value is determined as the loss value corresponding to the pixel, wherein the preset value may be 0.6, 0.5, etc. Specifically, the preset value may be set as required. In the present application, the preset value is 0.5. If it is determined that the third difference is not less than the preset difference threshold, the fourth difference between the absolute value of the third difference and the preset value is determined as the loss value corresponding to the pixel.

[0138] After determining the loss value corresponding to each pixel in each reference image sample, the sum of the loss values ​​corresponding to each pixel in each reference image sample can be determined as the second sub-loss value. In the present application, the first average loss value of the loss value corresponding to each pixel in each reference image sample can also be determined as the second sub-loss value.

[0139] Specifically, the second sub-loss value is determined according to the following formula:

[0140]

[0141]

[0142] Among them, L2 is the second sub-loss value, is the loss value corresponding to the i-th pixel in the k-th reference image sample, is the pixel value of the pixel corresponding to the i-th pixel in the k-th reference image sample on the second estimated image, I i,k is the pixel value corresponding to the i-th pixel in the k-th reference image sample, N is the total number of pixels contained in the reference image sample, which is also the total number of pixels contained in the second estimated image, and M is the total number of sample image pairs contained in the sample image set.

[0143] Embodiment 9:

[0144] In order to accurately determine the third sub-loss value, based on the above embodiments, in the embodiment of the present application, determining the third sub-loss value according to the first feature vector and the second feature vector corresponding to each sample image pair includes:

[0145] For each sample image pair, determine a first dot product of a first eigenvector and a second eigenvector corresponding to the sample image pair; and determine a second dot product of the first eigenvector corresponding to the sample image pair and second eigenvectors corresponding to other sample image pairs except the sample image pair, and determine a sum value corresponding to the second dot product;

[0146] A third sub-loss value is determined according to a ratio of the first dot product to the sum value.

[0147] In the present application, for each sample image pair, since the reference image sample and the target image sample in the sample image pair belong to the same video, and the reference image sample in the sample image pair and the target image samples in other sample image pairs except the sample image pair do not belong to the same video, the closer the first eigenvector and the second eigenvector corresponding to the sample image pair are, and the greater the difference between the first eigenvector corresponding to the sample image pair and the second eigenvector corresponding to other sample image pairs except the sample image pair, the more accurate the current image recognition model is, and the smaller the third sub-loss value is.

[0148] In the present application, in order to determine the third sub-loss value, for each sample image pair, the first dot product of the first eigenvector and the second eigenvector corresponding to the sample image pair can be determined, and the second dot product of the first eigenvector corresponding to the sample image pair and the second eigenvector corresponding to other sample image pairs except the sample image pair can be determined, and the sum value corresponding to each second dot product can be determined, and the third sub-loss value can be determined based on the ratio of the first dot product to the sum value.

[0149] In this application, the third sub-loss value can be determined according to the following formula:

[0150]

[0151]

[0152] Among them, L3 is the third sub-loss value, r i is the first eigenvector corresponding to the kth reference image sample, t i is the second eigenvector corresponding to the kth target image sample, wherein the kth reference image sample and the kth target image sample belong to the kth sample image pair, t j is the second eigenvector corresponding to the j-th target image sample, wherein the k-th reference image sample and the j-th target image sample do not belong to the same sample image pair, and M is the total number of sample image pairs included in the sample image set.

[0153] In the present application, after determining the first dot product and the second dot product, the third sub-loss value may be determined according to the following formula:

[0154]

[0155]

[0156] Among them, L3 is the third sub-loss value, r i is the first eigenvector corresponding to the kth reference image sample, t i is the second eigenvector corresponding to the kth target image sample, wherein the kth reference image sample and the kth target image sample belong to the kth sample image pair, t j is the second eigenvector corresponding to the j-th target image sample, wherein the k-th reference image sample and the j-th target image sample do not belong to the same sample image pair, and M is the total number of sample image pairs included in the sample image set.

[0157] Embodiment 10:

[0158] Figure 2 A schematic diagram of the structure of an image recognition model training device provided in some embodiments of the present application, the device comprising:

[0159] The processing module 201 is used to input each sample image pair in the sample image set into the original image recognition model in sequence, output the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair, and output the second feature vector corresponding to the target image sample in each sample image pair; wherein the reference image sample and the target image sample in the same sample image pair belong to the same video; and the reference image samples in different sample image pairs belong to different videos;

[0160] The determination module 202 is used to determine a first sub-loss value according to each reference image sample and a pixel value of a corresponding pixel point in the corresponding first estimated image; determine a second sub-loss value according to each reference image sample and a pixel value of a corresponding pixel point in the corresponding second estimated image; determine a third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair; determine a target loss value according to the first sub-loss value, the second sub-loss value and the third sub-loss value;

[0161] The training module 203 is used to train the original image recognition model according to the target loss value.

[0162] In a possible implementation, the determination module 202 is further used to obtain, for each pre-saved video, a reference image and a target image belonging to the video; input the reference image and the target image into a pre-trained foreground area acquisition model, and output a first foreground image corresponding to the reference image and a second foreground image corresponding to the target image; for each foreground image in the first foreground image and the second foreground image, determine a sub-foreground image corresponding to an object contained in the foreground image; determine the sub-foreground image of the first foreground image as a reference image sample in a sample image pair, and determine the sub-foreground image of the second foreground image as a target image sample in the sample image pair.

[0163] In a possible implementation, the processing module 201 is specifically used for the original image recognition model to perform a convolution operation on the reference image sample and the target image sample in each input sample image pair to determine the feature vector corresponding to each pixel; for each sample image pair, for each first pixel in the reference image sample in the sample image pair, according to the second pixel corresponding to the first pixel in the target image sample in the sample image pair, and a preset range, determine each third pixel within the set range including the second pixel; construct a first similarity matrix according to the similarity between the feature vectors of the first pixel and each third pixel; determine a first weight value corresponding to each third pixel according to the first similarity matrix and a preset normalization function, and determine a first estimated pixel value of the first pixel according to the pixel value of each third pixel and the corresponding first weight value; determine a first estimated image corresponding to the reference image sample according to the first estimated pixel value of each first pixel in the reference image sample.

[0164] In a possible implementation, the processing module 201 is specifically used to determine, for each sample image pair and for each first pixel in a reference image sample in the sample image pair, each fifth pixel within the set range including the fourth pixel based on the target image sample in the sample image pair, a preset number of fourth pixels corresponding to the target image samples in other sample image pairs other than the sample image pair, and a preset range; construct a second similarity matrix based on the similarity between the feature vectors of the first pixel and each fifth pixel; determine a second weight value corresponding to each fifth pixel based on the second similarity matrix and the preset normalization function, and determine a second estimated pixel value of the first pixel based on the pixel value of each fifth pixel and the corresponding second weight value; and determine a second estimated image corresponding to the reference image sample based on the second estimated pixel value of each first pixel in the reference image sample.

[0165] In a possible implementation, the processing module 201 is specifically used for the original image recognition model to sample the reference image sample in each sample image pair input to obtain a target feature block; and to perform pooling processing on the target feature block to obtain a first feature vector corresponding to the reference image sample.

[0166] In a possible implementation, the processing module 201 is specifically used for the original image recognition model to determine, for each sixth pixel point contained in the target feature block, according to the position of the sixth pixel point in the reference image sample, each seventh pixel point within a preset range containing the position in the target sample image; construct a third similarity matrix according to the similarity between the feature vector of the sixth pixel point and the feature vector corresponding to each seventh pixel point; determine the third weight corresponding to each seventh pixel point according to the third similarity matrix and the preset normalization function; determine the estimated position corresponding to the sixth pixel point in the target image sample according to the third weight corresponding to each seventh pixel point and the third position information corresponding to each seventh pixel point; determine the average estimated position according to the estimated position corresponding to each sixth pixel point, and determine the corresponding estimated feature block on the target image sample according to the average estimated position and the size of the target feature block; perform a pooling operation on the estimated feature block to obtain a second feature vector corresponding to the target image sample.

[0167] In a possible implementation, the determination module 202 is specifically used to determine, for each pixel in the reference image sample, a first difference between a pixel value of a corresponding pixel in the first estimated image and the pixel value of the pixel; determine whether the first difference is less than a preset difference threshold, and if so, determine the first product between the square of the first difference and a preset value as the loss value corresponding to the pixel; if not, determine the second difference between the absolute value of the first difference and the preset value as the loss value corresponding to the pixel; and determine a first sub-loss value based on a first average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

[0168] In a possible implementation, the determination module 202 is specifically used to determine, for each pixel in the reference image sample, a third difference between the pixel value of the corresponding pixel in the second estimated image and the pixel value of the pixel; determine whether the third difference is less than a preset difference threshold, and if so, determine the second product between the square of the third difference and the preset value as the loss value corresponding to the pixel; if not, determine the fourth difference between the absolute value of the third difference and the preset value as the loss value corresponding to the pixel; and determine the second sub-loss value based on the second average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

[0169] In a possible implementation, the determination module 202 is specifically used to determine, for each sample image pair, a first dot product of a first eigenvector and a second eigenvector corresponding to the sample image pair; and determine a second dot product of the first eigenvector corresponding to the sample image pair and second eigenvectors corresponding to other sample image pairs except the sample image pair, and determine a sum value corresponding to the second dot product; and determine a third sub-loss value based on a ratio of the first dot product to the sum value.

[0170] Embodiment 11:

[0171] Based on the above embodiments, some embodiments of the present application further provide an electronic device, such as Figure 3 As shown, it includes: a processor 301 , a communication interface 302 , a memory 303 and a communication bus 304 , wherein the processor 301 , the communication interface 302 , and the memory 303 communicate with each other via the communication bus 304 .

[0172] The memory 303 stores a computer program. When the program is executed by the processor 301, the processor 301 performs the following steps:

[0173] Input each sample image pair in the sample image set into the original image recognition model in sequence, output the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair, and output the second feature vector corresponding to the target image sample in each sample image pair; wherein the reference image sample and the target image sample in the same sample image pair belong to the same video; and the reference image samples in different sample image pairs belong to different videos;

[0174] Determine a first sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image; determine a second sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image; determine a third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair; determine a target loss value according to the first sub-loss value, the second sub-loss value and the third sub-loss value;

[0175] The original image recognition model is trained according to the target loss value.

[0176] Furthermore, the processor 301 is also used to obtain a reference image and a target image belonging to each pre-saved video; input the reference image and the target image into a pre-trained foreground area acquisition model, and output a first foreground image corresponding to the reference image and a second foreground image corresponding to the target image; for each foreground image in the first foreground image and the second foreground image, determine a sub-foreground image corresponding to an object contained in the foreground image; determine the sub-foreground image of the first foreground image as a reference image sample in a sample image pair, and determine the sub-foreground image of the second foreground image as a target image sample in the sample image pair.

[0177] Furthermore, the processor 301 is also used for the original image recognition model to perform convolution operations on the reference image sample and the target image sample in each input sample image pair to determine the feature vector corresponding to each pixel; for each sample image pair, for each first pixel in the reference image sample in the sample image pair, according to the second pixel corresponding to the first pixel in the target image sample in the sample image pair, and a preset range, determine each third pixel within the set range including the second pixel; construct a first similarity matrix based on the similarity between the feature vectors of the first pixel and each third pixel; determine the first weight value corresponding to each third pixel based on the first similarity matrix and a preset normalization function, and determine the first estimated pixel value of the first pixel based on the pixel value of each third pixel and the corresponding first weight value; determine the first estimated image corresponding to the reference image sample based on the first estimated pixel value of each first pixel in the reference image sample.

[0178] Furthermore, the processor 301 is also used to determine, for each sample image pair and for each first pixel in the reference image sample in the sample image pair, each fifth pixel within the set range including the fourth pixel based on the target image sample in the sample image pair, a preset number of fourth pixels corresponding to the target image samples in other sample image pairs other than the sample image pair, and a preset range; construct a second similarity matrix based on the similarity between the feature vectors of the first pixel and each fifth pixel; determine the second weight value corresponding to each fifth pixel based on the second similarity matrix and the preset normalization function, and determine the second estimated pixel value of the first pixel based on the pixel value of each fifth pixel and the corresponding second weight value; and determine the second estimated image corresponding to the reference image sample based on the second estimated pixel value of each first pixel in the reference image sample.

[0179] Furthermore, the processor 301 is also used for the original image recognition model to sample the reference image sample in each sample image pair input to obtain a target feature block; and to perform pooling processing on the target feature block to obtain a first feature vector corresponding to the reference image sample.

[0180] Furthermore, the processor 301 is also used for the original image recognition model to determine, for each sixth pixel point contained in the target feature block, according to the position of the sixth pixel point in the reference image sample, each seventh pixel point within a preset range containing the position in the target sample image; construct a third similarity matrix according to the similarity between the feature vector of the sixth pixel point and the feature vector corresponding to each seventh pixel point; determine the third weight corresponding to each seventh pixel point according to the third similarity matrix and the preset normalization function; determine the estimated position corresponding to the sixth pixel point in the target image sample according to the third weight corresponding to each seventh pixel point and the third position information corresponding to each seventh pixel point; determine the average estimated position according to the estimated position corresponding to each sixth pixel point, and determine the corresponding estimated feature block on the target image sample according to the average estimated position and the size of the target feature block; and perform a pooling operation on the estimated feature block to obtain a second feature vector corresponding to the target image sample.

[0181] Furthermore, the processor 301 is also used to determine, for each pixel point in the reference image sample, a first difference between a pixel value of a corresponding pixel point in the first estimated image and the pixel value of the pixel point; determine whether the first difference is less than a preset difference threshold, and if so, determine the first product between the square of the first difference and a preset value as the loss value corresponding to the pixel point; if not, determine the second difference between the absolute value of the first difference and the preset value as the loss value corresponding to the pixel point; and determine a first sub-loss value based on a first average loss value of the loss values ​​corresponding to each pixel point in the reference image sample.

[0182] Furthermore, the processor 301 is also used to determine, for each pixel in the reference image sample, a third difference between the pixel value of the corresponding pixel in the second estimated image and the pixel value of the pixel; determine whether the third difference is less than a preset difference threshold, and if so, determine the second product between the square of the third difference and the preset value as the loss value corresponding to the pixel; if not, determine the fourth difference between the absolute value of the third difference and the preset value as the loss value corresponding to the pixel; and determine the second sub-loss value based on the second average loss value of the loss value corresponding to each pixel in the reference image sample.

[0183] Furthermore, the processor 301 is also used to determine, for each sample image pair, a first dot product of a first eigenvector and a second eigenvector corresponding to the sample image pair; and determine a second dot product of the first eigenvector corresponding to the sample image pair and second eigenvectors corresponding to other sample image pairs except the sample image pair, and determine a sum value corresponding to the second dot product; and determine a third sub-loss value based on a ratio of the first dot product to the sum value.

[0184] The communication bus mentioned in the above server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0185] The communication interface 302 is used for communication between the electronic device and other devices.

[0186] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0187] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0188] Embodiment 12:

[0189] On the basis of the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by an electronic device, and when the program is run on the electronic device, the electronic device implements the following steps when executing:

[0190] The memory stores a computer program, and when the program is executed by the processor, the processor performs the following steps:

[0191] Input each sample image pair in the sample image set into the original image recognition model in sequence, output the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair, and output the second feature vector corresponding to the target image sample in each sample image pair; wherein the reference image sample and the target image sample in the same sample image pair belong to the same video; and the reference image samples in different sample image pairs belong to different videos;

[0192] Determine a first sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image; determine a second sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image; determine a third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair; determine a target loss value according to the first sub-loss value, the second sub-loss value and the third sub-loss value;

[0193] The original image recognition model is trained according to the target loss value.

[0194] Further, determining each sample image pair in the sample image set includes:

[0195] For each pre-saved video, a reference image and a target image belonging to the video are obtained; the reference image and the target image are input into a pre-trained foreground region acquisition model, and a first foreground image corresponding to the reference image and a second foreground image corresponding to the target image are output; for each foreground image in the first foreground image and the second foreground image, a sub-foreground image corresponding to an object contained in the foreground image is determined;

[0196] The sub-foreground image of the first foreground image is determined as a reference image sample in a sample image pair, and the sub-foreground image of the second foreground image is determined as a target image sample in the sample image pair.

[0197] Furthermore, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting a first estimated image corresponding to the reference image sample in each sample image pair includes:

[0198] The original image recognition model performs a convolution operation on the reference image sample and the target image sample in each input sample image pair to determine the feature vector corresponding to each pixel; for each sample image pair, for each first pixel in the reference image sample in the sample image pair, according to the second pixel corresponding to the first pixel in the target image sample in the sample image pair, and a preset range, determine each third pixel within the set range including the second pixel; construct a first similarity matrix based on the similarity between the feature vectors of the first pixel and each third pixel; determine the first weight value corresponding to each third pixel based on the first similarity matrix and a preset normalization function, and determine the first estimated pixel value of the first pixel based on the pixel value of each third pixel and the corresponding first weight value; determine the first estimated image corresponding to the reference image sample based on the first estimated pixel value of each first pixel in the reference image sample.

[0199] Furthermore, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting a second estimated image corresponding to the reference image sample in each sample image pair includes:

[0200] For each sample image pair, for each first pixel point in the reference image sample in the sample image pair, determine each fifth pixel point within the set range including the fourth pixel point based on the target image sample in the sample image pair, a preset number of fourth pixel points corresponding to the target image samples in other sample image pairs other than the sample image pair, and a preset range; construct a second similarity matrix based on the similarity between the feature vectors of the first pixel point and each fifth pixel point; determine the second weight value corresponding to each fifth pixel point based on the second similarity matrix and the preset normalization function, and determine the second estimated pixel value of the first pixel point based on the pixel value of each fifth pixel point and the corresponding second weight value; determine the second estimated image corresponding to the reference image sample based on the second estimated pixel value of each first pixel point in the reference image sample.

[0201] Furthermore, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting the first feature vector corresponding to the reference image sample in each sample image pair includes:

[0202] The original image recognition model samples the reference image sample in each input sample image pair to obtain a target feature block; and performs pooling processing on the target feature block to obtain a first feature vector corresponding to the reference image sample.

[0203] Furthermore, each sample image pair in the sample image set is sequentially input into the original image recognition model, and outputting the second feature vector corresponding to the target image sample in each sample image pair includes:

[0204] The original image recognition model determines, for each sixth pixel point included in the target feature block, each seventh pixel point within a preset range that includes the position in the target sample image according to the position of the sixth pixel point in the reference image sample; constructs a third similarity matrix according to the similarity between the feature vector of the sixth pixel point and the feature vector corresponding to each seventh pixel point; determines a third weight corresponding to each seventh pixel point according to the third similarity matrix and the preset normalization function; determines an estimated position corresponding to the sixth pixel point in the target image sample according to the third weight corresponding to each seventh pixel point and the third position information corresponding to each seventh pixel point;

[0205] Determine an average estimated position according to the estimated position corresponding to each sixth pixel point, and determine the corresponding estimated feature block on the target image sample according to the average estimated position and the target feature block size;

[0206] The estimated feature map blocks are pooled to obtain a second feature vector corresponding to the target image sample.

[0207] Further, determining the first sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image includes:

[0208] For each pixel in the reference image sample, determine a first difference between a pixel value of a corresponding pixel in the first estimated image and the pixel value of the pixel; determine whether the first difference is less than a preset difference threshold, and if so, determine a first product between the square of the first difference and a preset value as the loss value corresponding to the pixel; if not, determine a second difference between the absolute value of the first difference and the preset value as the loss value corresponding to the pixel;

[0209] A first sub-loss value is determined according to a first average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

[0210] Further, determining the second sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image includes:

[0211] For each pixel in the reference image sample, determine a third difference between the pixel value of the corresponding pixel in the second estimated image and the pixel value of the pixel; determine whether the third difference is less than a preset difference threshold, and if so, determine the second product between the square of the third difference and a preset value as the loss value corresponding to the pixel; if not, determine the fourth difference between the absolute value of the third difference and the preset value as the loss value corresponding to the pixel;

[0212] A second sub-loss value is determined according to a second average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

[0213] Further, determining the third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair includes:

[0214] For each sample image pair, determine a first dot product of a first eigenvector and a second eigenvector corresponding to the sample image pair; and determine a second dot product of the first eigenvector corresponding to the sample image pair and second eigenvectors corresponding to other sample image pairs except the sample image pair, and determine a sum value corresponding to the second dot product;

[0215] A third sub-loss value is determined according to a ratio of the first dot product to the sum value.

[0216] Since in the embodiment of the present application, the original image recognition model outputs the first estimated image and the second estimated image corresponding to the reference image sample in each sample image pair, and determines the first sub-loss value according to the pixel value of the corresponding pixel point in the first estimated image, and determines the second loss value according to the pixel value of the corresponding pixel point in the second estimated image, and performs pixel-level comparative learning, the original image recognition model also outputs the first eigenvector corresponding to the reference image sample in each sample image pair and the second eigenvector corresponding to the target image sample in each sample image pair, and determines the third sub-loss value according to the first eigenvector and the second eigenvector, and performs image-level comparative learning. Since both pixel-level comparative learning and image-level comparative learning are performed, the accuracy of the image recognition model is improved, and the significance or distinguishability of the object features in the image extracted by the image recognition model is improved.

[0217] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0218] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0219] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0220] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0221] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for training an image recognition model, characterized in that: The method comprises: Input each sample image pair in the sample image set into the original image recognition model in turn, output the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair, and output the second feature vector corresponding to the target image sample in each sample image pair; wherein the first estimated image is obtained by performing pixel-level contrast learning based on the reference image sample and the target image sample in the sample image pair; the second estimated image is obtained by performing pixel-level contrast learning based on the reference image sample in the sample image pair, the target image sample in the sample image pair and the target sample images in other sample image pairs except the sample image; the first feature vector is a feature vector estimated to characterize the reference image sample; the second feature vector is a feature vector estimated to characterize the target image sample by performing image-level contrast learning based on the reference image sample and the target sample image in the sample image pair; the reference image sample and the target image sample in the same sample image pair belong to the same video; the reference image samples in different sample image pairs belong to different videos; Determine a first sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image; determine a second sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image; determine a third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair; determine a target loss value according to the first sub-loss value, the second sub-loss value and the third sub-loss value; The original image recognition model is trained according to the target loss value.

2. The method according to claim 1, characterized in that Determining each sample image pair in the sample image set comprises: For each pre-saved video, a reference image and a target image belonging to the video are obtained; the reference image and the target image are input into a pre-trained foreground region acquisition model, and a first foreground image corresponding to the reference image and a second foreground image corresponding to the target image are output; for each foreground image in the first foreground image and the second foreground image, a sub-foreground image corresponding to an object contained in the foreground image is determined; The sub-foreground image of the first foreground image is determined as a reference image sample in a sample image pair, and the sub-foreground image of the second foreground image is determined as a target image sample in the sample image pair.

3. The method according to claim 1, characterized in that Inputting each sample image pair in the sample image set into the original image recognition model in sequence, and outputting the first estimated image corresponding to the reference image sample in each sample image pair comprises: The original image recognition model performs a convolution operation on the reference image sample and the target image sample in each input sample image pair to determine the feature vector corresponding to each pixel; for each sample image pair, for each first pixel in the reference image sample in the sample image pair, according to the second pixel corresponding to the first pixel in the target image sample in the sample image pair, and a preset range, determine each third pixel within the preset range containing the second pixel; construct a first similarity matrix based on the similarity between the feature vectors of the first pixel and each third pixel; determine the first weight value corresponding to each third pixel based on the first similarity matrix and a preset normalization function, and determine the first estimated pixel value of the first pixel based on the pixel value of each third pixel and the corresponding first weight value; determine the first estimated image corresponding to the reference image sample based on the first estimated pixel value of each first pixel in the reference image sample.

4. The method according to claim 3, characterized in that Inputting each sample image pair in the sample image set into the original image recognition model in sequence, and outputting a second estimated image corresponding to the reference image sample in each sample image pair comprises: For each sample image pair, for each first pixel point in the reference image sample in the sample image pair, determine each fifth pixel point within the preset range including the fourth pixel point based on the target image sample in the sample image pair, a preset number of fourth pixel points corresponding to the target image samples in other sample image pairs other than the sample image pair, and a preset range; construct a second similarity matrix based on the similarity between the feature vectors of the first pixel point and each fifth pixel point; determine the second weight value corresponding to each fifth pixel point based on the second similarity matrix and the preset normalization function, and determine the second estimated pixel value of the first pixel point based on the pixel value of each fifth pixel point and the corresponding second weight value; determine the second estimated image corresponding to the reference image sample based on the second estimated pixel value of each first pixel point in the reference image sample.

5. The method according to claim 1, characterized in that Inputting each sample image pair in the sample image set into the original image recognition model in turn, and outputting the first feature vector corresponding to the reference image sample in each sample image pair includes: The original image recognition model samples the reference image sample in each input sample image pair to obtain a target feature block; and performs pooling processing on the target feature block to obtain a first feature vector corresponding to the reference image sample.

6. The method according to claim 5, characterized in that Inputting each sample image pair in the sample image set into the original image recognition model in turn, and outputting the second feature vector corresponding to the target image sample in each sample image pair includes: The original image recognition model determines, for each sixth pixel point included in the target feature block, each seventh pixel point within a preset range that includes the position in the target sample image according to the position of the sixth pixel point in the reference image sample; constructs a third similarity matrix according to the similarity between the feature vector of the sixth pixel point and the feature vector corresponding to each seventh pixel point; determines a third weight corresponding to each seventh pixel point according to the third similarity matrix and the preset normalization function; determines an estimated position corresponding to the sixth pixel point in the target image sample according to the third weight corresponding to each seventh pixel point and the third position information corresponding to each seventh pixel point; Determine an average estimated position according to the estimated position corresponding to each sixth pixel point, and determine the corresponding estimated feature block on the target image sample according to the average estimated position and the target feature block size; The estimated feature map blocks are pooled to obtain a second feature vector corresponding to the target image sample.

7. The method according to claim 1, characterized in that The determining of the first sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding first estimated image comprises: For each pixel in the reference image sample, determine a first difference between a pixel value of a corresponding pixel in the first estimated image and the pixel value of the pixel; determine whether the first difference is less than a preset difference threshold, and if so, determine a first product between the square of the first difference and a preset value as the loss value corresponding to the pixel; if not, determine a second difference between the absolute value of the first difference and the preset value as the loss value corresponding to the pixel; A first sub-loss value is determined according to a first average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

8. The method according to claim 1, characterized in that The determining the second sub-loss value according to each reference image sample and the pixel value of the corresponding pixel point in the corresponding second estimated image comprises: For each pixel in the reference image sample, determine a third difference between the pixel value of the corresponding pixel in the second estimated image and the pixel value of the pixel; determine whether the third difference is less than a preset difference threshold, and if so, determine the second product between the square of the third difference and a preset value as the loss value corresponding to the pixel; if not, determine the fourth difference between the absolute value of the third difference and the preset value as the loss value corresponding to the pixel; A second sub-loss value is determined according to a second average loss value of the loss values ​​corresponding to each pixel in the reference image sample.

9. The method according to claim 1, characterized in that: The determining the third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair comprises: For each sample image pair, determine a first dot product of a first eigenvector and a second eigenvector corresponding to the sample image pair; and determine a second dot product of the first eigenvector corresponding to the sample image pair and second eigenvectors corresponding to other sample image pairs except the sample image pair, and determine a sum value corresponding to the second dot product; A third sub-loss value is determined according to a ratio of the first dot product to the sum value.

10. An image recognition model training device, characterized in that: The device comprises: A processing module, used for inputting each sample image pair in the sample image set into the original image recognition model in sequence, outputting the first estimated image, the second estimated image and the first feature vector corresponding to the reference image sample in each sample image pair, and outputting the second feature vector corresponding to the target image sample in each sample image pair; wherein the first estimated image is obtained by performing pixel-level contrast learning based on the reference image sample and the target image sample in the sample image pair; the second estimated image is obtained by performing pixel-level contrast learning based on the reference image sample in the sample image pair, the target image sample in the sample image pair and the target sample images in other sample image pairs except the sample image; the first feature vector is a feature vector estimated to characterize the reference image sample; the second feature vector is a feature vector estimated to characterize the target image sample by performing image-level contrast learning based on the reference image sample and the target sample image in the sample image pair; the reference image sample and the target image sample in the same sample image pair belong to the same video; the reference image samples in different sample image pairs belong to different videos; A determination module, configured to determine a first sub-loss value according to each reference image sample and a pixel value of a corresponding pixel point in the corresponding first estimated image; determine a second sub-loss value according to each reference image sample and a pixel value of a corresponding pixel point in the corresponding second estimated image; determine a third sub-loss value according to the first eigenvector and the second eigenvector corresponding to each sample image pair; determine a target loss value according to the first sub-loss value, the second sub-loss value, and the third sub-loss value; A training module is used to train the original image recognition model according to the target loss value.

11. The device according to claim 10, characterized in that The determination module is also used to obtain a reference image and a target image belonging to each pre-saved video; input the reference image and the target image into a pre-trained foreground area acquisition model, and output a first foreground image corresponding to the reference image and a second foreground image corresponding to the target image; for each foreground image in the first foreground image and the second foreground image, determine a sub-foreground image corresponding to an object contained in the foreground image; determine the sub-foreground image of the first foreground image as a reference image sample in a sample image pair, and determine the sub-foreground image of the second foreground image as a target image sample in the sample image pair.

12. The device according to claim 10, characterized in that The processing module is specifically used for the original image recognition model to perform a convolution operation on the reference image sample and the target image sample in each input sample image pair to determine the feature vector corresponding to each pixel; for each sample image pair, for each first pixel in the reference image sample in the sample image pair, according to the second pixel corresponding to the first pixel in the target image sample in the sample image pair, and a preset range, determine each third pixel within the preset range containing the second pixel; and construct a first similarity matrix according to the similarity between the feature vectors of the first pixel and each third pixel; Determine a first weight value corresponding to each third pixel point according to the first similarity matrix and a preset normalization function, and determine a first estimated pixel value of the first pixel point according to the pixel value of each third pixel point and the corresponding first weight value; A first estimated image corresponding to the reference image sample is determined according to the first estimated pixel value of each first pixel point in the reference image sample.

13. The device according to claim 12, characterized in that The processing module is specifically used to determine, for each sample image pair and for each first pixel point in the reference image sample in the sample image pair, each fifth pixel point within the preset range including the fourth pixel point according to the target image sample in the sample image pair, a preset number of fourth pixel points corresponding to the target image samples in other sample image pairs other than the sample image pair, and a preset range; and construct a second similarity matrix according to the similarity between the feature vectors of the first pixel point and each fifth pixel point; Determine a second weight value corresponding to each fifth pixel point according to the second similarity matrix and the preset normalization function, and determine a second estimated pixel value of the first pixel point according to the pixel value of each fifth pixel point and the corresponding second weight value; A second estimated image corresponding to the reference image sample is determined according to the second estimated pixel value of each first pixel point in the reference image sample.

14. The device according to claim 10, characterized in that The processing module is specifically used for the original image recognition model to sample the reference image sample in each sample image pair input to obtain a target feature block; and to perform pooling processing on the target feature block to obtain a first feature vector corresponding to the reference image sample.

15. The device according to claim 14, characterized in that The processing module is specifically used for the original image recognition model to determine, for each sixth pixel point contained in the target feature block, according to the position of the sixth pixel point in the reference image sample, each seventh pixel point within a preset range containing the position in the target sample image; construct a third similarity matrix according to the similarity between the feature vector of the sixth pixel point and the feature vector corresponding to each seventh pixel point; determine the third weight corresponding to each seventh pixel point according to the third similarity matrix and the preset normalization function; determine the estimated position corresponding to the sixth pixel point in the target image sample according to the third weight corresponding to each seventh pixel point and the third position information corresponding to each seventh pixel point; determine the average estimated position according to the estimated position corresponding to each sixth pixel point, and determine the corresponding estimated feature block on the target image sample according to the average estimated position and the size of the target feature block; and perform a pooling operation on the estimated feature block to obtain a second feature vector corresponding to the target image sample.

16. The device according to claim 10, characterized in that The determination module is specifically used to determine, for each pixel point in the reference image sample, a first difference between the pixel value of the corresponding pixel point in the first estimated image and the pixel value of the pixel point; determine whether the first difference is less than a preset difference threshold, and if so, determine the first product between the square of the first difference and a preset value as the loss value corresponding to the pixel point; if not, determine the second difference between the absolute value of the first difference and the preset value as the loss value corresponding to the pixel point; and determine a first sub-loss value based on a first average loss value of the loss values ​​corresponding to each pixel point in the reference image sample.

17. The device according to claim 10, characterized in that The determination module is specifically used to determine, for each pixel point in the reference image sample, a third difference between the pixel value of the corresponding pixel point in the second estimated image and the pixel value of the pixel point; determine whether the third difference is less than a preset difference threshold, and if so, determine the second product between the square of the third difference and the preset value as the loss value corresponding to the pixel point; if not, determine the fourth difference between the absolute value of the third difference and the preset value as the loss value corresponding to the pixel point; and determine the second sub-loss value based on the second average loss value of the loss value corresponding to each pixel point in the reference image sample.

18. The device according to claim 10, characterized in that The determination module is specifically used to determine, for each sample image pair, a first dot product of a first eigenvector and a second eigenvector corresponding to the sample image pair; and determine a second dot product of a first eigenvector corresponding to the sample image pair and second eigenvectors corresponding to other sample image pairs except the sample image pair, and determine a sum value corresponding to the second dot product; and determine a third sub-loss value based on a ratio of the first dot product to the sum value.

19. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory is used to store program instructions, and the processor is used to implement the steps of the image recognition model training method described in any one of claims 1 to 9 when executing the computer program stored in the memory.

20. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the steps of the image recognition model training method described in any one of claims 1 to 9.