Image feature extraction model training method and device, equipment and storage medium

Abstract

Embodiments of the present application disclose a kind of training method, device and equipment of image feature extraction model and storage medium, wherein the method comprises: obtaining at least two image sample groups;In any image sample group, based on the sample image of different spatial domain in image sample group, enhanced sample image is obtained by image fusion, and image sample group is updated based on enhanced sample image, and the enhanced training sample group corresponding to each image sample group is obtained;In each iterative training process of image feature extraction model, based on image feature extraction model, sample image in any two groups of enhanced training sample group is extracted, and the loss function is determined based on the image feature corresponding to sample image in any two groups of the enhanced training sample group, the parameter of the image feature extraction model of current iteration is adjusted based on loss function, until target image feature extraction model is obtained.Through the above technical solution, the feature extraction precision of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a training method, apparatus, device and storage medium for an image feature extraction model. Background Technology

[0002] Currently, image processing technology, as an effective means of extracting useful information from images, is widely used in various application scenarios. In many cases, feature extraction is required to capture information of interest from rich image data. With the rapid development of artificial intelligence technology, various neural network methods have been applied to improve the image feature extraction effect.

[0003] However, traditional methods for image feature extraction using neural network models have poor feature extraction accuracy for image groups with large spatial differences, which cannot meet user needs. Summary of the Invention

[0004] This invention provides a training method, apparatus, device, and storage medium for an image feature extraction model, in order to improve the feature extraction accuracy of the model.

[0005] In a first aspect, embodiments of the present invention provide a training method for an image feature extraction model, comprising:

[0006] At least two image sample groups are obtained, wherein each image sample group includes at least two sample images from different spatial domains;

[0007] Within any of the image sample groups, image fusion is performed based on sample images from different spatial domains in the image sample group to obtain an enhanced sample image, and the image sample group is updated based on the enhanced sample image to obtain an enhanced training sample group corresponding to each of the image sample groups.

[0008] In each iteration of training the image feature extraction model, image features are extracted from sample images in any two sets of the enhanced training sample groups based on the image feature extraction model, and a loss function is determined based on the image features corresponding to the sample images in the two sets of the enhanced training sample groups. The parameters of the image feature extraction model in the current iteration are adjusted based on the loss function until the target image feature extraction model is obtained.

[0009] Secondly, embodiments of the present invention also provide an image feature extraction method, including:

[0010] Obtain the image to be processed;

[0011] The image to be processed is input into the target image feature extraction model to obtain the feature extraction result;

[0012] The target image feature extraction model is trained using the image feature extraction model training method provided in any embodiment of the present invention.

[0013] Thirdly, embodiments of the present invention also provide a training apparatus for an image feature extraction model, comprising:

[0014] The sample acquisition module is used to acquire at least two image sample groups, wherein each image sample group includes at least two sample images from different spatial domains;

[0015] An enhanced sample determination module is used to perform image fusion based on sample images from different spatial domains in any of the image sample groups to obtain enhanced sample images, and to update the image sample groups based on the enhanced sample images to obtain enhanced training sample groups corresponding to each of the image sample groups.

[0016] The model training module is used to extract image features from sample images in any two sets of the enhanced training sample groups based on the image feature extraction model during each iteration of training the image feature extraction model, determine a loss function based on the image features corresponding to the sample images in the two sets of the enhanced training sample groups, and adjust the parameters of the image feature extraction model in the current iteration based on the loss function until the target image feature extraction model is obtained.

[0017] Fourthly, embodiments of the present invention also provide an image feature extraction apparatus, comprising:

[0018] The image acquisition module is used to acquire the image to be processed.

[0019] The feature extraction module is used to input the image to be processed into the target image feature extraction model to obtain the feature extraction result;

[0020] The target image feature extraction model is trained using the image feature extraction model training device provided in any embodiment of the present invention.

[0021] Fifthly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0022] One or more processors;

[0023] Storage device for storing one or more programs.

[0024] When the one or more programs are executed by the one or more processors, the one or more processors implement the training method or image feature extraction method of the image feature extraction model provided in any embodiment of the present invention.

[0025] In a sixth aspect, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method for the image feature extraction model or the image feature extraction method provided in any embodiment of the present invention.

[0026] The technical solution of this invention involves acquiring at least two image sample groups, each including at least two sample images from different spatial domains, indicating that the training samples include sample images from different spatial domains. Further, within any image sample group, image fusion is performed based on the sample images from different spatial domains to obtain an enhanced sample image. This enhanced sample image is then used to update the image sample group, resulting in enhanced training sample groups that are easier to learn the intrinsic relationships between samples. Furthermore, during each iteration of the image feature extraction model's training, image features are extracted from the sample images in any two enhanced training sample groups according to the model. A loss function is determined based on the image features corresponding to the sample images in any two enhanced training sample groups. The parameters of the image feature extraction model for the current iteration are adjusted based on the loss function until the target image feature extraction model is obtained. This training method enables the image feature extraction model to learn the intrinsic relationships between the features of sample images within and between different enhanced training sample groups, thereby improving the feature extraction accuracy of the image feature extraction model. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of exemplary embodiments of the present invention, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the drawings of the embodiments to be described in this invention, and not all of the drawings. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0028] Figure 1 This is a flowchart illustrating a training method for an image feature extraction model provided in Embodiment 1 of the present invention.

[0029] Figure 2 This is a flowchart illustrating a training method for an image feature extraction model provided in Embodiment 2 of the present invention.

[0030] Figure 3 This is a flowchart illustrating a training method for an image feature extraction model provided in Embodiment 3 of the present invention.

[0031] Figure 4 This is a schematic diagram of a loss calculation relationship provided in Embodiment 3 of the present invention;

[0032] Figure 5 This is a schematic flowchart of an image feature extraction method provided in Embodiment 4 of the present invention;

[0033] Figure 6 This is a schematic diagram of the structure of a training device for an image feature extraction model provided in Embodiment 5 of the present invention;

[0034] Figure 7 This is a schematic diagram of the structure of an image feature extraction device provided in Embodiment Six of the present invention;

[0035] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present invention. Detailed Implementation

[0036] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0037] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0038] Example 1

[0039] Figure 1 This is a flowchart illustrating a training method for an image feature extraction model provided in Embodiment 1 of the present invention. This embodiment is applicable to the training of feature extraction models for images in different spatial domains. The method can be executed by a training device for the image feature extraction model, which can be implemented by software and / or hardware and can be configured in a terminal and / or server to implement the training method for the image feature extraction model in the embodiment of the present invention.

[0040] like Figure 1 As shown, the method in this embodiment may specifically include:

[0041] S110. Obtain at least two image sample groups, wherein each image sample group includes at least two sample images from different spatial domains.

[0042] In this embodiment, the image sample set refers to pre-made training samples that can be used to train the image feature extraction model. The number of image sample sets can be two or more, and is not limited here. The image sample set can include multiple sample images from different spatial domains, where a spatial domain is also called image space. There are significant modal differences between sample images from different spatial domains. For example, at least two sample images from different spatial domains can be thermal infrared images and visible light images. The number of sample images can be two or more, and is not limited here. For example, the types of sample images can include various types, such as face images or body shape images.

[0043] Specifically, methods for obtaining image sample sets include, but are not limited to: acquiring images using infrared cameras and ordinary optical cameras, and creating image sample sets based on the acquired images; or retrieving pre-created image sample sets from a preset storage location. This embodiment does not limit the method for obtaining image sample sets.

[0044] S120. Within any of the image sample groups, image fusion is performed based on sample images from different spatial domains in the image sample group to obtain an enhanced sample image, and the image sample group is updated based on the enhanced sample image to obtain an enhanced training sample group corresponding to each of the image sample groups.

[0045] In this embodiment of the invention, in order to improve the accuracy of image recognition, the sample images within the image sample group are fused to obtain an enhanced sample image. The enhanced sample image refers to the image obtained by fusing sample images from different spatial domains within the image sample group.

[0046] It should be noted that augmented sample images are images in an intermediate state between sample images in different spatial domains. They have the comprehensive features of each sample image. Using augmented training sample sets containing augmented sample images to train the image feature extraction model can shorten the feature distance between sample images, enabling the image feature extraction model to learn the inherent relationship between sample images in different spatial domains, thereby improving the feature extraction accuracy of the image feature extraction model.

[0047] Furthermore, the image sample group is updated based on the enhanced sample images to obtain an enhanced training sample group containing the enhanced sample images. In this embodiment, the method for updating the image sample group includes various approaches. In some embodiments, enhanced sample images can be added to the image sample group, increasing the diversity of the training samples; in another embodiment, enhanced sample images can replace sample images in the image sample group, reducing the spatial domain differences between sample images within the image sample group, which is beneficial for the image feature extraction model to learn the intrinsic relationships between sample images. This embodiment does not limit the method for updating the image sample group.

[0048] Based on the above embodiments, sample images from different spatial domains within an image sample group can be fused to obtain an enhanced sample image. Specifically, based on the image fusion ratio, the pixel values ​​of corresponding pixels in the fused sample images are weighted to obtain the enhanced sample image. Here, the image fusion ratio refers to the fusion ratio of each sample image within the image sample group, which can be randomly set. For example, the fusion formula of the image fusion method includes:

[0049] x=λ×x i +(1-λ)×x j

[0050] Where x represents the enhanced sample image, x i Let x represent the first sample image. j Let represent the second sample image; λ represents the image fusion ratio, which is a random number between [0,1] and follows a beta distribution. The first sample image and the second sample image can be sample images with different spatial domains within the same image sample group.

[0051] S130. In each iteration of training the image feature extraction model, image features are extracted from sample images in any two groups of enhanced training samples based on the image feature extraction model, and a loss function is determined based on the image features corresponding to the sample images in any two groups of enhanced training samples. The parameters of the image feature extraction model in the current iteration are adjusted based on the loss function until the target image feature extraction model is obtained.

[0052] The image feature extraction model can be a deep learning network model, and the network architecture of the deep learning network is not limited here. Examples include feature extraction networks such as IR-50, ResNet50, or IR-152. The image feature extraction model can be used to extract image features from sample images in any two sets of augmented training samples. Specifically, image feature extraction can be performed simultaneously on sample images in each augmented training sample set, or sequentially according to the input order; this is not limited here. Simultaneous image feature extraction from all sample images can effectively improve the speed of feature extraction.

[0053] Specifically, in some embodiments, the loss function can be determined based on the image features of sample images within any two groups of augmented training samples. For example, the feature distance can be determined based on the image features of sample images within each augmented training sample group, and the loss function can be determined based on the feature distance between the two groups of augmented training samples. In other embodiments, the loss function can be determined based on the image features of sample images within any two groups of augmented training samples, as well as the image features of sample images between different augmented training sample groups. For example, the loss function can be determined based on the feature distance of image features within each group of augmented training samples, as well as the feature distance of image features of sample images between two groups of augmented training samples. The purpose of this setup is that image features within augmented training sample groups can be used as positive samples, and image features between different augmented training sample groups can be used as negative samples. This allows the model to effectively learn the intrinsic relationships between images in different spatial domains during the training process of the image feature extraction model, thereby improving the feature extraction accuracy of the model. Furthermore, by continuously adjusting the model parameters, the loss of the image feature extraction model gradually decreases and stabilizes, resulting in the target image feature extraction model. Feature distance refers to the distance between image features, which may include, but is not limited to, Euclidean distance, Manhattan distance, etc.

[0054] The technical solution of this invention involves acquiring at least two image sample groups, each including at least two sample images from different spatial domains, indicating that the training samples include sample images from different spatial domains. Further, within any image sample group, image fusion is performed based on the sample images from different spatial domains to obtain an enhanced sample image. This enhanced sample image is then used to update the image sample group, resulting in enhanced training sample groups that are easier to learn the intrinsic relationships between samples. Furthermore, during each iteration of the image feature extraction model's training, image features are extracted from the sample images in any two enhanced training sample groups according to the model. A loss function is determined based on the image features corresponding to the sample images in any two enhanced training sample groups. The parameters of the image feature extraction model for the current iteration are adjusted based on the loss function until the target image feature extraction model is obtained. This training method enables the image feature extraction model to learn the intrinsic relationships between the features of sample images within and between different enhanced training sample groups, thereby improving the feature extraction accuracy of the image feature extraction model.

[0055] Example 2

[0056] Figure 2This is a flowchart of the training method for the image feature extraction model provided in Embodiment 2 of the present invention. In this embodiment, based on any optional technical solution in the embodiments of the present invention, optionally, updating the image sample group based on the enhanced sample image to obtain an enhanced training sample group includes: replacing the sample images in the image sample group with the enhanced sample image to obtain the enhanced training sample group.

[0057] like Figure 2 As shown, the method in this embodiment may specifically include:

[0058] S210. Obtain at least two image sample groups, wherein each image sample group includes at least two sample images from different spatial domains.

[0059] S220. Within any of the image sample groups, image fusion is performed based on sample images from different spatial domains in the image sample group to obtain an enhanced sample image.

[0060] S230. Replace the sample images in the image sample group with the enhanced sample images to obtain the enhanced training sample group.

[0061] S240. In each iteration of training the image feature extraction model, image features are extracted from sample images in any two sets of the enhanced training sample groups based on the image feature extraction model, and a loss function is determined based on the image features corresponding to the sample images in any two sets of the enhanced training sample groups. The parameters of the image feature extraction model in the current iteration are adjusted based on the loss function until the target image feature extraction model is obtained.

[0062] In this embodiment, replacing the sample images in the image sample group with an enhanced sample image that possesses comprehensive sample image features can reduce the spatial domain differences between the sample images within the image sample group, which is beneficial for the image feature extraction model to learn the intrinsic relationships between the sample images. Specifically, the sample image replaced by the enhanced sample image can be any sample image in the image sample group, and there is no limitation here.

[0063] Based on the above embodiments, the sample image group includes at least a visible light image and an image in another spatial domain that is different from the spatial domain in which the visible light image is located. The step of replacing the sample images in the image sample group with the enhanced sample image to obtain an enhanced training sample group includes: replacing the image in the other spatial domain in the image sample group with the enhanced sample image to obtain an enhanced training sample group, wherein the enhanced training sample group includes the enhanced sample image and the visible light image.

[0064] Here, a visible light image refers to an image captured within the visible light range, while an image in a different spatial domain refers to an image whose spatial domain differs significantly from that of the visible light image. Furthermore, the visible light image contains richer features than the image in the other spatial domain, making them easier to extract. Optionally, the other spatial domain image and the visible light image can be thermal infrared images captured at the same time and location. Thermal infrared images can capture richer feature information than near-infrared images and are easier to acquire. Setting them to be captured at the same time and location has the advantage of ensuring that the resulting enhanced sample image retains the same features as both images and has minimal difference from the previous sample image.

[0065] Specifically, by replacing another spatial domain image in the image sample group with an enhanced sample image that retains the same features as both images, and retaining the visible light image, an enhanced training sample group containing both the enhanced sample image and the visible light image is obtained. This can reduce the spatial domain gap between sample images, which is beneficial for the image feature extraction model to learn the intrinsic relationship between sample images.

[0066] The technical solution of this invention replaces the sample images in the image sample group with enhanced sample images that have comprehensive features of the sample images. This reduces the spatial domain difference between the sample images in the image sample group, which is beneficial for the image feature extraction model to learn the intrinsic relationship between the sample images, thereby improving the feature extraction accuracy of the image feature extraction model.

[0067] Example 3

[0068] Figure 3 This is a flowchart of the training method for the image feature extraction model provided in Embodiment 3 of the present invention. Optionally, based on any optional technical solution in the embodiments of the present invention, the step of determining the loss function based on the image features corresponding to the sample images in any two sets of enhanced training sample groups includes: determining a first loss term based on the image features of the enhanced sample images in the first enhanced training sample group, the image features of the visible light images in the first enhanced training sample group, and the image features of the visible light images in the second enhanced training sample group; determining a second loss term based on the image features of the enhanced sample images in the second enhanced training sample group, the image features of the visible light images in the second enhanced training sample group, and the image features of the visible light images in the first enhanced training sample group; and determining a loss function based on the first loss term and the second loss term.

[0069] like Figure 3 As shown, the method in this embodiment may specifically include:

[0070] S310. Obtain at least two image sample groups, wherein each image sample group includes at least two sample images from different spatial domains.

[0071] S320. Within any of the image sample groups, image fusion is performed based on sample images from different spatial domains in the image sample group to obtain an enhanced sample image, and the image sample group is updated based on the enhanced sample image to obtain an enhanced training sample group corresponding to each of the image sample groups.

[0072] S330. In each iteration of training the image feature extraction model, image features are extracted from sample images in any two groups of the enhanced training sample groups based on the image feature extraction model.

[0073] S340. Determine the first loss term based on the image features of the enhanced sample images in the first enhanced training sample group, the image features of the visible light images in the first enhanced training sample group, and the image features of the visible light images in the second enhanced training sample group.

[0074] S350. Based on the image features of the enhanced sample images in the second enhanced training sample group, the image features of the visible light images in the second enhanced training sample group, and the image features of the visible light images in the first enhanced training sample group, determine the second loss term.

[0075] S360. Determine the loss function based on the first loss term and the second loss term.

[0076] S370. Adjust the parameters of the current iteration of the image feature extraction model based on the loss function until the target image feature extraction model is obtained.

[0077] In this embodiment, the first enhanced training sample group and the second enhanced training sample group can be two different image sample groups arbitrarily selected from all image sample groups.

[0078] Specifically, feature distances can be determined for the image features of the enhanced sample images in the first enhanced training sample group, the image features of the visible light images in the first enhanced training sample group, and the image features of the visible light images in the second enhanced training sample group. That is, the first loss term is determined based on the feature distances between different image features. Similarly, feature distances can be determined for the image features of the enhanced sample images in the second enhanced training sample group, the image features of the visible light images in the second enhanced training sample group, and the image features of the visible light images in the first enhanced training sample group. That is, the second loss term is determined based on the feature distances between different image features. Furthermore, the first and second loss terms can be directly added to obtain the loss function, or they can be added according to preset weights to obtain the loss function; this is not limited here.

[0079] It should be noted that the first and second loss terms utilize not only image features within the augmented training sample group but also image features between different augmented training sample groups. The purpose of this setting is that image features within the augmented training sample group can be used as positive samples, while image features between different augmented training sample groups can be used as negative samples. This brings the modalities of the sample images within the augmented training sample group closer together and widens the modalities of the sample images between different augmented training sample groups, which is more conducive to the image feature extraction model learning the intrinsic relationship between sample images within the same augmented training sample group.

[0080] Based on the above embodiments, determining the first loss term based on the image features of the enhanced sample images in the first enhanced training sample group, the image features of the visible light images in the first enhanced training sample group, and the image features of the visible light images in the second enhanced training sample group includes: determining first distance information based on the image features of the enhanced sample images in the first enhanced training sample group and the image features of the visible light images in the first enhanced training sample group; determining second distance information based on the image features of the enhanced sample images in the first enhanced training sample group and the image features of the visible light images in the second enhanced training sample group; and determining the first loss term based on the first distance information and the second distance information.

[0081] In this embodiment, the distance information is the same concept as the distance feature in the above embodiment, referring to the distance between image features corresponding to sample images. In this embodiment, by determining the first loss term through the first distance information within the first enhanced training sample group and the second distance information between the two enhanced training sample groups, the image feature extraction model can learn richer intrinsic relationships between sample images, thereby improving the feature extraction accuracy of the image feature extraction model.

[0082] Based on the above embodiments, determining the second loss term based on the image features of the enhanced sample images in the second enhanced training sample group, the image features of the visible light images in the second enhanced training sample group, and the image features of the visible light images in the first enhanced training sample group includes: determining third distance information based on the image features of the enhanced sample images in the second enhanced training sample group and the image features of the visible light images in the second enhanced training sample group; determining fourth distance information based on the image features of the enhanced sample images in the second enhanced training sample group and the image features of the visible light images in the first enhanced training sample group; and determining the second loss term based on the third distance information and the fourth distance information.

[0083] In this embodiment, similar to the first loss term, the second loss term is determined by the third distance information within the second enhanced training sample group and the third distance information between the two enhanced training sample groups. This allows the image feature extraction model to learn richer intrinsic relationships between sample images and improves the feature extraction accuracy of the image feature extraction model.

[0084] Based on the above embodiments, the labels of sample images in different spatial domains in any of the image sample groups are the same.

[0085] Understandably, the same label is assigned to sample images in different spatial domains within an image sample group, meaning that the labels of sample images in different image sample groups are different. Labels enable the differentiation of sample images, allowing the image feature extraction model to treat sample images with the same label as positive samples and sample images with different labels as negative samples during training, thereby improving the feature extraction accuracy of the image feature extraction model.

[0086] For example, such as Figure 4 The diagram illustrates the loss calculation relationship, where IR1 represents the augmented sample image in the first augmented training sample group, IR2 represents the augmented sample image in the second augmented training sample group, VIS1 represents the visible light image in the first augmented training sample group, and VIS2 represents the visible light image in the second augmented training sample group. The first augmented training sample group may include augmented sample images and visible light images with the same label; the second augmented training sample group includes augmented sample images and visible light images with the same label; the labels of the first and second augmented training sample groups are different.

[0087] Accordingly, the loss function includes:

[0088] L total =L1+L2

[0089]

[0090]

[0091] Where L1 represents the first loss term corresponding to the first augmented training sample group, L2 represents the second loss term corresponding to the second augmented training sample group, α and β are hyperparameters; f(·) represents the image features of the sample image; x a x represents the enhanced sample image. p This refers to a visible light image with the same label as the augmented sample image, i.e., a visible light image within the same augmented training sample group as the augmented sample image. n This represents a visible light image that is not labeled as an enhanced sample image, i.e., in the first loss function, x nFor the visible light images in the second enhanced training sample group, in the second loss function, x n The image represents the visible light image in the first enhanced training sample group; N represents the number of groups in the first enhanced training sample group, and M represents the number of groups in the second enhanced training sample group.

[0092] The technical solution of this invention utilizes not only image features within the augmented training sample group but also image features between different augmented training sample groups. Image features within the augmented training sample group can be used as positive samples, while image features between different augmented training sample groups can be used as negative samples. This brings the modalities of sample images within the augmented training sample group closer together and widens the modalities of sample images between different augmented training sample groups, which is more conducive to the image feature extraction model learning the intrinsic relationship between sample images within the same augmented training sample group.

[0093] Example 4

[0094] Figure 5 This is a flowchart illustrating an image feature extraction method provided in Embodiment 4 of the present invention. This embodiment is applicable to feature extraction of images in different spatial domains. The method can be executed by an image feature extraction device, which can be implemented by software and / or hardware and can be configured in a terminal and / or server to implement the image feature extraction method in the embodiments of the present invention.

[0095] like Figure 5 As shown, the method in this embodiment may specifically include:

[0096] S410. Obtain the image to be processed;

[0097] S420. Input the image to be processed into the target image feature extraction model to obtain the feature extraction result.

[0098] The target image feature extraction model is trained using the training method of any image feature extraction model according to the embodiments of the present invention.

[0099] Specifically, the training method for the image feature extraction model includes: acquiring at least two image sample groups, wherein each image sample group includes at least two sample images from different spatial domains, indicating that the training samples include sample images from different spatial domains; further, within any image sample group, image fusion is performed based on the sample images from different spatial domains in the image sample group to obtain enhanced sample images, and the image sample group is updated based on the enhanced sample images to obtain enhanced training sample groups corresponding to each image sample group that are easier to learn the intrinsic relationships between samples; further, in each iteration of training the image feature extraction model, image features are extracted from the sample images in any two enhanced training sample groups according to the image feature extraction model, and a loss function is determined based on the image features corresponding to the sample images in any two enhanced training sample groups. The parameters of the image feature extraction model in the current iteration are adjusted based on the loss function until the target image feature extraction model is obtained. This training enables the image feature extraction model to learn the intrinsic relationships between the features of sample images within the enhanced training sample groups and between different enhanced training sample groups, thereby improving the feature extraction accuracy of the image feature extraction model.

[0100] The technical solution of this embodiment acquires the image to be processed, inputs the image to be processed into the target image feature extraction model, and obtains the feature extraction result. This realizes the application of the target image feature extraction model, which can quickly extract image features online. The extracted image features can be used for image category recognition.

[0101] Example 5

[0102] Figure 6 This is a schematic diagram of the structure of the training device for the image feature extraction model provided in Embodiment 5 of the present invention. The training device for the image feature extraction model provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal and / or server to implement the training method of the image feature extraction model in the embodiments of the present invention. Specifically, the device may include:

[0103] The sample acquisition module 510 is used to acquire at least two image sample groups, wherein each image sample group includes at least two sample images from different spatial domains; the enhanced sample determination module 520 is used to perform image fusion based on the sample images from different spatial domains in any image sample group to obtain an enhanced sample image, and update the image sample group based on the enhanced sample image to obtain an enhanced training sample group corresponding to each image sample group; the model training module 530 is used to extract image features from sample images in any two groups of enhanced training sample groups based on the image feature extraction model during each iteration of training the image feature extraction model, determine a loss function based on the image features corresponding to the sample images in any two groups of enhanced training sample groups, and adjust the parameters of the image feature extraction model in the current iteration based on the loss function until the target image feature extraction model is obtained.

[0104] Based on any optional technical solution in the embodiments of the present invention, the enhanced sample determination module 520 may optionally include:

[0105] The sample augmentation unit is used to replace the sample images in the image sample group with the augmented sample images to obtain an augmented training sample group.

[0106] Based on any optional technical solution in the embodiments of the present invention, optionally, the sample image group includes at least a visible light image and an image in another spatial domain different from the spatial domain in which the visible light image is located, and the sample enhancement unit is specifically used for:

[0107] The enhanced sample image is used to replace the other spatial domain image in the image sample group to obtain an enhanced training sample group, wherein the enhanced training sample group includes the enhanced sample image and the visible light image.

[0108] Based on any optional technical solution in the embodiments of the present invention, optionally, the model training module 530 includes:

[0109] The first loss term determination unit is used to determine the first loss term based on the image features of the enhanced sample images in the first enhanced training sample group, the image features of the visible light images in the first enhanced training sample group, and the image features of the visible light images in the second enhanced training sample group.

[0110] The second loss term determination unit is used to determine the second loss term based on the image features of the enhanced sample images in the second enhanced training sample group, the image features of the visible light images in the second enhanced training sample group, and the image features of the visible light images in the first enhanced training sample group.

[0111] The loss function determination unit is used to determine the loss function based on the first loss term and the second loss term.

[0112] Optionally, based on any of the optional technical solutions in the embodiments of the present invention, the first loss term determination unit is specifically used for:

[0113] Based on the image features of the enhanced sample images in the first enhanced training sample group and the image features of the visible light images in the first enhanced training sample group, the first distance information is determined;

[0114] Based on the image features of the enhanced sample images in the first enhanced training sample group and the image features of the visible light images in the second enhanced training sample group, the second distance information is determined;

[0115] A first loss term is determined based on the first distance information and the second distance information;

[0116] And, the second loss term determination unit is specifically used for:

[0117] Based on the image features of the enhanced sample images in the second enhanced training sample group and the image features of the visible light images in the second enhanced training sample group, the third distance information is determined;

[0118] Based on the image features of the enhanced sample images in the second enhanced training sample group and the image features of the visible light images in the first enhanced training sample group, the fourth distance information is determined;

[0119] The second loss term is based on the third distance information and the fourth distance information.

[0120] Based on any optional technical solution in the embodiments of the present invention, optionally, the labels of sample images in different spatial domains in any of the image sample groups are the same.

[0121] The above-described apparatus can execute the training method of the image feature extraction model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the training method of the image feature extraction model.

[0122] Example 6

[0123] Figure 7 This is a schematic diagram of the image feature extraction device provided in Embodiment Six of the present invention. The image feature extraction device provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal and / or server to implement the image feature extraction method in the embodiments of the present invention. Specifically, the device may include:

[0124] The image acquisition module 610 is used to acquire the image to be processed; the feature extraction module 620 is used to input the image to be processed into the target image feature extraction model to obtain the feature extraction result.

[0125] The target image feature extraction model is trained using the image feature extraction model training method provided in any embodiment of the present invention.

[0126] The above-described apparatus can execute the image feature extraction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the image feature extraction method.

[0127] Example 7

[0128] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present invention. Figure 8 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 8 The electronic device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0129] like Figure 8 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0130] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0131] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0132] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0133] A program / utility 36 having a set (at least one) of program modules 26 may be stored, for example, in system memory 28. Such program modules 26 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 26 typically perform the functions and / or methods described in the embodiments of the present invention.

[0134] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the electronic device 12, and / or with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 8 As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 8 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0135] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a training method for an image feature extraction model provided in an embodiment of the present invention.

[0136] Example 8

[0137] Embodiment 8 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a training method for an image feature extraction model, the method comprising:

[0138] At least two image sample groups are obtained, wherein each image sample group includes at least two sample images from different spatial domains;

[0139] Within any of the image sample groups, image fusion is performed based on sample images from different spatial domains in the image sample group to obtain an enhanced sample image, and the image sample group is updated based on the enhanced sample image to obtain an enhanced training sample group corresponding to each of the image sample groups.

[0140] In each iteration of training the image feature extraction model, image features are extracted from sample images in any two sets of the enhanced training sample groups based on the image feature extraction model, and a loss function is determined based on the image features corresponding to the sample images in the two sets of the enhanced training sample groups. The parameters of the image feature extraction model in the current iteration are adjusted based on the loss function until the target image feature extraction model is obtained.

[0141] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0142] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0143] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0144] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0145] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for training an image feature extraction model, characterized in that, include: At least two image sample groups are obtained, wherein each image sample group includes at least two sample images from different spatial domains; Within any of the image sample groups, image fusion is performed based on sample images from different spatial domains within the image sample group to obtain an enhanced sample image. The image sample group is then updated based on the enhanced sample image to obtain an enhanced training sample group corresponding to each image sample group. Here, the enhanced sample image refers to the image obtained by fusing sample images from different spatial domains within the image sample group. In each iteration of training the image feature extraction model, image features are extracted from sample images in any two sets of the enhanced training sample groups based on the image feature extraction model, and a loss function is determined based on the image features corresponding to the sample images in any two sets of the enhanced training sample groups. The parameters of the image feature extraction model in the current iteration are adjusted based on the loss function until the target image feature extraction model is obtained. The step of updating the image sample groups based on the enhanced sample images to obtain the enhanced training sample groups corresponding to each image sample group includes: The enhanced sample images are used to replace the sample images in the image sample group to obtain the enhanced training sample group; The image sample group includes at least a visible light image and an image in another spatial domain distinct from the visible light image. The step of replacing the sample images in the image sample group with the enhanced sample images to obtain enhanced training sample groups corresponding to each image sample group includes: The enhanced sample image is used to replace the other spatial domain image in the image sample group to obtain an enhanced training sample group, wherein the enhanced training sample group includes the enhanced sample image and the visible light image; The step of determining the loss function based on the image features corresponding to the sample images in any two sets of the enhanced training sample groups includes: The first loss term is determined based on the image features of the enhanced sample images in the first enhanced training sample group, the image features of the visible light images in the first enhanced training sample group, and the image features of the visible light images in the second enhanced training sample group; The second loss term is determined based on the image features of the enhanced sample images in the second enhanced training sample group, the image features of the visible light images in the second enhanced training sample group, and the image features of the visible light images in the first enhanced training sample group; The loss function is determined based on the first loss term and the second loss term.

2. The method according to claim 1, characterized in that, The determination of the first loss term based on the image features of the enhanced sample images in the first enhanced training sample group, the image features of the visible light images in the first enhanced training sample group, and the image features of the visible light images in the second enhanced training sample group includes: Based on the image features of the enhanced sample images in the first enhanced training sample group and the image features of the visible light images in the first enhanced training sample group, the first distance information is determined; Based on the image features of the enhanced sample images in the first enhanced training sample group and the image features of the visible light images in the second enhanced training sample group, the second distance information is determined; A first loss term is determined based on the first distance information and the second distance information; And, determining the second loss term based on the image features of the enhanced sample images in the second enhanced training sample group, the image features of the visible light images in the second enhanced training sample group, and the image features of the visible light images in the first enhanced training sample group includes: Based on the image features of the enhanced sample images in the second enhanced training sample group and the image features of the visible light images in the second enhanced training sample group, the third distance information is determined; Based on the image features of the enhanced sample images in the second enhanced training sample group and the image features of the visible light images in the first enhanced training sample group, the fourth distance information is determined; The second loss term is determined based on the third distance information and the fourth distance information.

3. The method according to claim 1, characterized in that, The labels of sample images in different spatial domains within any of the image sample groups are the same.

4. An image feature extraction method, characterized in that, include: Obtain the image to be processed; The image to be processed is input into the target image feature extraction model to obtain the feature extraction result; The target image feature extraction model is trained using the image feature extraction model training method according to any one of claims 1-3.

5. A training device for an image feature extraction model, characterized in that, include: The sample acquisition module is used to acquire at least two image sample groups, wherein each image sample group includes at least two sample images from different spatial domains; An enhanced sample determination module is used to perform image fusion based on sample images from different spatial domains within any image sample group to obtain an enhanced sample image, and to update the image sample group based on the enhanced sample image to obtain an enhanced training sample group corresponding to each image sample group; wherein, the enhanced sample image refers to the image obtained by fusing sample images from different spatial domains within the image sample group; The model training module is used to extract image features from sample images in any two sets of the enhanced training sample groups based on the image feature extraction model during each iteration of training the image feature extraction model, determine the loss function based on the image features corresponding to the sample images in any two sets of the enhanced training sample groups, and adjust the parameters of the image feature extraction model in the current iteration based on the loss function until the target image feature extraction model is obtained. Enhanced sample determination module, including: The sample enhancement unit is used to replace the sample images in the image sample group with the enhanced sample images to obtain an enhanced training sample group; the image sample group includes at least a visible light image and an image in another spatial domain different from the spatial domain in which the visible light image is located; the sample enhancement unit is specifically used for: The enhanced sample image is used to replace the other spatial domain image in the image sample group to obtain an enhanced training sample group, wherein the enhanced training sample group includes the enhanced sample image and the visible light image; The model training module includes: The first loss term determination unit is used to determine the first loss term based on the image features of the enhanced sample images in the first enhanced training sample group, the image features of the visible light images in the first enhanced training sample group, and the image features of the visible light images in the second enhanced training sample group. The second loss term determination unit is used to determine the second loss term based on the image features of the enhanced sample images in the second enhanced training sample group, the image features of the visible light images in the second enhanced training sample group, and the image features of the visible light images in the first enhanced training sample group. The loss function determination unit is used to determine the loss function based on the first loss term and the second loss term.

6. An image feature extraction device, characterized in that, include: The image acquisition module is used to acquire the image to be processed. The feature extraction module is used to input the image to be processed into the target image feature extraction model to obtain the feature extraction result; The target image feature extraction model is trained using the training device for the image feature extraction model according to claim 5.

7. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the image feature extraction model as described in any one of claims 1-3, or the image feature extraction method as described in claim 4.

8. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the training method of the image feature extraction model as described in any one of claims 1-3, or the image feature extraction method as described in claim 4.

Images