Model training method, high-quality satellite image retrieval method and device

Abstract

The invention provides a model training method and a high-quality satellite image retrieval method and device, and the method comprises the steps: obtaining a first object type of each object in a sample satellite image through a first neural network model, and obtaining a first classification prediction value corresponding to each first object type; a first model loss value of the first neural network model can be further calculated according to the first object category and the first classification prediction value in combination with the sample object category of the sample satellite image, so that model parameters of the first neural network model can be adjusted through the first model loss value to obtain a target neural network model; as the processing of the sample satellite image is accurate to each object in the image, the prediction accuracy of the target neural network model can be improved, and when the target neural network model is used for processing the query image to retrieve the satellite image, the matching degree of the finally obtained target satellite image and the query image can be improved, so that the accuracy of satellite image retrieval is improved. And the accuracy of image retrieval query is improved.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a model training method, a high-quality satellite image retrieval method and a device. Background Art

[0002] In recent years, the rapid development of satellite technology has driven the growth of various satellite applications. As one of the main areas of satellite application, satellite remote sensing can obtain earth information all day, all day, high orbit, and large scale, playing an unparalleled role in information acquisition and providing decision support for major issues such as disaster prevention. Thanks to the development of satellite software and hardware technology, satellites have certain computing and storage capabilities and can store a large amount of observation data. However, due to the uniqueness of the space environment and the limitations of the satellite's own communication capabilities, how to successfully transmit the target data back to the ground station within the satellite's transit time has become an urgent problem that needs to be solved.

[0003] In related technologies, the content features of the query image and the image to be retrieved can be extracted through SIFT or deep learning methods based on the annotation information of the satellite remote sensing image data, and the similarity can be calculated to determine whether they belong to matching samples. Then, the target image data obtained by the query is sent to the ground station through the communication system to complete the image data transmission.

[0004] However, in the above method, the accuracy of the content features of the query image and the image to be retrieved is not high, the target image data obtained by the query often does not match the query image, and the accuracy of the image retrieval query is not high. Summary of the invention

[0005] In view of the above problems, the embodiments of the present application provide a model training method, a high-quality satellite image retrieval method, an apparatus, an electronic device and a readable storage medium to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of the present application provides a model training method, the method comprising: Inputting the sample satellite image into the first neural network model to obtain a first object category vector output by the first neural network model and a first classification prediction value corresponding to the first object category vector; wherein the sample satellite image includes at least one object; Determining a first model loss value of the first neural network model based on a sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; Based on the first model loss value, the model parameters of the first neural network model are adjusted to obtain a target neural network model; wherein the target neural network model is used to generate target object categories of the query image and target classification prediction values ​​corresponding to each of the target object categories.

[0007] Optionally, the first neural network model is composed of a convolutional neural network model, a Transformer-based bidirectional encoder representation model, and a Transformer-based encoder model, and the step of inputting the sample satellite image into the first neural network model to obtain a first object category vector output by the first neural network model and a first classification prediction value corresponding to the first object category vector includes: Inputting the sample satellite image into the convolutional neural network to obtain a sample satellite image feature vector output by the convolutional neural network model; Inputting each object category name into the Transformer-based bidirectional encoder representation model to obtain a sample object category vector output by the Transformer-based bidirectional encoder representation model; Splicing the sample satellite image feature vector and the sample object category vector to obtain a first sample splicing vector; Inputting the first sample splicing vector into the Transformer-based encoder to obtain a second sample splicing vector output by the Transformer-based encoder; A first object category vector is determined from the second sample splicing vector, and based on the first object category vector, a first object category corresponding to the sample satellite image and first classification prediction values ​​corresponding to each of the first object categories are determined.

[0008] Optionally, the determining a first model loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector and the first classification prediction value includes: Determining a binary cross entropy loss value of the first neural network model based on a sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; Determining a second model loss value of the first neural network model based on the first classification prediction value, the sample object category vector, and the first object category vector; Determining a maximum likelihood estimation loss value of the first neural network model based on the first classification prediction value; Based on the binary cross entropy loss value, the second model loss value and the maximum likelihood estimation loss value, a first model loss value of the first neural network model is determined.

[0009] Optionally, determining a second model loss value of the first neural network model based on the first classification prediction value, the sample object category vector and the first object category vector includes: Performing semantic transformation on the first object category vector to obtain a second object category vector; wherein the semantic space of the second object category vector is the same as the semantic space of the sample object category vector; Calculating a first cosine similarity between the second object category vector and the sample object category vector; Based on the first cosine similarity and the first classification prediction value, a second model loss value of the first neural network model is determined.

[0010] Optionally, determining a second model loss value of the first neural network model based on the first cosine similarity and the first classification prediction value includes: Determining a second cosine similarity corresponding to a clean sample satellite image and a third cosine similarity corresponding to a noise sample satellite image from the first cosine similarity; wherein the clean sample satellite image and the noise sample satellite image are both included in the sample satellite image, and the noise sample satellite image is generated based on the clean sample satellite image and the noise vector; Calculating a third model loss value of the first neural network model based on the second cosine similarity, the third cosine similarity and the noise vector; Based on the third model loss value and the first classification prediction value, a second model loss value of the first neural network model is determined.

[0011] Optionally, determining the maximum likelihood estimation loss value of the first neural network model based on the first classification prediction value includes: Determine first Gumbel Softmax values ​​corresponding to the first classification prediction values ​​respectively; When the first Gumbel Softmax value is greater than or equal to a first threshold, inputting a first object category vector corresponding to the first Gumbel Softmax value into a multilayer perceptron to obtain a second classification probability value output by the multilayer perceptron; Determining a first category identifier corresponding to the first object category vector based on the second classification probability value; Inputting the sample satellite image into the multi-layer perceptron to obtain a third classification probability value output by the multi-layer perceptron; Based on the third classification probability value, a maximum likelihood estimation loss value of the first neural network model is determined.

[0012] In a second aspect, an embodiment of the present application provides a high-quality satellite image retrieval method, the method comprising: Inputting the query image into the target neural network model, obtaining a first target object category vector output by the target neural network model and a first target classification prediction value corresponding to the first target object category vector; wherein the target neural network model is obtained based on any of the above-mentioned model training methods; Determining a first target object category corresponding to a first target classification prediction value that is greater than or equal to a first threshold as a query category identifier of the query image; Determining a query similarity identifier of the query image based on a cosine similarity between the first target object category vector and a standard category vector corresponding to a standard object category name; Based on the query category identifier and the query similarity identifier, a search is performed in an image database of the target satellite to obtain the target satellite image.

[0013] Optionally, searching in an image database of a target satellite based on the query category identifier and the query similarity identifier to obtain the target satellite image includes: Determine a first similarity between an image category identifier of each first satellite image included in an image database of the target satellite and the query category identifier, and a second similarity between an image similarity identifier of each first satellite image and the query similarity identifier; A first satellite image whose first similarity is greater than or equal to a second threshold and whose second similarity is greater than or equal to a third threshold is determined as a target satellite image.

[0014] In a third aspect, an embodiment of the present application provides a model training device, the device comprising: An input-output module, configured to input a sample satellite image into a first neural network model to obtain a first object category vector output by the first neural network model and a first classification prediction value corresponding to the first object category vector; wherein the sample satellite image includes at least one object; a determination module, configured to determine a first model loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; An adjustment module is used to adjust the model parameters of the first neural network model based on the first model loss value to obtain a target neural network model; wherein the target neural network model is used to generate a target object category of a query image and a target classification prediction value corresponding to each of the target object categories.

[0015] Optionally, the first neural network model is composed of a convolutional neural network model, a Transformer-based bidirectional encoder representation model and a Transformer-based encoder model, and the input-output module includes: A first input-output submodule, used to input a sample satellite image into the convolutional neural network to obtain a sample satellite image feature vector output by the convolutional neural network model; A second input-output submodule, configured to input each object category name into the Transformer-based bidirectional encoder representation model to obtain a sample object category vector output by the Transformer-based bidirectional encoder representation model; A splicing submodule, used for splicing the sample satellite image feature vector and the sample object category vector to obtain a first sample splicing vector; A third input-output submodule, configured to input the first sample splicing vector into the Transformer-based encoder to obtain a second sample splicing vector output by the Transformer-based encoder; The first determination submodule is used to determine a first object category vector from the second sample splicing vector, and based on the first object category vector, determine a first object category corresponding to the sample satellite image and a first classification prediction value corresponding to each of the first object categories.

[0016] Optionally, the determining module includes: A second determination submodule, configured to determine a binary cross entropy loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; A third determination submodule, configured to determine a second model loss value of the first neural network model based on the first classification prediction value, the sample object category vector, and the first object category vector; A fourth determination submodule, used to determine a maximum likelihood estimation loss value of the first neural network model based on the first classification prediction value; The fifth determination submodule is used to determine the first model loss value of the first neural network model based on the binary cross entropy loss value, the second model loss value and the maximum likelihood estimation loss value.

[0017] Optionally, the third determining submodule includes: A semantic transformation unit, configured to perform semantic transformation on the first object category vector to obtain a second object category vector; wherein the semantic space of the second object category vector is the same as the semantic space of the sample object category vector; a calculation unit, configured to calculate a first cosine similarity between the second object category vector and the sample object category vector; The first determining unit is used to determine a second model loss value of the first neural network model based on the first cosine similarity and the first classification prediction value.

[0018] Optionally, the first determining unit includes: A first determining subunit is used to determine a second cosine similarity corresponding to a clean sample satellite image and a third cosine similarity corresponding to a noise sample satellite image from the first cosine similarity; wherein the clean sample satellite image and the noise sample satellite image are both included in the sample satellite image, and the noise sample satellite image is generated based on the clean sample satellite image and the noise vector; A calculation subunit, configured to calculate a third model loss value of the first neural network model based on the second cosine similarity, the third cosine similarity and the noise vector; The second determining subunit is used to determine a second model loss value of the first neural network model based on the third model loss value and the first classification prediction value.

[0019] Optionally, the fourth determining submodule includes: A second determining unit, used to determine first Gumbel Softmax values ​​corresponding to the first classification prediction values ​​respectively; A first input-output unit is used for inputting a first object category vector corresponding to the first Gumbel Softmax value into a multilayer perceptron to obtain a second classification probability value output by the multilayer perceptron when the first Gumbel Softmax value is greater than or equal to a first threshold; a third determining unit, configured to determine a first category identifier corresponding to the first object category vector based on the second classification probability value; A second input-output unit is used to input the sample satellite image into the multi-layer perceptron to obtain a third classification probability value output by the multi-layer perceptron; The fourth determining unit is used to determine the maximum likelihood estimation loss value of the first neural network model based on the third classification probability value.

[0020] In a fourth aspect, an embodiment of the present application provides a high-quality satellite image retrieval device, the device comprising: An input-output module, used for inputting a query image into a target neural network model, and obtaining a first target object category vector output by the target neural network model and a first target classification prediction value corresponding to the first target object category vector; wherein the target neural network model is obtained based on the model training method according to any one of claims 1 to 6; A first determination module, configured to determine a second target object category corresponding to a first target classification prediction value that is greater than or equal to a first threshold as a query category identifier of the query image; A second determination module, configured to determine a query similarity identifier of the query image based on a cosine similarity between the first target object category vector and a standard category vector corresponding to a standard object category name; The retrieval module is used to search in the image database of the target satellite based on the query category identifier and the query similarity identifier to obtain the target satellite image.

[0021] Optionally, the retrieval module includes: A first determination submodule is used to determine a first similarity between an image category identifier of each first satellite image included in an image database of a target satellite and the query category identifier, and a second similarity between an image similarity identifier of each first satellite image and the query similarity identifier; The second determination submodule is configured to determine, as a target satellite image, a first satellite image whose first similarity is greater than or equal to a second threshold and whose second similarity is greater than or equal to a third threshold.

[0022] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the model training method as described in any one of the above, or the high-quality satellite image retrieval method.

[0023] In a sixth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the model training method as described in any one of the above items, or the high-quality satellite image retrieval method, is implemented.

[0024] The specific beneficial effects are: In the embodiment of the present application, the sample satellite image is input into the first neural network model to obtain the first object category output by the first neural network model and the first classification prediction value corresponding to each first object category, and the first model loss value of the first neural network model is determined based on the sample object category, the first object category and the first classification prediction value of the sample satellite image, and the model parameters of the first neural network model are adjusted based on the first model loss value to obtain the target neural network model; wherein the target neural network model is used to generate the target object category of the target query image and the target classification prediction value corresponding to each target object category based on the target query image, and the first pair of each object in the sample satellite image can be obtained through the first neural network model. The first object category and the first classification prediction value corresponding to each first object category can be respectively determined, and the first model loss value of the first neural network model can be further calculated according to the first object category and the first classification prediction value combined with the sample object category of the sample satellite image itself, so that the model parameters of the first neural network model can be adjusted according to the first model loss value to obtain the target neural network model. Since the sample satellite image is accurately processed to each object in the image, the prediction accuracy of the target neural network model finally obtained can be improved. When the target neural network model is used to process the query image to retrieve the satellite image, the matching degree between the target satellite image and the query image can be improved, thereby improving the accuracy of the image retrieval query. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 It is a flowchart of another model training method provided in an embodiment of the present application; Figure 2 It is a flowchart of another model training method provided in an embodiment of the present application; Figure 3 It is a flowchart of a high-quality satellite image retrieval method provided in an embodiment of the present application; Figure 4 It is a logic block diagram of a model training device provided in an embodiment of the present application; Figure 5 It is a logic block diagram of a high-quality satellite image retrieval device provided in an embodiment of the present application; Figure 6 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0028] Reference Figure 1 , Figure 1 A flow chart of a model training method provided in an embodiment of the present application, the method comprising: Step 101: input a sample satellite image into a first neural network model to obtain a first object category vector output by the first neural network model and a first classification prediction value corresponding to the first object category vector; wherein the sample satellite image includes at least one object.

[0029] In an embodiment of the present application, the objects in the sample satellite image may refer to some specific objects. For example, the sample satellite image may include roads, railways, houses, vehicles, fields, etc., which are all objects in the sample satellite image. Each object has a corresponding object category. The sample satellite image can be input into the first neural network model, so that the first object category vector output by the first neural network model and the first classification prediction value corresponding to the first object category vector can be obtained. The first neural network model can be a convolutional neural network model, a recurrent neural network model, or a combination of a convolutional neural network model and a recurrent neural network model. The first object category vector may include the category of each object in the sample satellite image, and the first classification prediction value corresponding to the first object category vector may refer to the probability that the sample satellite image contains each object category.

[0030] Step 102: Determine a first model loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector and the first classification prediction value.

[0031] In an embodiment of the present application, the sample object category vector of the sample satellite image can be used as the annotation information of the sample satellite image. The first model loss value of the first neural network model can be calculated based on the sample object category vector of the sample satellite image, the first object category vector output by the first neural network model, and the first classification prediction value. Among them, for any object category, two situations can be included: the sample satellite image contains the object category, or the sample satellite image does not contain the object category, so the first model loss value can be a binary cross entropy loss value. In addition, considering that the first object category and the sample object category may not completely overlap, the loss value of the first neural network model in object classification can also be designed based on the sample object category and the first object category, which is called the second model loss value. Thus, the first model loss value can also be a combination of the binary cross entropy loss value and the second model loss value. The combination method can be direct addition or addition with regularization parameters.

[0032] Step 103: Based on the first model loss value, adjust the model parameters of the first neural network model to obtain a target neural network model; wherein the target neural network model is used to generate target object categories of the query image and target classification prediction values ​​corresponding to each of the target object categories.

[0033] In an embodiment of the present application, the model parameters of the first neural network model can be adjusted according to the first model loss value, so that the target neural network model can be obtained. The adjustment direction of the above model parameters can be the direction of reducing the first model loss value. In addition, the adjustment process of the above model parameters can be repeated multiple times, that is, the process of steps 101 to 103 can be repeatedly executed to repeatedly train the first neural network model. When the training process of the first neural network model meets the preset convergence conditions, the training of the first neural network model can be stopped, and the first neural network model obtained after the last model parameter adjustment is used as the target neural network model. The target neural network model can be used to process the query image, so as to generate the target object category of the query image and the target classification prediction value corresponding to the target object category.

[0034] In an embodiment of the present application, a sample satellite image is input into a first neural network model to obtain a first object category output by the first neural network model and a first classification prediction value corresponding to each first object category, and a first model loss value of the first neural network model is determined based on the sample object category, the first object category and the first classification prediction value of the sample satellite image, and a model parameter of the first neural network model is adjusted based on the first model loss value to obtain a target neural network model; wherein the target neural network model is used to generate a target object category of a target query image and a target classification prediction value corresponding to each target object category based on a target query image, and the first neural network model can be used to obtain a first object category of each object in the sample satellite image. An object category, and first classification prediction values ​​corresponding to each first object category, respectively, and the first model loss value of the first neural network model can be further calculated according to the first object category and the first classification prediction value combined with the sample object category of the sample satellite image itself, so that the model parameters of the first neural network model can be adjusted by the first model loss value to obtain the target neural network model. Since the sample satellite image is accurately processed to each object in the image, the prediction accuracy of the target neural network model finally obtained can be improved. When the target neural network model is used to process the query image to retrieve the satellite image, the matching degree between the target satellite image and the query image can be improved, thereby improving the accuracy of the image retrieval query.

[0035] Reference Figure 2 , Figure 2 A flowchart of another model training method provided in an embodiment of the present application, the method may include: Step 201: input a sample satellite image into the convolutional neural network to obtain a sample satellite image feature vector output by the convolutional neural network model.

[0036] In an embodiment of the present application, the first neural network model may be composed of a convolutional neural network model, a Transformer-based bidirectional encoder representation model, and a Transformer-based encoder. The convolutional neural network model may be used to extract a feature vector of an image. The sample satellite image may be input into the convolutional neural network model in the first neural network model, thereby obtaining a sample satellite image feature vector output by the convolutional neural network model. The dimension of the sample satellite image feature vector may be determined based on the dimension of the sample satellite image and the number of image features. For example, if the dimension of the sample image is , the number of image features is d, then the dimension of the sample satellite image feature vector can be .

[0037] Step 202: Input each object category name into the Transformer-based bidirectional encoder representation model to obtain a sample object category vector output by the Transformer-based bidirectional encoder representation model.

[0038] In an embodiment of the present application, a bidirectional encoder representation model based on Transformer, namely a BERT (Bidirectional Encoder Representations from Transformers) model, can convert natural language text into numerical representation (which can be in the form of pure numerical values, vectors or matrices). Each object category name can be input into the BERT model in the first neural network model, so that a sample object category vector output by the BERT model can be obtained.

[0039] Step 203: concatenate the sample satellite image feature vector and the sample object category vector to obtain a first sample concatenated vector.

[0040] In an embodiment of the present application, the sample satellite image feature vector and the object category vector may be spliced ​​to obtain a first sample splicing vector, wherein the elements in the object category vector may be located in the second half of the first sample splicing vector, and the elements in the sample satellite image feature vector may be located in the first half of the first sample splicing vector.

[0041] For example, if the sample satellite image feature vector is , the object category vector is , then the first sample concatenation vector can be .

[0042] Step 204: input the first sample splicing vector into the Transformer-based encoder to obtain a second sample splicing vector output by the Transformer-based encoder.

[0043] In an embodiment of the present application, the self-attention mechanism of the Transformer-based encoder can be used to learn the correlation between object categories and image features. On this basis, the first splicing vector can be input into the Transformer-based encoder in the first neural network model, so that the second sample splicing vector output by the Transformer-based encoder can be obtained.

[0044] Continuing with the above example, the second sample concatenation vector can be .

[0045] Step 205: determine a first object category vector from the second sample splicing vector, and based on the first object category vector, determine a first object category corresponding to the sample satellite image and a first classification prediction value corresponding to each of the first object categories.

[0046] In an embodiment of the present application, a first object category vector can be obtained from the second sample splicing vector. After obtaining the first object category vector, the first object category corresponding to the sample satellite image and the first classification prediction values ​​corresponding to each first object category can be determined according to the first category vector.

[0047] Continuing with the above example, the first object category vector can be , which contains C first object categories in total. The first classification prediction value can be obtained by linearizing the first object category vector and then calculating it through the sigmoid function, which is expressed as ,in represents a linear function, represents the sigmoid function, .like , it means that the first object category does not include object category i. If , it means that the first object category may include objects of object category i.

[0048] In an embodiment of the present application, a sample satellite image is input into a convolutional neural network to obtain a sample satellite image feature vector output by the convolutional neural network model, each object category name is input into a Transformer-based bidirectional encoder representation model to obtain a sample object category vector output by the Transformer-based bidirectional encoder representation model, the sample satellite image feature vector and the sample object category vector are spliced ​​to obtain a first sample splicing vector, the first sample splicing vector is input into a Transformer-based encoder to obtain a second sample splicing vector output by the Transformer-based encoder, the first object category vector is determined from the second sample splicing vector, and based on the first object category vector, the first object category corresponding to the sample satellite image and the first classification prediction value corresponding to each first object category are determined. The sample satellite image can be processed step by step using a first neural network model obtained by combining a convolutional neural network, a Transformer-based bidirectional encoder representation model and a Transformer-based encoder to obtain a first object category vector, a first object category and a first classification prediction value, which fully utilizes the performance advantages of each mature model and improves the accuracy of the first object category vector, the first object category and the first classification prediction value to a certain extent.

[0049] Step 206: Determine a binary cross entropy loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value.

[0050] In an embodiment of the present application, the binary cross entropy loss value of the first neural network model can be calculated according to the sample object category vector, the first object category vector and the first classification prediction value of the sample satellite image, and the calculation method thereof is shown in the following formulas 1 and 2: (Formula 1) (Formula 2) In the above formula 1 and formula 2, Represents the binary cross entropy loss value, represents the binary cross entropy loss function, is the number of sample satellite images, C is the number of all object categories that the sample satellite images may contain, represents the first classification prediction value of category j contained in sample satellite image i, Indicates whether the sample satellite image i contains object category j, The value can be based on With the first threshold The size relationship is obtained if ,but , indicating that the sample satellite image i contains object category j, if ,but , indicating that the sample satellite image i does not contain the object category j. When performing calculations, the correspondence between the first classification prediction value and each object category contained in the sample satellite image can be determined according to the correspondence between the sample object category vector and each element in the first object category vector. Then, according to the above correspondence, the first classification prediction value can be substituted into Formula 1 and Formula 2 to calculate the binary cross entropy loss value of the first neural network model.

[0051] Step 207: Determine a second model loss value of the first neural network model based on the first classification prediction value, the sample object category vector and the first object category vector.

[0052] In an embodiment of the present application, the second model loss value of the first neural network model can be calculated according to the first classification prediction value, the sample object category vector of the sample satellite image, and the first object category vector, and the calculation method thereof is shown in the following formula 3 and formula 4: (Formula 3) (Formula 4) In the above formula 3 and formula 4, represents the loss value of the second model, represents the number of sample satellite images that do not contain the noise vector, Indicates that sample satellite image i contains the first classification prediction value of object category j, The sample feature representation of object category j in sample satellite image i can be obtained by transforming the name of object category j through the BERT model. The first feature representation representing the object category j in the sample satellite image i can be obtained by processing the sample satellite image by the first neural network model. The calculation process of Expression 4 is shown. When performing the calculation, the correspondence between the first classification prediction value and each object category included in the sample satellite image can be determined according to the correspondence between the sample object category vector and each element in the first object category vector, and then according to the above correspondence, the first classification prediction value can be substituted into Expressions 3 and 4 to calculate the binary cross entropy loss value of the first neural network model.

[0053] Optionally, step 207 may include the following sub-steps: Sub-step 2071 , performing semantic transformation on the first object category vector to obtain a second object category vector; wherein the semantic space of the second object category vector is the same as the semantic space of the sample object category vector.

[0054] In an embodiment of the present application, the first object category vector may be semantically transformed to obtain a second object category vector. During the transformation, the semantic space where the sample object category vector is located may be used as a reference space, and the transformation may be performed through a linear function between the semantic space of the first object category vector and the semantic space of the sample object category vector, so that the semantic space of the obtained second object category vector may be the same as the semantic space of the sample object category vector.

[0055] Sub-step 2072: Calculate the first cosine similarity between the second object category vector and the sample object category vector.

[0056] In an embodiment of the present application, the first cosine similarity between the second object category vector and the sample object category vector can be calculated, as shown in Formula 4: part.

[0057] Sub-step 2073, determining a second model loss value of the first neural network model based on the first cosine similarity and the first classification prediction value.

[0058] In an embodiment of the present application, the second model loss value of the first neural network model can be calculated according to the first cosine similarity and the first classification prediction value, and the calculation method thereof is shown in Formula 3 and Formula 4 above, wherein, That is the first cosine similarity, The meanings of the remaining parameter symbols can be referred to the embodiment content under step 207, and will not be repeated here.

[0059] Optionally, sub-step 2073 may include the following sub-steps: Sub-step A1: determining a second cosine similarity corresponding to a clean sample satellite image and a third cosine similarity corresponding to a noisy sample satellite image from the first cosine similarity; wherein the clean sample satellite image and the noisy sample satellite image are both included in the sample satellite image, and the noisy sample satellite image is generated based on the clean sample satellite image and a noise vector.

[0060] In an embodiment of the present application, the sample satellite image may include a clean sample satellite image and a noisy sample satellite image. The clean sample satellite image does not contain a noise vector, and the noisy sample satellite image can be obtained by adding a noise vector to the clean sample satellite image. Therefore, the first cosine similarity also includes the second cosine similarity corresponding to the clean sample satellite image and the third cosine similarity corresponding to the noisy sample satellite image. Thus, the second cosine similarity and the third cosine similarity can be determined from the first cosine similarity. The second cosine similarity can be expressed as , represents the cosine similarity of object category j contained in sample satellite image i, and the third cosine similarity can be expressed as , represents the cosine similarity of the noise object category j contained in the sample satellite image i.

[0061] Sub-step A2, calculating a third model loss value of the first neural network model based on the second cosine similarity, the third cosine similarity and the noise vector.

[0062] In an embodiment of the present application, the third model loss value of the first neural network model can be calculated according to the second cosine similarity, the third cosine similarity and the noise vector, and the calculation method thereof is shown in the following formula 5: (Formula 5) In the above formula 5, represents the loss value of the third model, The feature representation corresponding to the noise vector in object category j can be represented by the noise vector Obtained through conversion by BERT model. Indicates a positive number. Among them, the noise vector It can be set manually and added to the sample satellite image, so that the sample satellite image containing the noise vector can be obtained. The remaining parameter symbols can refer to the interpretation of the parameter symbols in equations 3 and 4, which will not be repeated here.

[0063] Sub-step A3, determining a second model loss value of the first neural network model based on the third model loss value and the first classification prediction value.

[0064] In an embodiment of the present application, the second loss value of the first neural network model can be calculated according to the third model loss value and the first classification prediction value, and the calculation method thereof is shown in Formula 6 below: (Formula 6) In the above formula 6, Represents the loss value of the third model. The interpretations of the remaining parameter symbols can be found in Formula 3 and Formula 4, which will not be repeated here.

[0065] In an embodiment of the present application, a second cosine similarity corresponding to a pure sample satellite image and a third cosine similarity corresponding to a noise sample satellite image are determined from the first cosine similarity; wherein the pure sample satellite image and the noise sample satellite image are both included in the sample satellite image, and the noise sample satellite image is generated based on the pure sample satellite image and the noise vector, a third model loss value of the first neural network model is calculated based on the second cosine similarity, the third cosine similarity and the noise vector, and a second model loss value of the first neural network model is determined based on the third model loss value and the first classification prediction value. The second model loss value of the first neural network model can be calculated using different image features of the pure sample satellite image and the noise sample satellite image, thereby improving the accuracy and reliability of the second model loss value to a certain extent, and the prediction performance of the target neural network model finally obtained can be improved.

[0066] In an embodiment of the present application, a second object category vector is obtained by performing a semantic transformation on a first object category vector; wherein the semantic space of the second object category vector is the same as the semantic space of the sample object category vector, and a first cosine similarity between the second object category vector and the sample object category vector is calculated. Based on the first cosine similarity and the first classification prediction value, a second model loss value of the first neural network model is determined. The first object category vector can be transformed by a semantic transformation method, and then the first cosine similarity between the second object category vector obtained after the transformation and the sample object category vector is calculated. Finally, the second model loss value of the first neural network model can be designed and calculated based on the first cosine similarity, thereby avoiding calculation errors caused by different semantic spaces and improving the accuracy of the second model loss value to a certain extent.

[0067] Step 208: Determine a maximum likelihood estimation loss value of the first neural network model based on the first classification prediction value.

[0068] In an embodiment of the present application, the maximum likelihood estimation loss value of the first neural network can be calculated according to the first classification prediction value, and the calculation method thereof is shown in the following formulas 5 to 7: (Formula 5) (Formula 6) (Formula 7) In the above formulas 5 to 7, represents the maximum likelihood estimation loss value, represents the number of sample satellite images, Represents a sample satellite image The number of object categories in , Represents a sample satellite image The maximum Gumbel Softmax value for object category j in . is the similarity label of object category j. is the Gumbel Softmax function, Indicates the first classification prediction value of the object category j contained in the sample satellite image. represents the temperature coefficient, k is the intrinsic parameter in the GumbelSoftmax function, .

[0069] Optionally, step 208 may include the following sub-steps: Sub-step 2081, determining first Gumbel Softmax values ​​corresponding to the first classification prediction values ​​respectively.

[0070] In the embodiment of the present application, the first GumbelSoftmax values ​​corresponding to the first classification prediction values ​​can be calculated first, and the calculation method thereof is shown in the above formula 7.

[0071] Sub-step 2082: when the first Gumbel Softmax value is greater than or equal to a first threshold, inputting a first object category vector corresponding to the first Gumbel Softmax value into a multilayer perceptron to obtain a second classification probability value output by the multilayer perceptron.

[0072] In an embodiment of the present application, if the first Gumbel Softmax value is greater than or equal to the first threshold, the first object category vector corresponding to the first Gumbel Softmax value can be input into the multilayer perceptron to obtain the second classification probability value output by the multilayer perceptron. The above process can be expressed in the form of the following formula 8: (Formula 8) In the above formula 8, represents the second classification probability value for object category j, , which serves as a logo. is the propagation function of the multilayer perceptron, is the feature representation of object category j in the second object category vector.

[0073] Sub-step 2083: determining a first category identifier corresponding to the first object category vector based on the second classification probability value.

[0074] In an embodiment of the present application, the first category representation corresponding to the first object category vector may be obtained according to the second classification probability value, as shown in the following formula 9: = (Formula 9) In the above formula 9, represents the largest first Gumbel Softmax value. The meanings of the other parameter symbols can be referred to the interpretations of the parameter symbols in Formula 6 and Formula 8, which will not be repeated here.

[0075] Sub-step 2084: input the sample satellite image into the multi-layer perceptron to obtain a third classification probability value output by the multi-layer perceptron.

[0076] In an embodiment of the present application, the sample satellite image may be input into a multi-layer perceptron to obtain a third classification probability value output by the multi-layer perceptron, which is expressed as shown in the following formula 10: (Formula 10) In the above formula 10, It can be used as a reference for multi-layer perceptron propagation. represents sample satellite image i, Represents the third classification probability value.

[0077] Sub-step 2085, determining the maximum likelihood estimation loss value of the first neural network model based on the third classification probability value.

[0078] In an embodiment of the present application, the maximum likelihood estimation loss value of the first neural network model can be calculated based on the third classification probability value. The calculation method is shown in the above formula 5, where the meaning of each parameter symbol can be referred to the embodiment content under step 208 and will not be repeated here.

[0079] In an embodiment of the present application, by determining the first Gumbel Softmax values ​​corresponding to the first classification prediction values, when the first Gumbel Softmax values ​​are greater than or equal to the first threshold value, the first object category vector corresponding to the first Gumbel Softmax value is input into the multilayer perceptron to obtain the second classification probability value output by the multilayer perceptron, and based on the second classification probability value, the first category identifier corresponding to the first object category vector is determined, and the sample satellite image is input into the multilayer perceptron to obtain the third classification probability value output by the multilayer perceptron. Based on the third classification probability value, the maximum likelihood estimation loss value of the first neural network model is determined, and the Gumbel Softmax value can be used to replace the first classification prediction value and the first threshold value for numerical judgment, and then the third classification probability value corresponding to the sample satellite image is obtained by the multilayer perceptron, and the maximum likelihood estimation loss value of the first neural network model can be calculated according to the third classification probability value. Since the GumbelSoftmax function is differentiable, using the Gumbel Softmax value to replace the discrete discontinuous first classification prediction value can improve the accuracy and reliability of the maximum likelihood estimation loss value to a certain extent, and can improve the prediction performance of the target neural network model finally obtained.

[0080] Step 209: Determine a first model loss value of the first neural network model based on the binary cross entropy loss value, the second model loss value, and the maximum likelihood estimation loss value.

[0081] In an embodiment of the present application, the first model loss value of the first neural network model can be calculated according to the binary cross entropy loss value, the second model loss value and the maximum likelihood estimation loss value, and the calculation method thereof is shown in the following formula 11: (Formula 11) In the above formula 11, is the loss value of the first model, is the binary cross entropy loss value, is the loss value of the second model, is the maximum likelihood estimation loss value, is the regularization parameter of the second model loss value, which is used to balance the proportion of the second model loss value in the first model loss value.

[0082] In an embodiment of the present application, a binary cross entropy loss value of a first neural network model is determined based on a sample object category vector, a first object category vector and a first classification prediction value of a sample satellite image; a second model loss value of the first neural network model is determined based on the first classification prediction value, the sample object category vector and the first object category vector; a maximum likelihood estimation loss value of the first neural network model is determined based on the first classification prediction value; and a first model loss value of the first neural network model is determined based on the binary cross entropy loss value, the second model loss value and the maximum likelihood estimation loss value. The binary cross entropy loss value, the second model loss value and the maximum likelihood estimation loss value of the first neural network model can be calculated, and the binary cross entropy loss value, the second model loss value and the maximum likelihood estimation loss value can be combined to obtain the first model loss value of the first neural network model, which improves the accuracy and availability of the first model loss value to a certain extent, and can improve the prediction performance of the target neural network model finally obtained.

[0083] Step 210: Based on the first model loss value, adjust the model parameters of the first neural network model to obtain a target neural network model.

[0084] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 103, which will not be repeated here.

[0085] Reference Figure 3 , Figure 3 : is a flow chart of a high-quality satellite image retrieval method provided in an embodiment of the present application, and the method may include: Step 301: input the query image into the target neural network model to obtain the first target object category vector output by the target neural network model and the first target classification prediction value corresponding to the first target object category vector; wherein the target neural network model is obtained based on any of the model training methods described above.

[0086] In an embodiment of the present application, the query image may be an image having a certain number of object categories, and the query image may be input into the target neural network model, thereby obtaining the first target object category vector output by the target neural network model and the target classification prediction value corresponding to the first target object category vector.

[0087] Step 302: Determine a first target object category corresponding to a first target classification prediction value that is greater than or equal to a first threshold as a query category identifier of the query image.

[0088] In an embodiment of the present application, the number of first target classification prediction values ​​can be multiple, corresponding to each object category respectively. The first target classification prediction value can be compared with a first threshold, and the first target object category corresponding to the first target classification prediction value greater than or equal to the first threshold is determined as the query category identifier of the query image, which is used to query the corresponding target satellite image in the target satellite.

[0089] Step 303: Determine a query similarity identifier of the query image based on the cosine similarity between the first target object category vector and a standard category vector corresponding to a standard object category name.

[0090] In an embodiment of the present application, the cosine similarity between the first target object category vector and the standard category vector corresponding to the standard object category name may be calculated, and then a query similarity identifier of the query image may be generated based on the cosine similarity.

[0091] For example, the method for generating the query similarity identifier is shown in Formula 12 below: (Formula 12) In formula 12, is the cosine similarity between the feature representation of the first target object category vector and the feature representation of the standard category vector, where i represents the position of the element in the vector, represents the feature representation of position i in the standard category vector, Represents the feature representation of position i in the first target object category vector. represents the floor function, is the preset proportional constant. For example, if hour, .

[0092] Step 304: Based on the query category identifier and the query similarity identifier, search in the target satellite image database to obtain the target satellite image.

[0093] In the embodiment of the present application, in the image database of the target satellite, the image category identifier and image similarity identifier of the first satellite image contained in the database may also be pre-generated according to the method of step 302 and step 303. Thus, the image database of the target satellite may be matched by querying the category identifier and querying the similarity identifier. If the match is successful, the matched first satellite image may be used as the target satellite image.

[0094] Optionally, step 304 may include the following sub-steps: Sub-step 3041 , determining a first similarity between the image category identifier of each first satellite image included in the image database of the target satellite and the query category identifier, and a second similarity between the image similarity identifier of each first satellite image and the query similarity identifier.

[0095] In an embodiment of the present application, a first similarity between an image category identifier of each first satellite image contained in an image database of a target satellite and the query category identifier, and a second similarity between an image similarity identifier of each first satellite image and the query similarity identifier can be calculated. The categories of the first similarity and the second similarity can be cosine similarity.

[0096] Sub-step 3042: determining the first satellite image whose first similarity is greater than or equal to the second threshold and whose second similarity is greater than or equal to the third threshold as the target satellite image.

[0097] In an embodiment of the present application, screening may be performed among the first satellite images, and first satellite images having a first similarity greater than or equal to a second threshold and a second similarity greater than or equal to a third threshold may be determined as target satellite images.

[0098] In an embodiment of the present application, by determining a first similarity between an image category identifier and a query category identifier of each first satellite image contained in an image database of the target satellite, and a second similarity between an image similarity identifier and a query similarity identifier of each first satellite image, a first satellite image whose first similarity is greater than or equal to a second threshold and whose second similarity is greater than or equal to a third threshold is determined as a target satellite image. The target satellite image can be screened from the first satellite images according to the first similarity between the query category identifier and the image category identifier of the first satellite image, and the second similarity between the query similarity identifier and the image similarity identifier of the first satellite image, thereby improving the matching degree between the target image and the query image and improving the accuracy of the target satellite image retrieval.

[0099] In an embodiment of the present application, a query image is input into a target neural network model to obtain a first target object category vector output by the target neural network model and a first target classification prediction value corresponding to the first target object category vector; wherein the target neural network model is obtained based on any of the above model training methods, and the first target object category corresponding to the first target classification prediction value greater than or equal to the first threshold is determined as the query category identifier of the query image, and the query similarity identifier of the query image is determined based on the cosine similarity between the first target object category vector and the standard category vector corresponding to the standard object category name, and the query similarity identifier is retrieved in the image database of the target satellite based on the query category identifier and the query similarity identifier to obtain the target satellite image, and the query category identifier and the query similarity identifier corresponding to the query image can be obtained through the target neural network model, and the query category identifier and the query similarity identifier can be used to retrieve the target satellite image in the image database, thereby obtaining the target satellite image, and since two types of query identifiers are used, the matching degree between the target satellite image and the query image can be improved to a certain extent, and the accuracy of the satellite image retrieval query can be improved.

[0100] Reference Figure 4 , Figure 4 A logical block diagram of a model training device provided in an embodiment of the present application, the device 400 may include: The input-output module 401 is used to input the sample satellite image into the first neural network model to obtain the first object category vector output by the first neural network model and the first classification prediction value corresponding to the first object category vector; wherein the sample satellite image includes at least one object; A determination module 402, configured to determine a first model loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; The adjustment module 403 is used to adjust the model parameters of the first neural network model based on the first model loss value to obtain a target neural network model; wherein the target neural network model is used to generate a target object category of a query image and a target classification prediction value corresponding to each of the target object categories.

[0101] Optionally, the first neural network model is composed of a convolutional neural network model, a Transformer-based bidirectional encoder representation model and a Transformer-based encoder model, and the input-output module 401 includes: A first input-output submodule, used to input a sample satellite image into the convolutional neural network to obtain a sample satellite image feature vector output by the convolutional neural network model; A second input-output submodule, configured to input each object category name into the Transformer-based bidirectional encoder representation model to obtain a sample object category vector output by the Transformer-based bidirectional encoder representation model; A splicing submodule, used for splicing the sample satellite image feature vector and the sample object category vector to obtain a first sample splicing vector; A third input-output submodule, configured to input the first sample splicing vector into the Transformer-based encoder to obtain a second sample splicing vector output by the Transformer-based encoder; The first determination submodule is used to determine a first object category vector from the second sample splicing vector, and based on the first object category vector, determine a first object category corresponding to the sample satellite image and a first classification prediction value corresponding to each of the first object categories.

[0102] Optionally, the determining module 402 includes: A second determination submodule, configured to determine a binary cross entropy loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; A third determination submodule, configured to determine a second model loss value of the first neural network model based on the first classification prediction value, the sample object category vector, and the first object category vector; A fourth determination submodule, used to determine a maximum likelihood estimation loss value of the first neural network model based on the first classification prediction value; The fifth determination submodule is used to determine the first model loss value of the first neural network model based on the binary cross entropy loss value, the second model loss value and the maximum likelihood estimation loss value.

[0103] Optionally, the third determining submodule includes: A semantic transformation unit, configured to perform semantic transformation on the first object category vector to obtain a second object category vector; wherein the semantic space of the second object category vector is the same as the semantic space of the sample object category vector; a calculation unit, configured to calculate a first cosine similarity between the second object category vector and the sample object category vector; The first determining unit is used to determine a second model loss value of the first neural network model based on the first cosine similarity and the first classification prediction value.

[0104] Optionally, the first determining unit includes: A first determining subunit is used to determine a second cosine similarity corresponding to a clean sample satellite image and a third cosine similarity corresponding to a noise sample satellite image from the first cosine similarity; wherein the clean sample satellite image and the noise sample satellite image are both included in the sample satellite image, and the noise sample satellite image is generated based on the clean sample satellite image and the noise vector; A calculation subunit, configured to calculate a third model loss value of the first neural network model based on the second cosine similarity, the third cosine similarity and the noise vector; The second determining subunit is used to determine a second model loss value of the first neural network model based on the third model loss value and the first classification prediction value.

[0105] Optionally, the fourth determining submodule includes: A second determining unit, used to determine first Gumbel Softmax values ​​corresponding to the first classification prediction values ​​respectively; A first input-output unit is used for inputting a first object category vector corresponding to the first Gumbel Softmax value into a multilayer perceptron to obtain a second classification probability value output by the multilayer perceptron when the first Gumbel Softmax value is greater than or equal to a first threshold; a third determining unit, configured to determine a first category identifier corresponding to the first object category vector based on the second classification probability value; A second input-output unit is used to input the sample satellite image into the multi-layer perceptron to obtain a third classification probability value output by the multi-layer perceptron; The fourth determining unit is used to determine the maximum likelihood estimation loss value of the first neural network model based on the third classification probability value.

[0106] The model training device provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0107] Reference Figure 5 , Figure 5 A logical block diagram of a model training device provided in an embodiment of the present application, the device 500 may include: An input-output module 501 is used to input a query image into a target neural network model to obtain a first target object category vector output by the target neural network model and a first target classification prediction value corresponding to the first target object category vector; wherein the target neural network model is obtained based on the model training method according to any one of claims 1 to 6; A first determination module 502 is used to determine a second target object category corresponding to a first target classification prediction value that is greater than or equal to a first threshold as a query category identifier of the query image; A second determination module 503 is used to determine a query similarity identifier of the query image based on the cosine similarity between the first target object category vector and a standard category vector corresponding to a standard object category name; The retrieval module 504 is used to search the image database of the target satellite based on the query category identifier and the query similarity identifier to obtain the target satellite image.

[0108] Optionally, the retrieval module 504 includes: A first determination submodule is used to determine a first similarity between an image category identifier of each first satellite image included in an image database of a target satellite and the query category identifier, and a second similarity between an image similarity identifier of each first satellite image and the query similarity identifier; The second determination submodule is configured to determine, as a target satellite image, a first satellite image whose first similarity is greater than or equal to a second threshold and whose second similarity is greater than or equal to a third threshold.

[0109] The model training device provided in the embodiment of the present application can achieve Figure 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0110] The model training device and high-quality satellite image retrieval device in the embodiment of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a GPUBOX, a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0111] The model training device and high-quality satellite image retrieval device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, a Linux operating system, a Windows operating system, etc., or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0112] The present application embodiment provides an electronic device, see Figure 6 The electronic device 60 includes: a processor 601, a memory 602, and a computer program 6021 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the program, the model training method of the aforementioned embodiment or the high-quality satellite image retrieval method is implemented.

[0113] The embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the model training method disclosed in the embodiment of the present application, or the steps in the high-quality satellite image retrieval method, are implemented.

[0114] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the processor is executed to implement the model training method disclosed in the embodiment of the present application, or the steps in the high-quality satellite image retrieval method.

[0115] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0116] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0119] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0120] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0121] The above is a detailed introduction to a model training method, a high-quality satellite image retrieval method and a device provided by the present application. In this article, specific examples are used to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for a person skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A model training method, characterized in that: The method comprises: Inputting the sample satellite image into the first neural network model to obtain a first object category vector output by the first neural network model and a first classification prediction value corresponding to the first object category vector; wherein the sample satellite image includes at least one object; Determining a first model loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; Based on the first model loss value, the model parameters of the first neural network model are adjusted to obtain a target neural network model; wherein the target neural network model is used to generate target object categories of the query image and target classification prediction values ​​corresponding to each of the target object categories.

2. The method according to claim 1, characterized in that The first neural network model is composed of a convolutional neural network model, a Transformer-based bidirectional encoder representation model, and a Transformer-based encoder model. The sample satellite image is input into the first neural network model to obtain a first object category vector output by the first neural network model and a first classification prediction value corresponding to the first object category vector, including: Inputting the sample satellite image into the convolutional neural network to obtain a sample satellite image feature vector output by the convolutional neural network model; Inputting each object category name into the Transformer-based bidirectional encoder representation model to obtain a sample object category vector output by the Transformer-based bidirectional encoder representation model; Splicing the sample satellite image feature vector and the sample object category vector to obtain a first sample splicing vector; Inputting the first sample splicing vector into the Transformer-based encoder to obtain a second sample splicing vector output by the Transformer-based encoder; A first object category vector is determined from the second sample splicing vector, and based on the first object category vector, a first object category corresponding to the sample satellite image and first classification prediction values ​​corresponding to each of the first object categories are determined.

3. The method according to claim 1, characterized in that The determining, based on the sample object category vector of the sample satellite image, the first object category vector and the first classification prediction value, a first model loss value of the first neural network model comprises: Determining a binary cross entropy loss value of the first neural network model based on a sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; Determining a second model loss value of the first neural network model based on the first classification prediction value, the sample object category vector, and the first object category vector; Determining a maximum likelihood estimation loss value of the first neural network model based on the first classification prediction value; Based on the binary cross entropy loss value, the second model loss value and the maximum likelihood estimation loss value, a first model loss value of the first neural network model is determined.

4. The method according to claim 3, characterized in that The determining, based on the first classification prediction value, the sample object category vector and the first object category vector, a second model loss value of the first neural network model comprises: Performing semantic transformation on the first object category vector to obtain a second object category vector; wherein the semantic space of the second object category vector is the same as the semantic space of the sample object category vector; Calculating a first cosine similarity between the second object category vector and the sample object category vector; Based on the first cosine similarity and the first classification prediction value, a second model loss value of the first neural network model is determined.

5. The method according to claim 4, characterized in that The determining, based on the first cosine similarity and the first classification prediction value, a second model loss value of the first neural network model comprises: Determining a second cosine similarity corresponding to a clean sample satellite image and a third cosine similarity corresponding to a noise sample satellite image from the first cosine similarity; wherein the clean sample satellite image and the noise sample satellite image are both included in the sample satellite image, and the noise sample satellite image is generated based on the clean sample satellite image and the noise vector; Calculating a third model loss value of the first neural network model based on the second cosine similarity, the third cosine similarity and the noise vector; Based on the third model loss value and the first classification prediction value, a second model loss value of the first neural network model is determined.

6. The method according to claim 3, characterized in that The determining, based on the first classification prediction value, a maximum likelihood estimation loss value of the first neural network model comprises: Determine first Gumbel Softmax values ​​corresponding to the first classification prediction values ​​respectively; When the first Gumbel Softmax value is greater than or equal to a first threshold, inputting a first object category vector corresponding to the first Gumbel Softmax value into a multilayer perceptron to obtain a second classification probability value output by the multilayer perceptron; Determining a first category identifier corresponding to the first object category vector based on the second classification probability value; Inputting the sample satellite image into the multi-layer perceptron to obtain a third classification probability value output by the multi-layer perceptron; Based on the third classification probability value, a maximum likelihood estimation loss value of the first neural network model is determined.

7. A high-quality satellite image retrieval method, characterized in that: The method comprises: Inputting the query image into the target neural network model, obtaining a first target object category vector output by the target neural network model and a first target classification prediction value corresponding to the first target object category vector; wherein the target neural network model is obtained based on the model training method according to any one of claims 1 to 6; Determining a first target object category corresponding to a first target classification prediction value that is greater than or equal to a first threshold as a query category identifier of the query image; Determining a query similarity identifier of the query image based on a cosine similarity between the first target object category vector and a standard category vector corresponding to a standard object category name; Based on the query category identifier and the query similarity identifier, a search is performed in an image database of the target satellite to obtain the target satellite image.

8. The method according to claim 7, characterized in that The step of searching in an image database of a target satellite based on the query category identifier and the query similarity identifier to obtain the target satellite image comprises: Determine a first similarity between an image category identifier of each first satellite image included in an image database of the target satellite and the query category identifier, and a second similarity between an image similarity identifier of each first satellite image and the query similarity identifier; A first satellite image whose first similarity is greater than or equal to a second threshold and whose second similarity is greater than or equal to a third threshold is determined as a target satellite image.

9. A model training device, characterized in that: The device comprises: An input-output module, configured to input a sample satellite image into a first neural network model to obtain a first object category vector output by the first neural network model and a first classification prediction value corresponding to the first object category vector; wherein the sample satellite image includes at least one object; a determination module, configured to determine a first model loss value of the first neural network model based on the sample object category vector of the sample satellite image, the first object category vector, and the first classification prediction value; An adjustment module is used to adjust the model parameters of the first neural network model based on the first model loss value to obtain a target neural network model; wherein the target neural network model is used to generate a target object category of a query image and a target classification prediction value corresponding to each of the target object categories.

10. A high-quality satellite image retrieval device, characterized in that: The device comprises: An input-output module, used for inputting a query image into a target neural network model, and obtaining a first target object category vector output by the target neural network model and a first target classification prediction value corresponding to the first target object category vector; wherein the target neural network model is obtained based on the model training method according to any one of claims 1 to 6; A first determination module, configured to determine a second target object category corresponding to a first target classification prediction value that is greater than or equal to a first threshold as a query category identifier of the query image; A second determination module, configured to determine a query similarity identifier of the query image based on a cosine similarity between the first target object category vector and a standard category vector corresponding to a standard object category name; The retrieval module is used to search in the image database of the target satellite based on the query category identifier and the query similarity identifier to obtain the target satellite image.

Images