Model training method and device, electronic equipment and storage medium

Abstract

A model training method and device, electronic equipment and storage medium are provided. The method comprises: obtaining first local feature information and first global feature information corresponding to a first image and second local feature information and second global feature information corresponding to a second image by using a first feature extraction model, and obtaining third global feature information corresponding to the first image and fourth global feature information corresponding to the second image by using a second feature extraction model; fusing by a multi-head cross attention network unit to obtain first target feature information and second target feature information; determining a first similarity between the first target feature information and the second target feature information, and a second similarity between the third global feature information and the fourth global feature information; determining first loss information and second loss information and training an initial double-branch network to obtain a target feature extraction model corresponding to the second feature extraction model. The disclosure improves the generalization ability of the model.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of machine learning, and particularly relates to a model training method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Visual place recognition (VPR) has a wide range of applications in the field of autonomous driving. In autonomous driving, technology can help vehicles recognize and locate the current road scene, which will help vehicles make more accurate driving decisions.

[0003] In VPR technology, image retrieval is an important component. When trying to identify and locate a certain scene from a dataset, image retrieval is usually needed to find the most similar scene image to the input image to be processed, so as to make more accurate driving decisions for vehicles. However, in related technologies, the generalization ability of the feature extraction model of the image is insufficient, which may lead to poor accuracy and robustness of image retrieval when dealing with a wide range of situations and complex environments in real-world environments. SUMMARY

[0004] Embodiments of the present disclosure provide an implementation scheme different from related technologies to solve the technical problem of poor accuracy and robustness of image retrieval caused by insufficient generalization ability of the feature extraction model of the image in related technologies.

[0005] In a first aspect, the present disclosure provides a model training method, comprising:

[0006] obtaining a training image set, and a first feature extraction model and a second feature extraction model in a preset initial double-branch network, wherein the training image set comprises at least one training image pair, each training image pair in the at least one training image pair comprises a first image, a second image, and a pre-set target similarity of the first image and the second image;

[0007] for each training image pair, using the first feature extraction model to respectively extract features of the first image and the second image in the training image pair, to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image, and using the second feature extraction model to respectively extract features of the first image and the second image, to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image;

[0008] The first target feature information is used for representing an enhanced global feature corresponding to the first image, and the second target feature information is used for representing an enhanced global feature corresponding to the second image.

[0009] The first similarity between the first target feature information and the second target feature information is determined by a first similarity calculation unit in the initial double-branch network, and the second similarity between the third global feature information and the fourth global feature information is determined by a second similarity calculation unit in the initial double-branch network.

[0010] The first similarity is taken as a predicted value, and the second similarity is taken as a true value to determine first loss information, and the second similarity is taken as a predicted value, and the target similarity is taken as a true value to determine second loss information.

[0011] The initial double-branch network is trained based on the first loss information and the second loss information to obtain a trained target double-branch network, and then a trained target feature extraction model corresponding to the second feature extraction model is obtained, and the target feature extraction model is used for feature extraction on a to-be-processed image.

[0012] In a second aspect, the present disclosure provides a model training device, comprising:

[0013] An acquisition unit is configured to acquire a training image set, and a first feature extraction model and a second feature extraction model in a preset initial double-branch network, wherein the training image set includes at least one training image pair, each training image pair in the at least one training image pair includes a first image, a second image, and a target similarity between the first image and the second image which is set in advance;

[0014] A feature extraction unit is configured to, for each training image pair, perform feature extraction on the first image and the second image in the training image pair respectively by using the first feature extraction model, to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image, and perform feature extraction on the first image and the second image respectively by using the second feature extraction model, to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image.

[0015] a feature fusion unit, configured to fuse the first global feature information and the second local feature information by a multi-head cross attention network unit in the initial double-branch network to obtain first target feature information corresponding to the first image, and fuse the second global feature information and the first local feature information by the multi-head cross attention network unit to obtain second target feature information corresponding to the second image, the first target feature information being used to represent enhanced global features corresponding to the first image, and the second target feature information being used to represent enhanced global features corresponding to the second image;

[0016] a first determination unit, configured to determine a first similarity between the first target feature information and the second target feature information by a first similarity calculation unit in the initial double-branch network, and determine a second similarity between the third global feature information and the fourth global feature information by a second similarity calculation unit in the initial double-branch network;

[0017] a second determination unit, configured to determine first loss information by taking the first similarity as a predicted value and the second similarity as a true value, and determine second loss information by taking the second similarity as a predicted value and the target similarity as a true value;

[0018] a training unit, configured to train the initial double-branch network based on the first loss information and the second loss information to obtain a trained target double-branch network, and further obtain a trained target feature extraction model corresponding to the second feature extraction model, the target feature extraction model being used to perform feature extraction on a to-be-processed image.

[0019] In a third aspect, the present disclosure provides an electronic device, comprising:

[0020] a processor; and

[0021] a memory configured to store executable instructions of the processor;

[0022] The processor is configured to execute any of the methods in the first aspect and possible implementation manners of the first aspect by executing the executable instructions.

[0023] In a fourth aspect, an embodiment of the present disclosure provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement any of the methods in the first aspect and possible implementation manners of the first aspect.

[0024] The disclosure provides obtaining a training image set and preset first and second feature extraction models in an initial double-branch network, wherein the training image set includes at least one training image pair, each of the at least one training image pair includes a first image, a second image, and a preset target similarity between the first image and the second image; for each training image pair, the first and second images in the training image pair are respectively subjected to feature extraction by using the first feature extraction model to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image, and the first and second images are respectively subjected to feature extraction by using the second feature extraction model to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image; the first global feature information and the second local feature information are fused by a multi-head cross attention network unit in the initial double-branch network to obtain first target feature information corresponding to the first image, and the second global feature information and the first local feature information are fused by the multi-head cross attention network unit to obtain second target feature information corresponding to the second image, the first target feature information is used to represent enhanced global features corresponding to the first image, and the second target feature information is used to represent enhanced global features corresponding to the second image; a first similarity between the first target feature information and the second target feature information is determined by a first similarity calculation unit in the initial double-branch network, and a second similarity between the third global feature information and the fourth global feature information is determined by a second similarity calculation unit in the initial double-branch network; the first similarity is taken as a prediction value, the second similarity is taken as a true value, first loss information is determined, and the second similarity is taken as a prediction value, and the target similarity is taken as a true value, second loss information is determined; the initial double-branch network is trained based on the first loss information and the second loss information to obtain a trained target double-branch network, and then a trained target feature extraction model corresponding to the second feature extraction model is obtained, the target feature extraction model is used for a scheme of feature extraction on a to-be-processed image, a multi-head cross attention mechanism can be introduced in the initial double-branch network to fuse multiple feature sources (such as global features and local features) to capture high-correlation features between two images, generate more abundant and more targeted enhanced global features, and then pass the enhanced global features to the second feature extraction model in the initial double-branch network through training of the initial double-branch network, so that the trained second feature extraction model (target feature extraction model) learns more abundant feature representations, the target feature extraction model can perform better in different image data sets and tasks, and then the generalization ability of the feature extraction model can be improved to improve the accuracy and robustness of image retrieval. BRIEF DESCRIPTION OF DRAWINGS

[0025] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0026] Figure 1 This is a schematic diagram of the structure of the model training system provided in the embodiments of this disclosure;

[0027] Figure 2 A first flowchart illustrating the model training method provided in this embodiment of the disclosure;

[0028] Figure 3 This is a second flowchart illustrating the model training method provided in an embodiment of the present disclosure;

[0029] Figure 4 A schematic diagram of the third process of the model training method provided in the embodiments of this disclosure;

[0030] Figure 5 A schematic diagram of the fourth process of the model training method provided in the embodiments of this disclosure;

[0031] Figure 6 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure;

[0032] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0033] Embodiments of this disclosure are described in detail below, with examples of these embodiments illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting it.

[0034] The terms "first" and "second," etc., used in the specification, claims, and drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the present disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] First, the following explains some terms in the embodiments of the present disclosure to facilitate understanding by those skilled in the art.

[0036] A convolutional neural network (CNN) is a type of feedforward neural network that contains convolutional computation and has a deep structure, and has the ability of representation learning.

[0037] Local image features are local representations of image features, which reflect the local characteristics of images and are suitable for image matching, retrieval and other applications.

[0038] Global image features refer to features that can represent the entire image. Global features are relative to image local features and are used to describe the overall features of images or targets such as color and shape.

[0039] Visual place recognition (VPR) has a wide range of applications in the field of autonomous driving. In autonomous driving, VPR technology can help vehicles recognize and locate the current road scene, which will help vehicles make more accurate driving decisions.

[0040] In VPR technology, image retrieval is an important part. When trying to identify and locate a certain scene from a dataset, image retrieval is usually used to find the most similar scene image to the input image to be processed, and then make more accurate driving decisions for the vehicle. However, in related technologies, the generalization ability of the feature extraction model of the image is insufficient, which may lead to poor accuracy and robustness of image retrieval when dealing with a wide range of real-world environments and complex environments. Therefore, the present disclosure provides a model training method and device, an electronic device and a storage medium to solve the technical problem of insufficient generalization ability of the feature extraction model of the image in related technologies, which may lead to poor accuracy and robustness of image retrieval.

[0041] The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0042] Figure 1 The structure diagram of the model training system provided by the embodiments of the present disclosure is shown in the figure, which includes a terminal 10 and a server 20, etc. The terminal 10 and the server 20 are connected through a network, such as a wired or wireless network connection, etc.

[0043] The terminal 10 can be a vehicle-mounted terminal device and can be used to display a graphical user interface. The terminal 10 can be used to interact with a user through a graphical user interface, for example, by downloading and installing a corresponding client on the terminal 10 and running it, for example, by calling and running a corresponding applet, for example, by logging into a website to present a corresponding image user interface, and the like. In the embodiments of the present disclosure, the terminal 10 can be used by a related object, such as a related person, to upload an image to be processed. In the model training stage, the terminal 10 can also be used by the related object to upload a training image set. The terminal 10 can send the training image set to the server 20, so that the server 20 trains the initial double-branch network and feeds back a target feature extraction model included in the target double-branch network obtained by training to the terminal 10.

[0044] Optionally, the server 20 can also store the target feature extraction model locally, and use the target feature extraction model to perform feature extraction on the image to be processed to obtain corresponding matching feature information. After determining the matching feature information, the server 20 can perform image feature matching and image retrieval based on the matching feature information.

[0045] Optionally, the terminal 10 itself can also directly train the initial double-branch network according to the training image set uploaded by the related object, obtain a target feature extraction model in the corresponding target double-branch network, and perform feature extraction on the image to be processed to obtain corresponding matching feature information after obtaining the image to be processed. Then, the terminal 10 can send the matching feature information to the server 20, so that the server 20 performs image feature matching and image retrieval based on the matching feature information.

[0046] Optionally, the terminal 10 can also send the target feature extraction model obtained by training to the server 20, so that the server 20 performs feature extraction on the image to be processed to obtain corresponding matching feature information after obtaining the image to be processed, and performs image feature matching and image retrieval based on the matching feature information.

[0047] The following describes the present solution by taking the server 20 for training the initial double-branch network as an example. Specifically, in the model training stage, the server 20 can be used to:

[0048] obtain a training image set, and a first feature extraction model and a second feature extraction model in a preset initial double-branch network, wherein the training image set includes at least one training image pair, each training image pair in the at least one training image pair includes a first image, a second image, and a preset target similarity between the first image and the second image;

[0049] For each training image pair, the first image and the second image in the training image pair are respectively subjected to feature extraction by using the first feature extraction model, to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image, and the first image and the second image are respectively subjected to feature extraction by using the second feature extraction model, to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image;

[0050] The first global feature information and the second local feature information are fused by using a multi-head cross attention network unit in the initial double-branch network, to obtain first target feature information corresponding to the first image, and the second global feature information and the first local feature information are fused by using the multi-head cross attention network unit, to obtain second target feature information corresponding to the second image, the first target feature information is used to represent enhanced global feature corresponding to the first image, and the second target feature information is used to represent enhanced global feature corresponding to the second image;

[0051] A first similarity between the first target feature information and the second target feature information is determined by using a first similarity calculation unit in the initial double-branch network, and a second similarity between the third global feature information and the fourth global feature information is determined by using a second similarity calculation unit in the initial double-branch network;

[0052] The first similarity is used as a predicted value, and the second similarity is used as a true value, to determine first loss information, and the second similarity is used as a predicted value, and the target similarity is used as a true value, to determine second loss information;

[0053] The initial double-branch network is trained based on the first loss information and the second loss information, to obtain a trained target double-branch network, and then a trained target feature extraction model corresponding to the second feature extraction model is obtained, and the target feature extraction model is used to perform feature extraction on a to-be-processed image.

[0054] The specific embodiments are described in detail below. It should be noted that the description order of the following embodiments is not used as a limitation on the priority order of the embodiments.

[0055] Embodiments of the present disclosure provide a model training method, which can be executed by a terminal or a server, or jointly executed by a terminal and a server. The model training method is taken as an example to be executed by a server for description. Figure 2A first flowchart of a model training method provided by an embodiment of the present disclosure is shown in FIG. 21. The method includes steps S21-S26.

[0056] In S21, a training image set and preset initial double-branch network including a first feature extraction model and a second feature extraction model are obtained. The training image set includes at least one training image pair. Each training image pair includes a first image, a second image, and a preset target similarity between the first image and the second image.

[0057] In some embodiments, the preset initial double-branch network is an untrained model. Figure 3 A second flowchart of the model training method provided by an embodiment of the present disclosure is shown in FIG. 22. Figure 3 As shown in FIG. 22, the initial double-branch network can include a first branch network and a second branch network. The first branch network can be an auxiliary branch, and the second branch network can be a main branch. The first feature extraction model can be included in the first branch network, and the second feature extraction model can be included in the second branch network. The first feature extraction model and the second feature extraction model are deep learning models and can be used to extract image features related to a task from input image data.

[0058] In some embodiments, the preset target similarity between the first image and the second image can be manually annotated.

[0059] In S22, for each training image pair, the first feature extraction model is used to extract features from the first image and the second image in the training image pair, respectively, to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image. In addition, the second feature extraction model is used to extract features from the first image and the second image, respectively, to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image.

[0060] Figure 4 A third flowchart of the model training method provided by an embodiment of the present disclosure is shown in FIG. 23. Figure 4 As shown in FIG. 23, the first feature extraction model includes a first network unit and a second network unit. Optionally, the first network unit can include a convolutional neural network backbone (CNN backbone) network, which can be composed of multiple convolutional layers and pooling layers. The second network unit can include an aggregator network.

[0061] In some embodiments, in S22, the feature extraction of the first image and the second image in the training image pair respectively by the first feature extraction model comprises S221-S222:

[0062] S221, feature extraction of the first image and the second image in the training image pair respectively by the first network unit in the first feature extraction model, to obtain the first local feature information corresponding to the first image and the second local feature information corresponding to the second image;

[0063] In some embodiments, the first local feature information (Q_local) corresponding to the first image (Query image) is used to indicate the local image features of the first image, which can be specifically represented as a plurality of first local descriptors of the first image; the second local feature information (C_local) corresponding to the second image (Candidate image) is used to indicate the local image features of the second image, which can be specifically represented as a plurality of second local descriptors of the second image. The local descriptor (Local Descriptor) can be used to describe the feature vector of the local region of the image.

[0064] The following describes the determination process of the first local feature information corresponding to the first image:

[0065] Taking the first image as an example, the first image is input into the CNN backbone network in the first network unit, and the output first semantic information feature map corresponding to the first image can be obtained. The first semantic information feature map is the intermediate feature representation of the first image generated by the CNN backbone network, which contains the abstract features of the first image at different levels. The first semantic information feature map can be specifically a feature image with a D*H*W dimension, where D represents the depth, H represents the height, and W represents the width. In the first semantic information feature map, each element can be understood as a feature detector, which corresponds to a small region in the first image. The depth (D) of the first semantic information feature map represents the number of feature detectors, which can increase with the deepening of the network level, and the height (H) and the width (W) can represent the size of the first semantic information feature map in space.

[0066] Further, the first semantic information feature map can be discretized into (H*W) first local descriptors, each first local descriptor having D-dimensional feature information, where the D-dimensional feature information means that each local descriptor contains D numerical values, which describe the features of the local region, and (H*W) first local descriptors can be taken as the first local feature information corresponding to the first image.

[0067] Correspondingly, the first network unit is used to extract features of the second image to obtain (H*W) second local feature descriptors corresponding to the second image, and then the (H*W) second local feature descriptors are taken as the specific process of the second local feature information corresponding to the second image. Similar to the above steps, details are not repeated here.

[0068] S222, the first local feature information is aggregated by a second network unit in the first feature extraction model to obtain first global feature information corresponding to the first image, and the second local feature information is aggregated by the second network unit to obtain second global feature information corresponding to the second image.

[0069] In some embodiments, Figure 4 The second network unit in the first feature extraction model can include an aggregator network, which can be used to integrate and aggregate feature information of different levels. The aggregator network can specifically include convolutional layers, pooling layers, fully connected layers, etc.

[0070] In some embodiments, the aggregator network can be used to aggregate the first local feature information and the second local feature information based on a preset fusion strategy. The preset fusion strategy can include at least one of the following:

[0071] Additive fusion: add local descriptors (first local descriptors / second local descriptors) one by one to obtain global descriptors (first global descriptors / second global descriptors).

[0072] Multiplicative fusion: multiply local descriptors (first local descriptors / second local descriptors) one by one to obtain global descriptors (first global descriptors / second global descriptors).

[0073] Attention mechanism: learn an attention weight to weight and fuse local descriptors (first local descriptors / second local descriptors) to obtain global descriptors (first global descriptors / second global descriptors).

[0074] Transform function: use a transform function (such as linear transform, nonlinear transform, etc.) to fuse local descriptors (first local descriptors / second local descriptors) into global descriptors (first global descriptors / second global descriptors).

[0075] In some embodiments, the first global feature information corresponding to the first image can be represented as Q_global, and the second global feature information corresponding to the second image can be represented as C_global.

[0076] In some optional embodiments provided in this disclosure, the second feature extraction model includes a third network unit and a fourth network unit. In S22, the second feature extraction model is used to extract features from the first image and the second image respectively, including S223-S224:

[0077] S223. The first image and the second image are extracted by the third network unit in the second feature extraction model to obtain the third local feature information corresponding to the first image and the fourth local feature information corresponding to the second image.

[0078] Figure 5 This is a schematic diagram of the fourth process of the model training method provided in the embodiments of this disclosure, as shown below. Figure 5 As shown, in some embodiments, the third network unit may have the same structure and function as the first network unit, and the specific model parameters may be the same or different. The specific process of extracting features from the first image and the second image respectively using the third network unit in the second feature extraction model in step S223 to obtain the third local feature information corresponding to the first image and the fourth local feature information corresponding to the second image can be found in step S221 above, where the first network unit in the first feature extraction model extracts features from the first image and the second image in the training image pair to obtain the first local feature information corresponding to the first image and the second local feature information corresponding to the second image. This process will not be repeated here.

[0079] In some embodiments, when the relevant parameters in the third network unit are the same as the relevant parameters in the first network unit, the third network unit and the first network unit can be the same network unit. In this case, the first local feature information corresponding to the first image is the same as the corresponding third local feature information, and the second local feature information corresponding to the second image is the same as the corresponding fourth local feature information.

[0080] S224. The third local feature information is aggregated by the fourth network unit in the second feature extraction model to obtain the third global feature information corresponding to the first image, and the fourth local feature information is aggregated by the fourth network unit to obtain the fourth global feature information corresponding to the second image.

[0081] In some embodiments, the fourth network unit can be the same as the second network unit in structure and role, and the specific model parameters can be different. The specific process of performing feature aggregation on the third local feature information through the fourth network unit in the second feature extraction model in step S224 to obtain the third global feature information corresponding to the first image, and performing feature aggregation on the fourth local feature information to obtain the fourth global feature information corresponding to the second image can be referred to the specific process of performing feature aggregation on the first local feature information through the second network unit in the first feature extraction model to obtain the first global feature information corresponding to the first image, and performing feature aggregation on the second local feature information through the second network unit to obtain the second global feature information corresponding to the second image, which will not be described herein again.

[0082] S23, performing fusion on the first global feature information and the second local feature information through a multi-head cross attention network unit in the initial double-branch network to obtain first target feature information corresponding to the first image, and performing fusion on the second global feature information and the first local feature information through the multi-head cross attention network unit to obtain second target feature information corresponding to the second image, the first target feature information being used to represent enhanced global feature corresponding to the first image, and the second target feature information being used to represent enhanced global feature corresponding to the second image;

[0083] In some embodiments, the multi-head cross attention network unit can be included in the first branch network as described above (for example, Figure 4 ).

[0084] In some embodiments, the first target feature information and the second target feature information can be feature vectors, and can be specifically represented as Q_global_att and C_global_att respectively.

[0085] In the above scheme, the first global feature information Q_global is a Q vector, the second local feature information C_local is a K vector and a V vector, and the input to the multi-head cross attention unit is a vector, which can be denoted as C_global_att; similarly, the second global feature information C_global is a Q vector, the first local feature information Q_local is a K vector and a V vector, and the input to the multi-head cross attention unit is a vector, which can be denoted as Q_global_att. Wherein, the K vector and the V vector are vectors used to calculate attention weight.

[0086] In the above scheme, the multi-head cross-attention network unit fuses the first global feature information of the first image and the second local feature information of the second image to obtain the first target feature information, and fuses the first local feature information of the first image and the second global feature information of the second image to obtain the second target feature information. By fusing multiple feature sources, the ability and richness of feature representation are improved, which helps to improve the representation ability and prediction performance of the initial double-branch network, and provides richer feature representation for downstream tasks such as image retrieval, image classification, target detection and segmentation.

[0087] S24, determining a first similarity between the first target feature information and the second target feature information through a first similarity calculation unit in the initial double-branch network, and determining a second similarity between the third global feature information and the fourth global feature information through a second similarity calculation unit in the initial double-branch network;

[0088] In some embodiments, the first similarity calculation unit can be included in the above-mentioned first branch network (such as Figure 4 ), and the second similarity calculation unit can be included in the above-mentioned second branch network (such as Figure 5 ). That is, the first branch network can output the first similarity, and the second branch network can output the second similarity (such as Figure 3 ).

[0089] In other embodiments, the first similarity calculation unit and the second similarity calculation unit can be the same, that is, they are the same unit.

[0090] In some embodiments, the first similarity calculation unit and the second similarity calculation unit can calculate the similarity between the two feature information based on a preset similarity measurement method, wherein the preset similarity measurement method can be any one of the following:

[0091] Cosine similarity: calculate the cosine similarity between features, the value range is [-1, 1], and the value closer to 1 indicates greater similarity.

[0092] Euclidean distance: calculate the Euclidean distance between features, the value range is non-negative real number, and the smaller the value, the more similar.

[0093] Jaccard similarity coefficient: calculate the Jaccard similarity coefficient between features, the value range is [0, 1], and the value closer to 1 indicates greater similarity.

[0094] S25, determining a first loss information by taking the first similarity as a prediction value and the second similarity as a true value, and determining a second loss information by taking the second similarity as a prediction value and the target similarity as a true value;

[0095] In some embodiments, the first loss information can be calculated by using a preset function, for example, the preset function can be a cross-entropy loss function.

[0096] In some embodiments, Figure 3 The first loss information in the formula (1) can be expressed as:

[0097] L1=-(y*log(p)+(1-y)*log(1-p))

[0098] Wherein, y represents the second similarity, p represents the first similarity, and the range of y and p can be in the interval [0, 1].

[0099] The smaller the value of the first loss information L1 is, the smaller the difference between the first similarity (predicted value) and the second similarity (true value) is, and the better the performance of the model is.

[0100] In some embodiments, Figure 3 The second loss information in the formula (2) can be expressed as:

[0101] L2=-(z*log(y)+(1-z)*log(1-y))

[0102] Wherein, z represents the target similarity, y represents the second similarity, and the range of z and y can be in the interval [0, 1].

[0103] The smaller the value of the first loss information L2 is, the smaller the difference between the second similarity (predicted value) and the target similarity (true value) is, and the better the performance of the model is.

[0104] S26, training the initial double-branch network based on the first loss information and the second loss information to obtain a trained target double-branch network, and further obtaining a trained target feature extraction model corresponding to the second feature extraction model, the target feature extraction model being used for feature extraction on a to-be-processed image.

[0105] In the training process, the related parameters of the initial double-branch network can be adjusted by minimizing the cross-entropy loss function.

[0106] In some optional embodiments provided by the present disclosure, in S26, the training of the initial double-branch network based on the first loss information and the second loss information includes S261-S264:

[0107] S261, obtaining a first preset weight corresponding to the first loss information and a second preset weight corresponding to the second loss information;

[0108] In some embodiments, the first preset weight and the second preset weight are added together to be 1, and the first preset weight and the second preset weight can be hyperparameters for adjusting the importance of different parts in the loss function. By adjusting the weights of the loss function, the initial double-branch network can pay more attention to specific prediction errors during the training process, thereby improving the performance of the initial double-branch network.

[0109] S262, determining a total loss result according to the first loss information, the second loss information, the first preset weight and the second preset weight;

[0110] In some optional embodiments provided in the present disclosure, in S262, the total loss result is determined according to the first loss information, the second loss information, the first preset weight and the second preset weight, including S2621-S2623:

[0111] S2621, calculating a first product of the first loss information and the first preset weight;

[0112] S2622, calculating a second product of the second loss information and the second preset weight;

[0113] S2623, taking the first product and the second product as the total loss result.

[0114] In some embodiments, the total loss result can be expressed as:

[0115] L0=w1*L1+w2*L2

[0116] wherein w1 represents the first preset weight, L1 represents the first loss information, w2 represents the second preset weight, and L2 represents the second loss information.

[0117] S263, if the total loss result is less than a preset threshold, completing the training of the initial double-branch network;

[0118] S264, if the total loss result is not less than the preset threshold, adjusting the parameters in the initial double-branch network, and returning to perform the steps of extracting features from the first image and the second image in the training image pair by using the first feature extraction model, and extracting features from the first image and the second image by using the second feature extraction model, until the total loss result is less than the preset threshold, and completing the training of the initial double-branch network.

[0119] In the training process, the second similarity output in the second branch is used as the predicted value, the first similarity output in the first branch is used as the true value, the cross-entropy function is used to calculate the first loss information value, and the self-distillation learning mode is used for gradient back propagation; the second similarity output in the second branch is used as the predicted value, and the target similarity set in advance is used as the true value, the cross-entropy function is used to calculate the second loss information, and the supervised learning mode is used for gradient back propagation. By training the two branches at the same time, the initial double-branch network can learn more comprehensive information, and the generalization ability of the initial double-branch network is improved.

[0120] In some embodiments, the initial double-branch network after training can be a target double-branch network. The target double-branch network can include a target feature extraction model, which is the second feature extraction model after training. The target feature extraction model can be used for feature extraction, feature matching, etc. of images.

[0121] In some optional embodiments provided in the present disclosure, for the process of using the target feature extraction model to extract features of a to-be-processed image, the method further includes S31-S33:

[0122] S31, extracting features of the to-be-processed image by the target feature extraction model to obtain to-be-matched feature information corresponding to the to-be-processed image, wherein the to-be-processed image can be one or more images;

[0123] S32, performing feature matching based on the to-be-matched feature information and a plurality of reference feature information included in a preset database to determine target feature information matched with the to-be-matched feature information from the plurality of reference feature information;

[0124] In some embodiments, the plurality of reference feature information in the preset database can correspond to a plurality of reference images.

[0125] S33, determining a reference image corresponding to the target feature information as a target image corresponding to the to-be-processed image.

[0126] In some embodiments, the above scheme can be used in VPR technology in automatic driving. The target feature extraction model can be better generalized to scenes that have not been seen before and have different weather, building elements and seasons from the training image set, to help the vehicle to recognize and locate the current road scene, and to help the vehicle to make more accurate driving decisions in large-scale urban driving environments, thereby improving the safety of vehicle automatic driving. In addition, the low dimension of the image descriptor in the scheme facilitates the storage of large-scale urban scene descriptors, and can effectively improve the efficiency of feature matching and image retrieval.

[0127] The disclosure provides an acquisition training image set and a preset initial double-branch network, wherein the training image set includes at least one training image pair, each training image pair in the at least one training image pair includes a first image, a second image, and a preset target similarity of the first image and the second image; for each training image pair, the first image and the second image in the training image pair are respectively subjected to feature extraction by using a first feature extraction model to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image, and the first image and the second image are respectively subjected to feature extraction by using a second feature extraction model to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image; the first global feature information and the second local feature information are fused by using a multi-head cross attention network unit in the initial double-branch network to obtain first target feature information corresponding to the first image, and the second global feature information and the first local feature information are fused by using the multi-head cross attention network unit to obtain second target feature information corresponding to the second image, the first target feature information is used for representing enhanced global feature corresponding to the first image, and the second target feature information is used for representing enhanced global feature corresponding to the second image; a first similarity between the first target feature information and the second target feature information is determined by using a first similarity calculation unit in the initial double-branch network, and a second similarity between the third global feature information and the fourth global feature information is determined by using a second similarity calculation unit in the initial double-branch network; the first similarity is used as a prediction value, the second similarity is used as a true value, first loss information is determined, and the second similarity is used as a prediction value, and the target similarity is used as a true value, second loss information is determined; the initial double-branch network is trained based on the first loss information and the second loss information to obtain a trained target double-branch network, and then a trained target feature extraction model corresponding to the second feature extraction model is obtained, the target feature extraction model is used for a feature extraction scheme of a to-be-processed image, a multi-head cross attention mechanism can be introduced in the initial double-branch network, a plurality of feature sources (such as global features and local features) are fused, high correlation features between two images are captured, more abundant and more targeted enhanced global features are generated, and then the enhanced global features are transmitted to the second feature extraction model in the initial double-branch network by training the initial double-branch network, so that the trained second feature extraction model (the target feature extraction model) learns more abundant feature representations, the target feature extraction model branch can focus on feature information that needs to be focused on in the matching process when only inputting a single picture, can perform better in different image data sets and tasks, and then the generalization ability and the robustness of the feature extraction model can be improved, and further, the accuracy of image retrieval can be improved.

[0128] Figure 6 A structural schematic diagram of a model training device provided for an exemplary embodiment of the present disclosure is shown in FIG. 1.

[0129] The device comprises:

[0130] An acquisition unit 61 is configured to acquire a training image set and preset first and second feature extraction models in an initial double-branch network, wherein the training image set comprises at least one training image pair, each training image pair comprises a first image, a second image, and a preset target similarity between the first and second images.

[0131] A feature extraction unit 62 is configured to, for each training image pair, perform feature extraction on the first and second images in the training image pair respectively by using the first feature extraction model to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image, and perform feature extraction on the first and second images respectively by using the second feature extraction model to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image.

[0132] A feature fusion unit 63 is configured to fuse the first global feature information and the second local feature information by using a multi-head cross-attention network unit in the initial double-branch network to obtain first target feature information corresponding to the first image, and fuse the second global feature information and the first local feature information by using the multi-head cross-attention network unit to obtain second target feature information corresponding to the second image, wherein the first target feature information is used to represent enhanced global feature corresponding to the first image, and the second target feature information is used to represent enhanced global feature corresponding to the second image.

[0133] A first determination unit 64 is configured to determine a first similarity between the first target feature information and the second target feature information by using a first similarity calculation unit in the initial double-branch network, and determine a second similarity between the third global feature information and the fourth global feature information by using a second similarity calculation unit in the initial double-branch network.

[0134] A second determination unit 65 is configured to determine first loss information by taking the first similarity as a predicted value and the second similarity as a true value, and determine second loss information by taking the second similarity as a predicted value and the target similarity as a true value.

[0135] The training unit 66 is configured to train the initial double-branch network based on the first loss information and the second loss information to obtain a trained target double-branch network, and further obtain a trained target feature extraction model corresponding to the second feature extraction model, where the target feature extraction model is configured to perform feature extraction on a to-be-processed image.

[0136] In some optional embodiments provided by the present disclosure, the initial double-branch network comprises a first branch network and a second branch network, where the first branch network comprises the first feature extraction model, the multi-head cross attention network unit and the first similarity calculation unit, and the second branch network comprises the second feature extraction model and the second similarity calculation unit.

[0137] In some optional embodiments provided by the present disclosure, the first feature extraction model comprises a first network unit and a second network unit, and the feature extraction unit 62 is specifically configured to:

[0138] perform feature extraction on the first image and the second image in the training image pair respectively by using the first network unit in the first feature extraction model, to obtain first local feature information corresponding to the first image and second local feature information corresponding to the second image;

[0139] perform feature aggregation on the first local feature information by using the second network unit in the first feature extraction model, to obtain first global feature information corresponding to the first image, and perform feature aggregation on the second local feature information by using the second network unit, to obtain second global feature information corresponding to the second image.

[0140] In some optional embodiments provided by the present disclosure, the second feature extraction model comprises a third network unit and a fourth network unit, and the feature extraction unit 62 is specifically configured to:

[0141] perform feature extraction on the first image and the second image respectively by using the third network unit in the second feature extraction model, to obtain third local feature information corresponding to the first image and fourth local feature information corresponding to the second image;

[0142] The third local feature information is aggregated by a fourth network unit in the second feature extraction model to obtain third global feature information corresponding to the first image, and the fourth local feature information is aggregated by the fourth network unit to obtain fourth global feature information corresponding to the second image.

[0143] In some optional embodiments provided by the present disclosure, when the training unit 66 is used to train the initial double-branch network based on the first loss information and the second loss information, the training unit 66 is specifically configured to:

[0144] obtain a first preset weight corresponding to the first loss information and a second preset weight corresponding to the second loss information;

[0145] determine a total loss result according to the first loss information, the second loss information, the first preset weight, and the second preset weight;

[0146] if the total loss result is less than a preset threshold, the training of the initial double-branch network is completed;

[0147] if the total loss result is not less than the preset threshold, the parameters in the initial double-branch network are adjusted, and the step of performing feature extraction on the first image and the second image in the training image pair by using the first feature extraction model and performing feature extraction on the first image and the second image by using the second feature extraction model is returned until the total loss result is less than the preset threshold, and the training of the initial double-branch network is completed.

[0148] In some optional embodiments provided by the present disclosure, when the training unit 66 is used to determine a total loss result according to the first loss information, the second loss information, the first preset weight, and the second preset weight, the training unit 66 is specifically configured to:

[0149] calculate a first product of the first loss information and the first preset weight;

[0150] calculate a second product of the second loss information and the second preset weight;

[0151] take a sum of the first product and the second product as the total loss result.

[0152] In some optional embodiments provided by the present disclosure, the apparatus is further configured to:

[0153] perform feature extraction on the to-be-processed image by using a target feature extraction model to obtain to-be-matched feature information corresponding to the to-be-processed image;

[0154] perform feature matching based on the to-be-matched feature information and a plurality of reference feature information included in a preset database to determine target feature information matched with the to-be-matched feature information from the plurality of reference feature information;

[0155] determine a reference image corresponding to the target feature information as a target image corresponding to the to-be-processed image.

[0156] It should be understood that the device embodiments and the method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, details are not described here. Specifically, the device can perform the above-mentioned method embodiments, and the foregoing and other operations and / or functions of each module in the device are respectively for the corresponding flow in each method in the above-mentioned method embodiments, and for the sake of brevity, details are not described here.

[0157] The device of the embodiments of the present disclosure is described above in the perspective of functional modules. It should be understood that the functional modules can be realized by hardware, instructions in the form of software, or a combination of hardware and software modules. Specifically, each step of the method embodiments in the embodiments of the present disclosure can be completed by integrated logic circuits of hardware in the processor and / or instructions in the form of software, and the steps of the method disclosed in combination with the embodiments of the present disclosure can be directly embodied as hardware code processor execution completion, or executed by a combination of hardware and software modules in the code processor. Alternatively, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above-mentioned method embodiments in combination with its hardware.

[0158] Figure 7 is a schematic block diagram of an electronic device provided by the embodiments of the present disclosure, which can include:

[0159] The memory 701 is used to store computer programs and transmit the program codes to the processor 702. In other words, the processor 702 can call and run the computer programs from the memory 701 to implement the method in the embodiments of the present disclosure.

[0160] For example, the processor 702 can be used to execute the above-mentioned method embodiments according to the instructions in the computer program.

[0161] In some embodiments of the present disclosure, the processor 702 can include but is not limited to:

[0162] general processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.

[0163] In some embodiments of the present disclosure, the memory 701 includes, but is not limited to:

[0164] volatile memory and / or non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a synch link DRAM (SLDRAM), and a Direct Rambus RAM (DR RAM).

[0165] In some embodiments of the present disclosure, the computer program can be divided into one or more modules, which are stored in the memory 701 and executed by the processor 702 to complete the method provided by the present disclosure. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device.

[0166] As shown in Figure 7 , the electronic device can further include:

[0167] A transceiver 703, which can be connected to the processor 702 or the memory 701.

[0168] The processor 702 can control the transceiver 703 to communicate with other devices, specifically, can send information or data to other devices, or receive information or data sent by other devices. The transceiver 703 can include a transmitter and a receiver. The transceiver 703 can further include an antenna, and the number of antennas can be one or more.

[0169] It should be understood that various components in the electronic device are connected through a bus system, which includes a data bus, a power supply bus, a control bus, and a state signal bus in addition to a data bus.

[0170] The present disclosure also provides a computer storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the method of the above method embodiments. Alternatively, the present disclosure embodiments also provide a computer program product containing instructions, which, when executed by a computer, enables the computer to perform the method of the above method embodiments.

[0171] When implemented by using software, the computer program product can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When loaded and executed by a computer, the computer program instructions produce the flow or function according to the present disclosure embodiments in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0172] Those skilled in the art can understand that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present disclosure.

[0173] In several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0174] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, i.e. can be located in one place or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. For example, the functional modules in each embodiment of the present disclosure can be integrated in one processing module, or each module can be physically present separately, or two or more modules can be integrated in one module.

[0175] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any skilled person in the art can easily think of changes or replacements within the technical range disclosed in the present disclosure, which should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A model training method, characterized in that, The method comprises the following steps: obtaining a training image set and preset initial first and second feature extraction models in a double-branch network, wherein the training image set comprises at least one training image pair, each training image pair comprises a first image, a second image, and a preset target similarity between the first and second images; for each training image pair, performing feature extraction on the first and second images in the training image pair respectively by using the first feature extraction model to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image, and performing feature extraction on the first and second images respectively by using the second feature extraction model to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image; fusing the first global feature information and the second local feature information by using a multi-head cross-attention network unit in the initial double-branch network to obtain first target feature information corresponding to the first image, and fusing the second global feature information and the first local feature information by using the multi-head cross-attention network unit to obtain second target feature information corresponding to the second image, wherein the first target feature information is used to represent enhanced global feature corresponding to the first image, and the second target feature information is used to represent enhanced global feature corresponding to the second image; determining a first similarity between the first target feature information and the second target feature information by using a first similarity calculation unit in the initial double-branch network, and determining a second similarity between the third global feature information and the fourth global feature information by using a second similarity calculation unit in the initial double-branch network; determining first loss information by taking the first similarity as a predicted value and the second similarity as a true value, and determining second loss information by taking the second similarity as a predicted value and the target similarity as a true value; training the initial double-branch network based on the first loss information and the second loss information to obtain a trained target double-branch network, and further obtaining a trained target feature extraction model corresponding to the second feature extraction model, wherein the trained target feature extraction model is used to perform feature extraction on a to-be-processed image.

2. The method of claim 1, wherein, The initial double-branch network comprises a first branch network and a second branch network, wherein the first branch network comprises the first feature extraction model, the multi-head cross-attention network unit, and the first similarity calculation unit, and the second branch network comprises the second feature extraction model and the second similarity calculation unit.

3. The method of claim 1, wherein, The first feature extraction model comprises a first network unit and a second network unit, and the feature extraction on the first and second images in each training image pair by using the first feature extraction model comprises: extracting features of the first image and the second image in the first image pair respectively by a first network unit in the first feature extraction model to obtain first local feature information corresponding to the first image and second local feature information corresponding to the second image; aggregating features of the first local feature information by a second network unit in the first feature extraction model to obtain first global feature information corresponding to the first image, and aggregating features of the second local feature information by the second network unit to obtain second global feature information corresponding to the second image.

4. The method of claim 1, wherein, The second feature extraction model includes a third network unit and a fourth network unit, and the extracting features of the first image and the second image by the second feature extraction model respectively includes: extracting features of the first image and the second image respectively by a third network unit in the second feature extraction model to obtain third local feature information corresponding to the first image and fourth local feature information corresponding to the second image; aggregating features of the third local feature information by a fourth network unit in the second feature extraction model to obtain third global feature information corresponding to the first image, and aggregating features of the fourth local feature information by the fourth network unit to obtain fourth global feature information corresponding to the second image.

5. The method of claim 1, wherein, The training of the initial double-branch network based on the first loss information and the second loss information includes: obtaining a first preset weight corresponding to the first loss information and a second preset weight corresponding to the second loss information; determining a total loss result according to the first loss information, the second loss information, the first preset weight and the second preset weight; if the total loss result is less than a preset threshold, the training of the initial double-branch network is completed; if the total loss result is not less than the preset threshold, adjusting parameters in the initial double-branch network, and returning to execute the step of extracting features of the first image and the second image in the training image pair by the first feature extraction model and extracting features of the first image and the second image by the second feature extraction model until the total loss result is less than the preset threshold, and the training of the initial double-branch network is completed.

6. The method of claim 5, wherein, The determining of the total loss result according to the first loss information, the second loss information, the first preset weight and the second preset weight includes: calculating a first product of the first loss information and the first preset weight; calculating a second product of the second loss information and the second preset weight; taking a sum of the first product and the second product as the total loss result.

7. The method of claim 1, wherein, The method further includes: extracting features of the to-be-processed image by a target feature extraction model to obtain to-be-matched feature information corresponding to the to-be-processed image; Based on the to-be-matched feature information and a plurality of reference feature information included in a preset database, feature matching is performed to determine target feature information matched with the to-be-matched feature information from the plurality of reference feature information; A reference image corresponding to the target feature information is determined as a target image corresponding to the to-be-processed image.

8. A model training apparatus, comprising: Comprise: An acquisition unit is configured to acquire a training image set, and a first feature extraction model and a second feature extraction model in a preset initial double-branch network, wherein the training image set includes at least one training image pair, each training image pair in the at least one training image pair includes a first image, a second image, and a target similarity degree of the first image and the second image preset in advance; A feature extraction unit is configured to, for each training image pair, perform feature extraction on the first image and the second image in the training image pair respectively by using the first feature extraction model, to obtain first local feature information and first global feature information corresponding to the first image, and second local feature information and second global feature information corresponding to the second image, and perform feature extraction on the first image and the second image respectively by using the second feature extraction model, to obtain third global feature information corresponding to the first image, and fourth global feature information corresponding to the second image; A feature fusion unit is configured to fuse the first global feature information and the second local feature information by using a multi-head cross-attention network unit in the initial double-branch network, to obtain first target feature information corresponding to the first image, and fuse the second global feature information and the first local feature information by using the multi-head cross-attention network unit, to obtain second target feature information corresponding to the second image, wherein the first target feature information is used to represent enhanced global feature corresponding to the first image, and the second target feature information is used to represent enhanced global feature corresponding to the second image; A first determination unit is configured to determine a first similarity degree between the first target feature information and the second target feature information by using a first similarity calculation unit in the initial double-branch network, and determine a second similarity degree between the third global feature information and the fourth global feature information by using a second similarity calculation unit in the initial double-branch network; A second determination unit is configured to determine first loss information by taking the first similarity degree as a predicted value and the second similarity degree as a true value, and determine second loss information by taking the second similarity degree as a predicted value and the target similarity degree as a true value; A training unit is configured to train the initial double-branch network based on the first loss information and the second loss information, to obtain a trained target double-branch network, and further obtain a trained target feature extraction model corresponding to the second feature extraction model, wherein the target feature extraction model is used to perform feature extraction on a to-be-processed image.

9. An electronic device, comprising: Comprise: A processor; And A memory configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1-7 via execution of the executable instructions.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the method of any one of claims 1-7.

Images