Image processing method, device, electronic device and storage medium

Abstract

The present disclosure provides an image processing method, device, electronic device and storage medium, wherein the image processing method comprises: acquiring an image to be stored; extracting a first information vector of the image to be stored; the first information vector comprises a first expected feature vector corresponding to the image to be stored in a multidimensional space, and a first variance vector corresponding to the image to be stored in the multidimensional space, wherein the first expected feature vector is a multidimensional vector, and the first variance vector is a one-dimensional vector; and storing the first information vector corresponding to the image to be stored.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the popularization and promotion of deep learning, the technology of recognition and retrieval based on image visual features has gradually been applied to various industries. At present, face recognition technology has been widely used in security and security-related fields. The application scenarios mainly include face control, static library detection, passerby library trajectory search, etc.

[0003] At present, the principle of face recognition based on deep learning is to extract the feature vector composed of the eigenvalues ​​of the face image, determine the similarity between the face images by comparing the distance between the feature vectors, and then identify the collected face image.

[0004] When facial images are recognized based on this method, if the collected facial images are occluded or blurred, the accuracy of the collected feature vectors is low, and the accuracy of determining the similarity between facial images based on the feature vectors is also low. The recognition of the facial images cannot be completed, and the facial images need to be re-photographed, resulting in low image recognition efficiency. Summary of the invention

[0005] The embodiments of the present disclosure at least provide an image processing solution.

[0006] In a first aspect, an embodiment of the present disclosure provides an image processing method, including:

[0007] Get the image to be stored;

[0008] Extracting a first information vector of the image to be stored; the first information vector includes a first expected feature vector corresponding to the image to be stored in a multidimensional space, and a first variance vector corresponding to the image to be stored in the multidimensional space, the first expected feature vector is a multidimensional vector, and the first variance vector is a one-dimensional vector;

[0009] The image to be stored corresponds to the first information vector.

[0010] In the implementation of the present disclosure, for each image to be stored that is acquired, the distribution interval corresponding to the image to be stored in the multidimensional space can be jointly represented based on the first expected feature vector and the first variance vector corresponding to the image to be stored, and the image library can be stored according to the distribution interval corresponding to the image to be stored. In this way, when the image to be compared is received later, for example, for face image recognition, the similarity between each image to be compared and the stored image can be identified based on the distribution interval corresponding to each pre-stored face image. In this way, the method of characterizing the similarity of images by the similarity of the distribution interval can reduce the influence of the image quality on the similarity accuracy, thereby improving the recognition efficiency.

[0011] Furthermore, the first information vector corresponding to each image pre-stored here includes a multi-dimensional first expected feature vector and a one-dimensional first variance vector. Compared with the multi-dimensional variance vector, the one-dimensional first variance vector occupies less storage memory, which can save storage space.

[0012] In a possible implementation, extracting the first information vector of the image to be stored includes:

[0013] Inputting the image to be stored into a feature map extraction network in a pre-trained uncertain neural network to obtain a feature map corresponding to the image to be stored;

[0014] Inputting the feature map into a feature value extraction network in the uncertain neural network to obtain the first expected feature vector composed of feature values ​​corresponding to feature points of the image to be stored;

[0015] Inputting the feature map into a variance extraction network in the uncertainty neural network to obtain the first variance vector corresponding to the image to be stored;

[0016] The first information vector of the image to be stored is determined based on the first expected feature vector and the first variance vector corresponding to the image to be stored.

[0017] In the embodiment of the present disclosure, it is proposed to introduce an uncertain neural network to extract a first information vector corresponding to the image to be stored, so as to obtain a one-dimensional first variance vector corresponding to the image to be stored. In this way, when performing image comparison and recognition, the quality requirements for the image to be compared can be reduced, thereby improving the recognition efficiency.

[0018] In a possible implementation manner, determining the first information vector of the image to be stored based on the first expected feature vector and the first variance vector corresponding to the image to be stored includes:

[0019] The first information vector is obtained by concatenating the first variance vector and the first expected feature vector; or,

[0020] After replacing the eigenvalue corresponding to the target dimension in the first expected eigenvector with the first variance vector, the first information vector is obtained.

[0021] In the disclosed embodiment, when storing the first information vector corresponding to the image to be stored, the one-dimensional first variance vector is concatenated with the first expected feature vector, or the eigenvalue of the target dimension in the first expected feature vector is replaced, so that storage space can be saved when storing the first information vector.

[0022] In a possible implementation, the image processing method further includes:

[0023] Obtaining an image to be compared;

[0024] Extracting a second information vector of the image to be compared; the second information vector includes a second expected feature vector corresponding to the image to be compared in a multidimensional space, and a second variance vector corresponding to the image to be compared in the multidimensional space, the second expected feature vector is a multidimensional vector, and the second variance vector is a one-dimensional vector;

[0025] Based on the second information vector of the image to be compared and the first information vectors of each of a plurality of images in a pre-constructed image library, a target image matching the image to be compared is determined.

[0026] In the disclosed embodiment, for the acquired image to be compared, the distribution interval corresponding to the image to be compared in the multidimensional space can be jointly represented based on the second expected feature vector and the second variance vector corresponding to the comparison image. In this way, the similarity between the image to be compared and the stored image can be identified based on the distribution intervals corresponding to each pre-stored facial image. The method of characterizing the similarity of images by the similarity of the distribution intervals can reduce the influence of the image quality on the similarity accuracy, thereby improving the recognition efficiency.

[0027] In a possible implementation, before determining the target image matching the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of the plurality of images in the image library, the image processing method further includes:

[0028] Based on the second expected feature vector of the image to be compared and the first expected feature vectors corresponding to the plurality of images in the image library, candidate images whose first similarity with the image to be compared is located in the first set number in a sequence sorted from high to low are screened out from the image library.

[0029] In the disclosed embodiment, in the process of searching for candidate images through expected feature vectors, an existing matrix calculation library can be used to increase the search speed, thereby quickly screening out candidate images.

[0030] In a possible implementation, determining a target image matching the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of a plurality of images in the image library includes:

[0031] Determine a second similarity between the image to be compared and each of the candidate images based on the second information vector of the image to be compared and the first information vector of each of the candidate images;

[0032] The candidate image having the highest second similarity with the image to be compared and whose second similarity is greater than a preset threshold is selected as the target image to be matched with the image to be compared.

[0033] In the disclosed embodiment, after obtaining a set number of candidate images, the target image that is most similar to the image to be compared is determined according to the first information vector of the image to be compared and the second information vector of each candidate image. This process does not require one-to-one comparison with each image in the image library, thereby improving the efficiency of determining the target image.

[0034] In a possible implementation, determining the second similarity between the image to be compared and each of the candidate images based on the second information vector of the image to be compared and the first information vector of each of the candidate images includes:

[0035] For each of the candidate images, determining the cosine similarity between the image to be compared and one of the candidate images based on the second expected feature vector corresponding to the image to be compared and the first expected feature vector corresponding to one of the candidate images in each of the candidate images;

[0036] A second similarity between the image to be compared and the one candidate image is determined based on the cosine similarity, the second variance vector corresponding to the image to be compared and the first variance vector corresponding to the one candidate image.

[0037] In the embodiment of the present disclosure, when a multi-dimensional variance vector is replaced by a scalar one-dimensional first variance vector to represent the degree of deviation between each feature vector corresponding to the image in the multi-dimensional space and the expected feature vector, the process of determining the similarity between the image to be compared and any candidate image in an uncertain manner can be simplified, thereby improving image recognition efficiency.

[0038] In a possible implementation, the image processing method further includes: training an uncertainty neural network model to obtain the trained uncertainty neural network;

[0039] The training of the uncertainty neural network model to obtain the trained uncertainty neural network includes:

[0040] Acquire a sample image set, wherein the sample image set includes multiple sample images and sample category information corresponding to each sample image;

[0041] Based on the sample image set and the sample category information corresponding to each sample image, training the feature map extraction network to be trained in the neural network to be trained and the feature value extraction network to be trained in the neural network to be trained to obtain the trained feature map extraction network and the feature value extraction network;

[0042] Based on the positive sample image pairs corresponding to the same sample category information and the negative sample image pairs corresponding to different sample category information contained in the sample image set, the variance extraction network to be trained in the neural network to be trained is trained to obtain the variance extraction network that has been trained.

[0043] In a possible implementation, after obtaining the trained feature map extraction network and the feature value extraction network, the image processing method further includes:

[0044] Inputting the sample image set into the feature map extraction network and the feature value extraction network in sequence to obtain a sample expected feature vector corresponding to each sample image;

[0045] The method of training the variance extraction network to be trained in the neural network to be trained based on the positive sample image pairs corresponding to the same sample category information and the negative sample image pairs corresponding to different sample category information contained in the sample image set to obtain the trained variance extraction network includes:

[0046] Input the sample image set and the positive sample image pair information and the negative sample image pair information contained in the sample image set into the variance extraction network to be trained in the neural network to be trained, and predict the predicted variance vector corresponding to each sample image; the positive sample image pair information corresponds to a pair of positive sample images with the same category information, and the negative sample image information corresponds to a pair of negative sample images with different category information;

[0047] Determine the loss function value corresponding to the variance extraction network to be trained based on the similarity between the predicted variance vectors corresponding to the positive sample image pairs and the sample expected feature vectors of the positive sample image pairs, and the similarity between the predicted variance vectors corresponding to the negative sample image pairs and the sample expected feature vectors of the negative sample image pairs;

[0048] Based on the loss function value, the network parameter value corresponding to the variance extraction network to be trained is adjusted to obtain the variance extraction network.

[0049] In the embodiments of the present disclosure, on the one hand, a one-dimensional variance vector is used to represent the degree of deviation between the feature vector corresponding to the image in the multidimensional space and the expected feature vector, which can simplify the process of determining the loss function value and improve the training speed of the uncertain neural network. On the other hand, by introducing positive sample image pairs and negative sample images to jointly train the one-dimensional variance vector, the accuracy of the variance extraction network can be further improved.

[0050] Furthermore, on the basis of improving the accuracy of the variance extraction network, a one-dimensional variance vector with higher accuracy can be extracted based on the variance extraction network. In this way, when the one-dimensional variance feature vector and the multi-dimensional expected feature vector are used to represent the information vector of the image to be stored for subsequent processing (such as image comparison, image retrieval, etc.), the accuracy and precision of the subsequent image processing are not affected.

[0051] In a second aspect, an embodiment of the present disclosure provides an image processing device, including:

[0052] An acquisition module, used for acquiring images to be stored;

[0053] an extraction module, configured to extract a first information vector of the image to be stored; the first information vector comprises a first expected feature vector corresponding to the image to be stored in a multidimensional space, and a first variance vector corresponding to the image to be stored in the multidimensional space, the first expected feature vector being a multidimensional vector, and the first variance vector being a one-dimensional vector;

[0054] A storage module is used to store the first information vector corresponding to the image to be stored.

[0055] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method described in the first aspect are performed.

[0056] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the image processing method described in the first aspect are executed.

[0057] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0059] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure is shown;

[0060] Figure 2 A flow chart of a method for determining a first information vector corresponding to an image to be stored provided by an embodiment of the present disclosure is shown;

[0061] Figure 3 A schematic diagram of the structure of an uncertain neural network provided by an embodiment of the present disclosure is shown;

[0062] Figure 4 A schematic diagram showing the occupancy of storage space when different first information vectors corresponding to an image are stored in the image library provided by the embodiment of the present disclosure;

[0063] Figure 5 A flow chart of a training method for an uncertain neural network provided by an embodiment of the present disclosure is shown;

[0064] Figure 6 A flow chart of a method for determining a target image that matches an image to be compared provided by an embodiment of the present disclosure is shown;

[0065] Figure 7 Another method flow chart of determining a target image to be matched with an image to be compared provided by an embodiment of the present disclosure is shown;

[0066] Figure 8 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure is shown;

[0067] Fig. 9 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0069] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0070] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0071] With the development of deep learning, image recognition technology has gradually emerged. Taking face recognition as an example, it has been applied to many fields, such as security control, target tracking, etc. In these fields, the category information corresponding to a large number of face images can be pre-stored to determine the category information corresponding to the received face image. Taking the security control field as an example, after receiving the image to be compared, the feature vector representing the facial features contained in the image to be compared can be extracted, and the feature vector can be compared with the feature vector corresponding to each face image in the pre-saved image library to determine the similarity between the image to be compared and each face image in the image library, and then the identity information corresponding to the image to be compared can be determined.

[0072] When determining the similarity of images based on feature vectors, if the image quality is poor, such as when there are occlusion or blur issues, the accuracy of the extracted feature vector is low. At this time, when determining the similarity between the image to be compared and each image in the image library based on the feature vector with low accuracy, it may be difficult to accurately find a facial image similar to the image to be compared. In this case, it is necessary to be prompted to re-upload the image to be compared with higher clarity, resulting in lower recognition efficiency.

[0073] Based on the above research, the present disclosure provides an image processing method, which can extract the expected feature vector corresponding to the image to be stored in the multidimensional space and the one-dimensional variance vector corresponding to the image in the multidimensional space for the acquired image to be stored. For example, the expected feature vector corresponding to the image to be stored can represent the position corresponding to the maximum probability density value of the image to be stored in the multidimensional space, and then combined with the variance vector corresponding to the image in the multidimensional space, the distribution interval corresponding to the image to be stored in the multidimensional space can be obtained. In this way, after the image to be stored is acquired, the information vector of the image to be stored can be stored, that is, the information vector including the expected feature vector and the variance vector corresponding to the one-dimensional variance vector of the image in the multidimensional space. Feature vector and variance vector. Later, after receiving the image to be compared, the similarity between the image to be compared and the stored image can also be determined according to the information vector of the image to be compared and the information vector of the stored image. That is, the similarity of the images can be determined by the similarity of the distribution intervals. This uncertain method can reduce the impact of image quality on the similarity accuracy, thereby improving recognition efficiency. In addition, the information vector corresponding to each image pre-stored here includes a multi-dimensional expected feature vector and a one-dimensional variance vector. Compared with the multi-dimensional variance vector, the one-dimensional variance vector occupies less storage memory, which can save storage space.

[0074] To facilitate understanding of this embodiment, an image processing method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the image processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example, a terminal device or a server or other processing device. In some possible implementations, the image processing method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0075] See also Figure 1 FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, and the image processing method includes the following S101 to S103:

[0076] S101, obtaining an image to be stored.

[0077] Exemplarily, the image to be stored proposed in the embodiment of the present disclosure is a captured face image. In different application fields, the source of the image to be stored may be different. For example, in a security control application scenario, the image to be stored may be captured by an image acquisition component pre-installed at the access control point.

[0078] S102, extracting a first information vector contained in the image to be stored; the first information vector includes an expected feature vector corresponding to the image to be stored in a multidimensional space, and a first variance vector corresponding to the image to be stored in the multidimensional space, wherein the first expected feature vector is a multidimensional vector, and the first variance vector is a one-dimensional vector.

[0079] Exemplarily, based on a eigenvalue extraction network in a pre-trained uncertain neural network, eigenvalues ​​corresponding to feature points of the face in the image to be stored, such as feature points representing the face and feature points representing the facial features, can be extracted to obtain a first expected feature vector composed of these eigenvalues.

[0080] Exemplarily, the multiple eigenvalues ​​included in the first expected eigenvector may correspond to multiple dimensions, and the image to be stored may be mapped to a multidimensional space according to the first expected eigenvector to obtain the position with the highest probability density value of the image to be stored in the multidimensional space.

[0081] Exemplarily, a first variance vector of one dimension corresponding to the image to be stored can be extracted according to the variance extraction network in a pre-trained uncertainty neural network. The first variance vector can represent the degree of deviation between most of the first eigenvectors corresponding to the image to be stored in the multidimensional space and the first expected eigenvector, that is, the multiple possible first eigenvectors corresponding to the image to be stored in the multidimensional space conform to the normal distribution with the first expected eigenvector as the mean and the first variance vector as the variance. Through this normal distribution, the distribution interval of the image to be stored in the multidimensional space can be obtained.

[0082] In addition, combined with the meaning of the variance value, when the first variance vector is larger, the first eigenvector corresponding to the image to be stored is more dispersed. Conversely, when the first variance vector is smaller, it means that most of the first eigenvectors corresponding to the image to be stored are concentrated near the first expected eigenvector. This means that the larger the first variance vector is, the more unstable the first eigenvector corresponding to the image to be stored is, and the lower the accuracy of the corresponding first expected eigenvector is. Conversely, the smaller the first variance vector is, the more stable the first eigenvector corresponding to the image to be stored is, and the higher the accuracy of the corresponding first expected eigenvector is. Therefore, the first variance vector can be used as an indicator to measure the accuracy of the first expected eigenvector. In this way, the first expected eigenvector and the first variance vector can be used as the first information vector of the image to be stored, so that the similarity between images can be determined based on the first information vector in the later stage.

[0083] Similar to the first expected feature vector, in general, when the image to be stored is subjected to feature extraction in a multi-dimensional space, the obtained first expected feature vector is multi-dimensional, and the corresponding variance feature vector is also multi-dimensional. For example, when the image features are extracted in a 256-dimensional space, the corresponding first expected feature vector is 256-dimensional, and the corresponding variance feature vector is also 256-dimensional. The present invention reduces the multi-dimensional variance vector to a one-dimensional first variance vector, and it has been verified that when the one-dimensional first variance feature vector and the multi-dimensional first expected feature vector are used to represent the first information vector of the image to be stored for subsequent processing (such as image comparison, image retrieval, etc.), the accuracy and precision of the subsequent image processing are not affected. In addition, since the number of variance values ​​contained in the one-dimensional first variance vector is much smaller than the number of variance values ​​contained in the multi-dimensional variance vector, when the one-dimensional first variance vector is used to constitute the first information vector of the image to be stored, the storage space can be reduced.

[0084] S103: Store a first information vector corresponding to the image to be stored.

[0085] After obtaining the first information vector corresponding to the image to be stored, it can be stored according to the storage requirements of different application scenarios. Taking the construction of a storage repository for face identity recognition as an example, when constructing the storage repository, the image to be stored and the corresponding first information vector can be associated and stored in the storage repository.

[0086] In addition, when constructing a storage repository for face recognition, in addition to the first information vector corresponding to the image to be stored, the category information corresponding to the image to be stored can also be stored. Specifically, the user identity corresponding to each image can be encoded. For example, for the company's access control system, the code corresponding to each employee in the company can be pre-stored, and the code corresponding to each employee can be stored as the category information of the employee.

[0087] In the implementation of the present disclosure, for each image to be stored that is acquired, the distribution interval corresponding to the image to be stored in the multidimensional space can be jointly represented based on the first expected feature vector and the first variance vector corresponding to the image to be stored, and the image library can be stored according to the distribution interval corresponding to the image to be stored. In this way, when the image to be compared is received later, for example, for face image recognition, the similarity between each image to be compared and the stored image can be identified based on the distribution interval corresponding to each pre-stored face image. In this way, the method of characterizing the similarity of images by the similarity of the distribution interval can reduce the influence of the image quality on the similarity accuracy, thereby improving the recognition efficiency.

[0088] Furthermore, the first information vector corresponding to each image pre-stored here includes a multi-dimensional first expected feature vector and a one-dimensional first variance vector. Compared with the multi-dimensional variance vector, the one-dimensional first variance vector occupies less storage memory, which can save storage space.

[0089] The above S101 to S103 will be described below in conjunction with specific embodiments.

[0090] With respect to the above S102, when extracting the first information vector contained in the image to be stored, such as Figure 2 As shown, the following S201 to S204 may be included:

[0091] S201, inputting the image to be stored into a feature map extraction network in a pre-trained uncertain neural network to obtain a feature map corresponding to the image to be stored;

[0092] S202, inputting the feature map into a feature value extraction network in an uncertain neural network to obtain a first expected feature vector composed of feature values ​​corresponding to feature points of the image to be stored;

[0093] S203, inputting the feature map into a variance extraction network in an uncertain neural network to obtain a first variance vector corresponding to the image to be stored;

[0094] S204: Determine a first information vector corresponding to the image to be stored based on the first expected feature vector and the first variance vector corresponding to the image to be stored.

[0095] Specifically, an uncertain neural network is introduced here, such as Figure 3 As shown, the uncertain neural network may include a feature map extraction network, a eigenvalue extraction network and a variance extraction network, wherein the feature map extraction network is used to extract feature maps for the image to be stored, and obtain feature maps corresponding to the image to be stored, and then the feature maps are respectively input into the eigenvalue extraction network and the variance extraction network to obtain a first expected feature vector composed of eigenvalues ​​of multiple feature points corresponding to the image to be stored, and a first variance vector corresponding to the image to be stored.

[0096] The first variance vector here is a one-dimensional vector and can be regarded as a scalar. After obtaining the first expected feature vector and the first variance vector corresponding to the image to be stored, the first information vector corresponding to the image to be stored can be constructed in a preset combination manner.

[0097] In the embodiment of the present disclosure, it is proposed to introduce an uncertain neural network to extract a first information vector corresponding to the image to be stored, so as to obtain a one-dimensional first variance vector corresponding to the image to be stored. In this way, when performing image comparison and recognition, the quality requirements for the image to be compared can be reduced, thereby improving the recognition efficiency.

[0098] Exemplarily, when determining the first information vector of the image to be stored based on the first expected feature vector and the first variance vector corresponding to the image to be stored, the following two methods may be included:

[0099] The first method: concatenate the first variance vector and the first expected feature vector to obtain a first information vector;

[0100] The second method is to obtain a first information vector by replacing the eigenvalue corresponding to the target dimension in the first expected eigenvector with the first variance vector.

[0101] For the first method, when splicing the first variance vector with the first expected feature vector, the first variance vector can be spliced ​​at the position after the last dimension of the first expected feature vector. For example, if the first expected feature vector is a feature vector with a dimension of 256, after splicing the first variance vector of one dimension, a first information vector with a dimension of 257 is obtained.

[0102] For the second method, the eigenvalue corresponding to the target dimension in the first expected feature vector can be directly replaced by the first variance vector of one dimension. For example, the first expected feature vector is a feature vector with a dimension of 256. After replacing the eigenvalue of the last dimension in the first expected feature vector with the first variance vector of one dimension, a first information vector with a dimension of 256 is obtained. The first information vector obtained in this way has the same dimension as the first expected feature vector, and no additional storage space is occupied.

[0103] Taking the second case above as an example, combined with Figure 4 As shown, (a) is the storage space occupied when the feature vector corresponding to each image is stored in the image library, (b) is the storage space occupied when the first expected feature vector corresponding to each image and the variance vector of the same dimension as the first expected feature vector are stored in the image library, and (c) is the storage space occupied when the first expected feature vector corresponding to each image and the first variance vector of one dimension are stored in the image library. It can be seen that compared with (a), although the variance vector is increased in (c), the additional storage space occupied is not increased, and the storage space occupied by (c) is smaller than that of (b).

[0104] In the disclosed embodiment, when storing the first information vector corresponding to the image to be stored, the one-dimensional first variance vector is concatenated with the first expected feature vector, or the eigenvalue of the target dimension in the first expected feature vector is replaced, so that storage space can be saved when storing the first information vector.

[0105] Specifically, with respect to the uncertainty neural network mentioned above, the image processing method provided by the embodiment of the present disclosure further includes: training the uncertainty neural model to obtain a trained uncertainty neural network;

[0106] Among them, the uncertainty neural model is trained to obtain the trained uncertainty neural network, such as Figure 5 As shown, it includes the following S301 to S303:

[0107] S301, obtaining a sample image set, where the sample image set includes multiple sample images and sample category information corresponding to each sample image.

[0108] The sample category information here is similar to the category information mentioned above and will not be repeated here.

[0109] S302, based on the sample image set and the sample category information corresponding to each sample image, train the feature map extraction network to be trained in the neural network to be trained and the feature value extraction network to be trained in the neural network to be trained to obtain the trained feature map extraction network and feature value extraction network.

[0110] Specifically, the sample image set can be grouped to obtain multiple groups of sample images, each group of sample images can contain multiple sample images, and starting from the first group of sample images, the group of sample images can be input into the feature map extraction network to be trained to obtain the predicted feature map corresponding to each sample image, and then the predicted feature map corresponding to the group of sample images can be input into the feature value extraction network to be trained to obtain the predicted category information, and then based on the sample category information and the predicted category information corresponding to the group of sample images, the function loss value of the loss function corresponding to the feature map extraction network to be trained and the feature value extraction network to be trained is determined, and then the network parameter values ​​of the feature map extraction network to be trained and the feature value extraction network to be trained are adjusted based on the function loss value, and then the next group of sample images is further replaced to continue the above operation until the set number of training times is reached, or the loss functions corresponding to the feature map extraction network to be trained and the feature value extraction network to be trained converge, to obtain the trained feature map extraction network and feature value extraction network.

[0111] S303, based on the positive sample image pairs corresponding to the same sample category information and the negative sample image pairs corresponding to different sample category information contained in the sample image set, the variance extraction network to be trained in the neural network to be trained is trained to obtain a trained variance extraction network.

[0112] Exemplarily, a positive sample image pair includes two sample images corresponding to the same sample category information, such as two sample images representing the same user "Zhang San" can be used as a pair of positive sample image pairs; a negative sample image pair includes two sample images corresponding to different sample category information, such as a sample image representing "Zhang San" and a sample image representing "Li Si" can be used as a pair of negative sample image pairs.

[0113] Specifically, after obtaining the trained feature map extraction network and feature value extraction network, the image processing method further includes:

[0114] The sample image set is input into the feature map extraction network and the eigenvalue extraction network in turn to obtain the sample expected feature vector corresponding to each sample image.

[0115] After obtaining the trained feature map extraction network and eigenvalue extraction network, the sample expected feature vector corresponding to each sample image can be further obtained based on the network parameter values ​​of the feature map extraction network and the network parameter values ​​of the eigenvalue extraction network. The meaning of the sample expected feature vector here is similar to the meaning of the first expected feature vector mentioned above, and will not be repeated here.

[0116] After obtaining the sample expected feature vector corresponding to each sample image, based on the positive sample image pairs corresponding to the same sample category information and the negative sample image pairs corresponding to different sample category information contained in the sample image set, the variance extraction network to be trained in the neural network to be trained is trained to obtain the trained variance extraction network, which may include:

[0117] S3031, input the sample image set and the positive sample image pair information and negative sample image pair information contained in the sample image set into the variance extraction network to be trained in the neural network to be trained, and predict the predicted variance vector corresponding to each sample image set; the positive sample image pair information corresponds to a pair of positive sample images with the same category information, and the negative sample image information corresponds to a pair of negative sample images with different category information.

[0118] Similarly, here, each group of sample images is grouped and input into the variance extraction network to be trained. Before inputting into the variance extraction network to be trained, the positive sample image pairs contained in the group of sample images can be annotated to obtain the positive sample image pair information contained in the group of sample images, and the negative sample image pairs contained in the group of sample images can be annotated to obtain the negative sample image pair information contained in the group of sample images.

[0119] S3032, based on the similarity between the predicted variance vectors corresponding to the positive sample image pairs and the sample expected feature vectors of the positive sample image pairs, and the similarity between the predicted variance vectors corresponding to the negative sample image pairs and the sample expected feature vectors of the negative sample image pairs, determine the loss function value corresponding to the variance extraction network to be trained.

[0120] As mentioned above, the sample feature vector corresponding to each sample image (the corresponding position in the multidimensional space) obeys the normal distribution. Here, we can introduce the normal distribution formula obeyed by the sample feature vector of the sample image. i And the sample image X j For example, sample image X i The normal distribution of the sample feature vector can be expressed by the following formula (1):

[0121] p(z i |X i )=N(μ i ,σ i 2 I) (1);

[0122] Among them, z i Represents the sample image X i Category information of i Represents the sample image X i The sample feature vector corresponding to the sample expected feature vector; σ i 2 Represents the sample image X i The sample variance vector corresponding to the sample eigenvector of ; I represents the unit diagonal matrix of the same dimension as the sample expected eigenvector.

[0123] Sample Image X j The normal distribution of the sample feature vector can be expressed by the following formula (2):

[0124] p(z j |X j )=N(μ j ,σ j 2 I) (2);

[0125] Among them, z j Represents the sample image X j Category information of j Represents the sample image X j The sample feature vector corresponding to the sample expected feature vector; σ j 2 Represents the sample image X jThe sample variance vector corresponding to the sample eigenvector of ; I represents the unit diagonal matrix of the same dimension as the sample expected eigenvector.

[0126] Furthermore, the sample image X i And the sample image X j The normal distribution of the difference between the sample feature vectors can be expressed by the following formula (3):

[0127] p(Δz ij |X i ,X j )=N(μ i -μ j ,(σ i 2 +σ j 2 )I) (3);

[0128] Where Δz ij Represents the sample image X i And the sample image X j The difference between the sample feature vectors.

[0129] The loss function value corresponding to the loss function in the variance extraction network to be trained can include the difference Δz between the sample feature vectors of the optimized positive sample image pair ij =0 probability density value p(Δz ij = 0), and maximize the negative sample image pair (sample image X m And the sample image X n )’s sample eigenvector difference Δz mn =2u m (where μ m Represents the sample image X m The probability density value p(Δz mn =2μ m ), so the variance extraction network to be trained can determine the loss function value corresponding to the loss function according to the following formula (4):

[0130] L=-(log(p(Δz ij =0)+λlog(p(Δz mn =2μ m ) (4);

[0131] Wherein, λ represents the weight of the negative sample image pair, which can be a preset parameter value.

[0132] Specifically, when a multi-dimensional variance vector (such as D dimension) is used to represent the degree of deviation between each position of the image mapped in the multi-dimensional space and the position with the highest probability density value, the loss function value L1 determined based on the positive sample image pair can be determined by the following formula (5):

[0133]

[0134] Where D represents the sample image X i And the sample image X j The dimension of the sample expected feature vector, l is a variable, indicating the lth dimension, μ i (l) Represents the sample image X i The eigenvalue of the lth dimension in the sample expected eigenvector; μ j (l) Represents the sample image X j The eigenvalue of the lth dimension in the sample expected eigenvector; Represents the sample image X i The variance value of the lth dimension in the sample variance vector; Represents the sample image X j The variance value of the lth dimension in the sample variance vector.

[0135] Specifically, when a one-dimensional variance vector is used to replace a multi-dimensional variance vector, that is, when a scalar variance value is used to represent the degree of deviation between each position of the image mapped in the multi-dimensional space and the position with the highest probability density value, the above formula (5) can be simplified to the following formula (6):

[0136]

[0137] Among them, cosθ represents the positive sample image pair (sample image X i And the sample image X j )’s sample expected feature vectors.

[0138] Similarly, when a multi-dimensional variance vector (such as D dimension) is used to represent the degree of deviation between each position of the image mapped in the multi-dimensional space and the position with the highest probability density value, the loss function value L2 determined based on the positive sample image pair can be determined by the following formula (7):

[0139]

[0140] Where D represents the sample image X m And the sample image X n The dimension of the sample expected feature vector, l is a variable, indicating the lth dimension, μ m (l) Represents the sample image Xm The eigenvalue of the lth dimension in the sample expected eigenvector; μ n (l) Represents the sample image X n The eigenvalue of the lth dimension in the sample expected eigenvector; Represents the sample image X m The variance value of the lth dimension in the sample variance vector; Represents the sample image X n The variance value of the lth dimension in the sample variance vector.

[0141] Specifically, when a one-dimensional variance vector is used to replace a multi-dimensional variance vector, that is, when a scalar variance value is used to represent the degree of deviation between each position of the image mapped in the multi-dimensional space and the position with the highest probability density value, the above formula (7) can be simplified to the following formula (8):

[0142]

[0143] In this way, the loss function value corresponding to the variance extraction network to be trained can be determined according to the above formulas (4), (6) and (8).

[0144] S3033, adjusting the network parameter values ​​corresponding to the variance extraction network to be trained based on the loss function value to obtain the variance extraction network.

[0145] The variance extraction network to be trained is continuously trained through multiple groups of sample images until the loss function value corresponding to the variance extraction network to be trained is less than a set threshold, or the training reaches a set number of times, and a trained variance extraction network is obtained.

[0146] In the embodiments of the present disclosure, on the one hand, a one-dimensional variance vector is used to represent the degree of deviation between the feature vector corresponding to the image in the multidimensional space and the expected feature vector, which can simplify the process of determining the loss function value and improve the training speed of the uncertain neural network. On the other hand, by introducing positive sample image pairs and negative sample images to jointly train the one-dimensional variance vector, the accuracy of the variance extraction network can be further improved.

[0147] Furthermore, on the basis of improving the accuracy of the variance extraction network, a one-dimensional variance vector with higher accuracy can be extracted based on the variance extraction network. In this way, when the one-dimensional variance feature vector and the multi-dimensional expected feature vector are used to represent the information vector of the image to be stored for subsequent processing (such as image comparison, image retrieval, etc.), the accuracy and precision of the subsequent image processing are not affected.

[0148] After obtaining the first information vector of the image to be stored, the first information vector corresponding to the image to be stored can be stored, so as to facilitate the subsequent recognition of the received image to be compared based on the first information vector corresponding to each stored image, such as Figure 6 As shown, the image processing method provided by the embodiment of the present disclosure further includes the following S401 to S403:

[0149] S401, obtaining an image to be compared.

[0150] Here, the image to be compared may be a face image to be identified.

[0151] S402, extracting a second information vector of the image to be compared; the second information vector includes a second expected feature vector corresponding to the image to be compared in the multidimensional space, and a second variance vector corresponding to the image to be compared in the multidimensional space, the second expected feature vector is a multidimensional vector, and the second variance vector is a one-dimensional vector.

[0152] The process of extracting the second information vector of the image to be compared here is similar to the process of extracting the first information vector of the image to be stored described above, and will not be repeated here.

[0153] S403, determining a target image that matches the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of a plurality of images stored in a pre-built image library.

[0154] The image to be compared is image X i , take one image X from multiple images in the image library j For example, when combining the multi-dimensional variance vector to determine the image X i and image X j The similarity S(X i ,X j ), it can be determined by the following formula (9):

[0155]

[0156] When a one-dimensional variance vector is used to replace a multi-dimensional variance vector, that is, a scalar variance value is used to represent the degree of deviation between each position of the image mapped in the multi-dimensional space and the position with the highest probability density value, the above formula (9) can be simplified to the following formula (10):

[0157]

[0158] When the target image that matches the image to be compared is determined by formula (10), the amount of calculation will be greatly reduced and the calculation speed will be improved compared with the method of formula (9). However, in the process of determining the target image, if the target image that matches the image to be compared is found among multiple images by formula (10), it is generally necessary to traverse all images in the image library to find the target image that is most similar to the image to be compared. This method is cumbersome. In view of this, the embodiment of the present disclosure proposes that before determining the target image that matches the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of the multiple images in the image library, the image processing method provided by the embodiment of the present disclosure also includes:

[0159] Based on the second expected feature vector of the image to be compared and the first expected feature vectors corresponding to the plurality of images in the image library, candidate images whose first similarity with the image to be compared is in the first set number in the descending order are screened from the image library.

[0160] It is proposed here to first determine the first similarity between the image to be compared and the multiple images based on the first expected feature vector of the image to be compared and the first expected feature vectors corresponding to the multiple images in the image library. When determining the first similarity based on the expected feature vector, a matrix calculation library that is convenient for calculation can be used to quickly determine the first similarity with the image to be compared and the first set number of candidate images in a descending order. For example, consider establishing a search index for the first expected feature vector corresponding to each image in the image library in a binary tree manner. Specifically, a kd tree index can be established. In this way, when calculating the similarity between the second expected feature vector of the image to be compared and the first expected feature vector of the image in the image library, the binary tree search algorithm can be used to exclude images that do not need to be compared with the image to be compared, thereby improving the efficiency of determining the candidate images.

[0161] In the disclosed embodiment, in the process of searching for candidate images based on the expected feature vector, the existing matrix calculation library can be used to improve the search speed, thereby quickly screening out the candidate images whose first similarity with the image to be compared is located in the first set number in the high to low sorting sequence, so as to facilitate the subsequent determination of the second similarity between the candidate image and the image to be compared according to the information vector, which can reduce the number of images to be compared, thereby improving the recognition efficiency of the images to be compared.

[0162] Specifically, based on the second expected feature vector of the image to be compared and the first expected feature vectors respectively corresponding to the plurality of images in the image library, when selecting from the image library the candidate images whose first similarity with the image to be compared is located in the first set number in the sequence sorted from high to low, the following steps S5011 to S5012 may be included:

[0163] S5011, using the first expected feature vectors of the multiple images in the image library as index nodes of the binary tree, searching for the first expected feature vectors of the multiple images according to the binary tree search algorithm, and calculating the first similarity between the searched first expected feature vectors of the multiple images and the second expected feature vector of the image to be compared;

[0164] S5012, after sorting the obtained multiple first similarities in descending order, select candidate images whose first similarities with the image to be compared are located in the first set number of the sequence sorted from high to low.

[0165] Specifically, each image in the image library can be mapped to a multidimensional space according to the first expected feature vector corresponding to each image in the image library, and then the space can be segmented according to the position corresponding to each image in the image library in the multidimensional space. After generating a binary tree index corresponding to the image contained in the image library according to the position corresponding to each image in the multidimensional space after the segmentation, the initial distance between any node and the image to be compared can be calculated based on the second expected feature vector corresponding to the image to be compared. The distance between the first expected feature vector of one of the child nodes of any node and the second expected feature vector of the image to be compared is less than the initial distance, and the distance between the first expected feature vector of another child node and the second expected feature vector of the image to be compared is greater than the initial distance. Therefore, when the initial distance is greater than the set distance threshold, that is, when the condition for screening images is not met, the search can be continued in the direction of the child node less than the initial distance, and the image in the direction of the child node greater than the initial distance can be excluded. In this way, there is no need to calculate the first similarity between all images in the image library and the image to be compared, thereby improving the speed of obtaining candidate images.

[0166] Here, the candidate images may be multiple or one. In the case of one, the image with the highest first similarity to the image to be compared is selected as the candidate image.

[0167] In the disclosed embodiment, the binary tree search index pre-constructed in the image library can be used to reduce the number of images to be compared, so that candidate images that meet the requirements can be found quickly.

[0168] Further, when determining a target image matching the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of the plurality of images in the image library, as follows: Figure 7 As shown, the following S501 to S502 may be included:

[0169] S501, determining a second similarity between the image to be compared and each candidate image based on a second information vector of the image to be compared and a first information vector of each candidate image;

[0170] S502: Select a candidate image whose second similarity with the image to be compared is the highest and whose second similarity is greater than a preset threshold as a target image to be matched with the image to be compared.

[0171] With respect to the above S501, when determining the second similarity between the image to be compared and each candidate image based on the second information vector of the image to be compared and the first information vector of each candidate image, the following S5011 to S5012 may be included:

[0172] S5011, for each candidate image, determining the cosine similarity between the image to be compared and one of the candidate images based on the second expected feature vector corresponding to the image to be compared and the first expected feature vector corresponding to one of the candidate images;

[0173] S5012: Determine a second similarity between the image to be compared and a candidate image based on the cosine similarity, a second variance vector corresponding to the image to be compared, and a first variance vector corresponding to a candidate image.

[0174] A candidate image may be any candidate image among a plurality of candidate images.

[0175] First, the cosine similarity between the image to be compared and each candidate image can be calculated based on the second expected feature vector corresponding to the image to be compared and the first expected feature vector corresponding to each candidate image. Then, the second similarity between the image to be compared and the candidate image can be determined by the above formula (10).

[0176] In the embodiment of the present disclosure, when a scalar one-dimensional variance vector is used instead of a multi-dimensional variance vector to represent the degree of deviation between each feature vector corresponding to an image in a multi-dimensional space and an expected feature vector, the process of determining the similarity between the image to be compared and any candidate image in an uncertain manner can be simplified, thereby improving image recognition efficiency.

[0177] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0178] Based on the same technical concept, an image processing device corresponding to the image processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0179] Reference Figure 8FIG. 6 is a schematic diagram of an image processing device 600 provided in an embodiment of the present disclosure, and the image processing device 600 includes:

[0180] An acquisition module 601 is used to acquire an image to be stored;

[0181] An extraction module 602 is used to extract a first information vector of the image to be stored; the first information vector includes a first expected feature vector corresponding to the image to be stored in the multidimensional space, and a first variance vector corresponding to the image to be stored in the multidimensional space, the first expected feature vector is a multidimensional vector, and the first variance vector is a one-dimensional vector;

[0182] The storage module 603 is used to store the first information vector corresponding to the image to be stored.

[0183] In a possible implementation, when the extraction module 602 is used to extract the first information vector of the image to be stored, it includes:

[0184] Inputting the image to be stored into a feature map extraction network in a pre-trained uncertain neural network to obtain a feature map corresponding to the image to be stored;

[0185] Inputting the feature map into a feature value extraction network in an uncertain neural network to obtain a first expected feature vector consisting of feature values ​​corresponding to feature points of the image to be stored;

[0186] Input the feature map into the variance extraction network in the uncertainty neural network to obtain a first variance vector corresponding to the image to be stored;

[0187] Based on the first expected feature vector and the first variance vector corresponding to the image to be stored, a first information vector of the image to be stored is determined.

[0188] In a possible implementation, when the extraction module 602 is used to determine the first information vector of the image to be stored based on the first expected feature vector and the first variance vector corresponding to the image to be stored, it includes:

[0189] After concatenating the first variance vector and the first expected eigenvector, a first information vector is obtained; or,

[0190] After replacing the eigenvalue corresponding to the target dimension in the first expected eigenvector with the first variance vector, a first information vector is obtained.

[0191] In a possible implementation, the image processing device 600 further includes a comparison module 604, and the comparison module 604 is used to:

[0192] Obtaining an image to be compared;

[0193] Extracting a second information vector of the image to be compared; the second information vector includes a second expected feature vector corresponding to the image to be compared in the multidimensional space, and a second variance vector corresponding to the image to be compared in the multidimensional space, the second expected feature vector is a multidimensional vector, and the second variance vector is a one-dimensional vector;

[0194] Based on the second information vector of the image to be compared and the first information vectors of each of the plurality of images in the pre-constructed image library, a target image matching the image to be compared is determined.

[0195] In a possible implementation manner, before determining the target image matching the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of the plurality of images in the image library, the comparison module 604 is further configured to:

[0196] Based on the second expected feature vector of the image to be compared and the first expected feature vectors corresponding to the plurality of images in the image library, candidate images whose first similarity with the image to be compared is located at least a set number before in the sorting sequence are screened from the image library.

[0197] In a possible implementation, when the comparison module 604 is used to determine a target image matching the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of the plurality of images in the image library, it includes:

[0198] Determine a second similarity between the image to be compared and each candidate image based on the second information vector of the image to be compared and the first information vector of each candidate image;

[0199] A candidate image having the highest second similarity with the image to be compared and a second similarity greater than a preset threshold is selected as a target image to be matched with the image to be compared.

[0200] In a possible implementation, when the comparison module 604 is used to determine the second similarity between the image to be compared and each candidate image based on the second information vector of the image to be compared and the first information vector of each candidate image, it includes:

[0201] For each candidate image, based on the second expected feature vector corresponding to the image to be compared and the first expected feature vector corresponding to one of the candidate images, determine the cosine similarity between the image to be compared and one of the candidate images;

[0202] Based on the cosine similarity, the second variance vector corresponding to the image to be compared and the first variance vector corresponding to the candidate image, a second similarity between the image to be compared and the candidate image is determined.

[0203] In a possible implementation, the image processing device 600 further includes a training module 605: the training module 605 is used to train the uncertainty neural network model to obtain a trained uncertainty neural network;

[0204] The training module 605, when used to train the uncertainty neural network model to obtain the trained uncertainty neural network, includes:

[0205] Obtain a sample image set, which includes multiple sample images and sample category information corresponding to each sample image;

[0206] Based on the sample image set and the sample category information corresponding to each sample image, the feature map extraction network to be trained in the neural network to be trained and the feature value extraction network to be trained in the neural network to be trained are trained to obtain the trained feature map extraction network and feature value extraction network;

[0207] Based on the positive sample image pairs corresponding to the same sample category information and the negative sample image pairs corresponding to different sample category information contained in the sample image set, the variance extraction network to be trained in the neural network to be trained is trained to obtain a trained variance extraction network.

[0208] In a possible implementation manner, after obtaining the trained feature map extraction network and feature value extraction network, the training module 605 is further used to:

[0209] The sample image set is sequentially input into the feature map extraction network and the eigenvalue extraction network to obtain the sample expected feature vector corresponding to each sample image;

[0210] Based on the positive sample image pairs corresponding to the same sample category information and the negative sample image pairs corresponding to different sample category information contained in the sample image set, the variance extraction network to be trained in the neural network to be trained is trained to obtain a trained variance extraction network, including:

[0211] The sample image set and the positive sample image pair information and the negative sample image pair information contained in the sample image set are input into the variance extraction network to be trained in the neural network to be trained, and the predicted variance vector corresponding to each sample image is predicted; the positive sample image pair information corresponds to a pair of positive sample images with the same category information, and the negative sample image information corresponds to a pair of negative sample images with different category information;

[0212] Based on the similarity between the predicted variance vectors corresponding to the positive sample image pairs and the sample expected feature vectors of the positive sample image pairs, and the similarity between the predicted variance vectors corresponding to the negative sample image pairs and the sample expected feature vectors of the negative sample image pairs, determining the loss function value corresponding to the variance extraction network to be trained;

[0213] The network parameter values ​​corresponding to the variance extraction network to be trained are adjusted based on the loss function value to obtain the variance extraction network.

[0214] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0215] Corresponds to Figure 1 The image processing method in the present disclosure also provides an electronic device 700, such as Figure 8 FIG. 7 is a schematic diagram of the structure of an electronic device 700 provided in an embodiment of the present disclosure, including:

[0216] Processor 71, memory 72, and bus 73; memory 72 is used to store execution instructions, including internal memory 721 and external memory 722; the internal memory 721 here is also called internal memory, which is used to temporarily store the calculation data in the processor 71, and the data exchanged with the external memory 722 such as a hard disk. The processor 71 exchanges data with the external memory 722 through the internal memory 721. When the electronic device 700 is running, the processor 71 and the memory 72 communicate through the bus 73, so that the processor 71 executes the following instructions: obtain the image to be stored; extract the first information vector of the image to be stored; the first information vector includes the first expected feature vector corresponding to the image to be stored in the multidimensional space, and the first variance vector corresponding to the image to be stored in the multidimensional space, the first expected feature vector is a multidimensional vector, and the first variance vector is a one-dimensional vector; store the first information vector corresponding to the image to be stored.

[0217] The present disclosure also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0218] The computer program product of the image processing method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the image processing method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0219] The present disclosure also provides a computer program, which implements any one of the methods of the aforementioned embodiments when executed by a processor. The computer program product can be implemented in hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0220] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0221] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0222] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0223] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0224] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Get the image to be stored; Extracting a first information vector of the image to be stored; The first information vector includes a first expected feature vector corresponding to the image to be stored in the multidimensional space, and a first variance vector corresponding to the image to be stored in the multidimensional space, the first expected feature vector is a multidimensional vector, and the first variance vector is a one-dimensional vector; storing the first information vector corresponding to the image to be stored; The extracting the first information vector of the image to be stored comprises: Inputting the image to be stored into a feature map extraction network in a pre-trained uncertain neural network to obtain a feature map corresponding to the image to be stored; Inputting the feature map into a feature value extraction network in the uncertain neural network to obtain the first expected feature vector composed of feature values ​​corresponding to feature points of the image to be stored; Inputting the feature map into a variance extraction network in the uncertainty neural network to obtain the first variance vector corresponding to the image to be stored; Determining the first information vector of the image to be stored based on the first expected feature vector and the first variance vector corresponding to the image to be stored; The determining the first information vector of the image to be stored based on the first expected feature vector and the first variance vector corresponding to the image to be stored includes: The first information vector is obtained by concatenating the first variance vector and the first expected feature vector; or, After replacing the eigenvalue corresponding to the target dimension in the first expected eigenvector with the first variance vector, the first information vector is obtained.

2. The image processing method according to claim 1, characterized in that: The image processing method further comprises: Obtaining an image to be compared; Extracting a second information vector of the image to be compared; the second information vector includes a second expected feature vector corresponding to the image to be compared in a multidimensional space, and a second variance vector corresponding to the image to be compared in the multidimensional space, the second expected feature vector is a multidimensional vector, and the second variance vector is a one-dimensional vector; Based on the second information vector of the image to be compared and the first information vectors of each of a plurality of images stored in a pre-constructed image library, a target image matching the image to be compared is determined.

3. The image processing method according to claim 2, characterized in that: Before determining a target image matching the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of the plurality of images in the image library, the image processing method further includes: Based on the second expected feature vector of the image to be compared and the first expected feature vectors corresponding to the plurality of images in the image library, candidate images whose first similarity with the image to be compared is located in the first set number in a sequence sorted from high to low are screened out from the image library.

4. The image processing method according to claim 3, characterized in that: The step of determining a target image matching the image to be compared based on the second information vector of the image to be compared and the first information vectors of each of the plurality of images in the image library comprises: Determine a second similarity between the image to be compared and each of the candidate images based on the second information vector of the image to be compared and the first information vector of each of the candidate images; The candidate image having the highest second similarity with the image to be compared and whose second similarity is greater than a preset threshold is selected as the target image to be matched with the image to be compared.

5. The image processing method according to claim 4, characterized in that: The determining, based on the second information vector of the image to be compared and the first information vector of each of the candidate images, a second similarity between the image to be compared and each of the candidate images comprises: For each of the candidate images, determining the cosine similarity between the image to be compared and one of the candidate images based on the second expected feature vector corresponding to the image to be compared and the first expected feature vector corresponding to one of the candidate images in each of the candidate images; Based on the cosine similarity, the second variance vector corresponding to the image to be compared and the first variance vector corresponding to one of the candidate images, a second similarity between the image to be compared and one of the candidate images is determined.

6. The image processing method according to claim 1, characterized in that: The image processing method further includes: training the uncertainty neural network model to obtain a trained uncertainty neural network; The training of the uncertainty neural network model to obtain a trained uncertainty neural network includes: Acquire a sample image set, wherein the sample image set includes multiple sample images and sample category information corresponding to each sample image; Based on the sample image set and the sample category information corresponding to each sample image, training the feature map extraction network to be trained in the neural network to be trained and the feature value extraction network to be trained in the neural network to be trained to obtain the trained feature map extraction network and the feature value extraction network; Based on the positive sample image pairs corresponding to the same sample category information and the negative sample image pairs corresponding to different sample category information contained in the sample image set, the variance extraction network to be trained in the neural network to be trained is trained to obtain the variance extraction network that has been trained.

7. The image processing method according to claim 6, characterized in that: After obtaining the trained feature map extraction network and the feature value extraction network, the image processing method further includes: Inputting the sample image set into the feature map extraction network and the feature value extraction network in sequence to obtain a sample expected feature vector corresponding to each sample image; The method of training the variance extraction network to be trained in the neural network to be trained based on the positive sample image pairs corresponding to the same sample category information and the negative sample image pairs corresponding to different sample category information contained in the sample image set to obtain the trained variance extraction network includes: Input the sample image set and the positive sample image pair information and the negative sample image pair information contained in the sample image set into the variance extraction network to be trained in the neural network to be trained, and predict the predicted variance vector corresponding to each sample image; the positive sample image pair information corresponds to a pair of positive sample images with the same category information, and the negative sample image information corresponds to a pair of negative sample images with different category information; Determine the loss function value corresponding to the variance extraction network to be trained based on the similarity between the predicted variance vectors corresponding to the positive sample image pairs and the sample expected feature vectors of the positive sample image pairs, and the similarity between the predicted variance vectors corresponding to the negative sample image pairs and the sample expected feature vectors of the negative sample image pairs; Based on the loss function value, the network parameter value corresponding to the variance extraction network to be trained is adjusted to obtain the variance extraction network.

8. An image processing device, characterized in that: include: An acquisition module, used for acquiring images to be stored; An extraction module, used for extracting a first information vector of the image to be stored; The first information vector includes a first expected feature vector corresponding to the image to be stored in the multidimensional space, and a first variance vector corresponding to the image to be stored in the multidimensional space, the first expected feature vector is a multidimensional vector, and the first variance vector is a one-dimensional vector; A storage module, used for storing the first information vector corresponding to the image to be stored; When the extraction module is used to extract the first information vector of the image to be stored, it includes: Inputting the image to be stored into a feature map extraction network in a pre-trained uncertain neural network to obtain a feature map corresponding to the image to be stored; Inputting the feature map into a feature value extraction network in the uncertain neural network to obtain the first expected feature vector composed of feature values ​​corresponding to feature points of the image to be stored; Inputting the feature map into a variance extraction network in the uncertainty neural network to obtain the first variance vector corresponding to the image to be stored; Determining the first information vector of the image to be stored based on the first expected feature vector and the first variance vector corresponding to the image to be stored; The extraction module, when used to determine the first information vector of the image to be stored based on the first expected feature vector and the first variance vector corresponding to the image to be stored, includes: The first information vector is obtained by concatenating the first variance vector and the first expected feature vector; or, After replacing the eigenvalue corresponding to the target dimension in the first expected eigenvector with the first variance vector, the first information vector is obtained.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are executed.

Images