Image processing method, image processing device and computer equipment

Abstract

The embodiment of the invention provides an image processing method, an image processing device and computer equipment, and the method comprises the steps: carrying out the similarity detection processing of a to-be-processed image and historical images in a historical database, obtaining a similarity detection result, and determining candidate images from the historical images according to the similarity detection result; determining a first feature map of the to-be-processed image and a second feature map of the candidate image; inputting the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result comprises matched local features of the to-be-processed image and the candidate image; and determining similar images of the to-be-processed image from the candidate images according to the number of matched local features included in the local feature matching result. According to the embodiment of the invention, the accuracy and efficiency of similar image retrieval can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, an image processing apparatus and a computer device. Background Art

[0002] Agricultural finance generally has fraud risks. Compared with small and micro enterprises and consumer finance, agricultural finance has the characteristics of older credit applicants, lower education levels, and less familiarity with smart devices. During the application process, the approval staff generally needs to assist in the preparation and submission of materials. In order to improve performance, some approval staff will use the information of other customers who have passed the credit approval because the applicant's relevant business information does not meet the access requirements. The traditional method is for credit review personnel to manually review the images submitted by customers to determine whether they are similar to the information of customers who have passed the credit approval. However, this method has low processing efficiency and is easily affected by subjective factors. It is difficult to identify the fraudulent images, resulting in low accuracy. Summary of the invention

[0003] The present application provides an image processing method, an image processing apparatus, a computer device, a computer-readable storage medium, and a computer program product, which can improve the accuracy and efficiency of similar image retrieval.

[0004] In one aspect, the present application provides an image processing method, the method comprising:

[0005] Performing similarity detection processing on the image to be processed and the historical images in the historical database to obtain a similarity detection result, and determining a candidate image from the historical images according to the similarity detection result;

[0006] Determine a first feature map of the image to be processed and a second feature map of the candidate image;

[0007] Inputting the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result includes matching local features of the image to be processed and the candidate image;

[0008] According to the number of matching local features included in the local feature matching result, a similar image to the image to be processed is determined from the candidate images.

[0009] On the other hand, the present application provides an image processing device, the device comprising:

[0010] A processing module is used to perform similarity detection processing on the image to be processed and the historical images in the historical database to obtain a similarity detection result, and determine a candidate image from the historical images according to the similarity detection result;

[0011] The processing module is further used to determine the first feature map of the image to be processed and the second feature map of the candidate image;

[0012] A local feature matching module, used for inputting the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result includes matching local features of the image to be processed and the candidate image;

[0013] The processing module is further configured to determine similar images of the image to be processed from the candidate images according to the number of matching local features included in the local feature matching result.

[0014] On the other hand, the present application provides a computer device, comprising: a processor, a storage device and a communication interface, wherein the processor, the communication interface and the storage device are interconnected, wherein the storage device stores executable program code, and the processor is used to call the executable program code to implement the image processing method.

[0015] On the other hand, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the program instructions are executed by a processor to implement the image processing method as described above.

[0016] On the other hand, the present application provides a computer program product, which includes a computer program or computer instructions, and the computer program or computer instructions are executed by a processor to implement the above-mentioned image processing method.

[0017] The present application performs a coarse-grained similarity detection process on the image to be processed and the historical images in the historical database to obtain a similarity detection result, and determines a candidate image from the historical image based on the similarity detection result to improve the efficiency of subsequent similar image retrieval. Then, the first feature map of the image to be processed and the second feature map of the candidate image are determined, and the first feature map and the second feature map are input into the local feature matching network for processing to obtain a local feature matching result. The matching local features that match the image to be processed and the candidate image can be obtained through fine-grained local feature matching. Then, based on the number of matching local features included in the local feature matching result, a similar image of the image to be processed is determined from the candidate image. Compared with manual image retrieval, the above method can automatically and efficiently perform similar image retrieval, and by combining coarse-grained similarity detection and fine-grained local feature matching, the accuracy of similar image retrieval is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 is a schematic diagram of the architecture of an image processing system provided by an exemplary embodiment of the present application;

[0020] Figure 2 is a flowchart of an image processing method provided by an exemplary embodiment of the present application;

[0021] Figure 3 is a flowchart of another image processing method provided by an exemplary embodiment of the present application;

[0022] Figure 4A is a structural schematic diagram of a feature extraction network provided by an exemplary embodiment of the present application;

[0023] Figure 4B is a schematic diagram of the structure of a residual block provided by an exemplary embodiment of the present application;

[0024] Figure 4C is a structural schematic diagram of a local feature matching network provided by an exemplary embodiment of the present application;

[0025] Figure 4D is a processing flow chart of similar image retrieval provided by an exemplary embodiment of the present application;

[0026] Figure 5 is a schematic block diagram of an image processing device provided by an exemplary embodiment of the present application;

[0027] Figure 6 It is a schematic block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be noted that the descriptions of "first", "second", etc. involved in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0030] It can be understood that in the specific implementation of the present application, data related to images to be processed, historical images, etc. are involved. When the above embodiments of the present application are applied to specific products or technologies, the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0031] This application will be specifically described by the following examples:

[0032] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an image processing system provided by an exemplary embodiment of the present application. Figure 1 As shown, the image processing system may specifically include a terminal device 101 and a server 102. The terminal device 101 and the server 102 are connected via a network, for example, via a local area network, a wide area network, a mobile Internet, and the like.

[0033] The terminal device 101 is also referred to as a terminal, user equipment (UE), access terminal, user unit, mobile device, user terminal, wireless communication device, user agent or user device. The terminal device may be a smart home appliance, a handheld device with wireless communication function (such as a smart phone, a tablet computer), a computing device (such as a personal computer (PC), a vehicle terminal, an intelligent voice interaction device, a wearable device or other smart device, but is not limited thereto. In the embodiment of the present application, the terminal device 101 may be the terminal device corresponding to the applicant who applies for credit.

[0034] Server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. In the embodiment of the present application, server 102 may be a management server for credit qualification approval.

[0035] In one implementable manner, the terminal device 101 can send application materials for credit application (such as an image to be processed) to the server 102. After receiving the image to be processed, the server 102 performs similarity detection processing on the image to be processed and historical images in a historical database (the historical images are application materials uploaded by customers who have been granted credit) to obtain a similarity detection result, and determines a candidate image from the historical images based on the similarity detection result; the server 102 determines a first feature map of the image to be processed and a second feature map of the candidate image; the server 102 inputs the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result includes matching local features between the image to be processed and the candidate image; the server 102 determines similar images of the image to be processed from the candidate images based on the number of matching local features included in the local feature matching result.

[0036] When the server 102 does not determine a similar image of the image to be processed from the historical database, the credit evaluation data of the applicant corresponding to the image to be processed is determined; when the credit evaluation data meets the credit conditions, the applicant corresponding to the image to be processed is determined as a credit customer; at the same time, the first feature map of the image to be processed and the first feature vector obtained by performing global feature extraction processing on the image to be processed are stored in the historical database. When the server 102 determines a similar image of the image to be processed from the historical database, the applicant corresponding to the image to be processed can be determined as an illegal applicant and the credit application of the customer is rejected.

[0037] It is understandable that the system architecture diagram described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. For example, the access method provided in the embodiment of the present application can be executed by the server 102, and can also be executed by other servers or server clusters that are different from the server 102 and can communicate with the terminal device 101 and / or the server 102. It is known to those skilled in the art that Figure 1 The number of terminal devices and servers in the embodiment is only illustrative. According to the business implementation needs, any number of terminal devices and servers can be configured. In addition, with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems. In the subsequent embodiments, the client will refer to the above-mentioned terminal device 101, and the server will refer to the above-mentioned server 102, which will not be repeated in the subsequent embodiments.

[0038] See also Figure 2 , Figure 2This is a flowchart of an image processing method provided by an exemplary embodiment of the present application. Taking the method applied to a server as an example for explanation, the method may include the following steps:

[0039] S201 , performing similarity detection processing on the image to be processed and the historical images in the historical database to obtain a similarity detection result, and determining a candidate image from the historical images according to the similarity detection result.

[0040] The image to be processed is an image that needs to be retrieved for similar images, such as a store photo submitted by a credit applicant to the server. The historical database includes multiple historical images, which are images uploaded by credited customers, such as store photos uploaded by credited customers. The same credited customer can upload one or more images and store one or more images in the historical database to facilitate subsequent similar image retrieval tasks.

[0041] In an embodiment of the present application, the server performs similarity detection processing on the image to be processed and the historical images in the historical database to obtain a similarity detection result. Through the similarity detection result, relatively similar images (a coarse-grained image similarity measurement) can be preliminarily screened out from the historical images in the historical database. The relatively similar images are candidate images, which can reduce the amount of calculation and time consumption in subsequent further screening, thereby improving the efficiency of similar image retrieval.

[0042] In one implementable manner, the similarity detection process can be implemented by one or more of the following methods, which may include: histogram similarity comparison of images, image hash algorithms (such as average hash algorithm, difference hash algorithm, perceptual hash algorithm), image structural similarity (SSIM), image feature extraction and matching, etc.

[0043] S202: Determine a first feature map of the image to be processed and a second feature map of the candidate image.

[0044] In an embodiment of the present application, the server obtains a first feature map of the image to be processed and a second feature map of the candidate image. The first feature map and the second feature map have high image feature distinction and representation capabilities, and are used for subsequent local feature matching processing.

[0045] In one implementable manner, the first feature map may be obtained by extracting features through a deep neural network, and the second feature map may be obtained by extracting features through a deep neural network. Since the feature map of each historical image can be stored in the historical database, and the historical images include candidate images, the second feature map may also be obtained by the server from the historical database.

[0046] S203, inputting the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result includes matching local features of the image to be processed and the candidate image.

[0047] The local feature matching network is a deep learning network structure used for image similarity matching. It extracts local features from two images respectively, and then compares and matches the local features in the two images. Local features are feature representations of local areas in an image that are unique and distinguishable. For example, local features can represent information such as texture, edges, and shapes in a specific area of ​​an image, and are represented by feature vectors. Matching local features refers to local feature points of an image that are determined to be mutually matching and similar. Matching local features can be used to determine the similarity between images.

[0048] In an embodiment of the present application, the server inputs the first feature map of the image to be processed and the second feature map of the candidate image into a local feature matching network for processing. Local feature matching can help identify similar local features between the image to be processed and the candidate image, providing a more fine-grained image similarity measurement, which helps to more accurately determine similar images.

[0049] S204: Determine similar images to the image to be processed from the candidate images according to the number of matching local features included in the local feature matching result.

[0050] In an embodiment of the present application, the server can measure the similarity between the image to be processed and the candidate image by the number of matching local features of the first feature map and the second feature map. When the number of matching local features of the candidate image meets a preset condition, the candidate image can be determined as a similar image, thereby realizing similar image retrieval.

[0051] Compared with manual image retrieval, the above method can automatically and efficiently retrieve similar images, and by combining coarse-grained similarity detection with fine-grained local feature matching, the accuracy of similar image retrieval can be improved, and the retrieval efficiency can be further improved. At the same time, it realizes the rapid screening and matching of similar images for large-scale historical image data.

[0052] See also Figure 3 , Figure 3This is a flowchart of an image processing method provided by an exemplary embodiment of the present application. Taking the method applied to a server as an example for explanation, the method may include the following steps:

[0053] S301 , performing global feature extraction processing on the image to be processed to obtain a first feature vector, and obtaining second feature vectors of a plurality of pre-stored historical images from a historical database.

[0054] In the embodiment of the present application, the feature vector of each historical image may be pre-stored in the historical database, and the server may directly obtain the second feature vectors of the plurality of pre-stored historical images from the historical database, thereby improving processing efficiency.

[0055] In an implementable manner, the server may further perform the following steps:

[0056] (1) Obtain historical images uploaded by the credit-granted customers, perform global feature extraction on the historical images, obtain second feature vectors, and store the second feature vectors of the historical images in a historical database.

[0057] (2) Input the historical image into the feature extraction network for processing to obtain a second feature map of the historical image, and store the second feature map of the historical image in a historical database.

[0058] In the above steps (1)-(2), the server performs global feature extraction processing on the historical image in advance to obtain the second feature vector, and inputs the historical image into the feature extraction network for processing to obtain the second feature map, and then stores the second feature vector and the second feature map in the historical database, so that when the server obtains the image to be processed later, it can quickly obtain the feature vector and the estimated feature map of each historical image from the historical database to perform related similar image retrieval tasks. The above method realizes the construction of the historical database, which stores the high-dimensional feature representation of each historical image uploaded by the credited customer, which is used for subsequent image similarity calculation and matching, so as to efficiently perform similar image retrieval and meet the retrieval performance requirements of similar image retrieval.

[0059] It should be noted that the method by which the server determines the second feature map and the second feature vector of the historical image may be the same as the method by which the server determines the first feature map and the first feature vector of the image to be processed, which will not be repeated here.

[0060] It should be noted that the historical database may include a first database and a second database. The first database may separately store the feature vector (such as the second feature vector) of each historical image, and the second database may separately store the feature map (such as the second feature map) of each historical image. At the same time, each feature vector in the first database is associated with the corresponding feature map in the second database, such as by associating through the identification of the historical image. This ensures the accuracy of obtaining the second feature vector and the second feature map of the historical image from the historical database, and ensures the orderliness of the data in the historical database.

[0061] S302 , respectively performing similarity calculation on the first feature vector and the plurality of second feature vectors to obtain a similarity detection result; the similarity detection result includes the similarity between the first feature vector and each of the second feature vectors.

[0062] S303 , determining a second feature vector whose similarity to the first feature vector is greater than a first threshold, and taking a historical image corresponding to the second feature vector whose similarity is greater than the first threshold as a candidate image.

[0063] In one implementable manner, the similarity calculation can use cosine similarity (Cosine Similarity). By setting a first threshold (the first threshold is set according to business needs and characteristics, such as the first threshold is set to 0.8), historical images with a similarity greater than the first threshold are screened out from historical images as candidate images.

[0064] In the above steps S302-S303, the server respectively calculates the similarity of the first feature vector and the plurality of second feature vectors to obtain similarity detection results, thereby realizing coarse-grained similarity detection. Then, the server uses the historical images corresponding to the second feature vectors in the similarity detection results whose similarity with the first feature vector is greater than the first threshold as candidate images obtained by coarse screening, and the number of candidate images is greater than or equal to 0.

[0065] S304: Input the image to be processed into a feature extraction network for processing to obtain a first feature map of the image to be processed, and obtain a second feature map of the pre-stored candidate image from a historical database.

[0066] In one implementable manner, the step of performing global feature extraction processing on the image to be processed to obtain the first feature vector in step S301, and the step of inputting the image to be processed into the feature extraction network for processing in step S304, can be respectively implemented through partial sub-networks in the shared feature extraction network. By inputting the image to be processed into the same feature extraction network to obtain the first feature map and the first feature vector of the image to be processed, it is possible to maintain consistency in feature representation, thereby better performing similarity calculation and image matching. In addition, by sharing partial sub-networks of the feature extraction network, the cost of repeated calculation and storage of features can be reduced, and repeated construction and training of multiple networks can be avoided. At the same time, only one feature extraction model needs to be maintained and updated, which makes it easier to deploy and manage the model.

[0067] Based on this, the shared feature extraction network can be composed of a backbone network and a neck network. The backbone network can be a Resnet-50 network, and the neck network can be an FPN network. Figure 4A , Figure 4A It is a structural diagram of a feature extraction network provided by an exemplary embodiment of the present application. The Resnet-50 network includes 5 stages, among which the structure of Stage 0 is relatively simple and can be regarded as a preprocessing of the input (such as the input image). The following 4 stages are all composed of BottleNeck modules with similar structures. The number of BottleNeck modules included in Stage 2, Stage 3, Stage 4, and Stage 5 are 3, 4, 6, and 3 respectively.

[0068] Among them, Stage 1 includes conv (convolution), Batch Norm (BN layer), RELU activation function and maxpool (maximum pooling layer). Stage 2, Stage 3, Stage 4, and Stage 5 are all obtained by combining two residual blocks in different ways. The two residual blocks are: Residual BottleNeck Block 1 and Residual BottleNeck Block 2. The network structures of Residual BottleNeck Block 1 and Residual BottleNeck Block 2 can be found in Figure 4B , Figure 4B is a schematic diagram of a residual block structure provided by an exemplary embodiment of the present application, ResidualBottleNeck Block 1 and Residual BottleNeck Block 2 correspond to Figure 4BThe network structures shown in (1) and (2) are not described in detail because the two residual blocks are not the main invention of this application.

[0069] The image to be processed is processed through the backbone network to obtain: output C3 after Stage 3, output C4 after Stage 4, and output C5 after Stage 5. Then C5 is processed through an adaptive average pool layer to finally obtain a feature vector, such as the first feature vector, and the size of the first feature vector may be 1*2048.

[0070] C3, C4, and C5 are processed through the neck network to obtain feature maps, such as the first feature map ( Figure 4A In P3), the size of the first feature map can be 256*(H / 8)*(W / 8).

[0071] In the embodiment of the present application, by removing the last FC layer and softmax operation of the Resnet-50 network, and FPN only uses the underlying C3, C4, and C5 to perform feature fusion and generate feature map P3, the network complexity can be reduced and the model processing efficiency can be improved.

[0072] It should be noted that historical images can also be processed by the above feature extraction network. At this time, Figure 4A The input image in is recorded as the historical image, and the second feature map and the second feature vector of the historical image are finally output, which will not be described here. The second feature map and the second feature vector of the historical image have the same size as the first feature map and the first feature vector, respectively, such as the size of the second feature map is 1*2048, and the second feature vector is 256*(H / 8)*(W / 8).

[0073] S305 , using the feature transformation subnetwork of the local feature matching network to perform feature transformation processing on the first feature map and the second feature map to obtain a first feature map after feature transformation and a second feature map after feature transformation.

[0074] In an embodiment of the present application, the server can perform feature transformation on the first feature map and the second feature map through feature transformation processing to improve their representation ability and feature matching performance to better capture the relative relationship between local features.

[0075] S306, using the feature matching sub-network of the local feature matching network, perform feature matching processing on the first feature map after feature transformation and the second feature map after feature transformation to obtain a matching degree matrix.

[0076] In the embodiment of the present application, the server can calculate a matching degree matrix through feature matching processing to measure the local feature similarity between the image to be processed and each candidate image, thereby achieving a fine-grained and more accurate image similarity measurement.

[0077] In one possible implementation, see Figure 4C , Figure 4C It is a structural diagram of a local feature matching network provided by an exemplary embodiment of the present application. The local feature matching network includes a feature transformation subnetwork and a feature matching subnetwork. The specific processing process can be as follows: the local feature matching network first flattens the first feature map and the second feature map respectively. The flattening operation is that the features at the same position on each channel are integrated into a group of features, and each position is processed separately to finally obtain the flattened features; the corresponding position code is added according to the flattened features; the first feature map is flattened and the features after the position code is added, and the second feature map is flattened and the features after the position code is added are input into the feature transformation subnetwork for processing. The feature transformation subnetwork can be a LoFTR module, which includes a self-attention layer (self-attention) and a cross-attention layer (cross-attention). The transformation operations corresponding to the self-attention layer and the cross-attention layer are repeated N times. In actual use, N can be 4.

[0078] The first feature map after feature transformation and the second feature map after feature transformation are obtained through the feature transformation subnetwork. The first feature map after feature transformation and the second feature map after feature transformation are processed by the feature matching subnetwork. For example, the feature matching subnetwork can be a matching layer processing network, and finally a two-dimensional matching degree matrix is ​​obtained.

[0079] It should be noted that the pre-trained weights used in the feature extraction network and the local feature matching network in the embodiment of the present application can be the weights obtained by training the LoFTR algorithm on the MegaDepth dataset. The structure of the network is based on the LoFTR algorithm. The network model structure provided in the embodiment of the present application removes a large number of branches that have little impact on performance. Therefore, the open source outdoor-ds weight file can be adapted to the network in this algorithm without retraining the algorithm, or only fine-tuning can achieve good feature extraction and local feature matching effects.

[0080] S307, determining a local feature matching result according to the matching degree matrix; the local feature matching result includes matching local features of the image to be processed and the candidate image.

[0081] In the embodiment of the present application, the server determines matching local features that match the image to be processed and the candidate image according to the matching matrix, and performs a more refined similarity measurement through the matching local features, which helps to more accurately determine similar images.

[0082] In an implementable manner, the element values ​​of any group of elements in the matching degree matrix represent the matching degree between any local feature vector of the image to be processed and each local feature vector of the candidate image.

[0083] It should be noted that any group of elements may refer to a row of elements or a column of elements in the matching matrix. The following will provide an exemplary description using any group of elements as a column of elements in the matching matrix. A column of elements in the matching matrix represents the degree of match between a local feature vector of the image to be processed and each local feature vector of the candidate image.

[0084] Exemplarily, the number of local feature vectors corresponding to the image to be processed and the number of local feature vectors of a candidate image are both s. Then, the element values ​​in the first column of the matching matrix respectively represent the matching degrees between the first local feature vector corresponding to the image to be processed and the s local feature vectors of the candidate image.

[0085] Based on this, the above step S307 can be implemented by the following steps:

[0086] (1) Determine candidate elements in the matching matrix, where the candidate elements are elements whose element values ​​are greater than a second threshold value among the elements included in the matching matrix.

[0087] In an embodiment of the present application, the server can filter out elements with lower matching degrees by screening out candidate elements with higher matching degrees from the matching degree matrix, thereby reducing subsequent calculations and processing, and focusing attention on local features that may be more similar, thereby improving matching efficiency.

[0088] Exemplarily, the second threshold may be set to 0.9, and the server uses elements in the matching degree matrix with a matching degree greater than 0.9 as candidate elements.

[0089] (2) When the element value of the candidate element is the maximum element value of its row and the maximum element value of its column, it is determined that the local feature vector corresponding to the candidate element in the image to be processed and the local feature vector corresponding to the candidate element in the candidate image are matching local features.

[0090] In an embodiment of the present application, the server determines whether the element value of the candidate element is the maximum element value of the row and the maximum element value of the column. If so, the server determines the two local feature vectors corresponding to the candidate element as matching local features. This ensures that the two local feature vectors have the maximum match degree with each other, and achieves the highest two-way match degree judgment, thereby improving the accuracy of the match and facilitating more accurate retrieval of similar images.

[0091] (3) Determine the local feature matching result based on the matching local features corresponding to the candidate images.

[0092] In an embodiment of the present application, for each candidate image, the server integrates the matching local features corresponding to the determined candidate image, and then further integrates the integration results of multiple candidate images to finally obtain a local feature matching result, which includes the matching local features corresponding to each candidate image.

[0093] S308: Determine the number of matching local features corresponding to each candidate image.

[0094] S309: Determine, among the candidate images, the candidate images for which the number of matching local features is greater than a third threshold as similar images to the image to be processed.

[0095] In the above steps S308-S309, the server can quantify the similarity between the image to be processed and each candidate image by determining the number of matching local features of each candidate image. When the number of matching local features of a candidate image is greater than the third threshold, the candidate image is determined as a similar image to the image to be processed. The above method improves the accuracy of similar image retrieval. The third threshold can be flexibly set according to business conditions, such as the third threshold can be set to 30.

[0096] In one possible implementation, the third threshold can be specifically set according to the image information corresponding to the candidate image. The server can perform image content detection on the candidate image to obtain the image category. If the identified candidate image is a store photo, the third threshold can be set to a larger value, because store photos usually have more feature elements, and a larger number of matching local features need to be matched before the image can be considered to be a similar image to the image to be detected; if the identified candidate image is a material photo (such as a shelf), the third threshold can be set to a smaller value, because material photos usually have fewer feature elements, and only a smaller number of matching local features need to be matched before the image can be considered to be a similar image to the image to be detected. The third thresholds corresponding to different image categories can be pre-set. Through the above method, the accuracy of similar image retrieval can be further improved.

[0097] In an implementable manner, the server may further perform the following steps:

[0098] (1) When no similar image to the image to be processed is determined, the credit evaluation data of the applicant corresponding to the image to be processed is determined.

[0099] In an embodiment of the present application, when no similar image to the image to be processed is determined, it means that the image to be processed submitted by the applicant does not appear in the historical database, indicating that the image to be processed has passed the review. The server can continue to conduct further credit qualification review on the applicant and obtain the credit evaluation data of the applicant corresponding to the image to be processed. The credit evaluation data indicates the degree of credit of the applicant.

[0100] In one implementable method, when the server fails to determine a similar image to the image to be processed, it can determine the credit evaluation data of the applicant corresponding to the image to be processed based on the relevant information of the applicant. The relevant information of the applicant may be the applicant's personal information, credit score, historical application records, etc., which improves the data accuracy and credibility of the credit evaluation data.

[0101] (2) When the credit evaluation data meets the credit conditions, the applicant corresponding to the image to be processed is determined as a credit customer.

[0102] In the embodiment of the present application, when the credit evaluation data meets the credit conditions, such as the credit evaluation data can be a credit evaluation score, and when the credit evaluation score is greater than a preset value, the server determines the applicant as a credit customer. The above method realizes the automatic judgment of credit qualifications, reduces the need for manual operation and manual intervention, improves the efficiency of credit decision-making, and speeds up the credit process and reduces customer waiting time.

[0103] (3) The first feature map of the image to be processed and the first feature vector obtained by performing global feature extraction processing on the image to be processed are stored in a history database.

[0104] In an embodiment of the present application, after determining that the applicant is a credit customer, the server stores the first feature map and the first feature vector corresponding to the image to be processed uploaded by the applicant in a historical database to ensure the integrity of the data in the historical database, so that when subsequent customers upload images to be verified for credit approval, all historical images pre-stored by all credit customers can be used to perform similar image retrieval tasks for the images to be verified, thereby ensuring the accuracy of credit decisions.

[0105] It should be noted that, when the first threshold, the second threshold, and the third threshold in the embodiment of the present application are set, the values ​​may be the same or different, and the embodiment of the present application is not limited to this.

[0106] The overall process of the similar image retrieval method provided in the embodiment of the present application is described below: Figure 4D , Figure 4D This is a processing flow chart of similar image retrieval provided by an embodiment of the present application. The historical images of the credited customers are processed by feature extraction to obtain a second feature vector and a second feature map. The second feature vector is stored in a feature vector database (such as a first database), and the second feature map is stored in a feature map database (such as a second database). When the image to be processed uploaded by the applicant is obtained, the image to be processed is subjected to feature extraction processing to obtain a first feature map and a first feature vector. Then, the first feature vector and the second feature vector of each historical image obtained from the first database are similarly calculated, and the historical image corresponding to the second feature vector with a similarity greater than a threshold is used as a candidate image. The second feature map corresponding to each candidate image is obtained from the second database, and the first feature map and the second feature map corresponding to each candidate image are input into the local feature matching network for processing to obtain a local feature matching result. The local feature matching result includes matching local features of the image to be processed and the candidate image; for each candidate image, when the number of matching local features corresponding to the candidate image is greater than the threshold, the candidate image is determined as a similar image to the image to be processed.

[0107] It should be noted that when a similar image to the image to be processed has not been determined, it is determined whether the applicant customer corresponding to the image to be processed is a credit customer. When it is determined that the applicant customer is a credit customer, the first feature vector of the image to be processed is stored in the feature vector database, and the first feature map of the image to be processed is stored in the feature map database.

[0108] The similar image retrieval method provided in the embodiment of the present application can be applied to agricultural finance. In agricultural finance, such as agricultural breeding or management, there is often a high regional concentration. It is relatively easy to obtain credit approval photos of other customers. The authenticity of uploaded photos cannot be accurately determined by GPS positioning. In addition, when different credit reviewers conduct audits, it is often difficult to distinguish the misused information. The similar image retrieval method provided in the embodiment of the present application can solve the above problems. It can quickly find images similar to the image to be processed in the image data of a large number of credit-granted customers. It can better identify fraudulent behavior in agricultural financial business and reduce the risk of fraud. At the same time, this method does not require any annotation, does not require training models, is quickly deployed, has high universality, and can be used for various businesses.

[0109] See also Figure 5 , Figure 5 : is a schematic block diagram of an image processing device provided in an embodiment of the present application. The image processing device may specifically include:

[0110] The processing module 501 is used to perform similarity detection processing on the image to be processed and the historical images in the history database to obtain a similarity detection result, and determine a candidate image from the historical images according to the similarity detection result;

[0111] The processing module 501 is further used to determine the first feature map of the image to be processed and the second feature map of the candidate image;

[0112] A local feature matching module 502 is used to input the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result includes matching local features of the image to be processed and the candidate image;

[0113] The processing module 501 is further configured to determine similar images of the image to be processed from the candidate images according to the number of matching local features included in the local feature matching result.

[0114] In one implementable manner, the processing module 501 is used to perform similarity detection processing on the image to be processed and the historical images in the history database to obtain a similarity detection result, and determine a candidate image from the historical images according to the similarity detection result, specifically for:

[0115] Performing global feature extraction processing on the image to be processed to obtain a first feature vector, and obtaining second feature vectors of a plurality of pre-stored historical images from a historical database;

[0116] Performing similarity calculation on the first feature vector and the plurality of second feature vectors respectively to obtain a similarity detection result; the similarity detection result includes the similarity between the first feature vector and each of the second feature vectors;

[0117] Determine a second feature vector whose similarity to the first feature vector is greater than a first threshold;

[0118] The historical image corresponding to the second feature vector whose similarity is greater than the first threshold is taken as a candidate image.

[0119] In an implementable manner, when the processing module 501 is used to determine the first feature map of the image to be processed and the second feature map of the candidate image, it is specifically used to:

[0120] Inputting the image to be processed into a feature extraction network for processing to obtain a first feature map of the image to be processed;

[0121] The second feature map of the candidate image stored in advance is obtained from the historical database.

[0122] In an implementable manner, when the local feature matching module 502 is used to input the first feature map and the second feature map into the local feature matching network for processing to obtain the local feature matching result, it is specifically used to:

[0123] Using the feature transformation subnetwork of the local feature matching network, the first feature map and the second feature map are subjected to feature transformation processing to obtain a first feature map after feature transformation and a second feature map after feature transformation;

[0124] Using the feature matching subnetwork of the local feature matching network, perform feature matching processing on the first feature map after the feature transformation and the second feature map after the feature transformation to obtain a matching degree matrix;

[0125] The local feature matching result is determined according to the above matching degree matrix.

[0126] In an implementable manner, the element values ​​of any group of elements in the matching degree matrix represent the matching degree between any local feature vector of the image to be processed and each local feature vector of the candidate image;

[0127] The local feature matching module 502, when used to determine the local feature matching result according to the above matching degree matrix, is specifically used to:

[0128] Determine candidate elements in the matching matrix, where the candidate elements are elements whose element values ​​are greater than a second threshold value among the elements included in the matching matrix;

[0129] When the element value of the candidate element is the maximum element value of the row and the maximum element value of the column, determining that the local feature vector corresponding to the candidate element in the image to be processed and the local feature vector corresponding to the candidate element in the candidate image are matching local features;

[0130] A local feature matching result is determined based on each matching local feature corresponding to the candidate image.

[0131] In an implementable manner, when the processing module 501 is used to determine a similar image of the image to be processed from the candidate images according to the number of matching local features included in the local feature matching result, it is specifically used to:

[0132] Determine the number of matching local features corresponding to each of the candidate images;

[0133] The candidate images whose number of matching local features is greater than a third threshold value among the candidate images are determined as similar images to the image to be processed.

[0134] In an implementable manner, the processing module 501 is further used for:

[0135] Acquire historical images uploaded by the credit-granted customer, perform global feature extraction processing on the historical images to obtain a second feature vector, and store the second feature vector of the historical images in the historical database;

[0136] The historical image is input into the feature extraction network for processing to obtain a second feature map of the historical image, and the second feature map of the historical image is stored in the historical database.

[0137] In an implementable manner, the processing module 501 is further used for:

[0138] When no similar image to the image to be processed is determined, determining the credit evaluation data of the applicant corresponding to the image to be processed;

[0139] When the credit evaluation data meets the credit conditions, the applicant corresponding to the image to be processed is determined as a credit customer;

[0140] The first feature map of the image to be processed and the first feature vector obtained by performing global feature extraction processing on the image to be processed are stored in the historical database.

[0141] It should be noted that the functions of each functional module of the image processing device in the embodiment of the present application can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment, which will not be repeated here.

[0142] See also Figure 6 , Figure 6 601, storage device 602 and communication interface 603. The processor 601, storage device 602 and communication interface 603 can exchange data.

[0143] The above-mentioned storage device 602 may include a volatile memory (volatile memory), such as a random-access memory (RAM); the storage device 602 may also include a non-volatile memory (non-volatile memory), such as a flash memory, a solid-state drive (SSD), etc.; the above-mentioned storage device 602 may also include a combination of the above-mentioned types of memory.

[0144] The processor 601 may be a central processing unit (CPU). In one embodiment, the processor 601 may also be a graphics processing unit (GPU). The processor 601 may also be a combination of a CPU and a GPU. In one embodiment, the storage device 602 is used to store program instructions, and the processor 601 may call the program instructions to perform the following operations:

[0145] Performing similarity detection processing on the image to be processed and the historical images in the historical database to obtain a similarity detection result, and determining a candidate image from the historical images according to the similarity detection result;

[0146] Determine a first feature map of the image to be processed and a second feature map of the candidate image;

[0147] Inputting the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result includes matching local features of the image to be processed and the candidate image;

[0148] According to the number of matching local features included in the local feature matching result, a similar image to the image to be processed is determined from the candidate images.

[0149] In an implementable manner, the processor 601 is used to perform similarity detection processing on the image to be processed and the historical images in the history database to obtain a similarity detection result, and determine the candidate image from the historical images according to the similarity detection result, specifically for:

[0150] Performing global feature extraction processing on the image to be processed to obtain a first feature vector, and obtaining second feature vectors of a plurality of pre-stored historical images from a historical database;

[0151] Performing similarity calculation on the first feature vector and the plurality of second feature vectors respectively to obtain a similarity detection result; the similarity detection result includes the similarity between the first feature vector and each of the second feature vectors;

[0152] Determine a second feature vector whose similarity to the first feature vector is greater than a first threshold;

[0153] The historical image corresponding to the second feature vector whose similarity is greater than the first threshold is taken as a candidate image.

[0154] In an implementable manner, when the processor 601 is used to determine the first feature map of the image to be processed and the second feature map of the candidate image, it is specifically used to:

[0155] Inputting the image to be processed into a feature extraction network for processing to obtain a first feature map of the image to be processed;

[0156] The second feature map of the candidate image stored in advance is obtained from the historical database.

[0157] In an implementable manner, when the processor 601 is used to input the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result, it is specifically used to:

[0158] Using the feature transformation subnetwork of the local feature matching network, the first feature map and the second feature map are subjected to feature transformation processing to obtain a first feature map after feature transformation and a second feature map after feature transformation;

[0159] Using the feature matching subnetwork of the local feature matching network, perform feature matching processing on the first feature map after the feature transformation and the second feature map after the feature transformation to obtain a matching degree matrix;

[0160] The local feature matching result is determined according to the above matching degree matrix.

[0161] In an implementable manner, the element values ​​of any group of elements in the matching degree matrix represent the matching degree between any local feature vector of the image to be processed and each local feature vector of the candidate image;

[0162] The processor 601, when used to determine the local feature matching result according to the matching degree matrix, is specifically used to:

[0163] Determine candidate elements in the matching matrix, where the candidate elements are elements whose element values ​​are greater than a second threshold value among the elements included in the matching matrix;

[0164] When the element value of the candidate element is the maximum element value of the row and the maximum element value of the column, determining that the local feature vector corresponding to the candidate element in the image to be processed and the local feature vector corresponding to the candidate element in the candidate image are matching local features;

[0165] A local feature matching result is determined based on each matching local feature corresponding to the candidate image.

[0166] In an implementable manner, when the processor 601 is used to determine a similar image of the image to be processed from the candidate images according to the number of matching local features included in the local feature matching result, the processor 601 is specifically used to:

[0167] Determine the number of matching local features corresponding to each of the candidate images;

[0168] The candidate images whose number of matching local features is greater than a third threshold value among the candidate images are determined as similar images to the image to be processed.

[0169] In an implementable manner, the processor 601 is further configured to:

[0170] Acquire historical images uploaded by the credit-granted customer, perform global feature extraction processing on the historical images to obtain a second feature vector, and store the second feature vector of the historical images in the historical database;

[0171] The historical image is input into the feature extraction network for processing to obtain a second feature map of the historical image, and the second feature map of the historical image is stored in the historical database.

[0172] In an implementable manner, the processor 601 is further configured to:

[0173] When no similar image to the image to be processed is determined, determining the credit evaluation data of the applicant corresponding to the image to be processed;

[0174] When the credit evaluation data meets the credit conditions, the applicant corresponding to the image to be processed is determined as a credit customer;

[0175] The first feature map of the image to be processed and the first feature vector obtained by performing global feature extraction processing on the image to be processed are stored in the historical database.

[0176] In a specific implementation, the processor 601, the storage device 602, and the communication interface 603 described in the embodiments of the present application can execute the embodiments of the present application. Figure 2 or Figure 3 The implementation described in the related embodiments of the provided image processing method can also be implemented in the embodiments of the present application. Figure 5 The implementation methods described in the relevant embodiments of the provided image processing device will not be repeated here.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0178] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the image processing device mentioned above, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the above Figure 2 , Figure 3 The method in the corresponding embodiment, therefore, will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on a computer device, or executed on multiple computer devices located in one location, or, executed on multiple computer devices distributed in multiple locations and interconnected by a communication network, and multiple computer devices distributed in multiple locations and interconnected by a communication network can constitute a blockchain system.

[0179] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device can perform the above Figure 2 , Figure 3 The method in the corresponding embodiment will therefore not be described in detail here.

[0180] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the above-mentioned program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The above-mentioned storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0181] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of this application are still within the scope of the invention.

Claims

1. An image processing method, characterized in that: The method comprises: Performing similarity detection processing on the image to be processed and the historical images in the historical database to obtain a similarity detection result, and determining a candidate image from the historical images according to the similarity detection result; Determining a first feature map of the image to be processed and a second feature map of the candidate image; Inputting the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result includes matching local features of the image to be processed and the candidate image; According to the number of matched local features included in the local feature matching result, a similar image to the image to be processed is determined from the candidate images.

2. The method according to claim 1, characterized in that The performing similarity detection processing on the image to be processed and the historical images in the historical database to obtain a similarity detection result, and determining a candidate image from the historical images according to the similarity detection result, comprises: Performing global feature extraction processing on the image to be processed to obtain a first feature vector, and obtaining second feature vectors of a plurality of pre-stored historical images from a historical database; Performing similarity calculation on the first feature vector and the plurality of second feature vectors respectively to obtain a similarity detection result; the similarity detection result includes the similarity between the first feature vector and each of the second feature vectors; Determine a second feature vector whose similarity to the first feature vector is greater than a first threshold; The historical image corresponding to the second feature vector whose similarity is greater than the first threshold is taken as a candidate image.

3. The method according to claim 2, characterized in that The determining the first feature map of the image to be processed and the second feature map of the candidate image includes: Inputting the image to be processed into a feature extraction network for processing to obtain a first feature map of the image to be processed; A pre-stored second feature map of the candidate image is obtained from the history database.

4. The method according to any one of claims 1 to 3, characterized in that The step of inputting the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result includes: Using the feature transformation subnetwork of the local feature matching network, performing feature transformation processing on the first feature map and the second feature map to obtain a first feature map after feature transformation and a second feature map after feature transformation; Using the feature matching subnetwork of the local feature matching network, performing feature matching processing on the first feature map after the feature transformation and the second feature map after the feature transformation to obtain a matching degree matrix; The local feature matching result is determined according to the matching degree matrix.

5. The method according to claim 4, characterized in that The element values ​​of any group of elements in the matching degree matrix represent the matching degree between any local feature vector of the image to be processed and each local feature vector of the candidate image; Wherein, determining the local feature matching result according to the matching degree matrix includes: Determine a candidate element in the matching matrix, wherein the candidate element is an element whose element value is greater than a second threshold value among the elements included in the matching matrix; When the element value of the candidate element is the maximum element value of the row and the maximum element value of the column, determining that the local feature vector corresponding to the candidate element in the image to be processed and the local feature vector corresponding to the candidate element in the candidate image are matching local features; A local feature matching result is determined according to each matching local feature corresponding to the candidate image.

6. The method according to any one of claims 1 to 3, characterized in that The determining, from the candidate images, a similar image to the image to be processed according to the number of matching local features included in the local feature matching result, comprises: Determine the number of matching local features corresponding to each of the candidate images; The candidate images whose number of matching local features is greater than a third threshold value in the candidate images are determined as similar images to the image to be processed.

7. The method according to claim 2 or 3, characterized in that: The method further comprises: Acquire historical images uploaded by the credit-granted customer, perform global feature extraction processing on the historical images to obtain a second feature vector, and store the second feature vector of the historical images in the historical database; The historical image is input into the feature extraction network for processing to obtain a second feature map of the historical image, and the second feature map of the historical image is stored in the historical database.

8. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When no similar image to the image to be processed is determined, determining the credit evaluation data of the applicant corresponding to the image to be processed; When the credit evaluation data meets the credit conditions, the applicant corresponding to the image to be processed is determined as a credit customer; The first feature map of the image to be processed and the first feature vector obtained by performing global feature extraction processing on the image to be processed are stored in the history database.

9. An image processing device, characterized in that: The device comprises: A processing module, used for performing similarity detection processing on the image to be processed and the historical images in the historical database to obtain a similarity detection result, and determining a candidate image from the historical images according to the similarity detection result; The processing module is further used to determine a first feature map of the image to be processed and a second feature map of the candidate image; A local feature matching module, used for inputting the first feature map and the second feature map into a local feature matching network for processing to obtain a local feature matching result; the local feature matching result includes matching local features of the image to be processed and the candidate image; The processing module is further used to determine similar images of the image to be processed from the candidate images according to the number of matching local features included in the local feature matching result.

10. A computer device, characterized in that: include: A processor, a storage device and a communication interface, wherein the processor, the communication interface and the storage device are connected to each other, wherein the storage device stores an executable program code, and the processor is used to call the executable program code to implement the image processing method according to any one of claims 1 to 8.

Images