Banking business image material detection method and system based on deep learning

Abstract

The embodiment of the invention provides a bank business image material detection method and system based on deep learning, and the method comprises the steps: obtaining a to-be-audited bank business image, and extracting a target feature file from the to-be-audited bank business image through a comparison text image pre-training model, according to the feature tensor of a reference non-scene image in an image database and the target feature tensor, carrying out image cleaning on the to-be-audited banking business image to generate a to-be-audited scene image set, according to a target Hamming code corresponding to the target feature tensor of the to-be-audited scene image, obtaining a candidate banking business image, and according to the candidate banking business image, carrying out image cleaning on the to-be-audited banking business image. And determining a risk business image according to the similarity between the feature tensor of each candidate banking business image and the feature tensor of the to-be-audited scene image, so that the processing efficiency of the banking business images is improved by quickly detecting the images by using a comparison text image pre-training model and a locality sensitive hashing algorithm.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a method and system for detecting banking business image materials based on deep learning. Background Art

[0002] In the bank loan business process, the review of loan image materials has always been an important issue that plagues the banking industry. Under the traditional review model, the data entry and review work involved in the loan business requires a lot of manpower and material resources. Among them, the various certificates and other related paper material images required for the loan are usually uploaded by the loan officer to the designated directory for review. However, this review method faces many challenges.

[0003] Due to the huge volume of bank loan business, each loan may involve multiple images, such as various ID images, paper material images, etc., so the workload of manual comparison is large. If the computer system traverses all images for pairwise comparison, it will not only require more computing resources, but also take a long time, resulting in low processing efficiency and unable to meet the needs of fast processing of banking business.

[0004] Therefore, how to improve the processing efficiency of banking business images is an urgent problem to be solved. Summary of the invention

[0005] In view of this, the purpose of this application is to provide a banking business image material detection method and system based on deep learning to improve the processing efficiency of banking business images.

[0006] In conjunction with the first aspect of the present application, a method for detecting banking business image materials based on deep learning is provided, the method comprising: Acquire a bank business image to be reviewed, wherein the bank business image to be reviewed includes a scene image and a non-scene image; Extracting a target feature file from the pending bank business image using a comparison text image pre-training model, wherein the target feature file includes a target feature tensor, the name of the target feature file is the image name of the pending bank business image, the key name of the target feature file is the absolute path of the pending bank business image, and the key value of the target feature file is the target feature tensor; Perform image cleaning on the bank business image to be reviewed according to the feature tensor of the reference non-scene image in the image database and the target feature tensor to generate a set of scene images to be reviewed, wherein the target feature files corresponding to the scene images to be reviewed in the set of scene images to be reviewed are saved in a first folder, and the folder name of the first folder is the business number corresponding to the scene images to be reviewed; Acquire a candidate banking business image according to the target Hamming code corresponding to the target feature tensor of the scene image to be reviewed, wherein the second folder where the candidate banking business image is located is different from the first folder; Calculating the similarity between the feature tensor of each of the candidate banking business images and the feature tensor of the scene image to be reviewed, and generating a similarity detection result; A first candidate banking business image having a similarity greater than a first preset similarity threshold value in the similarity detection result and the scene image to be reviewed are outputted, wherein the first candidate banking business image is a risky business image.

[0007] In a possible implementation of the first aspect, extracting a target feature file from the to-be-audited banking business image using a comparison text image pre-training model includes: Using the visual transformer encoder in the contrast text image pre-training model, the size of the to-be-audited banking image is adjusted to a target size, wherein the target size is a multidimensional array including image height, image width, and image channel number; Performing image segmentation on the to-be-audited banking business image adjusted to the target size to obtain a target block set of the to-be-audited banking business image, wherein the target block set includes a classification mark block, and the classification mark block is used to characterize the global features of the to-be-audited banking business image; Flattening the target blocks in the target block set and mapping them to the target embedding space through linear projection; Adding a position code to each of the target blocks in the target embedding space, wherein the position code is used to characterize the spatial information of the bank business image to be reviewed; Performing feature extraction on the target block with the position code added thereto through a multi-head self-attention mechanism to obtain a feature representation to be aggregated of the target block with the position code added thereto; Performing feature aggregation on the feature representation to be aggregated using a multi-layer perceptron algorithm to obtain the target feature tensor of the bank business image to be reviewed; The target feature file is generated according to the target feature tensor of the banking business image to be reviewed.

[0008] In a possible implementation of the first aspect, extracting features from the target block to which the position code is added by a multi-head self-attention mechanism to obtain a feature representation to be aggregated of the target block to which the position code is added includes: According to the target block to which the position code is added and a preset weight matrix, a query vector, a key vector, and a value vector of the target block under each header are obtained; Calculate and obtain the attention scores of the target block and other target blocks according to the query vector and the key vector; Normalizing the attention score of the target block using a softmax function to generate a normalized attention weight; Generate candidate feature representations under each head according to the attention weight and the value vector; The candidate feature representations under each head are concatenated to generate the feature representation to be aggregated.

[0009] In a possible implementation of the first aspect, the cleaning of the to-be-reviewed banking business image according to the feature tensor of the reference non-scene image in the image database and the target feature tensor to generate a to-be-reviewed scene image set includes: Acquire a plurality of the reference non-scene images in the image database, and extract feature files of the reference non-scene images, wherein the feature files of the reference non-scene images include feature tensors of the reference non-scene images; Calculating the cosine similarity of the feature tensor of the reference non-scene image and the target feature tensor of the to-be-audited banking business image to obtain a comparison result; If the comparison result is greater than or equal to a second preset similarity threshold, the to-be-reviewed banking business image corresponding to the comparison result is deleted to generate a to-be-reviewed scene image set.

[0010] In a possible implementation of the first aspect, acquiring the candidate banking business image according to the target Hamming code corresponding to the target feature tensor of the scene image to be reviewed includes: In a local sensitive hash high-dimensional feature space, according to a first target Hamming code corresponding to the target feature tensor of the scene image to be reviewed, a candidate banking business image is obtained using a local sensitive hash algorithm, wherein the first Hamming code of the candidate banking business image is the same as the first target Hamming code, and the second folder where the candidate banking business image is located is different from the first folder; or, Obtaining M second target Hamming codes of the scene image to be reviewed, wherein the M second target Hamming codes are generated by performing hash processing on the scene image to be reviewed according to M hash algorithms, wherein the hash algorithms include the local sensitive hash algorithm; The images corresponding to the second Hamming codes having at least N identical ones among the M second target Hamming codes in the image database are taken as the candidate banking business images, the second folder where the candidate banking business images are located is different from the first folder, and N is less than or equal to M.

[0011] In a possible implementation of the first aspect, calculating the similarity between the feature tensor of each candidate banking service image and the feature tensor of the to-be-audited scene image to generate a similarity detection result includes: According to the image structural features of the candidate banking service image, the target feature tensor is decomposed into at least two target feature tensor substructures, and the feature tensor of the to-be-audited scene image is decomposed into at least two feature tensor substructures; Respectively calculating the cosine similarity between the target feature tensor substructure and the feature tensor substructure belonging to the same image structure feature, and obtaining the similarity value between each target feature tensor substructure and each feature tensor substructure corresponding to each image structure feature; The similarity detection result is generated according to a preset weight, the number of the image structural features, and each of the similarity values, wherein the preset weight is related to the importance of the image structural features.

[0012] In a possible implementation of the first aspect, the method further includes: Periodically obtaining feedback results of the risky business image, wherein the feedback results represent the correctness of the risk judgment of the risky business image; Determining a threshold adjustment rule for the first preset similarity threshold according to the accuracy rate of the feedback results of the risky business image within a period; The first preset similarity threshold is updated according to the threshold adjustment rule.

[0013] In combination with the second aspect of the present application, a deep learning-based banking business image material detection system is provided, wherein the deep learning-based banking business image detection system includes a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions, and when the processor executes the machine-executable instructions, the deep learning-based banking business image material detection system implements the aforementioned deep learning-based banking business image material detection method.

[0014] In conjunction with the third aspect of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed, the aforementioned deep learning-based banking business image material detection method is implemented.

[0015] In conjunction with the fourth aspect of the present application, a computer program product is provided, which, when executed by a processor, implements the aforementioned deep learning-based banking business image material detection method.

[0016] In combination with any of the above aspects, by obtaining a bank business image to be reviewed, a target feature file is extracted from the bank business image to be reviewed using a comparative text image pre-training model, and the bank business image to be reviewed is cleaned according to the feature tensor of the reference non-scene image in the image database and the target feature tensor to generate a set of scene images to be reviewed, and candidate banking business images are obtained according to the target Hamming code corresponding to the target feature tensor of the scene image to be reviewed, and risky business images are determined based on the similarity between the feature tensor of each candidate banking business image and the feature tensor of the scene image to be reviewed, thereby improving the processing efficiency of banking business images by quickly detecting images using a comparative text image pre-training model and a locally sensitive hashing algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained by combining these drawings without paying creative work.

[0018] Figure 1 A flowchart of a deep learning-based banking business image material detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. According to the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0021] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] Figure 1 The flowchart of the banking business image material detection method based on deep learning provided by the embodiment of the present application is shown. It should be understood that in other embodiments, the order of some steps in the banking business image detection method based on deep learning of this embodiment can be shared with each other according to actual needs, or some steps can be omitted or maintained. The details of the banking business image material detection method based on deep learning include: Step S110, obtaining the bank business image to be reviewed.

[0023] The bank business images to be reviewed include scene images and non-scene images.

[0024] In this embodiment, the server is connected to the image storage system inside the bank, and this storage system stores various banking business image data. The server obtains the banking business images that need to be reviewed from this storage system according to the predetermined task instructions. For example, in the commercial loan business process, the enterprise submits a series of loan application materials, including scene images (such as site photos) and non-scene images (such as contracts, certificates, screenshots, etc.). After these images are uploaded to the bank's image storage system, the server retrieves and obtains these images from the storage system as banking business images to be reviewed.

[0025] Step S120, extracting a target feature file from the to-be-audited banking business image using a comparison text image pre-training model.

[0026] Among them, the target feature file includes a target feature tensor, the name of the target feature file is the image name of the bank business image to be reviewed, the key name of the target feature file is the absolute path of the bank business image to be reviewed, and the key value of the target feature file is the target feature tensor.

[0027] In this embodiment, the server uses a pre-trained deep learning model that is specifically used to compare the features of text and images to extract key information from the image. This model usually includes a visual transformer encoder that can adjust images of different sizes to a unified target size for model processing.

[0028] After resizing the image, the server splits the image into multiple blocks, each of which represents a part of the image and may contain specific features such as text, numbers, or specific graphic elements. These blocks are flattened and mapped to a high-dimensional feature space through linear projection, and the position information of each block is added to the feature through position encoding to preserve spatial information.

[0029] Next, the server processes these chunks using a multi-head self-attention mechanism. In this mechanism, each chunk is treated as a query vector and compared with other chunks (key vectors and value vectors) to calculate an attention score. These scores are normalized by a softmax function to generate attention weights, and then candidate feature representations are generated based on the weights and value vectors. Finally, all candidate feature representations are concatenated to form the feature representation to be aggregated.

[0030] The server aggregates these feature representations using a multi-layer perceptron algorithm and eventually generates the target feature tensor for each image. These tensors are saved as feature files, with the same file name as the image, the key name being the image's storage path, and the key value being the feature tensor.

[0031] Step S130, performing image cleaning on the to-be-audited banking business image according to the feature tensor of the reference non-scene image in the image database and the target feature tensor, to generate a to-be-audited scene image set.

[0032] The target feature file corresponding to the scene image to be reviewed in the scene image set to be reviewed is stored in a first folder, and the folder name of the first folder is the business number corresponding to the scene image to be reviewed.

[0033] In this embodiment, the server retrieves reference non-scene images from the image database and extracts their feature tensors. These reference images are pre-defined to help identify and exclude images that are not related to banking business. The server calculates the cosine similarity between the feature tensors of these reference images and the target feature tensors of the image to be reviewed. If the similarity exceeds a preset threshold, the server will consider the image to be reviewed to be a non-scene image and remove it from the set to be reviewed.

[0034] After cleaning, the remaining images are considered to be scenes related to banking services to be reviewed. The server saves the feature files of these images in the first folder named by the business number, and each folder corresponds to a specific banking business.

[0035] Step S140, obtaining a candidate banking business image according to the target Hamming code corresponding to the target feature tensor of the scene image to be reviewed.

[0036] The second folder where the candidate banking service image is located is different from the first folder.

[0037] In this embodiment, the server processes the target feature tensor of the scene image to be reviewed to obtain the corresponding target Hamming code. Then, the server searches the bank's image database to find images whose Hamming codes are the same as the target Hamming code of the scene image to be reviewed, and these images are used as candidate banking business images.

[0038] Since different businesses are stored in different folders, and the review of risk images is for different businesses, the second folder where these candidate banking business images are located is different from the first folder storing the target feature files corresponding to the scene images to be reviewed.

[0039] Step S150, calculating the similarity between the feature tensor of each of the candidate banking business images and the feature tensor of the scene image to be reviewed, and generating a similarity detection result.

[0040] In this embodiment, the server calculates the similarity between the feature tensor of the candidate banking service image and the feature tensor of the scene image to be reviewed. For example, the similarity detection result between the two can be determined by calculating cosine similarity, Euclidean distance similarity, etc.

[0041] Step S160, outputting the first candidate banking business image whose similarity in the similarity detection result is greater than a first preset similarity threshold, and the scene image to be reviewed.

[0042] The first candidate banking business image is a risky business image.

[0043] In this embodiment, the server checks the similarity detection results to find candidate banking business images whose similarity exceeds a first preset similarity threshold. These images are considered to be potential risk business images, and the server outputs them together with the scene images to be reviewed for further manual review or automatic processing.

[0044] Based on the above steps, by obtaining a bank business image to be reviewed, a target feature file is extracted from the bank business image to be reviewed using a contrasting text image pre-training model, and the bank business image to be reviewed is cleaned according to the feature tensor of the reference non-scene image in the image database and the target feature tensor to generate a set of scene images to be reviewed, and candidate banking business images are obtained according to the target Hamming code corresponding to the target feature tensor of the scene image to be reviewed, and a risky business image is determined based on the similarity between the feature tensor of each candidate banking business image and the feature tensor of the scene image to be reviewed, thereby improving the processing efficiency of banking business images by quickly detecting images using a contrasting text image pre-training model and a locally sensitive hashing algorithm.

[0045] In a possible implementation, the extracting a target feature file from the to-be-audited banking business image using a contrasting text image pre-training model includes: Using the visual transformer encoder in the contrast text image pre-training model, the size of the to-be-audited banking image is adjusted to a target size, wherein the target size is a multidimensional array including image height, image width, and image channel number; Performing image segmentation on the to-be-audited banking business image adjusted to the target size to obtain a target block set of the to-be-audited banking business image, wherein the target block set includes a classification mark block, and the classification mark block is used to characterize the global features of the to-be-audited banking business image; Flattening the target blocks in the target block set and mapping them to the target embedding space through linear projection; Adding a position code to each of the target blocks in the target embedding space, wherein the position code is used to characterize the spatial information of the bank business image to be reviewed; Performing feature extraction on the target block with the position code added thereto through a multi-head self-attention mechanism to obtain a feature representation to be aggregated of the target block with the position code added thereto; Performing feature aggregation on the feature representation to be aggregated using a multi-layer perceptron algorithm to obtain the target feature tensor of the bank business image to be reviewed; The target feature file is generated according to the target feature tensor of the banking business image to be reviewed.

[0046] In this embodiment, the server calls the visual transformer encoder in the pre-trained model for comparing text images to process the image of the bank business to be reviewed. For example, when the server receives a bank business image to be reviewed, the visual transformer encoder will adjust the size of the image to the target size, which is a multidimensional array including image height, image width, and number of image channels. Assume that the image of the bank business to be reviewed is adjusted to a target size of 224×224×3 (where 224 is the height and width, and 3 is the number of channels, corresponding to the common RGB color channel). The server performs image segmentation on the image of the bank business to be reviewed after being adjusted to the target size. Through a specific segmentation algorithm, the photo is divided into multiple small blocks, thereby obtaining a target block set of the image of the bank business to be reviewed. In this target block set, a special classification mark block is included. This classification mark block has a special meaning, and it can characterize the global features of the image of the bank business to be reviewed, such as the overall layout structure of the image, the distribution of key clause areas, and other overall information.

[0047] The server processes each block in the target block set. First, the blocks are flattened, that is, each block is converted from a two-dimensional or three-dimensional (depending on its original shape) image block to a one-dimensional vector form. Then, these flattened blocks are mapped to the target embedding space by linear projection mapping. The purpose of this operation is to convert the information of each block into a representation form that is more suitable for subsequent processing of the model. In this target embedding space, the features of the image block can be represented and calculated in a specific mathematical structure. In the target embedding space, the server adds a position code to each target block. Since the image is processed in blocks in the previous operation, the spatial position information between blocks may be partially lost. The role of position coding is to supplement this part of information to characterize the spatial information of the house mortgage contract photo. For example, through a specific coding method, it is possible to represent the position relationship such as whether a block is located in the upper left corner or the lower right corner in the original image, as well as spatial information such as the relative position order between different blocks.

[0048] Next, the server uses a multi-head self-attention mechanism to extract features from the target block with position encoding added, and uses a multi-layer perceptron algorithm to perform feature aggregation operations on the feature representation to be aggregated, generates a target feature tensor of the bank business image to be reviewed, and stores the target feature tensor of the bank business image to be reviewed in a target feature file corresponding to the bank business image to be reviewed.

[0049] In a possible implementation, the extracting features of the target block to which the position code is added by a multi-head self-attention mechanism to obtain a feature representation to be aggregated of the target block to which the position code is added includes: According to the target block to which the position code is added and a preset weight matrix, a query vector, a key vector, and a value vector of the target block under each header are obtained; Calculate and obtain the attention scores of the target block and other target blocks according to the query vector and the key vector; Normalizing the attention score of the target block using a softmax function to generate a normalized attention weight; Generate candidate feature representations under each head according to the attention weight and the value vector; The candidate feature representations under each head are concatenated to generate the feature representation to be aggregated.

[0050] In this embodiment, for each target block to which position encoding is added, the server calculates the query vector, key vector, and value vector of the target block under each head according to a predefined preset weight matrix. For example, for a multi-head self-attention mechanism with 8 heads, 8 sets of query vectors, key vectors, and value vectors are calculated for each target block. The calculation of these vectors is based on the linear transformation between the features of the target block and the preset weight matrix, and different heads will focus on different aspects of the feature information of the target block. Based on the calculated query vector and key vector, the server calculates the attention score of the target block and other target blocks. For each target block, its query vector will perform a dot product operation with the key vectors of all other target blocks (including itself) to obtain an attention score representing the degree of attention of the target block to other target blocks.

[0051] Next, the server uses the softmax function to normalize the attention scores of the target blocks and generates normalized attention weights. This step ensures that the sum of all attention weights is 1, so that the degree of attention of each target block to other target blocks is expressed in the form of a relative proportion. Based on the normalized attention weights and value vectors, the server generates candidate feature representations for each head. Specifically, each target block performs a weighted summation of the value vectors of other target blocks according to its attention weights to other target blocks to obtain a new feature representation under each head. The server concatenates the candidate feature representations obtained under each head to generate a feature representation to be aggregated of the target block with position encoding added. For example, if there are 8 heads, the candidate feature representation under each head is a vector of a specific dimension. These 8 vectors are concatenated together in sequence to obtain a higher-dimensional feature representation vector to be aggregated, which contains the comprehensive feature information of the target block at different attention levels.

[0052] In a possible implementation, the cleaning of the to-be-reviewed banking business image according to the feature tensor of the reference non-scene image in the image database and the target feature tensor to generate a set of to-be-reviewed scene images includes: Acquire a plurality of the reference non-scene images in the image database, and extract feature files of the reference non-scene images, wherein the feature files of the reference non-scene images include feature tensors of the reference non-scene images; Calculating the cosine similarity of the feature tensor of the reference non-scene image and the target feature tensor of the to-be-audited banking business image to obtain a comparison result; If the comparison result is greater than or equal to a second preset similarity threshold, the to-be-reviewed banking business image corresponding to the comparison result is deleted to generate a to-be-reviewed scene image set.

[0053] In this embodiment, the server is connected to the bank's image database, which stores a large amount of image data related to banking business. The server obtains several reference non-scene images from the image database. For example, when processing image review related to corporate loan business, the server obtains corporate business licenses, legal person ID cards, financial statements, etc. from the database as reference non-scene images.

[0054] For each reference non-scene image obtained, the server uses a specific feature extraction algorithm (this algorithm is consistent with the algorithm principle used to extract the target feature file for the bank business image to be reviewed, but it is processed separately for the reference non-scene image) to extract the feature file of these images. This feature file contains the feature tensor of the reference non-scene image. This feature tensor is a mathematical representation of the features of the reference non-scene image, which can comprehensively reflect various feature information of the image, such as texture, shape, content, etc. The server calculates the cosine similarity between the target feature tensor of each bank business image to be reviewed and the feature tensor of the reference non-scene image obtained previously. For example, for the feature tensor of a business license of an enterprise (reference non-scene image) and the target feature tensor of the business premises photo provided by the enterprise (bank business image to be reviewed), the server calculates the cosine similarity between them. If the cosine similarity value is high, it means that the two images have high similarity in certain features; if the cosine similarity value is low, it means that the features between the two images are quite different.

[0055] The server checks each calculated comparison result. If the comparison result (i.e., the cosine similarity value) is greater than or equal to the pre-set second preset similarity threshold, this means that the bank business image to be reviewed has a high similarity in features with the reference non-scene image, and this similarity may not conform to normal business logic or the data is abnormal. For example, if the cosine similarity between the business premises photo provided by the enterprise and the business license of the enterprise is too high, it may indicate a risk of data error or malicious tampering. In this case, the server deletes the bank business image to be reviewed corresponding to this comparison result.

[0056] After processing all the images of bank business to be reviewed, the remaining images constitute a set of scene images to be reviewed. For each scene image to be reviewed in this set, the server saves its corresponding target feature file in a first folder, and the name of this first folder is the business number corresponding to the scene image to be reviewed.

[0057] In a possible implementation manner, obtaining a candidate banking business image according to a target Hamming code corresponding to the target feature tensor of the scene image to be reviewed includes: In the locally sensitive hashing high-dimensional feature space, a candidate banking business image is obtained using a locally sensitive hashing algorithm according to the first target Hamming code corresponding to the target feature tensor of the scene image to be reviewed, the first Hamming code of the candidate banking business image is the same as the first target Hamming code, and the second folder where the candidate banking business image is located is different from the first folder.

[0058] In this embodiment, the server operates in the local sensitive hashing high-dimensional feature space for each scene image to be reviewed. First, the server generates a corresponding first target Hamming code based on the target feature tensor of the scene image to be reviewed using a hash function related to the local sensitive hashing algorithm. This Hamming code is a specific coding representation of the target feature tensor, which can represent the features of the image in a relatively compact manner in the high-dimensional feature space and has a certain local sensitivity, that is, similar images have a greater probability of having similar Hamming codes under this coding. The server searches in the bank's image database. For each first target Hamming code generated for the scene image to be reviewed, the server searches for candidate banking business images in the image database that are similar to the scene image to be reviewed, that is, banking business images whose first Hamming codes are the same as the first target Hamming codes of the scene image to be reviewed.

[0059] or, Obtaining M second target Hamming codes of the scene image to be reviewed, wherein the M second target Hamming codes are generated by performing hash processing on the scene image to be reviewed according to M hash algorithms, wherein the hash algorithms include the local sensitive hash algorithm; The images corresponding to the second Hamming codes having at least N identical ones among the M second target Hamming codes in the image database are taken as the candidate banking business images, the second folder where the candidate banking business images are located is different from the first folder, and N is less than or equal to M.

[0060] In this embodiment, the server obtains M second target Hamming codes for each scene image to be audited. The M second target Hamming codes here are generated by hashing the scene image to be audited using M hash algorithms. For example, these hash algorithms may include local sensitive hashing algorithms, random distribution hashing algorithms, etc. Exemplarily, M can be taken as 3, and the server uses a hash algorithm based on data distribution characteristics, a hash algorithm based on image structure, and a local sensitive hashing algorithm to hash the scene image to be audited, respectively, to obtain 3 second target Hamming codes. These different hash algorithms encode image features from different angles, and can capture the feature information of the image more comprehensively.

[0061] The server searches the image database for images corresponding to second Hamming codes that are identical to at least N of the M second target Hamming codes of the scene image to be reviewed as candidate banking images. For example, when M = 3 and N = 2, the server searches the image database for images whose Hamming codes are identical to at least 2 of the 3 second target Hamming codes of the scene image to be reviewed. If an image meets this condition, it is determined to be a candidate banking image. Similarly, the second folder where these candidate banking images are located is different from the first folder to distinguish image collections from different sources. This method based on multi-hash algorithm and multi-Hamming code matching can improve the accuracy and comprehensiveness of obtaining candidate banking images and avoid missing potentially similar images due to the limitations of a single hash algorithm.

[0062] In a possible implementation manner, the calculating the similarity between the feature tensor of each of the candidate banking service images and the feature tensor of the to-be-audited scene image to generate a similarity detection result includes: According to the image structural features of the candidate banking service image, the target feature tensor is decomposed into at least two target feature tensor substructures, and the feature tensor of the to-be-audited scene image is decomposed into at least two feature tensor substructures; Respectively calculating the cosine similarity between the target feature tensor substructure and the feature tensor substructure belonging to the same image structure feature, and obtaining the similarity value between each target feature tensor substructure and each feature tensor substructure corresponding to each image structure feature; The similarity detection result is generated according to a preset weight, the number of the image structural features, and each of the similarity values, wherein the preset weight is related to the importance of the image structural features.

[0063] In this embodiment, after the server obtains the candidate banking business images, it decomposes their feature tensors according to the image structural features of each of these images. For example, when processing real estate-related images in housing loan business, if the candidate banking business image is a photo of the exterior of a real estate, the server will decompose the target feature tensor of the candidate banking business image into at least two target feature tensor substructures based on the image structural features of the real estate structure (such as the main body of the house, doors and windows, roof, etc.) and the surrounding environment (such as gardens, roads, etc.). Each substructure corresponds to a specific part or a specific feature combination in the image, so that the similarity between different parts of the image can be analyzed more carefully. Similarly, the server also decomposes the feature tensor of the scene image to be reviewed according to the above-mentioned image structural features. For each pair of candidate banking business images and scene images to be reviewed, the server calculates the cosine similarity of the target feature tensor substructure and the feature tensor substructure belonging to the same image structural features.

[0064] The server generates a similarity detection result by fusing and calculating the preset weights related to the importance of the image structural features, the number of the image structural features, and the similarity values ​​calculated previously. In this way, the similarity detection result obtained by fusing different substructures comprehensively considers the similarity between different structural features of the image, and can more comprehensively and accurately reflect the overall similarity between the candidate banking business image and the scene image to be reviewed.

[0065] In a possible implementation, the method further includes: Periodically obtaining feedback results of the risky business image, wherein the feedback results represent the correctness of the risk judgment of the risky business image; Determining a threshold adjustment rule for the first preset similarity threshold according to the accuracy rate of the feedback results of the risky business image within a period; The first preset similarity threshold is updated according to the threshold adjustment rule.

[0066] In this embodiment, the server can obtain feedback results of risky business images from the bank's internal audit system or relevant business personnel according to a preset period (such as weekly, monthly, etc.). These risky business images are images that were previously determined to be risky business images and output according to similarity detection results during the bank business image detection process, including first candidate bank business images whose similarity to the scene image to be audited is greater than a first preset similarity threshold.

[0067] For example, within a one-month period, for images related to home mortgages that were previously identified as risky business images, the server will obtain feedback from professionals responsible for reviewing these images on the correctness of the risk judgment of these images. These feedback results may be recorded in a specific data format, such as a list containing image identifiers (such as image names or unique numbers) and whether the corresponding risk judgment is correct (such as "correct" or "incorrect").

[0068] After obtaining the feedback results of the risk business images within the period, the server calculates the accuracy of these feedback results. Based on the calculated accuracy, the server determines the threshold adjustment rule of the first preset similarity threshold. If the accuracy is high (for example, greater than a certain set high accuracy threshold, such as 0.85), it means that the current first preset similarity threshold setting is relatively reasonable and may not need to be adjusted significantly. However, if the accuracy is low (for example, less than 0.7), it may mean that there is a problem with the current threshold setting and it needs to be adjusted.

[0069] The server updates the first preset similarity threshold according to the threshold adjustment rule determined according to actual needs. The updated first preset similarity threshold will be applied to the similarity comparison step in the next cycle of banking business image detection. In this way, by continuously adjusting the threshold according to the feedback results of risky business images, the server can improve the accuracy and effectiveness of the banking business image detection method and better adapt to the changes in image data in banking business and the risk assessment needs in different business scenarios.

[0070] In the above embodiments, the deep learning-based banking imaging material detection system for executing the above method embodiments has at least one processor, a control module (chip set) coupled to at least one of the (at least one) processors, a memory coupled to the control module, a non-volatile memory (NVM) / storage device coupled to the control module, at least one input / output device coupled to the control module, and a network interface coupled to the control module.

[0071] The processor may include at least one single-core or multi-core processor, and the processor may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). For some alternative implementations, the banking business image material detection system based on deep learning can be used as the banking business image material detection system device based on deep learning such as the gateway described in the embodiments of the present application.

[0072] For some alternative embodiments, a deep learning-based banking imaging material detection system may include at least one computer-readable medium (e.g., a memory or NVM / storage device) having instructions and at least one processor integrated with the at least one computer-readable medium and configured to execute the instructions to implement a module to perform the actions described in the present disclosure.

[0073] For one embodiment, the control module may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) and / or any suitable device or component in communication with the control module.

[0074] The control module may include a memory controller module to provide an interface to the memory. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0075] The memory may be used, for example, to load and store data and / or instructions for a deep learning-based banking imaging material detection system. For one embodiment, the memory may include any suitable volatile memory, such as a suitable DRAM.

[0076] For one embodiment, the control module may include at least one input / output controller to provide an interface to the NVM / storage device and (at least one) input / output device.

[0077] For example, the NVM / storage device may be used to store data and / or instructions. The NVM / storage device may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (at least one) non-volatile storage device (e.g., at least one hard disk drive (HDD), at least one compact disk (CD) drive, and / or at least one digital versatile disk (DVD) drive).

[0078] The NVM / storage device may include storage resources that are physically part of the device on which the deep learning-based banking imaging material detection system is installed, or it may be accessible to the device without being part of the device. For example, the NVM / storage device may be accessed via (at least one) input / output device based on a network.

[0079] (At least one) input / output device may provide an interface for the deep learning-based banking image material detection system to communicate with any other appropriate device, and the input / output device may include a communication component, a phonetic component, a sensor component, etc. A network interface may provide an interface for the deep learning-based banking image material detection system to communicate based on at least one network, and the deep learning-based banking image material detection system may wirelessly communicate with at least one component of a wireless network based on any standard and / or protocol of at least one wireless network standard and / or protocol, such as accessing a wireless network based on a communication standard.

[0080] For one embodiment, at least one of the (at least one) processor may be loaded together with the logic of at least one controller of the control module (e.g., a memory controller module). For one embodiment, at least one of the (at least one) processor may be loaded together with the logic of at least one controller of the control module to form a system level load. For one embodiment, at least one of the (at least one) processor may be integrated on the same die with the logic of at least one controller of the control module. For one embodiment, at least one of the (at least one) processor may be integrated on the same die with the logic of at least one controller of the control module to form a system on chip (SoC).

[0081] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0082] An embodiment of the present invention discloses a computer-readable storage medium that stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the banking business image material detection method based on deep learning described in the aforementioned embodiment.

[0083] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the deep learning-based banking business image material detection method described in the aforementioned embodiment.

[0084] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0085] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to be computer-readable with or store data.

[0086] Finally, it should be noted that what is disclosed above is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A banking business image material detection method based on deep learning, characterized in that: include: Acquire a bank business image to be reviewed, wherein the bank business image to be reviewed includes a scene image and a non-scene image; Extracting a target feature file from the pending bank business image using a comparison text image pre-training model, wherein the target feature file includes a target feature tensor, the name of the target feature file is the image name of the pending bank business image, the key name of the target feature file is the absolute path of the pending bank business image, and the key value of the target feature file is the target feature tensor; Perform image cleaning on the bank business image to be reviewed according to the feature tensor of the reference non-scene image in the image database and the target feature tensor to generate a set of scene images to be reviewed, wherein the target feature files corresponding to the scene images to be reviewed in the set of scene images to be reviewed are saved in a first folder, and the folder name of the first folder is the business number corresponding to the scene images to be reviewed; Acquire a candidate banking business image according to the target Hamming code corresponding to the target feature tensor of the scene image to be reviewed, wherein the second folder where the candidate banking business image is located is different from the first folder; Calculating the similarity between the feature tensor of each of the candidate banking business images and the feature tensor of the scene image to be reviewed, and generating a similarity detection result; A first candidate banking business image having a similarity greater than a first preset similarity threshold value in the similarity detection result and the scene image to be reviewed are outputted, wherein the first candidate banking business image is a risky business image.

2. The banking business image material detection method based on deep learning according to claim 1 is characterized in that: The method of extracting a target feature file from the to-be-audited banking business image using a contrasting text image pre-training model includes: Using the visual transformer encoder in the contrast text image pre-training model, the size of the to-be-audited banking image is adjusted to a target size, wherein the target size is a multidimensional array including image height, image width, and image channel number; Performing image segmentation on the to-be-audited banking business image adjusted to the target size to obtain a target block set of the to-be-audited banking business image, wherein the target block set includes a classification mark block, and the classification mark block is used to characterize the global features of the to-be-audited banking business image; Flattening the target blocks in the target block set and mapping them to the target embedding space through linear projection; Adding a position code to each of the target blocks in the target embedding space, wherein the position code is used to characterize the spatial information of the bank business image to be reviewed; Performing feature extraction on the target block with the position code added thereto through a multi-head self-attention mechanism to obtain a feature representation to be aggregated of the target block with the position code added thereto; Performing feature aggregation on the feature representation to be aggregated using a multi-layer perceptron algorithm to obtain the target feature tensor of the bank business image to be reviewed; The target feature file is generated according to the target feature tensor of the banking business image to be reviewed.

3. The banking business image material detection method based on deep learning according to claim 2 is characterized in that: The step of extracting features from the target block to which the position code is added by a multi-head self-attention mechanism to obtain a feature representation to be aggregated of the target block to which the position code is added includes: According to the target block to which the position code is added and a preset weight matrix, a query vector, a key vector, and a value vector of the target block under each header are obtained; Calculate and obtain the attention scores of the target block and other target blocks according to the query vector and the key vector; Normalizing the attention score of the target block using a softmax function to generate a normalized attention weight; Generate candidate feature representations under each head according to the attention weight and the value vector; The candidate feature representations under each head are concatenated to generate the feature representation to be aggregated.

4. The banking business image material detection method based on deep learning according to claim 1 is characterized in that: The step of cleaning the to-be-audited banking business image according to the feature tensor of the reference non-scene image in the image database and the target feature tensor to generate a set of to-be-audited scene images includes: Acquire a plurality of the reference non-scene images in the image database, and extract feature files of the reference non-scene images, wherein the feature files of the reference non-scene images include feature tensors of the reference non-scene images; Calculating the cosine similarity of the feature tensor of the reference non-scene image and the target feature tensor of the to-be-audited banking business image to obtain a comparison result; If the comparison result is greater than or equal to a second preset similarity threshold, the to-be-reviewed banking business image corresponding to the comparison result is deleted to generate a to-be-reviewed scene image set.

5. The banking business image material detection method based on deep learning according to any one of claims 1 to 4, characterized in that: The step of obtaining a candidate banking business image according to the target Hamming code corresponding to the target feature tensor of the scene image to be reviewed includes: In a local sensitive hash high-dimensional feature space, according to a first target Hamming code corresponding to the target feature tensor of the scene image to be reviewed, a candidate banking business image is obtained using a local sensitive hash algorithm, wherein the first Hamming code of the candidate banking business image is the same as the first target Hamming code, and the second folder where the candidate banking business image is located is different from the first folder; or, Obtaining M second target Hamming codes of the scene image to be reviewed, wherein the M second target Hamming codes are generated by performing hash processing on the scene image to be reviewed according to M hash algorithms, wherein the hash algorithms include the local sensitive hash algorithm; The images corresponding to the second Hamming codes having at least N identical ones among the M second target Hamming codes in the image database are taken as the candidate banking business images, the second folder where the candidate banking business images are located is different from the first folder, and N is less than or equal to M.

6. The banking business image material detection method based on deep learning according to any one of claims 1 to 4, characterized in that: The calculating the similarity between the feature tensor of each candidate banking business image and the feature tensor of the scene image to be reviewed, and generating a similarity detection result, includes: According to the image structural features of the candidate banking service image, the target feature tensor is decomposed into at least two target feature tensor substructures, and the feature tensor of the to-be-audited scene image is decomposed into at least two feature tensor substructures; Respectively calculating the cosine similarity between the target feature tensor substructure and the feature tensor substructure belonging to the same image structure feature, and obtaining the similarity value between each target feature tensor substructure and each feature tensor substructure corresponding to each image structure feature; The similarity detection result is generated according to a preset weight, the number of the image structural features, and each of the similarity values, wherein the preset weight is related to the importance of the image structural features.

7. The banking business image material detection method based on deep learning according to any one of claims 1 to 4, characterized in that: The method further comprises: Periodically obtaining feedback results of the risky business image, wherein the feedback results represent the correctness of the risk judgment of the risky business image; Determining a threshold adjustment rule for the first preset similarity threshold according to the accuracy rate of the feedback results of the risky business image within a period; The first preset similarity threshold is updated according to the threshold adjustment rule.

8. A banking business image material detection system based on deep learning, characterized in that: It includes a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by a computer, the deep learning-based banking business image detection method described in any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, which, when executed by a computer, implement the deep learning-based banking business image material detection method described in any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images