Target sample identification method and device, equipment and storage medium

Abstract

The invention relates to the technical field of image processing, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a target sample identification method, device and equipment and a computer readable storage medium, the method comprises the following steps: generating a global feature vector according to scene category information and space structure information in each sample image; calculating the similarity between the global feature vectors of the target image and the other images, selecting the images with the similarity greater than a first threshold value as similar images, and converging all the similar images to generate a first candidate image group; calculating a background score according to the edge density feature and the texture feature in the background region of each sample image, and deleting the images with the background scores smaller than a second threshold value to form a second candidate image group; and selecting a sample image pair of which the number of matched key points is less than a third threshold and the similarity is greater than a fourth threshold from the second candidate image group as a target sample. According to the invention, the accuracy of identifying the target sample is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for target sample recognition. Background Technology

[0002] With the deep penetration of artificial intelligence technology into vertical industries, financial risk control and medical image analysis have become two key areas relying on visual similarity judgment. In the microfinance scenario, fraudulent organizations manipulate applicants to take qualification photos at the same fake location at different times, creating related samples scattered across batch review flows to evade system detection. In the medical image-assisted diagnosis scenario, the identification of difficult lesions relies on mining target samples (difficult case samples) with highly similar imaging features but different pathological details from massive historical cases to improve the ability of deep learning models to distinguish subtle signs of early lesions.

[0003] The core problem faced by these two key areas is how to accurately obtain potentially similar samples distributed across time and space from a full dataset of hundreds of thousands or even millions of data points, and effectively distinguish between "effective similarity with discriminative value" and "noise similarity without information content," so as to construct a high-quality target sample set to support the training of deep learning models.

[0004] Existing methods are generally limited to comparing local features of images within an image set, failing to construct a global sample association network that spans time, space, and data acquisition sources, resulting in the systematic omission of samples across batches.

[0005] Meanwhile, existing technologies rely solely on a single similarity threshold for screening, failing to differentiate the effective information content of images. This results in a large number of low-complexity images, such as white walls and solid-color backgrounds in the financial field, as well as images without lesions, such as normal tissue areas and blank films in the medical field, being included in the candidate set due to background similarity. This can easily lead to ineffective computational overhead in the subsequent fine screening stage, and also introduces intra-domain noise interference, diluting the deep learning model's ability to learn real fraud features or lesion evolution features, and reducing the signal-to-noise ratio of the target sample set.

[0006] In summary, existing technologies that employ local feature comparison and similarity threshold filtering methods are prone to low accuracy in selecting target samples from batch image sets. Therefore, there is an urgent need for a method that can improve target sample recognition. Summary of the Invention

[0007] In view of the above, it is necessary to provide a target sample identification method, the purpose of which is to avoid the problem that the existing technology uses local feature comparison and similarity threshold screening, which easily leads to low accuracy in screening target samples from batch image sets.

[0008] Firstly, a target sample identification method is provided, including: Receive a request to identify target samples in a set of sample images, extract scene category information and spatial structure information from each sample image in the set of sample images, and generate a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information; Each sample image is selected sequentially from the sample image set as the target image. The similarity between the target image and the global feature vectors of the remaining sample images in the sample image set is calculated. One or more sample images with a similarity greater than a first threshold are selected as the first candidate image group corresponding to the target image, until the first candidate image group corresponding to each sample image is completed. Identify the background region of each sample image in each first candidate image group, extract the edge density features and texture features of the background region, calculate the background score of the background region belonging to a solid color background based on the edge density features and texture features, and delete sample images with background scores less than a second threshold from each first candidate image group to form a second candidate image group; For each pair of sample images in the second candidate image group, keypoint detection and matching are performed. The number of matching keypoints between each pair of sample images is calculated, and the similarity between the global feature vectors of each pair of sample images is calculated. Sample image pairs with a number of matching keypoints less than the third threshold and a similarity greater than the fourth threshold are selected as target samples.

[0009] Secondly, a target sample identification device is provided, comprising: The receiving module is used to receive a request for target sample recognition of a sample image set, extract scene category information and spatial structure information from each sample image in the sample image set, and generate a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information. The retrieval module is used to sequentially select each sample image from the sample image set as the target image, calculate the similarity between the target image and the global feature vectors of the remaining sample images in the sample image set, and select one or more sample images with a similarity greater than a first threshold as the first candidate image group corresponding to the target image, until the first candidate image group corresponding to each sample image is completed. The deletion module is used to identify the background region of each sample image in each first candidate image group, extract the edge density features and texture features in the background region, calculate the background score of the background region belonging to a solid color background based on the edge density features and texture features, and delete sample images with background scores less than a second threshold from each first candidate image group to form a second candidate image group. The selection module is used to perform keypoint detection and matching on each pair of sample images in each second candidate image group, calculate the number of matching keypoints between each pair of sample images, calculate the similarity between the global feature vectors of each pair of sample images, and select sample image pairs with a number of matching keypoints less than the third threshold and a similarity greater than the fourth threshold as target samples.

[0010] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the target sample recognition method described above.

[0011] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the target sample recognition method described above.

[0012] Compared to existing technologies, this invention extracts scene category information and spatial structure information from each sample image, generating a global feature vector corresponding to each sample image. This generated global feature vector can simultaneously characterize both "what" and "how" the objects in the image are arranged, improving the accuracy of similarity judgment between images across batches and time periods. The similarity between the global feature vectors of each sample image and the remaining sample images is calculated to obtain similar images for each sample image. Based on these similar images, a first candidate image group is generated. This overcomes the limitations of existing technologies that are restricted to single-batch or local comparisons, enabling the discovery of potentially related samples scattered across different time periods.

[0013] Images with background scores lower than the second threshold are removed from each first candidate image group to form a second candidate image group. This effectively eliminates noisy samples with no discriminative value, which not only significantly reduces the invalid computational overhead of subsequent keypoint matching, but also improves the signal-to-noise ratio of the second candidate image group, preventing the model from being misled by meaningless background similarity.

[0014] The number of matching keypoints between each pair of sample images is calculated, and the similarity between the global feature vectors of each pair of sample images is also calculated. Sample image pairs with a number of matching keypoints less than a third threshold and a similarity greater than a fourth threshold are selected as target samples. By introducing a keypoint verification mechanism, even if two images are highly similar in global semantics, if their key details (such as the corners of documents, the outlines of lesions, and the brand logos of equipment) cannot be aligned (few matching points), they are judged as high-quality target samples that are "visually similar but substantially different".

[0015] This invention improves the accuracy of target sample identification by accurately filtering target samples from a batch of image sets through a "high global similarity + low local matching" approach. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of an application environment for a target sample identification method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a target sample identification method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a target sample identification device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention.

[0017] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0019] It should be noted that the descriptions involving "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0020] The target sample identification method provided in this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can receive requests for target sample recognition of a sample image set, extract scene category information and spatial structure information from each sample image in the sample image set, generate a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information; sequentially select each sample image from the sample image set as a target image, calculate the similarity between the target image and the global feature vectors of the remaining sample images in the sample image set, and select one or more sample images with a similarity greater than a first threshold as the first candidate image group corresponding to the target image, until the first candidate image group corresponding to each sample image is completed; The background region of each sample image in each first candidate image group is identified, and the edge density features and texture features of the background region are extracted. The background score of the background region belonging to a solid color background is calculated based on the edge density features and texture features. Sample images with background scores less than a second threshold are deleted from each first candidate image group to form a second candidate image group. Keypoint detection and matching are performed on each pair of sample images in each second candidate image group. The number of matching keypoints between each pair of sample images is calculated, and the similarity between the global feature vectors of each pair of sample images is calculated. Sample image pairs with a number of matching keypoints less than a third threshold and a similarity greater than a fourth threshold are selected as target samples.

[0021] This invention targets fields such as finance and healthcare. It extracts scene category and spatial structure information from each sample image, generating a global feature vector for each image. This global feature vector simultaneously characterizes both "what" and "how" the objects in the image are arranged, improving the accuracy of similarity assessment across batches and time periods. The similarity between the global feature vectors of each sample image and the remaining sample images is calculated to obtain similar images for each sample image. A first candidate image group is generated based on these similar images. This overcomes the limitations of existing technologies that are restricted to single-batch or local comparisons, enabling the discovery of potentially related samples scattered across different time periods.

[0022] Images with background scores below a second threshold are removed from each first candidate image group to form a second candidate image group. This effectively eliminates noisy samples with no discriminative value, significantly reducing the computational overhead of subsequent keypoint matching and significantly improving the signal-to-noise ratio of the second candidate image group, preventing the model from being misled by meaningless background similarity. The number of matching keypoints between each pair of sample images is calculated, and the similarity between the global feature vectors of each pair of sample images is also calculated. Sample image pairs with a number of matching keypoints less than a third threshold and a similarity greater than a fourth threshold are selected as target samples. By introducing a keypoint verification mechanism, even if two images are highly semantically similar globally, if their key details (such as document corners, lesion outlines, and equipment brand logos) cannot be aligned (few matching points), they are judged as high-quality target samples that are "visually similar but substantially different." This invention accurately filters target samples from a batch of image sets through a "high global similarity + low local matching" approach, thereby improving the accuracy of target sample recognition.

[0023] Reference Figure 2 The diagram shown is a schematic flowchart of a target sample identification method provided in an embodiment of the present invention. This method is executed by a device.

[0024] In this embodiment, a target sample identification method includes: S1. Receive a request to identify target samples in a set of sample images, extract scene category information and spatial structure information from each sample image in the set of sample images, and generate a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information.

[0025] In this embodiment, upon receiving a request to identify target samples in the sample image set, a global feature extraction operation is performed on each sample image in the sample image set, as follows: The pre-trained visual base model is loaded as the first deep learning model, which includes the DINOv3 model. The first deep learning model has been self-supervised pre-trained on large-scale general image datasets (such as ImageNet-22K, LAION, etc.), and has strong semantic understanding and spatial structure awareness capabilities. It can be used for feature extraction without fine-tuning for specific tasks.

[0026] Image preprocessing is performed on each sample image in the sample image set: each sample image in the sample image set is uniformly adjusted to the size required for model input (e.g., 518×518 pixels) and standardized (e.g., subtracting the mean and dividing by the standard deviation) to adapt to the input requirements of the first deep learning model. The sample image set can come from various business scenarios, such as user-uploaded ID documents, bank card images, contract signing pages, and photos of branch environments in the fintech field; and medical record images, snapshots of treatment rooms or wards, and patient self-taken medication records or wound recovery photos in the healthcare field.

[0027] The preprocessed image is input into the visual backbone network (VisionTransformer, ViT) of the first deep learning model. The visual backbone network captures scene category information and spatial structure information in the image simultaneously during the forward propagation process through a multi-layer self-attention mechanism.

[0028] Scene category information refers to the high-level semantic category corresponding to the overall image (such as "office", "hospital clinic", "residential living room" etc.), which reflects the understanding of the global semantic context. In financial risk control scenarios, it can distinguish between "bank counter" and "private residence" to assist in anti-fraud judgment. In medical review, it can identify "home care environment" and "formal clinic" to assess service compliance.

[0029] Spatial structure information refers to the relative positions, layout relationships, and geometric structures of objects in an image (such as "tables and chairs are arranged in an L-shape", "door is located on the left side of the image", "equipment is concentrated in the lower right corner", etc.).

[0030] Using semantic scene type and spatial relationships between objects as initial features, a high-dimensional dense vector with a fixed dimension (e.g., 768 or 1024) is obtained by performing global average pooling on all initial features. This vector integrates the scene category information and spatial structure information mentioned above, serving as the global feature vector for each sample image.

[0031] Target samples (difficult samples) refer to image pairs that are highly similar semantically or visually, but have substantial differences in key details. These target samples pose a "deceptive" challenge to the model—they look very similar (e.g., consistent background, layout, scene category), but actually belong to different categories (e.g., different people, different lesions, different real medical visits, etc.). If the model cannot distinguish them, it will lead to misjudgment.

[0032] This invention fully utilizes the powerful visual representation capabilities learned by the first deep learning model (DINOv3 model) under unsupervised conditions. It can extract global feature vectors with both semantic discriminativeness and structural sensitivity from massive images without relying on manual annotation, thus laying a high-quality feature foundation for subsequent cross-batch and cross-temporal target sample mining. It is especially suitable for high-risk scenarios such as gang fraud identification in the fintech field (such as submitting different identity materials multiple times at the same address) and detection of fraudulent medical treatment or insurance fraud in the medical and health field (such as repeatedly using the same background photo to impersonate different patients).

[0033] In one embodiment, the step of extracting scene category information and spatial structure information from each sample image in the sample image set, and generating a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information, includes: Each sample image is input into the visual extraction network of a pre-defined first deep learning model; The self-attention mechanism layer of the visual extraction network is used to extract scene category information and spatial structure information from each sample image as initial features. The initial features are compressed using the global pooling layer of the visual extraction network to generate a high-dimensional dense vector, which is then used as the global feature vector for each sample image.

[0034] Each sample image is input into the visual extraction network of the first deep learning model. The visual extraction network refers to the Vision Transformer (ViT) or its variants (such as the architecture used by DINOv3). The visual extraction network processes images using patches as basic units, rather than the sliding convolution of traditional CNNs.

[0035] By using a self-attention layer, associations can be established between any two image patches in each sample image, thereby capturing long-distance dependencies and contextual semantics in each sample image. The self-attention mechanism layer of the visual extraction network extracts scene category information and spatial structure information from each sample image as initial features. Scene category information refers to high-level semantic understanding, such as determining whether an image is a "hospital ward," "bank counter," or "family living room"; spatial structure information refers to low-level structural information, such as "the ID card is located in the upper center of the image" and "the IV stand is arranged vertically with the hospital bed."

[0036] The initial features are compressed using a global pooling layer to achieve feature aggregation from local to global. The output is a high-dimensional dense vector of fixed length (e.g., 768-dimensional), which retains the overall scene semantics ("what kind of scene") and also integrates the internal structural layout ("how things are arranged").

[0037] This invention utilizes a visual extraction network to extract global feature vectors. Through these global feature vectors, it can not only "understand" the scene (semantics) of an image, but also "understand" how the objects within it are arranged (structure). This design is the technical foundation for this invention to achieve accurate target sample mining across batches and across time and space—because only by simultaneously capturing semantics and structure can we effectively distinguish between "truly relevant fraudulent / lesion samples" and "noise samples with only similar backgrounds."

[0038] S2. Select each sample image from the sample image set as the target image in sequence, calculate the similarity between the target image and the global feature vectors of the remaining sample images in the sample image set, and select one or more sample images with a similarity greater than a first threshold as the first candidate image group corresponding to the target image, until the first candidate image group corresponding to each sample image is completed.

[0039] In this embodiment, after obtaining the global feature vector corresponding to each sample image in the sample image set, a coarse screening stage is entered. A preset second deep learning model (such as the Faiss vector retrieval framework) is used to perform nearest neighbor retrieval on each sample image in the sample image set to obtain similar images. Based on these similar images, a preliminary sample set (first candidate image group) with high visual semantic similarity is constructed. Specifically: Each sample image in the sample image set is processed sequentially. (i=1,2,...,N, where N is the total number of images) is used as the current target image.

[0040] Acquire target image Corresponding global feature vector And calculate its relationship with the remaining N. Global feature vector of an image { The similarity between} is calculated, and the resulting similarity value is between [ The values ​​are between 1 and 1. The closer the value is to 1, the more similar the two images are in terms of semantic scene and spatial structure.

[0041] A first threshold is set (e.g., the first threshold is 0.85), which can be determined based on business requirements or optimization of the validation set. Images with a similarity greater than the first threshold are selected from the remaining images to obtain similar images corresponding to the target image. All similar images of the target image are combined to generate the first candidate image group of the target image. Repeat the above process until every sample image in the sample image set has been processed as a target image, and obtain the first candidate image group corresponding to each sample image. The first candidate image group consists of samples with high visual semantic similarity.

[0042] In the fintech field, for example, if a user uploads a loan application photo with a background of "white wall + simple desk," generating a first candidate image group corresponding to each sample image can quickly recall all images with highly similar scene layouts from the historical application database, even if these applications are scattered across different dates and review batches, thus exposing potential group fraud connections. Similarly, in the healthcare field, when a patient uploads a photo of wound recovery, generating a first candidate image group corresponding to each sample image can automatically match other medical images taken in the same home environment in the past, providing preliminary clues for subsequent judgment on whether there is photo reuse, falsification of medical records, or other similar behaviors.

[0043] This invention generates a first candidate image group corresponding to each sample image, achieving efficient coarse screening of massive images. It effectively breaks through the bottleneck of traditional methods being limited to comparison within a single batch, providing a candidate pool with high recall for subsequent refined difficult example mining, while avoiding the computational explosion problem caused by full pairwise comparison.

[0044] S3. Identify the background region of each sample image in each first candidate image group, extract the edge density features and texture features in the background region, calculate the background score of the background region belonging to a solid color background based on the edge density features and texture features, and delete sample images with background scores less than the second threshold from each first candidate image group to form a second candidate image group.

[0045] In this embodiment, a preset third deep learning model (such as the ICNet image complexity evaluation network) is used to identify the background region of each sample image in each first candidate image group. The background region is cropped from the corresponding image according to the background region mask. The edge density features and texture features in the background region are extracted using the third deep learning model. The weighted sum of the edge density features and texture features is then used to obtain the background score of the background region.

[0046] Images with background scores below the second threshold are removed (such as images with image complexity below the second threshold, i.e., white walls, solid color backgrounds, etc., which contain almost no information, and images that are mistakenly included in the second candidate set due to similar backgrounds are removed, thereby reducing the invalid computational overhead in subsequent stages and improving the quality of the target sample set), resulting in a high-quality second candidate image group.

[0047] In the fintech field, obtaining a high-quality second candidate image set can help filter out a large number of loan application images that are mistakenly identified as similar samples simply because of similar backgrounds (such as white walls or solid-color backgrounds), thus improving the purity of the model training dataset. Alternatively, in medical image-assisted diagnosis scenarios, obtaining a high-quality second candidate image set can help exclude images without lesion characteristics, such as normal tissue areas or blank films, allowing deep learning models to focus more on learning subtle signs of early lesions.

[0048] By obtaining a high-quality second candidate image set, this invention not only improves the accuracy of target sample recognition, but also enhances the information content of the constructed target sample set and the model training efficiency.

[0049] In one embodiment, identifying the background region of each sample image in each first candidate image group includes: Feature maps of each sample image in the first candidate image group at different resolutions are extracted respectively; The feature maps are upsampled, and the spatial positions and proportions of the upsampled feature maps are aligned in a preset spatial coordinate system. The aligned feature maps are then stitched together to generate a composite feature representation containing image spatial context information. Based on the composite feature representation, a preset foreground-background semantic segmentation module is used to predict the probability that each pixel in the feature map belongs to the background or the foreground. A binary background region mask is generated based on the probability, and the background region mask is used to identify the background region in each sample image.

[0050] By using the multi-scale convolutional branches of a pre-defined third deep learning model (such as the ICNet image complexity evaluation network), feature maps of each sample image in the first candidate image group are extracted at different resolutions. This step captures the detailed information of the image at different scales in order to better understand the spatial context of the image.

[0051] Feature maps are upsampled, aligned, and then fused to generate a composite feature representation that includes image spatial context information. Upsampling is a process of enlarging the feature map; common methods include interpolation and transposed convolution. The goal is to restore or increase the spatial resolution of the feature map, allowing it to contain more detailed information. Aligning spatial positions and scales ensures that the spatial positions and scales of the upsampled feature maps are consistent within a predefined spatial coordinate system. This is to guarantee that feature maps from different scales or levels accurately reflect the positional relationships of objects or regions in the original image, avoiding errors caused by scale changes, rotation, or other factors.

[0052] Concatenating to generate a composite feature representation: The aligned feature maps are concatenated in a specific way (such as along the channel dimension) to generate a composite feature representation containing richer spatial context information. This composite representation is helpful for subsequent tasks, such as object detection and segmentation, because it can provide a more comprehensive scene understanding. These feature maps are then fused to form a comprehensive composite feature representation.

[0053] Based on the composite feature representation, the probability of each pixel belonging to the background or the foreground is predicted using the foreground-background semantic segmentation module of the third deep learning model. A binary background region mask is generated based on the probability, and the background region mask is used to identify the background region in each sample image.

[0054] The foreground-background semantic segmentation module is used in image processing tasks to distinguish foreground objects from background regions in an image. It performs pixel-level classification to determine whether each pixel belongs to the foreground (i.e., the object or region of interest) or the background (i.e., the region of non-interest). In practical applications, it can help identify and separate important parts of an image, such as distinguishing lesion areas from normal tissue areas in medical image analysis, and potentially differentiating useful information areas from useless background areas in financial risk control materials.

[0055] In one embodiment, extracting the edge density features in the background region includes: The corresponding background region is cropped from the corresponding image based on the background region mask of each sample image; The number of effective edge pixels in the background region is calculated using a preset edge detection algorithm, and the ratio of the number of effective edge pixels to the area of ​​the background region is used as the edge density feature in the background region.

[0056] The corresponding background region is cropped from the background region mask of each sample image, and the number of effective edge pixels in the background region is calculated using a preset edge detection algorithm as the edge density feature of the background region, as follows: The sample image is processed using a background region mask, retaining the parts marked as background in the mask while setting the foreground parts (i.e., non-background parts) to zero or ignoring them, thus obtaining an image region that contains only background information.

[0057] The cropped background region is analyzed using a preset edge detection algorithm (such as the Canny edge detector or the Sobel operator) to identify and label all valid edge pixels within the region. The cropped background region is then input into the selected edge detection algorithm, which outputs a binary edge map. In this edge map, pixels with a value of 1 represent detected valid edge points, while pixels with a value of 0 represent no detected edges.

[0058] After obtaining the edge map, the number of valid edge pixels within the background region is counted: traverse the entire edge map and count the number of pixels with a value of 1. This number reflects the edge density within the background region, i.e., how much detail or structural variation there is in the background region. Background regions with more detail or structural variation typically have a higher number of valid edge pixels. The ratio of the number of valid edge pixels to the area of ​​the background region is the edge density feature of the background region.

[0059] By calculating the edge density features in the background region, the visual complexity of the background region in each sample image can be quantified. This quantification method helps to evaluate the background score in subsequent steps and select high-quality target samples accordingly, thereby improving the quality and effectiveness of the model training set.

[0060] In one embodiment, extracting texture features from the background region includes: Convert the background region from a color image to a grayscale image; Identify pixel pairs that meet preset conditions from the grayscale image; The gray-level co-occurrence matrix of the gray-level image is generated based on the pixel pairs; Extract at least one of contrast, energy, or entropy from the gray-level co-occurrence matrix as a texture feature in the background region.

[0061] In one embodiment, identifying pixel pairs that satisfy preset conditions from the grayscale image includes: Based on the preset conditions, the distance and direction corresponding to the pixels in the grayscale image are set for retrieval; By traversing all pixels in the grayscale image based on the distance and direction, pixel pairs that satisfy the preset conditions are obtained.

[0062] The background area is converted from the original color image to a grayscale image. This is done because grayscale images simplify the calculation process and can effectively preserve the brightness information in the image, which is very important for subsequent texture analysis.

[0063] Determine the pixel pair conditions used to generate the Gray-Level Co-occurrence Matrix (GLCM), which includes setting the distance and orientation between pixels. Setting distance and orientation: Select one or more distances and orientations based on preset conditions. For example, you can select a distance of 1, and the orientation can be one or all of the following four directions: horizontal (0°), vertical (90°), bottom left to top right (-45°), and bottom right to top left (+45°).

[0064] For each pixel, iterate through the grayscale image and find its corresponding pixel pair according to the set distance and direction. If the grayscale value difference between two pixels meets the requirements, the pair of pixels is considered a valid pair. This step needs to be repeated for every pixel in the image.

[0065] Based on all the identified pixel pairs that meet the criteria, a gray-level co-occurrence matrix (GLCM) is constructed. Specifically, a matrix is ​​created where the rows and columns represent different gray levels. Then, for each pair of pixels, GLCM element values ​​corresponding to the gray values ​​of the two pixels are added. The resulting gray-level co-occurrence matrix (GLCM) reflects the co-occurrence frequency of pixel pairs with different gray levels in the same direction and at the same distance.

[0066] A series of texture features, such as contrast, energy, or entropy, are extracted from the generated gray-level co-occurrence matrix. Contrast: Measures the degree of local variation in an image; high contrast implies greater gray-level differences. Energy: Also known as the second moment of the angle, it reflects the uniformity and smoothness of the image texture; higher energy indicates a more uniform image. Entropy: Represents the amount of information or uncertainty in an image; high entropy indicates that the image contains more detail and variation.

[0067] This method precisely extracts quantitative indicators reflecting the texture features of background regions. These texture features help to further distinguish background regions with different visual complexities, thereby improving the accuracy and efficiency of target sample selection. This method is particularly suitable for fields such as financial risk control and medical image analysis to enhance the model's ability to learn true fraud features or lesion evolution features.

[0068] S4. Perform keypoint detection and matching on each pair of sample images in each second candidate image group, calculate the number of matching keypoints between each pair of sample images, and calculate the similarity between the global feature vectors of each pair of sample images. Select sample image pairs with a number of matching keypoints less than the third threshold and a similarity greater than the fourth threshold as target samples.

[0069] In this embodiment, two images are selected sequentially from each second candidate image group as a pair of images. For the selected image pairs, a preset key point detection algorithm (such as SIFT, SURF, or LightGlue) is used to identify the key points in each sample image and their corresponding descriptor information.

[0070] Keypoints are local locations in an image that possess significant visual features and are easily and reliably detected by algorithms. These locations typically correspond to "unique" or "information-rich" regions in the image, such as corners, edge intersections, areas of dramatic texture changes, or repetitive but structurally stable local patterns. Descriptor information is the fundamental information used to compare the similarity of keypoints between different images.

[0071] Based on keypoint and descriptor information, keypoint matching is performed between two images. By comparing the descriptors of keypoints, it is determined which keypoints correspond to each other in the two images. For example, by calculating the distance between descriptor information (such as Euclidean distance), and judging whether the keypoints are matching points according to a preset distance threshold, a successfully matched keypoint pair is found.

[0072] Calculate the number of matching keypoints (keypoint pairs) between each pair of sample images, and calculate the similarity between the global feature vectors of each pair of sample images. Select sample image pairs with a number of matching keypoints less than the third threshold and a similarity greater than the fourth threshold as target samples.

[0073] In one embodiment, the keypoint detection and matching of each pair of sample images in each second candidate image group includes: Select two images from the second candidate image group in sequence as image pairs; Obtain the key points and descriptor information of the key points in each image pair; Based on the key points and the descriptor information, a key point matching operation is performed between each image pair to obtain successfully matched key point pairs.

[0074] This invention combines global visual structural information with local keypoint matching results to accurately identify visually similar target samples that contain subtle differences, thereby improving the training quality and robustness of deep learning models. This is particularly important in the fields of financial risk control and medical image analysis, as it helps improve the model's ability to learn complex patterns.

[0075] In the fintech field, such image pairs may manifest as follows: multiple loan application materials upload office environment photos with highly similar overall layouts (such as "white walls + L-shaped desks + green plants"), resulting in high global semantic similarity; however, the number of key point matches is extremely small, indicating that details such as the brand of desks and chairs, equipment models, and wall decorations are inconsistent. This often suggests that multiple applicants are using the same fake set to commit group fraud.

[0076] In the healthcare field, a similar situation may occur in patients' self-taken wound recovery records: both photos have the same background of "bed by the window + the same medicine bottle", which is highly similar in meaning, but the texture of the lesion area, the edge of the dressing or the skin folds and other key local structures cannot be aligned, and the matching points are sparse, suggesting that one of the photos may be fake or used by a different patient, which poses a risk of insurance fraud.

[0077] This invention identifies image pairs that meet the following criteria: if the number of matching key points is less than a third threshold, it means that although the images look very similar, they have significant differences in detail; and if their global feature vector similarity is greater than a fourth threshold, it indicates that the two have a high degree of consistency in high-level semantics. Image pairs that meet the above conditions are labeled as target samples and added to the final target sample set.

[0078] like Figure 3 The diagram shown is a schematic diagram of a target sample identification device provided in an embodiment of the present invention.

[0079] The target sample identification device 100 of the present invention can be installed in a device. Depending on the functions implemented, the target sample identification device 100 may include a receiving module 110, a retrieval module 120, a deletion module 130, and a selection module 140. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the device processor and perform a fixed function, and which are stored in the device's memory.

[0080] In this embodiment, the functions of each module / unit are as follows: The receiving module 110 is used to receive a request for target sample recognition of a sample image set, extract scene category information and spatial structure information from each sample image in the sample image set, and generate a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information. The retrieval module 120 is used to sequentially select each sample image from the sample image set as the target image, calculate the similarity between the target image and the global feature vectors of the remaining sample images in the sample image set, and select one or more sample images with a similarity greater than a first threshold as the first candidate image group corresponding to the target image, until the first candidate image group corresponding to each sample image is completed. The deletion module 130 is used to identify the background region of each sample image in each first candidate image group, extract the edge density features and texture features in the background region, calculate the background score of the background region belonging to a solid color background based on the edge density features and texture features, and delete sample images with background scores less than a second threshold from each first candidate image group to form a second candidate image group. The selection module 140 is used to perform key point detection and matching on each pair of sample images in each second candidate image group, calculate the number of matching key points between each pair of sample images, calculate the similarity between the global feature vectors of each pair of sample images, and select sample image pairs with a number of matching key points less than the third threshold and a similarity greater than the fourth threshold as target samples.

[0081] In one embodiment, the receiving module 110 is specifically used for: Each sample image is input into the visual extraction network of a pre-defined first deep learning model; The self-attention mechanism layer of the visual extraction network is used to extract scene category information and spatial structure information from each sample image as initial features. The initial features are compressed using the global pooling layer of the visual extraction network to generate a high-dimensional dense vector, which is then used as the global feature vector for each sample image.

[0082] In one embodiment, the deletion module 130 is specifically used for: Feature maps of each sample image in the first candidate image group at different resolutions are extracted respectively; The feature maps are upsampled, and the spatial positions and proportions of the upsampled feature maps are aligned in a preset spatial coordinate system. The aligned feature maps are then stitched together to generate a composite feature representation containing image spatial context information. Based on the composite feature representation, a preset foreground-background semantic segmentation module is used to predict the probability that each pixel in the feature map belongs to the background or the foreground. A binary background region mask is generated based on the probability, and the background region mask is used to identify the background region in each sample image.

[0083] In one embodiment, the deletion module 130 is specifically used for: The corresponding background region is cropped from the corresponding image based on the background region mask of each sample image; The number of effective edge pixels in the background region is calculated using a preset edge detection algorithm, and the ratio of the number of effective edge pixels to the area of ​​the background region is used as the edge density feature in the background region.

[0084] In one embodiment, the deletion module 130 is specifically used for: Convert the background region from a color image to a grayscale image; Identify pixel pairs that meet preset conditions from the grayscale image; The gray-level co-occurrence matrix of the gray-level image is generated based on the pixel pairs; Extract at least one of contrast, energy, or entropy from the gray-level co-occurrence matrix as a texture feature in the background region.

[0085] In one embodiment, the deletion module 130 is specifically used for: Based on the preset conditions, the distance and direction corresponding to the pixels in the grayscale image are set for retrieval; By traversing all pixels in the grayscale image based on the distance and direction, pixel pairs that satisfy the preset conditions are obtained.

[0086] In one embodiment, module 140 is selected and is specifically used for: Select two images from the second candidate image group in sequence as image pairs; Obtain the key points and descriptor information of the key points in each image pair; Based on the key points and the descriptor information, a key point matching operation is performed between each image pair to obtain successfully matched key point pairs.

[0087] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a target sample recognition method on the server side.

[0088] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a target sample recognition method on the client side.

[0089] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Receive a request to identify target samples in a set of sample images, extract scene category information and spatial structure information from each sample image in the set of sample images, and generate a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information; Each sample image is selected sequentially from the sample image set as the target image. The similarity between the target image and the global feature vectors of the remaining sample images in the sample image set is calculated. One or more sample images with a similarity greater than a first threshold are selected as the first candidate image group corresponding to the target image, until the first candidate image group corresponding to each sample image is completed. Identify the background region of each sample image in each first candidate image group, extract the edge density features and texture features of the background region, calculate the background score of the background region belonging to a solid color background based on the edge density features and texture features, and delete sample images with background scores less than a second threshold from each first candidate image group to form a second candidate image group; For each pair of sample images in the second candidate image group, keypoint detection and matching are performed. The number of matching keypoints between each pair of sample images is calculated, and the similarity between the global feature vectors of each pair of sample images is calculated. Sample image pairs with a number of matching keypoints less than the third threshold and a similarity greater than the fourth threshold are selected as target samples.

[0090] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0091] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0093] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; if any software tools or components other than those of our company appear in the embodiments of the application, they are merely for illustrative purposes and do not represent actual use; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for identifying target samples, characterized in that, The method includes: Receive a request to identify target samples in a set of sample images, extract scene category information and spatial structure information from each sample image in the set of sample images, and generate a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information; Each sample image is selected sequentially from the sample image set as the target image. The similarity between the target image and the global feature vectors of the remaining sample images in the sample image set is calculated. One or more sample images with a similarity greater than a first threshold are selected as the first candidate image group corresponding to the target image, until the first candidate image group corresponding to each sample image is completed. Identify the background region of each sample image in each first candidate image group, extract the edge density features and texture features of the background region, calculate the background score of the background region belonging to a solid color background based on the edge density features and texture features, and delete sample images with background scores less than a second threshold from each first candidate image group to form a second candidate image group; For each pair of sample images in the second candidate image group, keypoint detection and matching are performed. The number of matching keypoints between each pair of sample images is calculated, and the similarity between the global feature vectors of each pair of sample images is calculated. Sample image pairs with a number of matching keypoints less than the third threshold and a similarity greater than the fourth threshold are selected as target samples.

2. The target sample identification method as described in claim 1, characterized in that, The step of extracting scene category information and spatial structure information from each sample image in the sample image set, and generating a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information, includes: Each sample image is input into the visual extraction network of a pre-defined first deep learning model; The self-attention mechanism layer of the visual extraction network is used to extract scene category information and spatial structure information from each sample image as initial features. The initial features are compressed using the global pooling layer of the visual extraction network to generate a high-dimensional dense vector, which is then used as the global feature vector for each sample image.

3. The target sample identification method as described in claim 1, characterized in that, The identification of the background region of each sample image in each first candidate image group includes: Feature maps of each sample image in the first candidate image group at different resolutions are extracted respectively; The feature maps are upsampled, and the spatial positions and proportions of the upsampled feature maps are aligned in a preset spatial coordinate system. The aligned feature maps are then stitched together to generate a composite feature representation containing image spatial context information. Based on the composite feature representation, a preset foreground-background semantic segmentation module is used to predict the probability that each pixel in the feature map belongs to the background or the foreground. A binary background region mask is generated based on the probability, and the background region mask is used to identify the background region in each sample image.

4. The target sample identification method as described in claim 1, characterized in that, The extraction of edge density features in the background region includes: The corresponding background region is cropped from the corresponding image based on the background region mask of each sample image; The number of effective edge pixels in the background region is calculated using a preset edge detection algorithm, and the ratio of the number of effective edge pixels to the area of ​​the background region is used as the edge density feature in the background region.

5. The target sample identification method as described in claim 1, characterized in that, The extraction of texture features from the background region includes: Convert the background region from a color image to a grayscale image; Identify pixel pairs that meet preset conditions from the grayscale image; The gray-level co-occurrence matrix of the gray-level image is generated based on the pixel pairs; Extract at least one of contrast, energy, or entropy from the gray-level co-occurrence matrix as a texture feature in the background region.

6. The target sample identification method as described in claim 5, characterized in that, The step of identifying pixel pairs that meet preset conditions from the grayscale image includes: Based on the preset conditions, the distance and direction corresponding to the pixels in the grayscale image are set for retrieval; By traversing all pixels in the grayscale image based on the distance and direction, pixel pairs that satisfy the preset conditions are obtained.

7. The target sample identification method as described in claim 1, characterized in that, The step of performing keypoint detection and matching on each pair of sample images in each second candidate image group includes: Select two images from the second candidate image group in sequence as image pairs; Obtain the key points and descriptor information of the key points in each image pair; Based on the key points and the descriptor information, a key point matching operation is performed between each image pair to obtain successfully matched key point pairs.

8. A target sample identification device, characterized in that, The device includes: The receiving module is used to receive a request for target sample recognition of a sample image set, extract scene category information and spatial structure information from each sample image in the sample image set, and generate a global feature vector corresponding to each sample image based on the scene category information and the spatial structure information. The retrieval module is used to sequentially select each sample image from the sample image set as the target image, calculate the similarity between the target image and the global feature vectors of the remaining sample images in the sample image set, and select one or more sample images with a similarity greater than a first threshold as the first candidate image group corresponding to the target image, until the first candidate image group corresponding to each sample image is completed. The deletion module is used to identify the background region of each sample image in each first candidate image group, extract the edge density features and texture features in the background region, calculate the background score of the background region belonging to a solid color background based on the edge density features and texture features, and delete sample images with background scores less than a second threshold from each first candidate image group to form a second candidate image group. The selection module is used to perform keypoint detection and matching on each pair of sample images in each second candidate image group, calculate the number of matching keypoints between each pair of sample images, calculate the similarity between the global feature vectors of each pair of sample images, and select sample image pairs with a number of matching keypoints less than the third threshold and a similarity greater than the fourth threshold as target samples.

9. A device, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a target sample identification program that can be executed by the at least one processor, the target sample identification program being executed by the at least one processor to enable the at least one processor to perform the target sample identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a target sample identification program, which can be executed by one or more processors to implement the target sample identification method as described in any one of claims 1 to 7.

Images