Method for identifying specific background picture based on AI large model
By combining SAM2-UNet and SigLIP 2 models, high-precision segmentation and feature extraction of intermediary office backgrounds were achieved, solving the problem that traditional anti-fraud systems cannot identify batch applications under the same background, and improving the identification accuracy and efficiency of the anti-fraud system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-22
- Publication Date
- 2026-03-24
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional anti-fraud systems cannot identify batch application behavior of different users under the same background. Background feature extraction is inaccurate, and traditional CNN feature extraction models perform poorly in fine-grained background similarity tasks, making it difficult to capture common features in intermediary office scenarios.
The SAM2-UNet model is used for person and background segmentation. The SigLIP 2 image encoder is used to extract the background embedding, and the multilayer perceptron (MLP) is used for supervised training to output risk scores and mediator labels. The segmentation results are processed by combining L2 normalization and morphological operations to construct an efficient background recognition method.
It achieves high-precision identification and risk scoring of intermediary office backgrounds, supports real-time risk control, and can identify and filter suspected intermediary groups, thus improving the accuracy and efficiency of the anti-fraud system.
Smart Images
Figure CN121725291A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology with specific backgrounds, and more particularly to a method for recognizing images with specific backgrounds based on a large AI model. Background Technology
[0002] With the popularization of mobile internet financial services, the phenomenon of black market intermediaries using false identity information, forged documents, and group application on behalf of others to apply for financial services in bulk is becoming increasingly serious. Traditional anti-fraud systems mainly rely on behavioral data, device fingerprints, or OCR information for risk identification, but they are difficult to model the "environmental consistency" feature, that is, they cannot identify the behavior of different users applying in bulk in the same background (such as the same intermediary office).
[0003] In recent years, the development of large AI models and image segmentation technology has provided new approaches for extracting complex scene information from images. Meta's open-source Segment Anything Model 2 (SAM2) has zero-shot segmentation capabilities, enabling high-precision semantic region segmentation of images; meanwhile, the multimodal model SigLIP 2 can extract high-dimensional image semantic vectors, achieving efficient feature representation. These technologies provide a feasible foundation for "background feature recognition".
[0004] Current anti-fraud image detection mainly focuses on "face liveness detection" or "image tampering detection," lacking modeling of background environment features, which leads to the following problems: 1. Inability to distinguish whether different applicants are in the same scene: Traditional models ignore the background and only judge whether the face is real, which makes it impossible to identify photos taken in batches by intermediary organizations in the same office.
[0005] 2. Inaccurate background feature extraction: Traditional segmentation algorithms based on threshold or saliency detection are difficult to remove human figures in complex environments.
[0006] 3. Insufficient feature representation ability: Traditional CNN feature extraction models perform poorly in fine-grained background similarity tasks and have difficulty capturing common features in intermediary office scenarios. Summary of the Invention
[0007] The purpose of this invention is to address the problem in the existing technology of not being able to identify batch applications made by different users under the same background, and to propose a method for recognizing specific background images based on a large AI model.
[0008] The technical solution of this invention: a method for recognizing specific background images based on a large AI model, comprising the following steps: S1. Receive the liveness detection image uploaded by the user. The input image includes the person and their environmental background. S2. The SAM2-UNet model is used for person and background segmentation. The SAM2 part is responsible for feature encoding, and the U-Net structure is responsible for progressive upsampling to achieve pixel-level fine segmentation. The output consists of two parts: foreground (person area) and background (environment area). S3. Use the SigLIP 2 image encoder to extract the background embedding; S4. Perform L2 normalization on the extracted background embedding to eliminate feature scale differences and ensure the stability of similarity calculation. S5. Supervised training is performed using a multilayer perceptron (MLP). The input is the background embedding vector, and the output is the risk score and the intermediary label. The classifier learns the feature distribution of "intermediary office environment" through historical labeled samples to predict new samples. S6. Encapsulate the classifier results and output them to the downstream risk control system.
[0009] Preferably, in S1, the liveness detection images need to undergo standardized preprocessing before being uploaded to the model.
[0010] Preferably, in S2, if the mask of the segmented output contains noise and small holes, morphological operations are applied to smooth the edges and fill the small holes.
[0011] Preferably, in S3, the SigLIP 2 model is based on the ViT structure, which can generate multi-scale semantic features and obtain a unified feature vector of the background through average pooling.
[0012] Preferably, the extraction of background embedding using the SigLIP 2 image encoder specifically includes the following steps: S31, Input processing: The pure background image is directly input into the SigLIP 2 image encoder; S32, Feature aggregation: The SigLIP 2 ViT encoder outputs a feature sequence; S33, [CLS] Token features: The vector of [CLS] token is directly used as a global image representation; S34, Average pooling: The features of all image blocks are averaged as another global representation.
[0013] Preferably, in S5, the multilayer perceptron (MLP) structure includes an input layer, a hidden layer, and an output layer; the input layer has the same dimension as the SigLIP 2 feature vector, there are 1-2 hidden layers with decreasing number of neurons in each layer, using the ReLU activation function and Dropout, and the output layer has 2 neurons, corresponding to the "intermediate background" and "normal background", using the Softmax activation function.
[0014] Preferably, in S5, the training data includes positive samples and negative samples. Positive samples are application images of confirmed intermediary fraud cases, and the background feature vectors are obtained after processing in S2-S4. Negative samples are background feature vectors of application images from real, dispersed, normal users.
[0015] Preferably, in S6, for applications with extremely high risk scores, enhanced verification can be triggered in real time or the application can be rejected directly. The background feature vectors of all applications are stored in a vector database. When a suspicious intermediary case is found, all applications with highly similar background features in history can be retrieved in reverse, thereby uncovering the entire fraud gang.
[0016] Compared with existing technologies, the present invention has the following beneficial technical effects: it proposes a specific background image recognition scheme based on an AI large model, which extracts background features and normalizes them by combining advanced image segmentation models and multimodal embedding models, and then completes the recognition and risk scoring of intermediary office backgrounds through a trainable classifier, thereby assisting anti-fraud business in identifying and filtering suspected intermediary groups. Attached Figure Description
[0017] Figure 1 This is a flowchart of one embodiment of the present invention. Detailed Implementation
[0018] Example 1; as Figure 1 As shown, the present invention proposes a method for recognizing specific background images based on a large AI model, comprising the following steps: S1. Receive the liveness detection image uploaded by the user. The input image includes the person and their environmental background. S2. The SAM2-UNet model is used for person and background segmentation. The SAM2 part is responsible for feature encoding, and the U-Net structure is responsible for progressive upsampling to achieve pixel-level fine segmentation, outputting two parts: foreground (person region) and background (environment region). This is a key step in achieving "background stripping," aiming to generate a high-precision binary mask that separates the "person" (foreground) and "environment" (background) in the image at the pixel level. The SAM2-UNet combination is an efficient choice. SAM2, as the encoder, with its powerful Visual Transformer (ViT) backbone network, can understand complex scene semantics and provide rich features for segmentation. The U-Net decoder uses skip connections to fuse features from different levels of SAM2, progressively upsampling to finally output a fine pixel-level segmentation map. This is more suitable for automatic full-image segmentation of fixed categories (person / background) than simply using SAM2's cue-based segmentation mode. S3. Use the SigLIP 2 image encoder to extract the background embedding. The goal of this step is to convert the "pure background image" obtained in the previous step into a high-dimensional, dense semantic vector (embedding). This vector should be able to represent the essential features of the environment, such as the style of the desk, the color of the wall, the texture of the curtains, and the color tone of the lights. SigLIP 2 is an ideal choice. Compared with the traditional CLIP, SigLIP is trained using Sigmoid loss, which performs better in zero-shot classification and image retrieval tasks, and is also more computationally efficient. Its ViT structure can capture the semantic relationships between different regions in the image. It should be noted that the extraction and storage of background features of user images must strictly comply with data security and privacy protection regulations, and data anonymization and encrypted storage must be properly implemented. S4. Perform L2 normalization on the extracted background embedding to eliminate feature scale differences and ensure the stability of similarity calculation. That is, divide the vector by its magnitude to make its magnitude 1. Assuming the extracted feature vector is v, the normalized vector is... The core objectives are twofold: first, to eliminate dimensions and focus on direction. After normalization, the inner product of vectors equals the cosine similarity, which makes subsequent similarity calculations and classifier training more stable. The model focuses more on the "direction" (semantic content) of features in space rather than their absolute "length" (feature strength). Second, to lay the foundation for similarity retrieval. This is a key step in building a "background database" for group identification. When it is necessary to determine whether the background of a new application is similar to a certain intermediary den in the past, it is only necessary to calculate the cosine similarity of the normalized vectors of the two, which is efficient and accurate. S5. Supervised training is performed using a multilayer perceptron (MLP). The input is the background embedding vector, and the output is the risk score (0-1) and the mediator label. The classifier learns the feature distribution of "mediator office environment" through historical labeled samples to predict new samples. This is a supervised learning module whose task is to learn the distribution differences between "mediator office environment" and "normal user environment" in the feature space and make a binary classification judgment. Cold start problem: When there is a lack of labeled data in the early stage, self-supervised or semi-supervised learning can be considered. First, a large amount of unlabeled data is used to let the model learn the diversity of "normal background", and then the anomaly is identified by fine-tuning with small samples. S6. Encapsulate the classifier results and output them to the downstream risk control system in a structured format: { "user_id":"123456789", "application_id":"app_20231208001", "background_risk_score": 0.87, "is_intermediary_background":true, "segmentation_mask_url":"https: / / ... / mask_123.png", / / Optional, for auditing "feature_vector":[0.12,-0.05,...,0.33] / / Optional, used to store in the background feature library }
[0019] Example 2: This invention proposes a method for recognizing specific background images based on a large AI model. Compared to Example 1, this example details S1 and S2. In S1, liveness detection images need to undergo standardized preprocessing before being uploaded to the model. This includes: Size normalization: Adjusting all input images to a fixed resolution (e.g., 512x512 or 224x224) to meet the input requirements of subsequent SAM2 and SigLIP 2 models. Note that the original aspect ratio should be maintained during size adjustment to avoid distorting background information; Color normalization: Normalizing pixel values (e.g., from [0,255] to [0,1] or using the mean and standard deviation of ImageNet); Metadata association: Each image must be strongly associated with unique user ID, application time, device fingerprint, IP address, and other business metadata. This allows for fusion and decision-making with other dimensions of data (behavior, device, network) when the background recognition module outputs a risk signal; Quality filtering: A simple image quality detector is used to filter out extremely blurry, overly dark, overly bright, or severely damaged images to avoid invalid input interfering with subsequent complex models.
[0020] In S2, if the output mask after segmentation contains noise and small holes, morphological operations (such as closing operations) are applied to smooth the edges and fill the holes, ensuring that the "person" region in the mask is connected. Connectivity analysis can be used to retain only the largest connected region as the "person," and the rest as the "background," to eliminate interfering objects that are missegmented (such as a water glass on a table that might be mistaken for part of the foreground). Application of the segmentation results: Using the generated background mask, a bitwise AND operation is performed with the original image to obtain a pure background image, in which the person region has been set to a solid color (such as black or white). This "pure" background image will be used as the input for the next module.
[0021] Example 3; This invention proposes a method for recognizing specific background images based on a large AI model. Compared to Example 1, this example details step S3. The SigLIP 2 model is based on the ViT structure and can generate multi-scale semantic features. It obtains a unified feature vector of the background through average pooling (or other aggregation methods).
[0022] Extracting background embedding using the SigLIP 2 image encoder involves the following steps: S31, Input Processing: Directly input the pure background image into the SigLIP 2 image encoder. Note that if the human figure area in the background image is filled with a specific color, it is necessary to evaluate whether this color will interfere with the model. Generally, neutral colors (black / gray / white) are a safe choice. S32, Feature Aggregation: The SigLIP 2 ViT encoder will output a feature sequence (e.g., [197,768], which contains the features of a [CLS] token and the features of 196 image patches). S33, [CLS] Token Features: Directly using the vector of the [CLS] token as the global image representation is the simplest and most efficient method. It has already aggregated global information and is designed specifically for aggregating global semantics. Average pooling can be used as a comparative experiment for subsequent optimization. S34, Average Pooling: Perform average pooling on the features of all image patches as another global representation.
[0023] Furthermore, the Multilayer Perceptron (MLP) structure includes an input layer, hidden layers, and an output layer. The input layer has the same dimension as the SigLIP2 feature vector (e.g., 768), with 1-2 hidden layers, each with a decreasing number of neurons (e.g., 512->256), using the ReLU activation function and Dropout (to prevent overfitting). The output layer has 2 neurons, corresponding to "intermediate background" and "normal background," and uses the Softmax activation function; or 1 neuron, using the Sigmoid activation function, directly outputting a risk probability of 0-1.
[0024] Example 4: This invention proposes a method for recognizing specific background images based on a large AI model. Compared to Example 1, this example details steps S5 and S6. In S5, the training data includes positive and negative samples. Positive samples are application images from confirmed cases of intermediary fraud, processed by S2-S4 to obtain background feature vectors. Negative samples are background feature vectors from application images of real, dispersed, normal users. The diversity of negative samples is crucial, needing to cover various scenarios such as homes, outdoors, cafes, and real offices to prevent the model from misclassifying "all offices" as intermediaries. Additionally, cross-entropy loss is used as the loss function. Two training techniques are employed: first, hard example mining: during training, focus on samples misclassified by the model (hard examples) to strengthen learning; second, threshold determination: the model outputs a probability value, and an optimal decision threshold (e.g., 0.7) needs to be determined on the validation set using precision-recall (PRC) or ROC curves. Values above this threshold are considered "intermediary backgrounds," reflecting the business's risk preference.
[0025] In S6, for applications with extremely high risk scores, enhanced verification or direct rejection can be triggered in real time. The background feature vectors of all applications are stored in a vector database. When a suspicious intermediary case is discovered, all applications with highly similar background features (cosine similarity > 0.9) in the history can be retrieved in reverse, thereby uncovering the entire fraud gang. This is one of the greatest values of this solution. In an optional embodiment, rule engine fusion is performed: "intermediary background label" is used as a strong feature and input into a more complex risk control model or rule engine along with traditional rules such as device clustering, IP clustering, and application time patterns for comprehensive decision-making. System iteration is performed: cases misjudged by the model (especially false positives of normal user appeals) are fed back, labeled, and added to the training set to continuously optimize the model.
[0026] In summary, this invention, when used, combines the hierarchical Transformer encoder of SAM2 with the U-Net decoding structure to achieve high-precision person-background segmentation, effectively eliminating interference from people and retaining only the background area, ensuring the consistency of subsequent features; it uses the state-of-the-art multimodal model SigLIP 2 to extract semantic background features, possessing cross-language and cross-scene generalization capabilities; it combines image embedding with traditional machine learning models (XGBoost / MLP) to build a low-cost, high-performance classification system, outputting risk scores and labels, supporting batch judgment and real-time risk control; it is scalable and applicable to different business scenarios, such as background classification in offices, homes, and outdoors.
[0027] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.
Claims
1. A method for recognizing specific background images based on a large AI model, characterized in that, Includes the following steps: S1. Receive the liveness detection image uploaded by the user. The input image includes the person and their environmental background. S2. The SAM2-UNet model is used for person and background segmentation. The SAM2 part is responsible for feature encoding, and the U-Net structure is responsible for progressive upsampling to achieve pixel-level fine segmentation. The output consists of two parts: foreground (person area) and background (environment area). S3. Use the SigLIP 2 image encoder to extract the background embedding; S4. Perform L2 normalization on the extracted background embedding to eliminate feature scale differences and ensure the stability of similarity calculation. S5. Supervised training is performed using a multilayer perceptron (MLP). The input is the background embedding vector, and the output is the risk score and the intermediary label. The classifier learns the feature distribution of "intermediary office environment" through historical labeled samples to predict new samples. S6. Encapsulate the classifier results and output them to the downstream risk control system.
2. The method for recognizing specific background images based on a large AI model according to claim 1, characterized in that, In S1, liveness detection images need to undergo standardized preprocessing before being uploaded to the model.
3. The method for recognizing specific background images based on a large AI model according to claim 1, characterized in that, In S2, if the output mask after segmentation contains noise and small holes, morphological operations are applied to smooth the edges and fill the holes.
4. The method for recognizing specific background images based on a large AI model according to claim 1, characterized in that, In S3, the SigLIP 2 model is based on the ViT structure and can generate multi-scale semantic features. It obtains a unified feature vector of the background through average pooling.
5. The method for recognizing specific background images based on a large AI model according to claim 4, characterized in that, Extracting background embedding using the SigLIP 2 image encoder The process includes the following steps: S31, Input processing: The pure background image is directly input into the SigLIP 2 image encoder; S32, Feature aggregation: The SigLIP 2 ViT encoder outputs a feature sequence; S33, [CLS] Token features: The vector of [CLS] token is directly used as a global image representation; S34, Average pooling: The features of all image blocks are averaged as another global representation.
6. The method for recognizing specific background images based on a large AI model according to claim 1, characterized in that, In S5, the multilayer perceptron (MLP) structure includes an input layer, hidden layers, and an output layer. The input layer has the same dimension as the SigLIP 2 feature vector. There are 1-2 hidden layers with decreasing number of neurons in each layer, using the ReLU activation function and Dropout. The output layer has 2 neurons, corresponding to the "intermediate background" and "normal background", and uses the Softmax activation function.
7. The method for recognizing specific background images based on a large AI model according to claim 1, characterized in that, In S5, the training data includes positive samples and negative samples. Positive samples are application images from confirmed cases of intermediary fraud, and the background feature vectors are obtained after processing by S2-S4. Negative samples are background feature vectors of application images from real, dispersed, normal users.
8. The method for recognizing specific background images based on a large AI model according to claim 1, characterized in that, In S6, for applications with extremely high risk scores, enhanced verification can be triggered in real time or the application can be rejected directly. The background feature vectors of all applications are stored in a vector database. When a suspicious intermediary case is discovered, all applications with highly similar background features in history can be retrieved in reverse, thereby uncovering the entire fraud ring.