Picture emotion recognition anti-fraud method and device, server and storage medium

Abstract

The invention relates to the technical field of information processing, and discloses a picture emotion recognition anti-fraud method and device, a server and a storage medium, and the method comprises the following steps: S110: obtaining picture or audio data uploaded by a user; s120, detecting a forgery trace or a tampering behavior of the picture by using an image processing algorithm; s130, verifying the authenticity of the user identity information through a face comparison technology; s140, analyzing an emotional state in a user interaction process, and identifying abnormal emotional fluctuation; s150, synthesizing the image detection result and the emotion analysis result to generate a fraud risk score; and S160, triggering a further verification or alarm mechanism according to the risk score, and freezing the account or notifying an administrator. According to the picture emotion recognition anti-fraud method and device, the server and the storage medium provided by the invention, the fraud risk is automatically evaluated by analyzing the picture and emotion submitted by the user, and the user verification accuracy and efficiency of finance, e-commerce and social platforms are improved.

Description

Technical Field

[0001] The present invention relates to the field of information processing, and in particular to an image emotion recognition anti-fraud method, device, server and storage medium. Background Art

[0002] At present, most Internet financial platforms rely on rule engines and text analysis to carry out anti-fraud identification. However, as fraud methods become more and more complex, fraudsters frequently use forged images, false identity information, and carefully crafted rhetoric. These advanced fraud behaviors often make it difficult for traditional anti-fraud technology to quickly detect and identify them. Therefore, how to use more intelligent technical means, such as image recognition technology and sentiment analysis technology, to improve the accuracy of anti-fraud identification has become an extremely urgent task and demand facing Internet financial platforms.

[0003] To solve the above problems, this application proposes an image emotion recognition anti-fraud method, device, server and storage medium. Summary of the invention

[0004] (I) Purpose of the invention

[0005] In order to solve the technical problems existing in the background technology, the present invention proposes an anti-fraud method, device, server and storage medium for image emotion recognition. The present invention automatically evaluates fraud risks by analyzing images and emotions submitted by users, thereby improving the accuracy and efficiency of user verification in finance, e-commerce and social platforms.

[0006] (II) Technical solution

[0007] To solve the above problems, the present invention provides an anti-fraud method for image emotion recognition, comprising the following steps:

[0008] S110: Acquire the picture or audio data uploaded by the user;

[0009] S120: Detecting forgery or tampering of the image using an image processing algorithm;

[0010] S130: Verify the authenticity of the user's identity information through face comparison technology;

[0011] S140: Analyze the emotional state of the user during the interaction process and identify abnormal emotional fluctuations;

[0012] S150: Generate a fraud risk score by integrating the image detection result and the sentiment analysis result;

[0013] S160: Trigger further verification or alarm mechanisms, account freezing or administrator notification based on risk scores.

[0014] Preferably, in S110, the data includes:

[0015] Photos of the front and back of the user's ID card;

[0016] A live photo of the user holding his ID card;

[0017] Audio data generated when company specialists communicate with users.

[0018] Preferably, in S120, the Paddle framework is used in combination with the CNN model to perform image tampering detection, and the detection steps are as follows:

[0019] S1201, data preparation;

[0020] S1202, data preprocessing;

[0021] S1203, constructing a model;

[0022] S1204, training model;

[0023] S1205, model evaluation;

[0024] S1206: Model deployment.

[0025] Preferably, in S130, the Paddle framework is used in combination with the ArcFace model to perform face comparison, and the comparison steps are as follows:

[0026] S1301, data preparation: import the images filtered in S120, collect face data, and pre-process the face images;

[0027] S1302, load model: use Paddle deep learning framework to load the pre-trained ArcFace model;

[0028] S1303, feature extraction: Use the face detection algorithm Dlib to detect the face position in the image and extract the face area. Input the face image into the ArcFace model. The model will output a feature vector of a fixed length. This vector can be used to represent the features of the face.

[0029] S1304, feature comparison: calculating feature vectors and similarity;

[0030] S1305, determining a threshold: determining a suitable threshold based on experimental data, and when the similarity between two feature vectors exceeds the threshold, it is considered that the two images belong to the same person;

[0031] S1306: Save the face comparison data to the database.

[0032] Preferably, in S140, the emotion2vec speech emotion base model is used to perform emotion analysis, and the emotion analysis steps are as follows:

[0033] S1401, input voice data: import the voice file in S13, and divide the voice file into multiple paragraphs according to voice pauses;

[0034] S1402, analyzing the emotions of multiple paragraphs, and then obtaining the result of each paragraph;

[0035] S1403, using a prediction algorithm to guess the final result: arithmetic sum is performed to calculate the total value of the emotional probability (score) of different paragraphs, and the score and the maximum emotion are taken;

[0036] S1404: Save the sentiment analysis results to a database.

[0037] Preferably, in S150, the data returned from S130 and S140 are comprehensively analyzed to formulate risk scoring logic and determine the risk scoring threshold.

[0038] Preferably, in S160, a DingTalk alarm is used to notify the administrator.

[0039] An image emotion recognition anti-fraud device, comprising:

[0040] User picture and audio information acquisition module, used to obtain the user's ID card picture information, living body picture information and audio information;

[0041] An image detection module, used to detect image forgery and tampering traces;

[0042] Face matching module, used to verify the authenticity of user identity information;

[0043] Emotional analysis module, used to identify the user's emotional state and whether there are risky behaviors;

[0044] Fraud risk assessment module, which formulates risk assessment logic based on the face comparison module and sentiment analysis module;

[0045] The alarm module sets alarm thresholds based on risk assessment logic and notifies administrators in the form of DingTalk alarms.

[0046] A server, comprising:

[0047] At least one CPU;

[0048] At least one GPU;

[0049] The storage device is used to store one or at least one program. When the one or at least one program is executed by the one or at least one processor, the one or at least one processor implements the above-mentioned anti-bombing identification method.

[0050] A computer-readable storage medium stores a computer program, which implements the above-mentioned anti-bombing identification method when executed by a processor.

[0051] The above technical solution of the present invention has the following beneficial technical effects:

[0052] By analyzing the authenticity of the image information submitted by the user, combining sentiment analysis technology to identify the user's psychological state, and combining the results of image and sentiment analysis, a fraud risk assessment is generated. The system is triggered when the user submits information for identity verification, and improves the accuracy and efficiency of anti-fraud identification through automated processing. This method is suitable for user verification and risk control in financial technology, e-commerce and social platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a process of an image emotion recognition anti-fraud method provided in the first embodiment of the present invention;

[0054] Figure 2 A schematic diagram of detailed steps of an anti-fraud method for image emotion recognition provided in Embodiment 2 of the present invention;

[0055] Figure 3 A schematic diagram of the structure of an image emotion recognition anti-fraud device provided in Embodiment 3 of the present invention;

[0056] Figure 4 A schematic diagram of the structure of a server provided in Embodiment 4 of the present invention.

[0057] Attached figure numbers: 301, user picture and audio information acquisition module; 302, image detection module; 303, face comparison module; 304, emotion analysis module; 305, fraud risk assessment module; 306, alarm module; 401, server; 402, external device; 403, CPU; 404, GPU; 405, storage device; 406, cache memory; 407, RAM; 408, storage system; 409, display; 410, I / O interface; 411, program product. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0059] Embodiment 1

[0060] A method for anti-fraud by image emotion recognition includes the following steps:

[0061] S110: Acquire the image or audio data uploaded by the user.

[0062] Specifically, the pictures uploaded by users include pictures of the front and back of their ID cards, live photos of users holding their ID cards, and audio data generated by communication between company specialists and users.

[0063] S120: Detect traces of forgery or tampering of the image using an image processing algorithm.

[0064] Specifically, the Paddle framework is used in combination with the CNN model for image tampering detection. The steps mainly include: data preparation, preparing a data set containing real and tampered images. The real data set downloads real photos of users from the file system, including the front and back of the ID card, and live photos of the user holding the ID card; data preprocessing, image scaling and normalization of the image data; model building, using the Paddle framework to build a CNN model; model training, loading the real data set and the tampered image data set, training the model, and optimizing the model using the Adam optimizer; model evaluation, using the cross entropy loss function to evaluate the model; model deployment: deploy the model to server 401 and provide it for internal use in the form of an API.

[0065] S130. Verify the authenticity of the user's identity information through face comparison technology.

[0066] Specifically, the Paddle framework is used in combination with the ArcFace model for face comparison. The comparison steps include: data preparation, importing pictures filtered by S120, collecting face data, preprocessing face images, and ensuring image quality. Loading the model, using the Paddle deep learning framework to load the pre-trained ArcFace model. Feature extraction, using the face detection algorithm Dlib to detect the face position in the image and extract the face area. Input the face image into the ArcFace model, and the model will output a feature vector of a fixed length, which can be used to represent the features of the face. Feature comparison, calculate the feature vector and similarity. Determine the threshold, determine a suitable threshold based on experimental data, and when the similarity between two feature vectors exceeds this threshold, it is considered that the two images belong to the same person. Save the face comparison data to the database.

[0067] S140: Analyze the emotional state of the user during the interaction process and identify abnormal emotional fluctuations.

[0068] Specifically, the emotion2vec speech emotion base model is used for emotion analysis, and the emotion analysis steps are as follows: input speech data, import the speech file in S130, and divide the speech file into multiple paragraphs according to speech pauses. Analyze the emotions of multiple paragraphs, and then get the results of each paragraph. Use the prediction algorithm to guess the final result, calculate the total value of the emotion probability (score) of different paragraphs by arithmetic summation, and take the score and the maximum emotion. Save the emotion analysis results to the database.

[0069] S150, generating a fraud risk score by integrating the image detection result and the sentiment analysis result.

[0070] Specifically, the risk scoring logic is formulated by comprehensively analyzing the data returned from S130 and S140 to determine the risk scoring threshold.

[0071] S160. Trigger further verification or alarm mechanism, account freezing or administrator notification based on risk score.

[0072] Specifically, the administrator is notified by using a DingTalk alarm.

[0073] Embodiment 2 is a further refinement of the above embodiment 1.

[0074] A method for anti-fraud by image emotion recognition includes the following steps:

[0075] S201. Obtain the image and audio data uploaded by the user from the file system.

[0076] Specifically, the image data includes pictures of the front and back of the user’s ID card, live pictures of the user holding the ID card, and audio data generated when company specialists communicate with customers.

[0077] S202. Build a CNN model to detect image tampering.

[0078] Specifically, the steps include:

[0079] S2021. Data preparation: Prepare a real dataset and a tampered dataset of user photos. The real dataset downloads the user's real photos including the front and back of the ID card and the live photo of the ID card held in hand from the file system and saves them locally. Manually modify the real pictures and save them as tampered datasets, and save them locally.

[0080] S2022. Data preprocessing

[0081] Specifically, image scaling, normalization, and data enhancement are mainly used for data preprocessing.

[0082] Image scaling: Use the image processing function provided by Paddle for scaling. The formula is as follows:

[0083] Assume that the original image size is H×W and the target size is H′×W′:

[0084] I resized = resize(I,H′,W′)

[0085] Normalization: Normalize the image pixel value I to the interval [-1,1], the formula is as follows:

[0086]

[0087] where μ and σ are the mean and standard deviation respectively.

[0088] Optional, data augmentation. Use Paddle's image augmentation capabilities, such as rotation, flipping, scaling, etc., to randomly apply a series of transformations, such as rotating by an angle of θ:

[0089] I augmented =rotate(I,θ)

[0090] S2023. Build the model

[0091] Specifically, use the pre-trained model provided by Paddle as initialization to build a CNN architecture. Convolutional layer, formula:

[0092] F conv =ReLU(Conv2D(I normalized ,W conv )+b conv )

[0093] Among them, Conv2D represents a two-dimensional convolution operation, Wconv and bconv are the weight and bias of the convolution kernel respectively, and ReLU is the activation function. Pooling layer, formula:

[0094] F pool =MaxPool2D(F conv ,kernel_size=2,stride=2)

[0095] Here MaxPool2D represents the maximum pooling operation, kernel_sizekernel_size and stride are the size and stride of the pooling window respectively. Fully connected layer, formula:

[0096] F fc =ReLU(Linear(F flatten ,W fc )+b fc )

[0097] Among them, Fflatten is to flatten the output of the convolutional layer into a one-dimensional vector, Linear represents the fully connected operation, and Wfc and bfc are the weights and bias items of the fully connected layer. Output layer, formula:

[0098] P output =Softmax(Linear(F fc , W ont )+b out )

[0099] Here the Softmax function converts the output of the fully connected layer into a probability distribution.

[0100] S2024, training model

[0101] Specifically, the Adam optimizer is used to optimize the model. The Adam optimizer formula is as follows:

[0102] m t =β 1 m t-1 +(1-β 1 ) t

[0103]

[0104]

[0105]

[0106]

[0107] Where mt and vt are the first-order and second-order momentum estimates, α is the learning rate, β1 and β2 are momentum parameters, and ∈ is a small constant (usually 10^-8)

[0108] S2025, Model Evaluation

[0109] Specifically, the cross entropy loss function is used for model evaluation, formula:

[0110]

[0111] Among them, yiyi is the true label, Poutput,iPoutput,i is the predicted probability

[0112] S2026. Model deployment

[0113] Specifically, the model is saved and deployed to the server 401, and provided for internal use in the form of an API.

[0114] S203: Building ArcFace model for face comparison

[0115] Specifically, the steps of face comparison are as follows:

[0116] S2031 Data preparation: Import the images filtered by S202, collect face data, and pre-process the face images to ensure image quality.

[0117] S2032 Load model: Use the Paddle deep learning framework to load the pre-trained ArcFace model.

[0118] S2033 Feature Extraction: Use the face detection algorithm Dlib to detect the face position in the image and extract the face area. Input the face image into the ArcFace model, and the model will output a fixed-length feature vector, which can be used to represent the features of the face.

[0119] S2034 Feature comparison: Calculate feature vectors and similarities.

[0120] S2035 Determine a threshold: Determine a suitable threshold based on experimental data. When the similarity between two feature vectors exceeds this threshold, it is considered that the two images belong to the same person.

[0121] S204: Save face comparison score

[0122] Specifically, the comparison score is saved in a database.

[0123] S205. Construct emotion2vec speech emotion base model for emotion analysis

[0124] Specifically, the emotion2vec speech emotion base model is used for emotion analysis, and the emotion analysis steps are as follows: input speech data, import the speech file in S130, and divide the speech file into multiple paragraphs according to speech pauses. Analyze the emotions of multiple paragraphs, and then get the results of each paragraph. Use the prediction algorithm to guess the final result, calculate the total value of the emotion probability (score) of different paragraphs by arithmetic sum, and take the score and the maximum emotion.

[0125] S206. Save emotion results

[0126] The result of S205 is saved in the database.

[0127] S207. Generate fraud score

[0128] The fraud scoring threshold is determined based on the results of S204 and S206.

[0129] S208. Save fraud score

[0130] Save fraud scores for historical data query.

[0131] S209. Set alerts based on fraud scores

[0132] Set alarms based on fraud score thresholds and notify administrators via DingTalk alarms.

[0133] Example 3

[0134] An image emotion recognition anti-fraud device includes a user image and audio information acquisition module 301, an image detection module 302, a face comparison module 303, an emotion analysis module 304, a fraud risk assessment module 305, and an alarm module 306, wherein:

[0135] User picture and audio information acquisition module 301, used to obtain the user's ID card picture information, living body picture information and audio information;

[0136] An image detection module 302, used to detect image forgery and tampering traces;

[0137] The face comparison module 303 is used to verify the authenticity of the user's identity information;

[0138] Emotional analysis module 304, used to identify the emotional state of the user and whether there is risky behavior;

[0139] The fraud risk assessment module 305 formulates risk assessment logic based on the face comparison module 303 and the sentiment analysis module 304;

[0140] The alarm module 306 sets alarm thresholds according to the risk assessment logic and notifies the administrator in the form of DingTalk alarms;

[0141] Embodiment 4

[0142] A server, Figure 4 A block diagram of an exemplary server 401 suitable for implementing embodiments of the present invention is shown. Figure 4 The server 401 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0143] like Figure 4 As shown, server 401 is in the form of a general server. The components of server 401 include

[0144] But not limited to: one or at least one CPU403, one or at least one GPU404, storage device 405, IO component 410.

[0145] The CPU 403 is connected to a display 409 via an I / O 410 .

[0146] The server 401 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the server 401, including volatile and non-volatile media, removable and non-removable media.

[0147] The storage device 405 may include computer system readable media in the form of volatile memory, such as random access memory (Random Access Memory), RAM 407 and cache memory 406. The storage system 408 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 408 may be used to read and write to a non-removable, non-volatile solid state drive (SSD). The storage device 405 may include at least one program product 411, which has at least one program module, which is configured to perform the functions of various embodiments of the present invention.

[0148] The processor 403 executes various functional applications and data processing by running the program stored in the storage device 405, for example, implementing the anti-spoofing identification method provided by any embodiment of the present invention, which may include:

[0149] Get the picture or audio data uploaded by the user;

[0150] Use image processing algorithms to detect forgery or tampering of images;

[0151] Verify the authenticity of user identity information through face comparison technology;

[0152] Analyze the emotional state of users during interaction and identify abnormal emotional fluctuations;

[0153] Combining image detection results and sentiment analysis results to generate a fraud risk score;

[0154] Trigger further verification or alarm mechanisms, account freezing or administrator notification based on risk scores.

[0155] Embodiment 5

[0156] Embodiment 5 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the anti-bombing identification method provided by any embodiment of the present invention is implemented. The method may include:

[0157] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: including hard disk drives (HDDs) and solid-state drives (SSDs), USB flash drives, SD cards, CDs, DVDs and Blu-ray discs, magnetic tapes, main memory (RAM), read-only memory (ROM), network storage devices, cloud storage, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.

[0158] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, electrical signals, acoustic signals, digital signals, or any suitable combination thereof. The computer-readable medium may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0159] The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, optical cable, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​(such as Java, Python, C++). The program code may be executed entirely on the user's computer, as a stand-alone software package, or as a stand-alone software package.

[0160] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

Claims

1. A method for anti-fraud by image emotion recognition, characterized in that: The following steps are involved: S110: Acquire the picture or audio data uploaded by the user; S120: Detecting forgery or tampering of the image using an image processing algorithm; S130: Verify the authenticity of the user's identity information through face comparison technology; S140: Analyze the emotional state of the user during the interaction process and identify abnormal emotional fluctuations; S150: Generate a fraud risk score by integrating the image detection result and the sentiment analysis result; S160: Trigger further verification or alarm mechanisms, account freezing or administrator notification based on risk scores.

2. The image emotion recognition anti-fraud method according to claim 1, characterized in that: In S110, the data includes: Photos of the front and back of the user's ID card; A live photo of the user holding his ID card; Audio data generated when company specialists communicate with users.

3. The image emotion recognition anti-fraud method according to claim 1, characterized in that: In S120, the Paddle framework is used in combination with the CNN model for image tampering detection. The detection steps are as follows: S1201, data preparation; S1202, data preprocessing; S1203, constructing a model; S1204, training model; S1205, model evaluation; S1206: Model deployment.

4. The image emotion recognition anti-fraud method according to claim 1, characterized in that: In S130, the Paddle framework is used in combination with the ArcFace model for face comparison. The comparison steps are as follows: S1301, data preparation: import the images filtered in S120, collect face data, and pre-process the face images; S1302, load model: use Paddle deep learning framework to load the pre-trained ArcFace model; S1303, feature extraction: Use the face detection algorithm Dlib to detect the face position in the image and extract the face area. Input the face image into the ArcFace model. The model will output a feature vector of a fixed length. This vector can be used to represent the features of the face. S1304, feature comparison: calculating feature vectors and similarity; S1305, determining a threshold: determining a suitable threshold based on experimental data, and when the similarity between two feature vectors exceeds the threshold, it is considered that the two images belong to the same person; S1306: Save the face comparison data to the database.

5. The image emotion recognition anti-fraud method according to claim 1, characterized in that: In S140, the emotion2vec speech emotion base model is used to perform emotion analysis, and the emotion analysis steps are as follows: S1401, input voice data: import the voice file in S13, and divide the voice file into multiple paragraphs according to voice pauses; S1402, analyzing the emotions of multiple paragraphs, and then obtaining the result of each paragraph; S1403, using a prediction algorithm to guess the final result: arithmetic sum is performed to calculate the total value of the emotional probability (score) of different paragraphs, and the score and the maximum emotion are taken; S1404: Save the sentiment analysis results to a database.

6. The image emotion recognition anti-fraud method according to claim 1, characterized in that: In S150, the data returned from S130 and S140 are comprehensively analyzed to formulate risk scoring logic and determine the risk scoring threshold.

7. The image emotion recognition anti-fraud method according to claim 1, characterized in that: In S160, the administrator is notified by using a DingTalk alarm.

8. An image emotion recognition anti-fraud device, characterized in that: include: A user picture and audio information acquisition module (301) is used to acquire the user's ID card picture information, living body picture information and audio information; An image detection module (302) is used to detect image forgery and tampering traces; A face comparison module (303) is used to verify the authenticity of the user's identity information; Emotional analysis module (304), used to identify the emotional state of the user and whether there is risky behavior; A fraud risk assessment module (305) formulates risk assessment logic based on the face comparison module (303) and the sentiment analysis module; The alarm module (306) formulates the alarm threshold according to the risk assessment logic and notifies the administrator in the form of a DingTalk alarm.

9. A server, characterized in that: include: At least one CPU; At least one GPU; A storage device for storing one or at least one program, when the one or at least one program is executed by the one or at least one processor, the one or at least one processor implements the anti-bombing identification method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the anti-fraud identification method as described in any one of claims 1-7 is implemented.

Images