Information early warning method, system, device, terminal equipment, medium and program product

Abstract

The application relates to an information early warning method, system and device, terminal equipment, a medium and a program product, and relates to the fields of information security and biological identification. The method comprises the following steps: inputting multiple frames of face images of a user in a preset time period into an expression prediction model to obtain a facial emotion prediction result of the user, analyzing feature information of different positions in the face in the preset time period according to the multiple frames of face images, determining a physiological emotion prediction result of the user according to the feature information, determining whether the current operation of the user is a suspected fraud behavior through the facial emotion prediction result and the physiological emotion prediction result, and outputting suspected fraud early warning information if the current operation of the user is a suspected fraud behavior. The method can output suspected fraud early warning information before the user transfers money to the other party, a staff member can check the current operation of the user according to the suspected fraud early warning information, and when the checking result is a fraud behavior, the staff member can discover abnormal fraud behaviors in time and inform the user to stop the money transfer behavior, so that a series of fraud behaviors can be prevented.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, and in particular to an information early warning method, system, device, terminal equipment, medium and program product. BACKGROUND

[0002] Self-service terminal systems are commonly used in people's lives and are used by fraudsters as fraud tools. For example, fraudsters often use the other party's concern for their relatives, fake the other party's relatives' missing appearance, pretend to be the other party's relatives' friends, and ask the other party to transfer money to themselves on a self-service terminal system to help find the other party's relatives.

[0003] However, the related art does not provide a method that can timely warn the other party's operation on the self-service terminal system as a suspected fraud. SUMMARY

[0004] Therefore, it is necessary to provide an information early warning method, system, terminal device, equipment, medium and program product to solve the above technical problems.

[0005] In a first aspect, an information early warning method is provided, and the method includes:

[0006] inputting a plurality of frames of facial images of a user in a preset time period into an expression prediction model to obtain a facial emotion prediction result of the user;

[0007] analyzing feature information of different positions in the face in the preset time period according to the plurality of frames of facial images, and determining a physiological emotion prediction result of the user according to the feature information;

[0008] determining whether a current operation of the user is a suspected fraud behavior through the facial emotion prediction result and the physiological emotion prediction result;

[0009] if the current operation of the user is a suspected fraud behavior, outputting a suspected fraud early warning information; the suspected fraud early warning information is used to indicate to check the current operation of the user.

[0010] In one embodiment, the different positions in the face include an eye region and a forehead region in the face.

[0011] According to the plurality of frames of facial images, analyzing the feature information of the different positions in the face in the preset time period includes:

[0012] detecting the eye region in the plurality of frames of facial images to obtain eye feature information of the eye region of the user in the preset time period; detecting the forehead region in the plurality of frames of facial images to obtain skin data information of the forehead region of the user in the preset time period, and determining heart rate feature information of the user according to the skin data information.

[0013] In one of the embodiments, the eye feature information includes a blink frequency; the eye region in the plurality of face images is detected to obtain the eye feature information of the user in the preset time period, including:

[0014] According to the eye region in the plurality of face images, the eye state of the user in each face image is obtained;

[0015] According to the eye state of the user in each face image, the total number of blinks of the user in the preset time period is determined; wherein the eye state in the closed state indicates that a blink occurs once;

[0016] According to the total number of blinks and the duration of the preset time period, the blink frequency of the user is determined.

[0017] In one of the embodiments, according to the eye region in the plurality of face images, the eye state of the user in each face image is obtained, including:

[0018] For each face image, the eye position is determined according to the eye region in the face image;

[0019] The size information of the user's eyes is determined according to the eye position, and the size information includes an aspect ratio;

[0020] If the aspect ratio is 0, the eye state of the user in the face image is determined to be closed.

[0021] In one of the embodiments, the skin data information includes a blood volume pulse signal;

[0022] The forehead region in the plurality of face images is detected to obtain the skin data information of the user in the forehead region in the preset time period, and the heart rate feature information of the user is determined according to the skin data information, including:

[0023] According to the forehead region in the plurality of face images, the forehead and the vertex position of the user are determined;

[0024] The blood volume pulse signal of the forehead and the vertex position of the user in the preset time period is obtained by non-contact sensor measurement;

[0025] The heart rate feature information of the user is determined through the peak value in the blood volume pulse signal.

[0026] In one of the embodiments, the heart rate feature information of the user is determined through the peak value in the blood volume pulse signal, including:

[0027] According to the blood volume pulse signal and the preset blood volume threshold, the heartbeat blood volume signal in the blood volume pulse signal is determined;

[0028] The number of peak values in the heartbeat blood volume signal is obtained;

[0029] According to the number of peaks in the heartbeat blood volume signal and the length of the preset time period, the heart rate feature information of the user is determined.

[0030] In one of the embodiments, the physiological emotion prediction result of the user is determined according to the feature information, including:

[0031] The feature information at different positions in the face is respectively assigned a weight;

[0032] The feature information at different positions in the face and the weight of each feature information are input into a classifier for classification processing to obtain the physiological emotion prediction result of the user.

[0033] In one of the embodiments, the current operation of the user is determined to be a suspected fraud behavior through the facial emotion prediction result and the physiological emotion prediction result, including:

[0034] The facial emotion prediction result and the physiological emotion prediction result are weighted and summed to obtain an emotion prediction result;

[0035] If the emotion prediction result is greater than an abnormal emotion threshold, it is determined that the current operation of the user is a suspected fraud behavior; otherwise, it is determined that the current operation of the user is not a suspected fraud behavior.

[0036] In a second aspect, an information warning system, the system includes: an image acquisition device and a terminal device;

[0037] The image acquisition device is configured to acquire a face image of a user;

[0038] The terminal device is configured to execute the steps of the method in any of the embodiments of the first aspect.

[0039] In a third aspect, an information warning device, the device includes:

[0040] A prediction module is configured to input a plurality of face images of a user in a preset time period into an expression prediction model to obtain a facial emotion prediction result of the user;

[0041] An analysis module is configured to analyze feature information of different positions in a face in a preset time period according to a plurality of face images, and determine a physiological emotion prediction result of the user according to the feature information;

[0042] A determination module is configured to determine whether a current operation of the user is a suspected fraud behavior through the facial emotion prediction result and the physiological emotion prediction result;

[0043] A warning information acquisition module is configured to output a suspected fraud warning information when the result of the determination module is that the current operation of the user is a suspected fraud behavior; the suspected fraud warning information is used to indicate that the current operation of the user is checked.

[0044] Fourthly, a terminal device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method in any of the embodiments of the first aspect described above.

[0045] Fifthly, this application provides a readable storage medium. This readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0046] Sixthly, this application provides a computer program product. This computer program product includes a computer program that, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0047] The aforementioned information warning methods, systems, devices, terminal equipment, media, and program products allow the equipment to input multiple frames of a user's facial images within a preset time period into an expression prediction model to obtain the user's facial emotion prediction results. Based on the multiple frames of facial images, the system analyzes the feature information of different locations on the face within the preset time period and determines the user's physiological emotion prediction results based on this feature information. By combining the facial emotion prediction results and the physiological emotion prediction results, the system determines whether the user's current operation is suspected of being fraudulent. If the user's current operation is suspected of being fraudulent, a suspected fraud warning message is output. Using this method, suspected fraud warning messages can be output before a user transfers money to another party. Staff can then verify the user's current operation based on the suspected fraud warning message. If the verification result indicates fraudulent behavior, the system can promptly detect the abnormal fraudulent activity and inform the user to stop the transfer, thereby preventing a series of fraudulent activities. Attached Figure Description

[0048] Figure 1 This is a diagram illustrating the application environment of an information warning method in one embodiment;

[0049] Figure 2 This is a flowchart illustrating an information warning method in one embodiment;

[0050] Figure 3 This is a flowchart illustrating the steps of analyzing feature information of different locations on a face within a preset time period based on multiple frames of face images in one embodiment.

[0051] Figure 4 This is a flowchart illustrating the steps of detecting the eye region in multiple frames of face images and obtaining the eye feature information of the user's eye region within a preset time period, as described in another embodiment.

[0052] Figure 5 This is a flowchart illustrating the steps of obtaining the user's eye state in each frame of a face image based on the eye region in multiple frames of face images, as described in another embodiment.

[0053] Figure 6 FIG. 2 shows a flowchart of a process for detecting a forehead region in a plurality of face images to determine a heart rate feature information of a user in another embodiment;

[0054] Figure 7 FIG. 3 shows a flowchart of a process for determining a heart rate feature information of a user by peaks in a blood volume pulse signal in another embodiment;

[0055] Figure 8 FIG. 4 shows a flowchart of a process for determining a physiological emotion prediction result of a user according to a feature information in another embodiment;

[0056] Figure 9 FIG. 5 shows a block diagram of a structure of an information warning device in an embodiment;

[0057] Figure 10 FIG. 6 shows an internal structure diagram of a terminal device in an embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0059] The information warning method provided by the present application can be applied to Figure 1 The information warning system shown in FIG. 1. The information warning system includes an image acquisition device and a terminal device. Optionally, the image acquisition device can be a camera, a video camera, a scanner, or other electronic devices (such as mobile phones, tablet computers, etc.) with a photographing function, Figure 1 The camera is taken as an example to illustrate the information warning system; the terminal device can be a stand-alone server or a server cluster composed of multiple servers, and can be but is not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The image acquisition device and the terminal device can be communicatively connected, and the communication mode can be Wi-Fi, mobile network or Bluetooth connection, etc.

[0060] In order to avoid users falling into fraudulent behavior, after the user goes to a location with a self-service terminal system, the facial emotion and physiological emotion of the user can be predicted during the money transfer process through the self-service terminal system, so as to determine whether the current operation of the user is a suspected fraudulent behavior according to the prediction result, and if it is determined that the current operation of the user is a suspected fraudulent behavior, a warning information can be outputted for timely checking with the user. Therefore, in an embodiment, as Figure 2 shown in FIG. 2, an information warning method is provided, and the method is applied to Figure 1The terminal device in the method is taken as an example for illustration, including the following steps.

[0061] S100, inputting the multiple frames of face images of the user in the preset time period into the expression prediction model to obtain a facial emotion prediction result of the user.

[0062] Specifically, the self-service terminal system can be understood as an Automated Teller Machine (ATM). Optionally, the self-service terminal system can be in-line and off-line; the in-line one is generally set beside or inside any bank, and has a security door to ensure safe use; the off-line one is generally set in places with large flow of people, such as railway stations, shopping malls, commercial streets, etc. Optionally, the in-line self-service terminal system is generally managed by the bank; the off-line self-service terminal system is generally managed by the corresponding bank headquarters.

[0063] It should be noted that the image acquisition device can be arranged at any position on the self-service terminal system, as long as it can acquire the front face image of the user during the use of the self-service terminal system. The face image acquired by the image acquisition device can be a color image or a black and white image. The sizes of the multiple frames of face images acquired by the image acquisition device can be equal or not equal. If the sizes of the multiple frames of face images acquired are not equal, the multiple frames of face images are cropped into face images with equal sizes before being input into the expression prediction model. In the process of cropping the face images, it is ensured that each frame of face image includes a face region.

[0064] Optionally, if the multiple frames of face images of the user in the preset time period are color images, the multiple frames of face images can be first subjected to grayscale processing and normalization processing before being input into the expression prediction model for processing. If the multiple frames of face images of the user in the preset time period are black and white images, the multiple frames of face images can be directly input into the expression prediction model for processing. The specific steps of the information warning method will be described below taking the face images input into the expression prediction model as black and white images as an example.

[0065] The preset time period can be any time period from the start time of the user's operation on the self-service terminal system to the time before the user completes the remittance transfer to the other party on the self-service terminal system. Optionally, the other party is a fraud party. The image acquisition device can acquire multiple frames of face images in the preset time period; the acquisition intervals between each frame of face image can be equal or not equal.

[0066] It can be understood that the above expression prediction model can be a pre-trained expression prediction model; the expression prediction model can be at least one of a convolutional neural network model, a recurrent neural network model, and an adversarial neural network model. In the training process of the expression prediction model, the facial images of different users in different time periods can be used as a training set to train the initial expression prediction model through the training set, and a pre-trained expression prediction model is obtained.

[0067] It can also be understood that the terminal device can input the facial images of different users in different time periods in the training set into the initial expression prediction model to obtain an expression prediction result, calculate the prediction error value between the expression prediction result and the standard expression prediction result through a loss function, and update the initial network parameters in the initial expression prediction model according to the prediction error value. The above training steps are iterated until the prediction error value meets the preset error threshold or the number of iterations reaches the preset iteration threshold, and a pre-trained expression prediction model is obtained. The above standard expression prediction result is the gold standard for network training.

[0068] Optionally, the facial emotion prediction result can be any combination of multiple emotions such as happiness, anger, anxiety, calmness, sadness, confusion, excitement, fear, and sadness, and the prediction proportion corresponding to each emotion. Optionally, the sum of the prediction proportions corresponding to each emotion can be equal to 100%.

[0069] In this embodiment, the above expression prediction model can be a convolutional neural network model; the convolutional neural network model is composed of three convolutional layers, one fully connected layer and one activation layer. The above expression prediction model can also be a network model composed of a Chebyshev graph convolution network (GCN), a regular convolution network (CNN) and a fully connected layer; the model can first process irregular facial images in a preset time period through the Chebyshev GCN, and then use the regular CNN to extract higher-level features in the processed facial images, and then input the extracted features into the fully connected layer, and use the softmax function to predict and output the facial emotion prediction result of the user. Optionally, the above expression prediction model can also be other network models other than the above model structure, and this embodiment does not limit the structure.

[0070] S200, according to the plurality of facial images, analyzing the feature information of different positions in the face in the preset time period, and determining the physiological emotion prediction result of the user according to the feature information.

[0071] Specifically, the terminal device can analyze the feature information of different positions in the face in the preset time period according to the number of pixel points in each frame of face image of the user, and can also analyze the feature information of different positions in the face in the preset time period according to the proportion of the number of pixel points of the face region in each frame of face image of the user in the face image. Alternatively, the terminal device can also input multiple frames of face images into the analysis network model to analyze the feature information of different positions in the face in the preset time period.

[0072] It should be noted that the different positions in the face described above can be the cheek region, chin region, mouth region, eyebrow region, etc. of the face. The feature information corresponds to the different positions in the face. If the different positions in the face are the cheek region of the face, the feature information can be blood volume feature information; if the different positions in the face are the chin region, mouth region or eyebrow region of the face, the feature information can be chin feature information, mouth feature information and eyebrow feature information, respectively.

[0073] Further, the terminal device can perform arithmetic operations, comparisons, analyses and / or classifications, etc. on the feature information corresponding to the obtained frames of face images to obtain the physiological emotion prediction result of the user. Alternatively, the terminal device can also input the feature information into a physiological emotion prediction model to obtain the physiological emotion prediction result of the user.

[0074] Optionally, the above arithmetic operations can be addition, subtraction, multiplication, division, exponential operation and / or logarithmic operation, etc. Optionally, the comparison process can be understood as a process of comparing the feature information corresponding to different positions of the face in the face image. Optionally, the analysis process can be understood as a process of comprehensive analysis of the feature information corresponding to all frames of face images. Optionally, the classification process can be understood as a process of classification of the feature information corresponding to all frames of face images. Optionally, the physiological emotion prediction model can be a pre-trained physiological emotion prediction model.

[0075] It can be understood that the physiological emotion prediction result of the user can be a prediction result of blood pressure, pulse, heartbeat, etc. and the proportion corresponding to various prediction results; the sum of the proportions corresponding to various prediction results can be equal to 100%.

[0076] S300, determine whether the current operation of the user is a suspected fraudulent behavior through the facial emotion prediction result and the physiological emotion prediction result.

[0077] Specifically, the terminal device performs arithmetic operations, analyses, feature comparisons, etc. on the facial emotion prediction result and the physiological emotion prediction result to determine whether the current operation of the user is a suspected fraudulent behavior.

[0078] The step of determining whether the current operation of the user is the suspected fraud behavior by the facial emotion prediction result and the physiological emotion prediction result in S300 includes: obtaining an emotion prediction result by weighted summation of the facial emotion prediction result and the physiological emotion prediction result; determining that the current operation of the user is the suspected fraud behavior if the emotion prediction result is greater than an abnormal emotion threshold; or determining that the current operation of the user is not the suspected fraud behavior.

[0079] Optionally, weights are assigned to various emotions in the facial emotion prediction result according to the prediction proportions of the various emotions in the facial emotion prediction result, or a weight classification algorithm is used to respectively assign weights to the facial emotion prediction result and the physiological emotion prediction result; and the sum of the weights of the various emotions in the facial emotion prediction result can be equal to 1. Meanwhile, weights are assigned to various emotions in the physiological emotion prediction result according to the prediction proportions of the various emotions in the physiological emotion prediction result; and the sum of the weights of the various emotions in the physiological emotion prediction result can be equal to 1.

[0080] It should be noted that the terminal device can obtain an emotion prediction result by weighted summation of the facial emotion prediction result, the physiological emotion prediction result, the weights of the various emotions in the facial emotion prediction result, and the weights of the various emotions in the physiological emotion prediction result. The emotion prediction result can be a numerical value.

[0081] Further, the terminal device can compare the emotion prediction result with an abnormal emotion threshold, and determine whether the current operation of the user is the suspected fraud behavior according to a comparison result. The abnormal emotion threshold can be a critical emotion prediction value corresponding to the current operation of the user being the suspected fraud behavior and the non-suspected fraud behavior. Optionally, the comparison result can be that the emotion prediction result is equal to the abnormal emotion threshold, the emotion prediction result is greater than the abnormal emotion threshold, or the emotion prediction result is less than the abnormal emotion threshold.

[0082] S400, if the current operation of the user is the suspected fraud behavior, outputting a suspected fraud warning information; the suspected fraud warning information is used to instruct to check the current operation of the user.

[0083] In this embodiment, if it is determined that the emotion prediction result is greater than the abnormal emotion threshold, it can be determined that the current operation of the user is the suspected fraud behavior. Optionally, if the current operation of the user is suspected of fraud behavior, the current operation of the user can be determined as the suspected fraud behavior.

[0084] It should be noted that if it is determined that the current operation of the user is a suspected fraud behavior, a suspected fraud warning information can be output. Optionally, the suspected fraud warning information can be a suspected fraud warning signal, a suspected fraud warning view information or a suspected fraud warning text information, of course, it can also be a combination information of the suspected fraud warning signal, the suspected fraud warning view information and the suspected fraud warning text information.

[0085] It can be understood that if the self-service terminal system is in a line, after the staff of the bank to which the self-service terminal system belongs receives the suspected fraud warning information, the staff can go to the self-service terminal system to check the user operating the self-service terminal system to determine whether the current operation of the user is a real money transfer behavior or a fraud behavior. Optionally, the staff can determine whether the current operation of the user is a real money transfer behavior or a fraud behavior according to actual experience and the description of the user. If the staff determines that the current operation of the user is a real money transfer behavior, the user can be informed to stop the money transfer behavior in time to avoid being deceived, protect the user's safety, and reduce the user's economic loss.

[0086] In the above information warning method, the terminal device can input the multiple frames of face images of the user in the preset time period into the expression prediction model to obtain a facial emotion prediction result of the user, analyze feature information of different positions in the face in the preset time period according to the multiple frames of face images, and determine a physiological emotion prediction result of the user according to the feature information. By the facial emotion prediction result and the physiological emotion prediction result, it is determined whether the current operation of the user is a suspected fraud behavior. If the current operation of the user is a suspected fraud behavior, a suspected fraud warning information is output. The above method can output a suspected fraud warning information before the user transfers money to the other party. The staff can check the current operation of the user according to the received suspected fraud warning information. When the checking result is a fraud behavior, the abnormal fraud behavior can be found in time. The user can be informed to stop the money transfer behavior in time, so that a series of fraud behaviors can be prevented, the safety of the user can be protected, and the economic loss of the user caused by the fraud behavior can be greatly reduced. At the same time, the above method can stop loss in time to avoid affecting the physical health of the user due to the loss of the economy of the user caused by the fraud behavior.

[0087] In some scenarios, in order to analyze the feature information of different positions in the face, different positions in the face which have significant changes in blood volume or running features due to emotions can be processed. Therefore, in an embodiment, the different positions in the face include an eye region and a forehead region in the face. Figure 3 As shown in S200, the step of analyzing the feature information of different positions in the face in the preset time period according to the multiple frames of face images can include:

[0088] S210, detecting the eye region in the plurality of face images to obtain eye feature information of the eye region of the user in a preset time period.

[0089] Specifically, the terminal device can use a target detection algorithm to detect the eye region in each face image to obtain the eye feature information of the eye region of the user in the preset time period. Optionally, the target detection algorithm can be an image target positioning algorithm, such as a local feature point detection method, a scale-invariant feature transform algorithm, a speed-up robust features algorithm, and the like. Optionally, the above-mentioned eye feature information can be eye movement information, a blink duration, and the like.

[0090] Illustratively, if the local feature point detection method is used to detect the eye region in each face image, the distance between the feature points of the upper boundary of the eye region and the feature points of the lower boundary of the eye region in the face image can be detected. Optionally, the feature points of the upper boundary of the eye region are at least one, and the feature points of the lower boundary of the eye region are also at least one.

[0091] Optionally, if the feature points of the upper boundary of the eye region and the feature points of the lower boundary of the eye region are both one, the feature points of the upper boundary of the eye region and the feature points of the lower boundary of the eye region are both the midpoint of the corresponding boundary. If the feature points of the upper boundary of the eye region and the feature points of the lower boundary of the eye region are both two, the two feature points of the upper boundary of the eye region can divide the upper boundary of the eye region into three equal parts, and the two feature points of the lower boundary of the eye region can divide the lower boundary of the eye region into three equal parts. If the feature points of the upper boundary of the eye region and the feature points of the lower boundary of the eye region are other quantities, the specific setting positions are similar, and will not be described here. The setting positions of the feature points of the upper boundary of the eye region and the feature points of the lower boundary of the eye region correspond.

[0092] S220, detecting the forehead region in the plurality of face images to obtain skin data information of the forehead region of the user in a preset time period, and determining heart rate feature information of the user according to the skin data information.

[0093] Specifically, the terminal device can use a target detection algorithm to detect the forehead region in each face image to obtain the skin data information of the forehead region of the user in a preset time period. At this time, the target detection algorithm can also be an image target positioning algorithm, such as a local feature point detection method, a scale-invariant feature transform algorithm, a speed-up robust features algorithm, and the like. Optionally, the above-mentioned skin feature information can be skin type, secretion condition of sebaceous glands, water content of the stratum corneum, pH value on the surface of the stratum corneum, and the like.

[0094] Further, the terminal device can perform arithmetic operation, analysis, and / or feature comparison processing, etc. on the skin feature information to obtain the heart rate feature information of the user.

[0095] The above information warning method can obtain eye feature information and heart rate feature information, and then determine a physiological emotion prediction result according to the eye feature information and the heart rate feature information, and comprehensively consider the area emotion prediction result and the physiological emotion prediction result, so as to accurately determine whether the current operation of the user is a suspected fraud behavior, and further output suspected fraud warning information when it is determined that the current operation of the user is a suspected fraud behavior, so that the staff can timely check the current operation of the user before the user transfers money to the other party, and when the checking result is a fraud behavior, the user can be informed to stop the money transfer behavior in time, the user can be prevented from being deceived, the safety of the user is ensured, and the economic loss of the user caused by the fraud behavior is greatly reduced.

[0096] The following will introduce how to process the plurality of face images in the preset time period to obtain the eye feature information when the eye feature information is the blink frequency.

[0097] In an embodiment, the eye feature information includes a blink frequency; as Figure 4 As shown in the above S210, the step of detecting the eye region in the plurality of face images and obtaining the eye feature information of the eye region of the user in the preset time period can be implemented by the following steps:

[0098] S211, obtaining the eye state of the user in each frame of face image according to the eye region in the plurality of face images.

[0099] Specifically, the terminal device can perform arithmetic operation, analysis, comparison, cropping and other processing on the eye region in the plurality of face images to obtain the eye state of the user in each frame of face image. Optionally, the eye state of the user can include a closed state and an open state. In this embodiment, the eye state of the user other than the closed state is referred to as the open state.

[0100] In some scenarios, in order to determine the significant features of the eye region of the user, so as to help improve the accuracy of the physiological emotion prediction result. In an embodiment, as Figure 5 As shown in the above S211, the step of obtaining the eye state of the user in each frame of face image according to the eye region in the plurality of face images can include:

[0101] S2111, determining the eye position according to the eye region in the face image for each frame of face image.

[0102] In this embodiment, in the process of detecting the eye region in each frame of face image by using the local feature point detection method, two feature points can be marked at the two corners of each eye respectively while marking the feature points of the upper boundary and the lower boundary of the eye region.

[0103] It can be understood that the terminal device can detect feature points of the upper boundary and the lower boundary of the eyes and feature points of the corners of the eyes in each frame of face image by using a local feature point detection method, to determine the eye position. Optionally, the positions of the feature points of the upper boundary of the eyes, the feature points of the lower boundary of the eyes, and the feature points of the corners of the eyes are the eye position.

[0104] S2112, determine the size information of the eyes of the user according to the eye position, the size information including an aspect ratio.

[0105] Optionally, the terminal device can determine the width and the height of the eyes of the user in each frame of face image according to the detected eye position in each frame of face image, i.e., the size information of the eyes of the user. Further, the aspect ratio of the eyes of the user can be obtained by dividing the height of the eyes by the width of the eyes.

[0106] S2113, if the aspect ratio is 0, determine that the eye state of the user in the face image is a closed state.

[0107] It can also be understood that if the terminal device determines that the aspect ratio of the eyes of the user in the current frame of face image is 0, it can be determined that the eye state of the user in the face image is a closed state, indicating that the user blinks at the moment corresponding to the current frame. At this time, it can be determined that the user blinks once.

[0108] S212, determine the total number of blinks of the user in a preset time period according to the eye state of the user in each frame of face image. When the eye state is a closed state, it means that the user blinks once.

[0109] Specifically, the terminal device can determine the eye state of the user in each frame of face image, then calculate the total number of frames of face images in which the eye state is a closed state, and determine the number of frames of face images in which the eye state is a closed state as the total number of blinks of the user in the preset time period.

[0110] S213, determine the blink frequency of the user according to the total number of blinks and the length of the preset time period.

[0111] It should be noted that the terminal device can perform arithmetic operations on the total number of blinks and the length of the preset time period to determine the blink frequency of the user. Optionally, the arithmetic operations can be addition, subtraction, multiplication, exponentiation, and / or logarithm operations, etc.

[0112] In this embodiment, the terminal device can divide the total number of blinks by the length of the preset time period to obtain the blink frequency of the user in the preset time period. Optionally, the length of the preset time period can be equal to the length of time corresponding to the end time minus the start time of the preset time period.

[0113] The information early warning method can acquire the eye state of the user in each frame of face image according to the eye region in the multiple frames of face image, determine the total number of blinks of the user in the preset time period according to the eye state of the user in each frame of face image, determine the blink frequency of the user according to the total number of blinks and the duration of the preset time period, and then comprehensively consider the blink frequency of the user and the heart rate feature information, so as to improve the accuracy of the determined physiological emotion prediction result.

[0114] In some scenarios, in order to improve the accuracy of the determined physiological emotion prediction result, the heart rate feature information of the user can also be determined, and the physiological emotion prediction result is further determined according to the blink frequency of the user and the heart rate feature information. In some embodiments, the skin data information includes a blood volume pulse signal; as Figure 6 As shown in the S220, the steps of detecting the forehead region in the multiple frames of face image, acquiring the skin data information of the forehead region of the user in the preset time period, and determining the heart rate feature information of the user according to the skin data information can be implemented by the following steps:

[0115] S221, determining the forehead and temple position of the user according to the forehead region in the multiple frames of face image.

[0116] Specifically, since the skin of the forehead region is relatively smooth and thin, the influence on the user when speaking or making expressions is small, so the forehead region is generally selected to acquire the skin data information, and further, the heart rate feature information of the user can be accurately acquired according to the skin data information.

[0117] Optionally, the terminal device can adopt a target detection algorithm to detect the forehead region in each frame of face image and determine the forehead and temple position of the user.

[0118] In this embodiment, the forehead and temple position of the user can be determined by the following detection process:

[0119] Firstly, the terminal device can perform denoising processing (such as median filtering for smoothing denoising) on each frame of face image, and then adopt the maximum inter-class variance method to determine a separation threshold to separate the face and background in the denoised face image, so as to facilitate edge detection, and the size of the denoised face image is MxN.

[0120] Furthermore, the terminal device can use an integral projection localization algorithm to locate the face in the denoised face image. Specifically, the denoised face image is subjected to vertical integral projection to obtain a vertical integral projection curve. Then, the vertical integral projection curve is smoothed using a Bezier curve to obtain a smooth curve. Next, the point with the maximum gradient value within x∈[0, 0.5N] of the smooth curve is found; this point represents the left boundary of the face in the denoised face image. The point with the minimum gradient value within [0, 0.5N] is then found; this point represents the right boundary of the face in the denoised face image. Simultaneously, a similar horizontal integral projection can be used to obtain the upper boundary of the face. Based on the ratio of forehead height to face width of 0.35:1, the user's forehead region can be roughly determined, thus locating the user's forehead in the denoised face image.

[0121] Of course, other detection processes can also be used to locate the user's forehead in this embodiment, and this embodiment does not limit this.

[0122] S222. The blood volume pulse signal of the user's forehead position within a preset time period is obtained by non-contact sensor measurement.

[0123] Specifically, the terminal device can send the user's forehead position to a non-contact sensor. The non-contact sensor then uses remote photoelectric volume scanning technology to measure the blood volume pulse signal at the user's forehead position within a preset time period. The non-contact sensor then sends the measured blood volume pulse signal to the terminal device.

[0124] It is understood that non-contact sensors can be infrared sensors, laser sensors, electromagnetic wave sensors, or ultrasonic sensors, etc., and this embodiment does not limit them.

[0125] S223. Determine the user's heart rate characteristics by analyzing the peak values ​​in the blood volume pulse signal.

[0126] Specifically, the blood volume pulse signal within a preset time period can be represented by a pulse waveform diagram, through which the changes in the heart rate blood volume signal within the preset time period can be observed.

[0127] Optionally, the terminal device can acquire the peak values ​​in the blood volume pulse signal, and then perform arithmetic operations, analysis, comparison, feature extraction, and other processing on all the peak values ​​in the blood volume pulse signal to obtain the user's heart rate characteristic information.

[0128] Among them, such as Figure 7 As shown, the step in S223 above, which determines the user's heart rate characteristic information through the peak value in the blood volume pulse signal, may include:

[0129] S2231, determine the heartbeat blood volume signal in the blood volume pulse signal according to the blood volume pulse signal and the preset blood volume threshold.

[0130] Specifically, the blood volume pulse signal in the preset time period has a direct current signal and an alternating current signal. The blood volume pulse signal in one or more sub-time periods in the preset time period is a direct current signal, and the blood volume pulse signal in another one or more sub-time periods in the preset time period is an alternating current signal. The sub-time periods corresponding to the direct current signal and the alternating current signal are different.

[0131] It can be understood that the terminal device can compare the blood volume pulse signal with the preset blood volume threshold, and determine the heartbeat blood volume signal in the blood volume pulse signal according to the comparison result. Optionally, the comparison result can be that the blood volume pulse signal is greater than the preset blood volume threshold, the blood volume pulse signal is equal to the preset blood volume threshold, or the blood volume pulse signal is less than the preset blood volume threshold.

[0132] It can also be understood that the alternating current signal in the blood volume pulse signal in the preset time period is a signal generated by the change of the heartbeat blood volume, and the direct current signal in the blood volume pulse signal in the preset time period corresponds to the heartbeat blood volume that does not change. Therefore, the heartbeat blood volume signal can be obtained from the blood volume pulse signal in the preset time period.

[0133] S2232, obtain the number of peaks in the heartbeat blood volume signal.

[0134] Specifically, the terminal device can obtain the peaks in the heartbeat blood volume signal, and then obtain the number of peaks in the heartbeat blood volume signal. Optionally, the number of peaks in the heartbeat blood volume signal can be greater than 1.

[0135] S2233, determine the heart rate feature information of the user according to the number of peaks in the heartbeat blood volume signal and the length of the preset time period.

[0136] It should be noted that the terminal device can perform arithmetic operation on the number of peaks in the heartbeat blood volume signal and the length of the preset time period to obtain the heart rate feature information of the user.

[0137] In this embodiment, the terminal device can obtain the length of the sub-time period corresponding to the direct current signal in the blood volume pulse signal in the preset time period, and then subtract the length of the sub-time period corresponding to the direct current signal from the length of the preset time period to obtain the length of the heartbeat blood volume signal. Further, by dividing the number of peaks in the heartbeat blood volume signal by the length of the heartbeat blood volume signal, the heart rate feature information of the user is obtained.

[0138] The information early warning method can obtain heart rate feature information of the user, and then determine a physiological emotion prediction result according to the eye feature information and the heart rate feature information, and comprehensively consider the area emotion prediction result and the physiological emotion prediction result, so as to accurately determine whether the current operation of the user is a suspected fraud behavior, and further output suspected fraud early warning information when it is determined that the current operation of the user is a suspected fraud behavior, so that the staff can timely check the current operation of the user before the user transfers money to the other party, and when the checking result is a fraud behavior, the user can be informed to stop the money transfer behavior in time, the user can be prevented from being deceived, the safety of the user is ensured, and the economic loss of the user caused by the fraud behavior is greatly reduced.

[0139] In order to comprehensively consider the eye feature information and the heart rate feature information and obtain more accurate physiological emotion prediction results, different weights need to be assigned to different feature information, so as to calculate the physiological emotion prediction result of the user according to the weights. Therefore, in an embodiment, as shown in Figure 8 the step of determining the physiological emotion prediction result of the user according to the feature information in S200 can include:

[0140] S230, respectively assigning weights to the feature information at different positions in the face.

[0141] Specifically, the terminal device can use entropy weight method, AHP hierarchical analysis method, principal component method and / or priority graph method, etc. to respectively assign weights to the feature information at different positions in the face.

[0142] In this embodiment, the ReliefF algorithm can be used to respectively assign weights to the feature information at different positions in the face, that is, to obtain the weight of the blink frequency and the weight of the heart rate feature information.

[0143] S240, inputting the feature information at different positions in the face and the weights of the feature information into a classifier for classification processing to obtain the physiological emotion prediction result of the user.

[0144] It can be understood that the terminal device can input the blink frequency, the heart rate feature information, the weight of the blink frequency and the weight of the heart rate feature information of the user into the classifier, and the classifier uses K-nearest neighbor, Bayesian, linear regression, logistic regression, decision tree, random forest, neural network, etc. algorithm for classification processing to obtain the physiological emotion prediction result of the user. Optionally, the above classifier can be a naive Bayesian classifier, a selection tree classifier, a neural network classifier, etc.

[0145] In this embodiment, the above classifier can be a decision tree classifier, which is a pre-trained classifier. In the training process, the decision tree classifier can use pre-pruning and post-pruning methods to prevent overfitting of the decision tree classifier.

[0146] The information early warning method can obtain a physiological emotion prediction result, further comprehensively consider the area emotion prediction result and the physiological emotion prediction result, and accurately determine whether the current operation of the user is a suspected fraud behavior, and further output a suspected fraud early warning information when it is determined that the current operation of the user is a suspected fraud behavior, so that the staff can timely check the current operation of the user before the user transfers money to the other party, and the user can be informed to stop the money transfer behavior in time when the checking result is a fraud behavior, the user can be prevented from being deceived, the safety of the user is ensured, and the economic loss of the user caused by the fraud behavior is greatly reduced.

[0147] Continuing to refer to Figure 1 Another embodiment provides a specific structure diagram of an information early warning system, as shown in the figure. The information early warning system comprises an image acquisition device and a terminal device.

[0148] The image acquisition device is configured to acquire a face image of the user.

[0149] The terminal device is configured to execute the above Figures 2-8 The method in any one of the corresponding embodiments.

[0150] Specifically, the terminal device can input the multiple face images of the user acquired by the image acquisition device within a preset time period into an expression prediction model to obtain a facial emotion prediction result of the user, analyze feature information of different positions in the face within the preset time period according to the multiple face images, and determine a physiological emotion prediction result of the user according to the feature information. The current operation of the user is determined to be a suspected fraud behavior through the facial emotion prediction result and the physiological emotion prediction result. If the current operation of the user is a suspected fraud behavior, a suspected fraud early warning information is outputted to indicate that the current operation of the user is checked.

[0151] The information early warning system provided in this embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0152] It should be understood that, although Figures 2-8 The steps in the flowchart are displayed in sequence according to the arrows, but these steps are not necessarily executed in sequence according to the arrows. Unless otherwise stated in this article, the execution of these steps has no strict sequence limitation, and these steps can be executed in other orders. Moreover, Figures 2-8 At least part of the steps in the flowchart can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or steps or stages in other steps.

[0153] In one embodiment, as shown in Figure 9 An information early warning device is provided, comprising: a prediction module 11, an analysis module 12, a determination module 13 and an early warning information acquisition module 14, wherein:

[0154] The prediction module 11 is configured to input a plurality of frames of face images of a user in a preset time period into an expression prediction model to obtain a facial emotion prediction result of the user.

[0155] The analysis module 12 is configured to analyze feature information of different positions in the face in the preset time period according to the plurality of frames of face images, and determine a physiological emotion prediction result of the user according to the feature information.

[0156] The determination module 13 is configured to determine whether a current operation of the user is a suspected fraud behavior by the facial emotion prediction result and the physiological emotion prediction result.

[0157] The early warning information acquisition module 14 is configured to output a suspected fraud early warning information when the result of the determination module is that the current operation of the user is a suspected fraud behavior; the suspected fraud early warning information is used to instruct to check the current operation of the user.

[0158] The information early warning device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0159] In one embodiment, the different positions in the face include an eye region and a forehead region in the face; the analysis module includes an eye region detection unit and a forehead region detection unit, wherein:

[0160] The eye region detection unit is configured to detect the eye region in the plurality of frames of face images to obtain eye feature information of the eye region of the user in the preset time period.

[0161] The forehead region detection unit is configured to detect the forehead region in the plurality of frames of face images to obtain skin data information of the forehead region of the user in the preset time period, and determine heart rate feature information of the user according to the skin data information.

[0162] The information early warning device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0163] In one embodiment, the eye feature information includes a blink frequency; the eye region detection unit includes an eye state acquisition subunit, a total number of blinks determination subunit and a blink frequency determination subunit, wherein:

[0164] An eye state obtaining sub-unit is configured to obtain an eye state of the user in each frame of face image according to the eye region in the plurality of frames of face image;

[0165] A total number of blinking times determining sub-unit is configured to determine a total number of blinking times of the user in the preset time period according to the eye state of the user in each frame of face image; wherein the eye state in the closing state indicates that one blink occurs;

[0166] A blinking frequency determining sub-unit is configured to determine a blinking frequency of the user according to the total number of blinking times and the length of the preset time period.

[0167] The information warning device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described herein.

[0168] In one of the embodiments, the eye state obtaining sub-unit is specifically configured to determine the eye position according to the eye region in the face image for each frame of face image; determine the size information of the eyes of the user according to the eye position, the size information including an aspect ratio; and if the aspect ratio is 0, determine that the eye state of the user in the face image is in the closing state.

[0169] The information warning device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described herein.

[0170] In one of the embodiments, the skin data information includes a blood volume pulse signal; and the forehead region detecting unit includes a forehead position determining sub-unit, a signal determining sub-unit and a heart rate information determining sub-unit, wherein:

[0171] The forehead position determining sub-unit is configured to determine the forehead position of the user according to the forehead region in the plurality of frames of face image;

[0172] The signal determining sub-unit is configured to obtain the blood volume pulse signal of the forehead position of the user in the preset time period through non-contact sensor measurement;

[0173] The heart rate information determining sub-unit is configured to determine the heart rate feature information of the user through the peak value in the blood volume pulse signal.

[0174] The information warning device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described herein.

[0175] In one of the embodiments, the heart rate information determining sub-unit is specifically configured to determine a heartbeat blood volume signal in the blood volume pulse signal according to the blood volume pulse signal and a preset blood volume threshold; obtain the number of peak values in the heartbeat blood volume signal; and determine the heart rate feature information of the user according to the number of peak values in the heartbeat blood volume signal and the length of the preset time period.

[0176] The information early warning device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0177] In one of the embodiments, the analysis module 12 further comprises a weight distribution unit and a classification processing unit, wherein:

[0178] The weight distribution unit is configured to respectively distribute weights to the feature information at different positions in the face.

[0179] The classification processing unit is configured to input the feature information at different positions in the face and the weights of the feature information to the classifier for classification processing to obtain the physiological emotion prediction result of the user.

[0180] The information early warning device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0181] In one of the embodiments, the determination module 13 is specifically configured to perform weighted summation on the facial emotion prediction result and the physiological emotion prediction result to obtain an emotion prediction result, and determine that the current operation of the user is a suspected fraud behavior if the emotion prediction result is greater than an abnormal emotion threshold value, and otherwise, determine that the current operation of the user is not a suspected fraud behavior.

[0182] The information early warning device provided in the embodiment can execute the method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0183] The specific limitations of the information early warning device can refer to the limitations of the information early warning method described above, which will not be described here. Each module in the information early warning device can be realized by software, hardware and their combination in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the terminal device in hardware form, or can be stored in the memory in the terminal device in the form of software, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0184] In one embodiment, a terminal device is provided, which can be a server, and the internal structure diagram thereof can be as shown in Figure 10As shown in the figure. The terminal device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the terminal device is used to store the face image of the user. The network interface of the terminal device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement an information early warning method.

[0185] Those skilled in the art can understand that, Figure 10 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0186] In one embodiment, a terminal device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps:

[0187] Inputting a plurality of face images of the user in a preset time period into an expression prediction model to obtain a facial emotion prediction result of the user;

[0188] According to the plurality of face images, analyzing feature information of different positions in the face in the preset time period, and determining a physiological emotion prediction result of the user according to the feature information;

[0189] Determining whether the current operation of the user is a suspected fraud behavior through the facial emotion prediction result and the physiological emotion prediction result;

[0190] If the current operation of the user is a suspected fraud behavior, outputting a suspected fraud early warning information; the suspected fraud early warning information is used to indicate to check the current operation of the user.

[0191] In one embodiment, a readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0192] Inputting a plurality of face images of the user in a preset time period into an expression prediction model to obtain a facial emotion prediction result of the user;

[0193] According to the plurality of face images, analyzing feature information of different positions in the face in the preset time period, and determining a physiological emotion prediction result of the user according to the feature information;

[0194] Determine, through the facial emotion prediction result and the physiological emotion prediction result, whether the current operation of the user is a suspected fraud behavior;

[0195] If the current operation of the user is a suspected fraud behavior, output a suspected fraud warning information; the suspected fraud warning information is used to indicate to check the current operation of the user.

[0196] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the following steps:

[0197] Input the multiple frames of face images of the user in a preset time period into an expression prediction model to obtain a facial emotion prediction result of the user;

[0198] According to the multiple frames of face images, analyze feature information of different positions in the face in the preset time period, and determine a physiological emotion prediction result of the user according to the feature information;

[0199] Determine, through the facial emotion prediction result and the physiological emotion prediction result, whether the current operation of the user is a suspected fraud behavior;

[0200] If the current operation of the user is a suspected fraud behavior, output a suspected fraud warning information; the suspected fraud warning information is used to indicate to check the current operation of the user.

[0201] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0202] Any combination of the technical features in the above embodiments can be made, and for the sake of brevity, not all possible combinations are described above, however, as long as the combination of the technical features does not exist in contradiction, it shall be considered within the scope of the present disclosure.

[0203] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An information early warning method, characterized in that, The method includes: Multiple frames of facial images of the user within a preset time period are input into the expression prediction model to obtain the facial emotion prediction result of the user. The facial emotion prediction result includes multiple emotions such as happiness, anger, anxiety, calmness, sadness, confusion, excitement, fear and grief, as well as the prediction percentage of each emotion. Based on the multi-frame facial images, the blink frequency of the eye region in the face within the preset time period is analyzed. The ReliefF algorithm is used to assign weights to the blink frequency and heart rate feature information respectively. The user's blink frequency, heart rate feature information, the weight of blink frequency, and the weight of heart rate feature information are all input into the classifier for classification processing to obtain the user's physiological emotion prediction result. The heart rate feature information is determined based on the skin data information of the user's forehead region within the preset time period. The emotion prediction result is obtained by weighting and summing the facial emotion prediction results, the physiological emotion prediction results, the weights of various emotions in the facial emotion prediction results, and the weights of various emotions in the physiological emotion prediction results. If the emotion prediction result is greater than the preset abnormal emotion threshold, then the user's current operation is determined to be a suspected fraudulent behavior, and a suspected fraud warning message is output; the suspected fraud warning message is used to instruct the user's current operation to be verified.

2. The method according to claim 1, characterized in that, The method further includes: The eye region in the multi-frame face images is detected to obtain the eye feature information of the user's eye region within the preset time period; The forehead region in the multi-frame face images is detected to obtain skin data information of the user's forehead region within the preset time period, and the user's heart rate feature information is determined based on the skin data information.

3. The method according to claim 2, characterized in that, Determining the user's blink frequency includes: Based on the eye region in the multi-frame face images, obtain the user's eye state in each frame of face images; Based on the user's eye state in each frame of the face image, determine the total number of blinks of the user within the preset time period; wherein, a blink occurs when the eye state is closed. The user's blinking frequency is determined based on the total number of blinks and the duration of the preset time period.

4. The method according to claim 3, characterized in that, The step of obtaining the user's eye state in each frame of the face image based on the eye region in the multi-frame face image includes: For each frame of the face image, the position of the eyes is determined based on the eye region in the face image; The size information of the user's eyes is determined based on the eye position, and the size information includes the aspect ratio; If the aspect ratio is 0, then the user's eyes in the face image are determined to be in a closed state.

5. The method according to any one of claims 2-4, characterized in that, The skin data information includes blood volume and pulse signals; The step of detecting the forehead region in the multi-frame facial images, obtaining skin data information of the user's forehead region within the preset time period, and determining the user's heart rate feature information based on the skin data information includes: The position of the user's forehead is determined based on the forehead region in the multi-frame facial images; The blood volume pulse signal of the user's forehead position within the preset time period is obtained by non-contact sensor measurement. The user's heart rate characteristics are determined by the peak value in the blood volume pulse signal.

6. The method according to claim 5, characterized in that, Determining the user's heart rate characteristic information by analyzing the peak values ​​in the blood volume pulse signal includes: Based on the blood volume pulse signal and the preset blood volume threshold, the cardiac blood volume signal in the blood volume pulse signal is determined; Obtain the number of peak values ​​in the cardiac blood volume signal; The user's heart rate characteristic information is determined based on the number of peaks in the heart rate blood volume signal and the duration of the preset time period.

7. An information early warning system, characterized in that, The system includes: an image acquisition device and a terminal device; The image acquisition device is used to acquire the user's facial image; The terminal device is used to perform the steps of the method described in any one of claims 1-6.

8. An information early warning device, characterized in that, The device includes: The prediction module is used to input multiple frames of facial images of the user within a preset time period into the expression prediction model to obtain the facial emotion prediction result of the user. The facial emotion prediction result is a variety of emotions such as happiness, anger, anxiety, calmness, sadness, confusion, excitement, fear and grief, as well as the prediction percentage of each emotion. The analysis module is used to analyze the blink frequency of the eye region in the face within the preset time period based on the multi-frame face images, and to assign weights to the blink frequency and heart rate feature information using the ReliefF algorithm. The user's blink frequency, heart rate feature information, blink frequency weight, and heart rate feature information weight are all input into the classifier for classification processing to obtain the user's physiological emotion prediction result. The heart rate feature information is determined based on the skin data information of the user's forehead region within the preset time period. The determination module is used to perform a weighted summation based on the facial emotion prediction results, the physiological emotion prediction results, the weights of various emotions in the facial emotion prediction results, and the weights of various emotions in the physiological emotion prediction results to obtain the emotion prediction results. The warning information acquisition module is used to determine that the user's current operation is a suspected fraudulent behavior if the emotion prediction result is greater than a preset abnormal emotion threshold, and then outputs a suspected fraud warning information; the suspected fraud warning information is used to instruct the user's current operation to be verified.

9. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-6.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

11. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-6.

Images