Visual risk control method and device based on non-invasive respiratory signal detection

Abstract

The invention relates to a visual risk control method and device based on non-invasive detection of respiratory signals. The method comprises the steps of collecting image data of an upper body area of a user in a remote authentication process of the user; capturing a breathing signal of the user from the image data; constructing a breathing curve of the user based on the breathing signal; extracting respiration characteristics of the user from the respiration curve; the breathing characteristics comprise at least one of breathing frequency and breathing amplitude; detecting whether the user has an abnormal state or not according to the breathing frequency and / or the breathing amplitude; and if the user has the abnormal state, triggering a verification measure for the user. According to the scheme provided by the invention, the abnormal state of the user can be effectively detected, the detection precision and robustness are improved, and the risk control capability is enhanced.

Description

Technical Field

[0001] The present application relates to the field of remote wind control technology, and in particular to a visual wind control method and device based on non-invasive detection of respiratory signals. Background Art

[0002] In the remote video authentication process, identifying the authenticity of the user's identity and abnormal behavior (such as forged identity, fraudulent intention, etc.) is one of the key challenges.

[0003] The related technologies mainly rely on the analysis of visual behavioral features or voice features. However, visual behavioral features are sensitive to light, user facial clarity and environmental interference, and cannot capture changes in the user's intrinsic physiological signals. Voice features are significantly affected by background noise and are ineffective for non-voice scenarios. Therefore, the related technologies have limited ability to detect abnormal user conditions. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a visual risk control method and device based on non-invasive detection of respiratory signals, which can effectively detect the abnormal state of the user, improve detection accuracy and robustness, and enhance risk control capabilities.

[0005] The first aspect of the present application provides a visual wind control method based on non-invasive detection of respiratory signals, the method comprising: During the remote authentication process of the user, image data of the upper body area of ​​the user is collected through a camera; capturing a breathing signal of the user from the image data; Based on the breathing signal, construct a breathing curve of the user; Extracting a breathing feature of the user from the breathing curve; the breathing feature includes at least one of a breathing frequency and a breathing amplitude; Detecting whether the user is in an abnormal state according to the breathing frequency and / or the breathing amplitude; If the user is in an abnormal state, verification measures for the user are triggered.

[0006] In one embodiment, detecting whether the user is in an abnormal state according to the breathing frequency and / or the breathing amplitude includes: Determining whether the respiratory frequency is within a set respiratory frequency range of a standard physiological model, and determining whether the respiratory amplitude is within a set respiratory amplitude range of a standard physiological model; If the breathing frequency is not within the set breathing frequency range, and / or the breathing amplitude is not within the set breathing amplitude range, it is determined that the user is in an abnormal state.

[0007] In one embodiment, the camera is a common camera; and capturing the user's breathing signal from the image data includes: capturing the upper body displacement of the user from the image data collected by the ordinary camera; the upper body displacement is any one of chest displacement and neck displacement; Determine the pixel brightness of the upper body area of ​​the user in the image data, and determine the optical flow change type of the image data; If the optical flow change type is caused by breathing, and the pixel brightness changes, it is determined that the upper body displacement is caused by breathing, and the upper body displacement is used as the user's breathing signal.

[0008] In another embodiment, the camera is a thermal imaging camera; and capturing the user's breathing signal from the image data comprises: capturing the gas temperature of the nasal area or the mouth area of ​​the user from the image data collected by the thermal imaging camera; If the gas temperature changes, the change in the gas temperature is used as the user's breathing signal.

[0009] In one embodiment, before capturing the user's breathing signal from the image data, the method further includes: An adaptive filtering algorithm is used to perform noise reduction processing on the image data.

[0010] In one embodiment, if the user is in an abnormal state, triggering verification measures for the user includes: If the user is in an abnormal state, identifying the abnormal behavior of the user through an abnormal behavior detection model; Feedback information is generated according to the abnormal behavior, and the feedback information is sent to a remote monitoring platform to trigger verification measures of the remote monitoring platform on the user.

[0011] In one embodiment, the method further comprises: If the user is in an abnormal state, optimization information is generated according to the abnormal state; the optimization information is used to remind the user to optimize the current posture and the current environment.

[0012] A second aspect of the present application provides a visual wind control device based on non-invasive detection of respiratory signals, the device comprising: An image data acquisition module, used to acquire image data of the upper body area of ​​the user during the remote authentication process of the user; A breathing signal capturing module, used for capturing the user's breathing signal from the image data; A breathing curve construction module, used to construct a breathing curve of the user based on the breathing signal; A breathing feature extraction module, used to extract the user's breathing feature from the breathing curve; the breathing feature includes at least one of a breathing frequency and a breathing amplitude; an abnormal state detection module, used for detecting whether the user is in an abnormal state according to the breathing frequency and / or the breathing amplitude; The verification measure triggering module is used to trigger verification measures for the user if the user is in an abnormal state.

[0013] A third aspect of the present application provides an electronic device, including: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0014] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0015] The technical solution provided by this application may have the following beneficial effects: The solution provided by the present application, in the process of remote authentication of the user, collects image data of the upper body area of ​​the user; captures the user's breathing signal from the image data; constructs the user's breathing curve based on the breathing signal; extracts the user's breathing characteristics from the breathing curve; the breathing characteristics include at least one of breathing frequency and breathing amplitude; detects whether the user is in an abnormal state according to the breathing frequency and / or breathing amplitude; if the user is in an abnormal state, triggers verification measures for the user. The present application does not rely on additional hardware equipment, and can capture the user's breathing signal only through a camera, thereby achieving convenient and natural physiological state monitoring, which is suitable for general remote authentication scenarios. The breathing signal is a subtle but significant physiological signal. Therefore, obtaining breathing characteristics such as breathing frequency and breathing amplitude from the breathing signal can effectively reflect whether the user is in a physiological state such as tension and anxiety, thereby effectively detecting the user's abnormal state, improving detection accuracy and robustness, and immediately triggering verification measures when an abnormal state is detected, thereby enhancing risk control capabilities.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail the exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0018] Figure 1 It is a flow chart of a visual wind control method based on non-invasive detection of respiratory signals shown in an embodiment of the present application; Figure 2 is another flow chart of a visual wind control method based on non-invasive detection of respiratory signals shown in an embodiment of the present application; Figure 3 is a schematic diagram of the structure of a visual wind control device based on non-invasive detection of respiratory signals shown in an embodiment of the present application; Figure 4 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0020] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0022] The related technologies mainly rely on the analysis of visual behavior features or voice features, and can also monitor the user's physiological signals to detect the user's abnormal state. However, the related technologies have limited ability to detect the user's abnormal state. The specific analysis is as follows: A. Analysis of visual behavior characteristics: Implementation method: Analyze the user's visual behavior characteristics based on technologies such as user facial expressions, eye tracking or posture detection.

[0023] There are disadvantages: it is sensitive to light, user's facial clarity and environmental interference, and cannot capture changes in the user's intrinsic physiological signals.

[0024] B. Analysis of speech features: Implementation method: Analyze the emotional fluctuations and tension signals in the user's voice.

[0025] Disadvantages: It is significantly affected by background noise and is ineffective in non-speech scenarios.

[0026] C. Traditional physiological signal monitoring: Implementation: Use wearable devices to measure physiological signals such as heart rate and body temperature.

[0027] Disadvantages: It requires additional hardware support, increases user costs and operational complexity, and is not suitable for common remote authentication scenarios.

[0028] In response to the above problems, the embodiments of the present application provide a visual risk control method based on non-invasive detection of respiratory signals. The method does not rely on additional hardware equipment and can capture the user's respiratory signals only through a camera, thereby achieving convenient and natural physiological state monitoring. The method is suitable for common remote authentication scenarios. The respiratory signal is a subtle but significant physiological signal. Therefore, respiratory characteristics such as respiratory frequency and respiratory amplitude are obtained from the respiratory signal, which can effectively reflect whether the user has physiological states such as tension and anxiety, thereby effectively detecting the user's abnormal state, improving detection accuracy and robustness, and immediately triggering verification measures when an abnormal state is detected, thereby enhancing risk control capabilities.

[0029] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0030] Figure 1 It is a flow chart of a visual wind control method based on non-invasive detection of respiratory signals shown in an embodiment of the present application.

[0031] See also Figure 1 The visual risk control method based on non-invasive detection of respiratory signals of the present application includes: S110, during the remote authentication process of the user, image data of the upper body area of ​​the user is collected through a camera.

[0032] When a user needs a video interview or a loan application, the embodiment of the present application can remotely authenticate the user, where remote authentication refers to the process of identity verification of a remote user through a network or other communication methods, and its main purpose is to ensure that the identity of the remote user is authentic and reliable and to prevent unauthorized access and operation.

[0033] During the remote authentication process, the embodiment of the present application can use a camera to collect image data of the user's upper body area in real time, wherein the upper body area may include at least one of the chest area, neck area, nasal area and mouth area.

[0034] S120, capturing a user's breathing signal from the image data.

[0035] As a key physiological signal, breathing signals contain rich emotional and status information. However, their potential in the field of risk control has not been fully explored. Therefore, there is still much room for improvement in the accuracy and robustness of detecting abnormal user states in related technologies. New detection methods are needed to enhance risk control capabilities.

[0036] In this regard, after obtaining image data of the upper body area of ​​the user, the embodiment of the present application can capture the user's breathing signal from the image data, so as to use the breathing signal to obtain the user's emotion and state information during the remote authentication process.

[0037] S130, constructing a breathing curve of the user based on the breathing signal.

[0038] The embodiment of the present application can construct a user's breathing curve by analyzing the timing of the breathing signal. Specifically, the breathing signal may include an exhalation signal and an inhalation signal. The embodiment of the present application can construct a user's breathing curve based on the exhalation signal and the inhalation signal of the user in a continuous period of time, wherein the horizontal axis of the breathing curve is time, and the vertical axis is the user's breathing state, and the breathing state may include two states of exhalation and inhalation, and the breathing curve is a waveform curve.

[0039] S140, extracting a user's breathing characteristics from the breathing curve; the breathing characteristics include at least one of a breathing frequency and a breathing amplitude.

[0040] After constructing the user's breathing curve, the embodiment of the present application can extract key breathing features from the breathing curve, and the breathing features may include at least one of breathing frequency and breathing amplitude. Among them, breathing frequency can also be called breathing rate, which refers to the number of times the user breathes per unit time, so if the breathing curve is tighter, the extracted breathing frequency is larger, otherwise it is smaller. Breathing amplitude can also be called breathing intensity, which refers to the amplitude of expansion and contraction of the chest or abdomen during the user's breathing process. The breathing amplitude can be reflected by the difference between the peak and the trough of the breathing curve, so if the difference between the peak and the trough of the breathing curve is larger, the extracted breathing amplitude is larger, otherwise it is smaller.

[0041] S150, detecting whether the user is in an abnormal state according to the breathing frequency and / or breathing amplitude.

[0042] In the embodiment of the present application, a first frequency threshold, a second frequency threshold, a first amplitude threshold, and a second amplitude threshold can be pre-set, wherein the first frequency threshold is greater than the second frequency threshold, and the first amplitude threshold is greater than the second amplitude threshold.

[0043] In one example, if the breathing frequency is greater than the first frequency threshold, it means that the user is breathing rapidly, which can reflect that the user is in a physiological state such as tension, anxiety, etc., and thus it can be detected that the user is in an abnormal state.

[0044] In another example, if the breathing frequency is less than the second frequency threshold, it means that the user is bradypnea, which can reflect that the user is in a physiological state such as tension, anxiety, etc., and thus it can be detected that the user is in an abnormal state.

[0045] In another example, if the breathing amplitude is greater than the first amplitude threshold, it means that the user is breathing strongly, which can reflect that the user is in a physiological state such as tension, anxiety, etc., and thus it can be detected that the user is in an abnormal state.

[0046] In another example, if the breathing amplitude is less than the second amplitude threshold, it means that the user is breathing intermittently, which can reflect that the user is in a physiological state such as tension, anxiety, etc., and thus it can be detected that the user is in an abnormal state.

[0047] S160: If the user is in an abnormal state, a verification measure for the user is triggered.

[0048] When an abnormal state of a user is detected, the embodiment of the present application can immediately trigger verification measures for the user, such as immediately providing analysis results to the remote monitoring platform in real time for risk control or further verification, thereby enhancing risk control capabilities.

[0049] As can be seen from this example, the solution provided by this application, during the remote authentication process of the user, collects image data of the upper body area of ​​the user; captures the user's breathing signal from the image data; constructs the user's breathing curve based on the breathing signal; extracts the user's breathing characteristics from the breathing curve; the breathing characteristics include at least one of the breathing frequency and breathing amplitude; detects whether the user is in an abnormal state according to the breathing frequency and / or breathing amplitude; if the user is in an abnormal state, triggers verification measures for the user. This application does not rely on additional hardware equipment, and can capture the user's breathing signal only through a camera, thereby achieving convenient and natural physiological state monitoring, which is suitable for general remote authentication scenarios. The breathing signal is a subtle but significant physiological signal. Therefore, obtaining breathing characteristics such as breathing frequency and breathing amplitude from the breathing signal can effectively reflect whether the user is in a physiological state such as tension and anxiety, thereby effectively detecting the user's abnormal state, improving detection accuracy and robustness, and immediately triggering verification measures when an abnormal state is detected, thereby enhancing risk control capabilities.

[0050] Figure 2 It is another flow chart of the visual wind control method based on non-invasive detection of respiratory signals shown in the present application.

[0051] See also Figure 2 The visual risk control method based on non-invasive detection of respiratory signals of the present application includes: S210, during the remote authentication process of the user, image data of the upper body area of ​​the user is collected through a camera.

[0052] When a user needs a video interview or loan application, the embodiment of the present application can remotely authenticate the user. During the remote authentication process, the embodiment of the present application can use a camera to collect image data of the upper body area of ​​the user in real time, wherein the camera can include at least one of a normal camera and a thermal imaging camera.

[0053] It should be noted that thermal imaging cameras mainly use the infrared characteristics of objects to capture and detect infrared radiation from objects. In thermal imaging cameras, infrared radiation is detected by tiny probes in the sensor and converted into temperature data. By processing these data, thermal images of objects can be obtained. Ordinary cameras mainly rely on optical imaging to obtain images, and the images they can obtain are mainly reflection images of targets under visible light.

[0054] S220, performing noise reduction processing on the image data using an adaptive filtering algorithm.

[0055] In order to ensure the accuracy of the respiratory signal, the embodiment of the present application may use an adaptive filtering algorithm to perform noise reduction processing on environmental interference (such as light changes or background motion) in the image data.

[0056] In a specific implementation, the embodiment of the present application can grayscale the image data, then perform variance calculation on the pixel values ​​around each pixel in the image data, and then determine the local filter size and weight of each pixel based on the calculated variance value, so as to apply the local filter to each pixel in the image data and calculate the filtered pixel value, thereby obtaining the image data after noise reduction processing.

[0057] S230, capturing a user's breathing signal from the image data.

[0058] After obtaining the image data after noise reduction processing, the embodiment of the present application can capture the user's breathing signal from the image data after noise reduction processing, so as to evaluate the user's physiological state by analyzing the user's breathing signal.

[0059] In one embodiment, capturing a user's breathing signal from image data may include: The displacement of the user's upper body is captured from image data collected by an ordinary camera; the upper body displacement is any one of chest displacement and neck displacement; the pixel brightness of the upper body area of ​​the user in the image data is determined, and the optical flow change type of the image data is determined; if the optical flow change type belongs to the optical flow change type caused by breathing, and the pixel brightness changes, it is determined that the upper body displacement belongs to the micro-motion caused by breathing, and the upper body displacement is used as the user's breathing signal.

[0060] If the image data is acquired by an ordinary camera, the embodiment of the present application can use the user's upper body displacement to obtain the user's breathing signal. Specifically, the image data may include multiple consecutive image frames. If the image data is acquired by an ordinary camera, the embodiment of the present application can calculate the position difference between a certain part in the user's upper body area in two adjacent image frames, and then add all the position differences to capture the user's upper body displacement. The upper body area may include at least one of the chest area and the neck area, so the upper body displacement may be a chest displacement or a neck displacement.

[0061] The displacement of the upper body may be micro-movement caused by breathing, or it may be micro-movement caused by other factors (such as the user's own movement). Therefore, the embodiment of the present application can combine pixel brightness changes and optical flow analysis technology for further judgment.

[0062] In a specific implementation, taking the middle part of the chest area of ​​the user as an example, assuming that the image data includes image frame A, image frame B and image frame C, the embodiment of the present application calculates the position difference △ of the middle part in image frame A and image frame B. AB , and calculate the position difference △ between the middle part in image frame B and image frame CBC , the chest displacement S of the user is calculated based on the two position differences. In order to determine whether the upper body displacement S is caused by breathing, the embodiment of the present application can determine the pixel brightness of the middle part in image frame A, image frame B and image frame C respectively. In addition, the embodiment of the present application can input image frame A, image frame B and image frame C into a pre-trained classification model in pairs, and the classification model determines the optical flow change type f from image frame A to image frame B based on the overall optical flow of image frame A and the overall optical flow of image frame B. AB , and based on the overall optical flow of image frame B and the overall optical flow of image frame C, determine the optical flow change type f from image frame B to image frame C BC .

[0063] In one example, if the optical flow change type f AB It belongs to the type of optical flow change caused by breathing, and the optical flow change type f BC If the optical flow change is also caused by breathing, the embodiment of the present application can further judge based on the change of pixel brightness. Specifically, if the pixel brightness of the middle part in image frame A is different from that in image frame B, and the pixel brightness of the middle part in image frame B is also different from that in image frame C, it can be determined that the upper body displacement S is caused by breathing, so the upper body displacement S can be used as the user's breathing signal.

[0064] In another example, if the optical flow change type f AB It belongs to the type of optical flow change caused by breathing, while the optical flow change type f BC If the optical flow change is not caused by breathing, the present embodiment can discard the image frame C and the position difference △ BC , at this time, the user's chest displacement S is the position difference △ AB Furthermore, if the pixel brightness of the middle part in the image frame A is different from that in the image frame B, the upper body displacement S (i.e., the position difference △ AB ) is a micro-motion caused by breathing, so the upper body can be displaced by S (i.e., the position difference △ AB ) as the user's breathing signal.

[0065] In yet another example, if the optical flow change type f AB It belongs to the type of optical flow change caused by breathing, and the optical flow change type f BCIf the optical flow change is also caused by breathing, the embodiment of the present application can further judge based on the change of pixel brightness. Specifically, if the pixel brightness of the middle part in image frame A is different from that in image frame B, and the pixel brightness of the middle part in image frame B is the same as that in image frame C, the embodiment of the present application can discard image frame C and the position difference △ BC , at this time, the user's chest displacement S is the position difference △ AB , so that the upper body displacement S (i.e., the position difference △ AB ) is a micro-motion caused by breathing, so the upper body can be displaced by S (i.e., the position difference △ AB ) as the user's breathing signal.

[0066] In another embodiment, capturing a user's breathing signal from image data may include: The gas temperature of the user's nasal area or mouth area is captured from the image data collected by the thermal imaging camera; if the gas temperature changes, the change in gas temperature is used as the user's breathing signal.

[0067] If the image data is acquired by a thermal imaging camera, the embodiment of the present application can use the change between the temperature of the gas exhaled by the user and the temperature of the gas inhaled by the user to obtain the user's breathing signal. Specifically, the imaging effect of the thermal imaging camera is mainly manifested as a thermal image of the target, which can well reflect the change in the surface temperature of the target. Therefore, the embodiment of the present application can capture the gas temperature of the user's nasal area or mouth area from the image data acquired by the thermal imaging camera.

[0068] Generally, the temperature of the gas exhaled by the user is higher than the temperature of the gas inhaled by the user. Therefore, if the gas temperature changes, it can indicate that the user has completed a breath. Therefore, the embodiment of the present application can use the change in gas temperature as the user's breathing signal.

[0069] It should be noted that the method of capturing breathing signals using a thermal imaging camera does not require detecting the user's upper body displacement, and is therefore very suitable for scenes with complex backgrounds or user movement.

[0070] S240, constructing a breathing curve of the user based on the breathing signal.

[0071] The breathing signal may include an exhalation signal and an inhalation signal. The embodiment of the present application may construct a user's breathing curve based on the user's exhalation signal and inhalation signal over a continuous period of time, wherein the horizontal axis of the breathing curve is time, and the vertical axis is the user's breathing state. The breathing state may include both exhalation and inhalation states, and the breathing curve is a waveform curve.

[0072] S250, extracting a user's breathing feature from the breathing curve; the breathing feature includes at least one of a breathing frequency and a breathing amplitude.

[0073] After constructing the user's breathing curve, the embodiment of the present application can extract key breathing features from the breathing curve, and the breathing features can include at least one of breathing frequency and breathing amplitude. If the breathing curve is tighter, the extracted breathing frequency is larger, and vice versa. If the difference between the peak and the trough of the breathing curve is larger, the extracted breathing amplitude is larger, and vice versa.

[0074] S260: Detect whether the user is in an abnormal state based on the breathing frequency and / or breathing amplitude.

[0075] In one example, if the breathing frequency is greater than the first frequency threshold, it means that the user is breathing rapidly, which can reflect that the user is in a physiological state such as tension, anxiety, etc., and thus it can be detected that the user is in an abnormal state.

[0076] In another example, if the breathing frequency is less than the second frequency threshold, it means that the user is bradypnea, which can reflect that the user is in a physiological state such as tension, anxiety, etc., and thus it can be detected that the user is in an abnormal state.

[0077] In another example, if the breathing amplitude is greater than the first amplitude threshold, it means that the user is breathing strongly, which can reflect that the user is in a physiological state such as tension, anxiety, etc., and thus it can be detected that the user is in an abnormal state.

[0078] In another example, if the breathing amplitude is less than the second amplitude threshold, it means that the user is breathing intermittently, which can reflect that the user is in a physiological state such as tension, anxiety, etc., and thus it can be detected that the user is in an abnormal state.

[0079] The first frequency threshold is greater than the second frequency threshold, and the first amplitude threshold is greater than the second amplitude threshold.

[0080] In one embodiment, detecting whether the user is in an abnormal state according to the breathing frequency and / or the breathing amplitude may include: Determine whether the breathing frequency is within the set breathing frequency range of the standard physiological model, and determine whether the breathing amplitude is within the set breathing amplitude range of the standard physiological model; if the breathing frequency is not within the set breathing frequency range, and / or the breathing amplitude is not within the set breathing amplitude range, it is determined that the user is in an abnormal state.

[0081] The embodiment of the present application can determine whether the user is in an abnormal state by combining a pre-trained deep learning model (such as a decision tree, a neural network) with a standard breathing pattern. Specifically, the embodiment of the present application can compare the breathing frequency extracted in real time with the set breathing frequency range of the standard physiological model through the deep learning model, and compare the breathing amplitude extracted in real time with the set breathing amplitude range of the standard physiological model. The deep learning model identifies whether the user is breathing abnormally based on these two comparison results, thereby determining whether the user is in an abnormal state.

[0082] It should be noted that the standard breathing pattern may refer to the breathing frequency and breathing amplitude of the user in a calm breathing state, which is characterized by moderate frequency, appropriate depth, stable rhythm and natural smoothness.

[0083] It should be noted that the upper limit value of the respiratory frequency range is set to the first frequency threshold, the lower limit value of the respiratory frequency range is set to the second frequency threshold, the upper limit value of the respiratory amplitude range is set to the first amplitude threshold, and the lower limit value of the respiratory amplitude range is set to the second amplitude threshold.

[0084] In one example, if the breathing frequency is not within the set breathing frequency range, it means that the user is breathing rapidly or bradypnea, and the deep learning model can identify that the user's breathing is abnormal, thereby determining that the user is in an abnormal state.

[0085] In another example, if the breathing amplitude is not within the set breathing amplitude range, it means that the user is breathing strongly or breathing intermittently, then the deep learning model can identify that the user's breathing is abnormal, thereby determining that the user is in an abnormal state.

[0086] In another example, in order to improve the recognition accuracy, the embodiment of the present application can determine that the user is in an abnormal state only when the breathing frequency and breathing amplitude meet the corresponding conditions. Specifically, if the breathing frequency is not within the set breathing frequency range, and the breathing amplitude is not within the set breathing amplitude range, it means that the user is breathing shallowly and fast or breathing shallowly and slowly or breathing deeply and fast or breathing deeply and slowly, then the deep learning model can identify that the user's breathing is abnormal, thereby determining that the user is in an abnormal state.

[0087] S270: If the user is in an abnormal state, identify the user's abnormal behavior through an abnormal behavior detection model.

[0088] If the deep learning model identifies that the user's breathing is abnormal, the embodiment of the present application can further combine a pre-trained abnormal behavior detection model to identify the user's abnormal behavior, wherein the abnormal behavior may include at least one of identity forgery and fraudulent intent.

[0089] In a specific implementation, the abnormal behavior detection model can identify whether the user has unnatural movements or unnatural speech. Among them, unnatural movements can be, for example, the user frequently scratching his head, or the user's eyes flickering, or the user whispering with the person next to him. Unnatural speech can be, for example, the user stumbling when speaking, or the user communicating with the person next to him about irrelevant content. When the user has unnatural movements or unnatural speech, the abnormal behavior detection model can identify the user's abnormal behavior.

[0090] S280, generating feedback information based on the abnormal behavior, and sending the feedback information to the remote monitoring platform to trigger the remote monitoring platform's verification measures on the user.

[0091] After identifying the user's abnormal behavior, the embodiment of the present application can generate feedback information based on the abnormal behavior, so that the feedback information can carry the abnormal behavior (forged identity / fraud intention), and then immediately send the feedback information to the remote monitoring platform to trigger the remote monitoring platform's verification measures on the user, thereby enhancing risk control capabilities and improving risk control efficiency.

[0092] In one embodiment, the method may further include: If the user is in an abnormal state, optimization information is generated based on the abnormal state; the optimization information is used to remind the user to optimize the current posture and current environment.

[0093] Although the user's breathing is abnormal, it may just be that the user is nervous, and there is no abnormal behavior such as forging identity or fraudulent intent. Therefore, the embodiment of the present application can help the user optimize the current posture and current environment through real-time feedback to reduce the possibility of authentication failure. Specifically, the embodiment of the present application can generate optimization information based on the abnormal state, and then broadcast the optimization information through the microphone voice, so that the user can optimize the current posture and current environment according to the optimization information, thereby reducing the possibility of authentication failure, thereby improving user satisfaction and authentication efficiency.

[0094] It can be seen that the embodiments of the present application utilize non-contact visual technology to capture tiny displacements of the user's chest area or neck area through an ordinary camera, or to capture temperature changes in the user's nasal area or mouth area through a thermal imaging camera, thereby capturing the user's breathing signal in real time and analyzing the user's physiological state based on the breathing signal.

[0095] It can be seen that the embodiment of the present application separates a weak breathing signal from background noise through image processing and timing signal algorithm, and uses the breathing signal to construct the user's breathing curve, so as to extract breathing characteristics such as breathing frequency and breathing amplitude from the breathing curve, and compare these breathing characteristics with the standard physiological model respectively, so as to detect the user's abnormal state (such as tension or fraud).

[0096] It can be seen that the embodiment of the present application utilizes non-invasive respiratory signal detection to expand the remote identity authentication capability, thereby improving risk control accuracy and user experience, while requiring no additional hardware and having the advantages of low cost and easy deployment.

[0097] It can be seen from this example that the solution provided by this application can be both real-time, convenient and adaptable, and provides an efficient and intelligent solution for remote risk control scenarios. The solution provided by this application may include but is not limited to the following beneficial effects: 1. Without relying on additional hardware equipment, the user's breathing signal can be extracted only through the camera, thereby realizing convenient and natural physiological state monitoring, which is suitable for general remote authentication scenarios.

[0098] 2. Capture and analyze the user's breathing characteristics in real time, and immediately provide feedback to the remote monitoring platform when it is determined that the user is in an abnormal state, thereby improving the security and efficiency of the remote authentication process.

[0099] 3. As a subtle but significant physiological signal, breathing signals can effectively supplement the deficiencies of related technologies (such as analysis of visual behavioral features and speech features), especially in abnormal state detection scenarios.

[0100] 4. Applicable to a variety of remote authentication scenarios, such as identity verification, behavior monitoring and fraud risk identification, without requiring significant changes to the user's equipment or operating procedures.

[0101] 5. Help users optimize their posture and environment through real-time feedback, reduce the possibility of authentication failure, and improve user satisfaction and authentication efficiency.

[0102] 6. Enhance user experience, without the need for users to wear or operate the device, ensuring that the detection process is smooth and natural without interfering with user behavior.

[0103] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a visual wind control device based on non-invasive detection of respiratory signals, an electronic device, a computer-readable storage medium and corresponding embodiments.

[0104] Figure 3 It is a schematic diagram of the structure of a visual wind control device based on non-invasive detection of respiratory signals shown in an embodiment of the present application.

[0105] See also Figure 3 The present application provides a visual wind control device based on non-invasive detection of respiratory signals, which may include: The image data acquisition module 310 is used to acquire image data of the upper body area of ​​the user through a camera during the remote authentication process of the user; A breathing signal capturing module 320, for capturing a user's breathing signal from the image data; A breathing curve construction module 330, used to construct a breathing curve of the user based on the breathing signal; A breathing feature extraction module 340 is used to extract the user's breathing features from the breathing curve; the breathing features include at least one of breathing frequency and breathing amplitude; The abnormal state detection module 350 is used to detect whether the user is in an abnormal state according to the breathing frequency and / or breathing amplitude; The verification measure triggering module 360 ​​is used to trigger verification measures for the user if the user is in an abnormal state.

[0106] In one embodiment, the abnormal state detection module 350 may include: A respiratory characteristic judgment submodule is used to judge whether the respiratory frequency is within the set respiratory frequency range of the standard physiological model, and to judge whether the respiratory amplitude is within the set respiratory amplitude range of the standard physiological model; The abnormal state determination submodule is used to determine that the user is in an abnormal state if the breathing frequency is not within the set breathing frequency range and / or the breathing amplitude is not within the set breathing amplitude range.

[0107] In one embodiment, the camera is a common camera; the respiratory signal capturing module 320 may include: The upper body displacement capturing submodule is used to capture the upper body displacement of the user from the image data collected by the ordinary camera; the upper body displacement is any one of the chest displacement and the neck displacement; An auxiliary parameter determination submodule, used to determine the pixel brightness of the upper body area of ​​the user in the image data, and to determine the type of optical flow change of the image data; The first definition submodule of the breathing signal is used to determine that the upper body displacement is a micro-motion caused by breathing if the optical flow change type belongs to the optical flow change type caused by breathing and the pixel brightness changes, and use the upper body displacement as the user's breathing signal.

[0108] In another embodiment, the camera is a thermal imaging camera; the respiratory signal capture module 320 may include: A gas temperature capturing submodule, used to capture the gas temperature of the user's nasal cavity or mouth area from the image data collected by the thermal imaging camera; The second definition submodule of the breathing signal is used to use the change of the gas temperature as the user's breathing signal if the gas temperature changes.

[0109] In one embodiment, before capturing the user's breathing signal from the image data, the device may further include: The noise reduction processing module is used to perform noise reduction processing on image data using an adaptive filtering algorithm.

[0110] In one embodiment, the verification measure triggering module 360 ​​may include: The abnormal behavior identification submodule is used to identify the abnormal behavior of the user through the abnormal behavior detection model if the user is in an abnormal state; The verification measure triggering submodule is used to generate feedback information based on abnormal behavior and send the feedback information to the remote monitoring platform to trigger the remote monitoring platform's verification measures for the user.

[0111] In one embodiment, the device may further include: The optimization module is used to generate optimization information based on the abnormal state if the user is in an abnormal state; the optimization information is used to remind the user to optimize the current posture and current environment.

[0112] As can be seen from this example, the solution provided by this application, during the remote authentication process of the user, collects image data of the upper body area of ​​the user; captures the user's breathing signal from the image data; constructs the user's breathing curve based on the breathing signal; extracts the user's breathing characteristics from the breathing curve; the breathing characteristics include at least one of the breathing frequency and breathing amplitude; detects whether the user is in an abnormal state according to the breathing frequency and / or breathing amplitude; if the user is in an abnormal state, triggers verification measures for the user. This application does not rely on additional hardware equipment, and can capture the user's breathing signal only through a camera, thereby achieving convenient and natural physiological state monitoring, which is suitable for general remote authentication scenarios. The breathing signal is a subtle but significant physiological signal. Therefore, obtaining breathing characteristics such as breathing frequency and breathing amplitude from the breathing signal can effectively reflect whether the user is in a physiological state such as tension and anxiety, thereby effectively detecting the user's abnormal state, improving detection accuracy and robustness, and immediately triggering verification measures when an abnormal state is detected, thereby enhancing risk control capabilities.

[0113] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0114] Figure 4 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0115] See also Figure 4 , the electronic device 400 includes a memory 410 and a processor 420.

[0116] The processor 420 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0117] The memory 410 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 420 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at runtime. In addition, the memory 410 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 410 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0118] The memory 410 stores executable codes, and when the executable codes are processed by the processor 420 , the processor 420 can execute part or all of the methods described above.

[0119] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0120] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0121] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A visual wind control method based on non-invasive detection of respiratory signals, characterized in that: The method comprises: During the remote authentication process of the user, image data of the upper body area of ​​the user is collected through a camera; capturing a breathing signal of the user from the image data; Based on the breathing signal, construct a breathing curve of the user; Extracting a breathing feature of the user from the breathing curve; the breathing feature includes at least one of a breathing frequency and a breathing amplitude; Detecting whether the user is in an abnormal state according to the breathing frequency and / or the breathing amplitude; If the user is in an abnormal state, verification measures for the user are triggered.

2. The method according to claim 1, characterized in that The detecting whether the user is in an abnormal state according to the breathing frequency and / or the breathing amplitude includes: Determining whether the respiratory frequency is within a set respiratory frequency range of a standard physiological model, and determining whether the respiratory amplitude is within a set respiratory amplitude range of a standard physiological model; If the breathing frequency is not within the set breathing frequency range, and / or the breathing amplitude is not within the set breathing amplitude range, it is determined that the user is in an abnormal state.

3. The method according to claim 1, characterized in that The camera is a common camera; The capturing of the user's breathing signal from the image data comprises: capturing the upper body displacement of the user from the image data collected by the ordinary camera; the upper body displacement is any one of chest displacement and neck displacement; Determine the pixel brightness of the upper body area of ​​the user in the image data, and determine the optical flow change type of the image data; If the optical flow change type is caused by breathing, and the pixel brightness changes, it is determined that the upper body displacement is caused by breathing, and the upper body displacement is used as the user's breathing signal.

4. The method according to claim 1, characterized in that: The camera is a thermal imaging camera; and capturing the user's breathing signal from the image data includes: capturing the gas temperature of the nasal area or the mouth area of ​​the user from the image data collected by the thermal imaging camera; If the gas temperature changes, the change in the gas temperature is used as the user's breathing signal.

5. The method according to claim 1, characterized in that Before capturing the user's breathing signal from the image data, the method further includes: An adaptive filtering algorithm is used to perform noise reduction processing on the image data.

6. The method according to claim 1, characterized in that If the user is in an abnormal state, triggering verification measures for the user includes: If the user is in an abnormal state, identifying the abnormal behavior of the user through an abnormal behavior detection model; Feedback information is generated according to the abnormal behavior, and the feedback information is sent to a remote monitoring platform to trigger verification measures of the remote monitoring platform on the user.

7. The method according to claim 1, characterized in that The method further comprises: If the user is in an abnormal state, optimization information is generated according to the abnormal state; the optimization information is used to remind the user to optimize the current posture and the current environment.

8. A visual wind control device based on non-invasive detection of respiratory signals, characterized in that: The device comprises: An image data acquisition module, used to acquire image data of the upper body area of ​​the user through a camera during the remote authentication process of the user; A breathing signal capturing module, used for capturing the user's breathing signal from the image data; A breathing curve construction module, used to construct a breathing curve of the user based on the breathing signal; A breathing feature extraction module, used to extract the user's breathing feature from the breathing curve; the breathing feature includes at least one of a breathing frequency and a breathing amplitude; an abnormal state detection module, used for detecting whether the user is in an abnormal state according to the breathing frequency and / or the breathing amplitude; The verification measure triggering module is used to trigger verification measures for the user if the user is in an abnormal state.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 7.

Images