Integrated multi-mode physiological behavior signal acquisition system and method for lie detection scene

Abstract

The invention belongs to the technical field of brain-computer interfaces and physiological and psychological measurement, and particularly relates to an integrated multi-mode physiological behavior signal acquisition system and method for a lie detection scene. The system comprises an electroencephalogram acquisition module used for detecting an electroencephalogram signal generated by a central nervous system; the peripheral physiological acquisition module comprises a first physiological acquisition sub-module and a second physiological acquisition sub-module, and a second sensor unit of the first physiological acquisition sub-module comprises two groups of first surface electrode patches used for detecting a skin electric signal and an electromyographic signal respectively and a miniature optical measurer used for detecting a pulse signal; a third sensor unit of the second physiological acquisition sub-module comprises a second surface electrode patch used for detecting electrocardiosignals and a respiration induction plethysmography belt used for detecting respiration signals. And the expression acquisition module is used for detecting the macroscopic facial expression image data of the subject. The system solves the problems of low system integration level and portability, insufficient lie detection task specificity and the like in the prior art.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain-computer interface and physiological and psychological measurement, and particularly relates to an integrated multi-modal physiological behavior signal acquisition system and method for a lie detection scene. BACKGROUND

[0002] Lie detection technology is of great significance in judicial interrogation and public security, and its core goal is to identify the authenticity of the subject's statement, thereby providing an objective basis for key tasks such as criminal investigation and intelligence identification. This technology is undergoing profound changes from peripheral indirect measurement to central direct decoding, from single indicator judgment to multi-information fusion.

[0003] Traditional lie detection mainly relies on the monitoring and analysis of peripheral physiological parameters such as respiration, heart rate, and skin conductance, or non-contact behaviors such as facial expressions. Peripheral physiological parameters are a manifestation of peripheral nervous system activity, and facial expressions are controlled by the central nervous system but are still indirect behavioral manifestations. Both can reflect the emotional stress and cognitive load of the subject when lying to some extent, but their inherent defects of low generalizability, subjective consciousness control, and interference from unrelated emotions and abnormal physiological states severely restrict the accuracy and reliability of the detection results. For example, subjects can deliberately affect such signals through professional psychological training, leading to recognition failure; traditional lie detection also cannot determine whether the measured signals are caused by anxiety, fear, hypoglycemia, or other problems.

[0004] To overcome the above limitations, the focus of research in the field has further shifted to the direct decoding of central nervous activity in the brain. Neuroimaging technologies such as electroencephalography and functional magnetic resonance imaging can directly capture neural physiological activity closely related to the cognitive processing of lies, such as neural activation related to covert information recognition and true response inhibition. This type of neural signal originates from the high-level cognitive function area of the brain and is difficult to be fully controlled by the subject's consciousness, thereby providing a more direct and reliable physiological basis for achieving high-specificity lie detection, effectively compensating for the shortcomings of traditional peripheral physiological indicators or facial expressions.

[0005] Currently, multi-modal methods that integrate multiple signals have become a key development focus in the field. A single signal source, whether it is facial expression, peripheral physiology, or brain signal, is difficult to fully depict the complex psychological and physiological process of lying. Therefore, integrating central nervous signals (such as electroencephalography) with peripheral physiological signals (such as electrocardiogram, electromyogram, and skin conductance), and combining facial expression behavior data for comprehensive analysis, has become an inevitable path to improve system performance. This multi-modal architecture can effectively reduce the impact of individual differences and significantly enhance the system's anti-interference and countermeasures capabilities through information complementation.

[0006] Currently, there are some technical solutions related to the construction of multi-modal signal acquisition systems, and attempts have been made to apply them to lie detection or cognitive state exploration tasks: Chinese patent document CN115316943A discloses a head-mounted multi-physiological parameter detection system and its detection method. The technical solution aims to conveniently and non-invasively obtain the multi-channel physiological parameters of impedance, blood oxygen, pulse rate and body temperature of the subject through an integrated head-mounted device.

[0007] Chinese patent document CN116965830A discloses a passenger comfort evaluation system and method based on multi-modal physiological data. The system integrates a set of human physiological feedback acquisition devices, which can collect various signals including electroencephalogram, electrocardiogram, electromyogram, skin electricity, eye movement and skin temperature, and quantitatively evaluate the comfort level of the passenger based on these signals.

[0008] Chinese patent document CN119679411A provides a multi-modal portable cognitive screening device. The device can record the multi-modal behavior and physiological data of the user's eye movement trajectory, electroencephalogram, scale data, facial expression and gait characteristics during the test and training process.

[0009] Chinese patent document CN115299947A discloses a psychological scale confidence evaluation method and system based on multi-modal physiological data, which integrates micro-expression, electrocardiogram, electroencephalogram, eye movement, skin electricity and other multi-modal data to evaluate the confidence of whether the subject is lying.

[0010] Chinese patent document CN119523439A discloses a wearable multi-channel physiological parameter acquisition system. The system can meet the needs of stable acquisition of physiological data such as brain electrical activity, blood oxygen, skin electricity, pulse, body temperature and eye movement of the pilot in special environments such as air flight or ground driving.

[0011] Chinese patent document CN113951886A discloses a magnetoencephalogram generation system and lie detection decision system. The generation and decision system comprehensively collects the magnetoencephalogram, heart rate, respiratory rate, skin conductance and micro-expression data of the subject, and quantifies the probability of answering questions truthfully through statistical analysis of multi-modal information.

[0012] However, these existing technologies still have many limitations in system integration, portability and specificity of lie detection tasks, which are specifically manifested as: 1) Insufficient system integration and portability.

[0013] Some existing systems (such as patent document CN115299947A) employ a collaborative working mode of multiple independent devices, which not only increases the overall system complexity but also directly impacts the user experience due to the cumbersome donning process, potentially causing discomfort to test subjects and affecting the stability of signal acquisition and data quality. Furthermore, existing devices (such as patent documents CN116965830A, CN115299947A, and CN113951886A) suffer from large size, relatively dispersed components, or high environmental requirements, making them difficult to integrate into portable carriers. This severely limits deployment to fixed scenarios such as laboratories, hindering their suitability for changing scenarios requiring rapid response, such as on-site investigations and mobile law enforcement.

[0014] 2) Lack of adaptability to lie detection scenarios.

[0015] On the one hand, existing lie detection systems lack comprehensive coverage of relevant indicators, failing to fully collect key physiological and behavioral signals such as EEG, skin conductance, pulse, electromyography, electrocardiography, respiration, and facial expressions (e.g., patent documents CN115316943A, CN116965830A, CN119679411A, CN115299947A, CN119523439A). This makes it difficult to support systematic lie detection judgments; for example, the lack of EEG and electromyography signals may lead to distorted lie detection conclusions in adversarial scenarios. On the other hand, existing devices lack specific optimization designs for lie detection scenarios, especially in the EEG acquisition module. Electrode configuration fails to balance acquisition effectiveness with system power consumption (e.g., patent documents CN116965830A, CN119679411A, CN115299947A, CN119523439A). Redundant electrodes increase system complexity and cost, while insufficient electrodes can cause the loss of signals in key brain regions, directly affecting detection accuracy.

[0016] Based on this, the present invention designs an integrated multimodal physiological behavior signal acquisition system and method for lie detection scenarios, in order to overcome the limitations of existing technologies in terms of system integration, portability, and specialization of lie detection tasks. Summary of the Invention

[0017] This invention aims to overcome at least one of the defects of the prior art and provide an integrated multimodal physiological and behavioral signal acquisition system for lie detection scenarios. By improving module integration and wearability, it constructs an integrated acquisition system that can be integrated into a suitcase carrier. At the same time, in response to the special needs of lie detection applications, it fully covers key physiological indicators such as EEG, skin conductance, pulse, electromyography, electrocardiography, and respiration, as well as behavioral indicators such as facial expressions. Furthermore, it optimizes the configuration of EEG electrodes to achieve a balance between acquisition effect and power consumption cost.

[0018] The application also discloses a method applied to the integrated multi-modal physiological behavior signal collection system for the polygraph scene.

[0019] The detailed technical scheme of the application is as follows: An integrated multi-modal physiological behavior signal collection system for a polygraph scene comprises: An electroencephalogram collection module is used for detecting brain electrical signals generated by the central nervous system. A peripheral physiological collection module is used for detecting physiological signals regulated by the peripheral nervous system, and comprises a physiological collection sub-module I and a physiological collection sub-module II. An expression collection module is used for detecting macro facial expression image data of a subject. The physiological collection sub-module I comprises a second sensor unit and a second signal processing unit, and the second sensor unit is in communication connection with the second signal processing unit. The physiological collection sub-module II comprises a third sensor unit and a third signal processing unit, and the third sensor unit is in communication connection with the third signal processing unit. The electroencephalogram collection module, the peripheral physiological collection module and the expression collection module are in communication connection with a mobile workstation.

[0020] According to the application, the electroencephalogram collection module comprises a first sensor unit and a first signal processing unit, and the first sensor unit is in communication connection with the first signal processing unit.

[0021] According to the application, when the skin electrical signal of the subject is collected, one group of first surface electrode patches of the second sensor unit is placed on the middle finger and the ring finger of the subject.

[0022] According to the application, preferably, during the collection of the pulse signal of the subject, the micro optical measuring device of the second sensor unit is fixed to the fingertip of the index finger of the subject through an adjustable bandage; the micro optical measuring device is internally provided with a light source and a photoelectric detector, and detects the periodic fluctuation of light intensity caused by the change of blood volume through the photoplethysmography method to obtain the pulse signal.

[0023] According to the application, preferably, during the collection of the electrocardio signal of the subject, the second surface electrode patch of the third sensor unit is placed on the chest and back of the subject; during the collection of the respiration signal of the subject, the respiration sensing volume plethysmography band of the third sensor unit is wrapped around the chest of the subject, and the bandage is internally embedded with a flexible strain sensor to detect the change of the bandage tension caused by the fluctuation of the chest during the respiration of the subject, and convert the change of the bandage tension into a corresponding electrical signal.

[0024] According to the application, preferably, the first signal processing unit, the second signal processing unit and the third signal processing unit each include a microcontroller assembly, and an analog front-end assembly, a communication assembly, a power management assembly and a state indication assembly electrically connected with the microcontroller assembly; wherein: The analog front-end assembly is used for lead detection, amplification, analog-to-digital conversion, filtering and common-mode rejection operation on the signals received thereby, and transmits the processed signals to the microcontroller assembly; The communication assembly is used for establishing communication between the microcontroller assembly and the mobile workstation; The power management assembly is used for powering the whole of the corresponding first signal processing unit, second signal processing unit or third signal processing unit; The microcontroller assembly is used for coordinating and controlling the working of the analog front-end assembly, the communication assembly, the power management assembly and the state indication assembly.

[0025] According to the application, preferably, in the first signal processing unit: the microcontroller assembly realizes its functions based on an STM32F429 chip; the analog front-end assembly realizes its functions based on an ADS1299 chip; the communication assembly realizes wireless communication through an MQTT protocol based on an ESP8266 chip; the power management assembly realizes its functions through an SGM2082 voltage stabilizer and a MAX865 direct current conversion chip; and the state indication assembly realizes its functions by using an OLED screen. And / or, in the second signal processing unit: the microcontroller component implements its function based on the STM32F429 chip; the analog front-end component implements its function based on the MAX30102 pulse oximetry chip and the ADS1292 electrical signal chip; the communication component implements wireless communication via the MQTT protocol based on the ESP8266 chip; the power management component implements its function through the TPS7A02 voltage regulator and the XL4015 DC-DC converter chip; and the status indication component uses an OLED screen to implement its function. And / or, in the third signal processing unit: the microcontroller component implements its function based on the STM32F429 chip; the analog front-end component implements the ECG acquisition function based on the ADS1298R chip and the respiration acquisition function based on the ADS124S08 chip; the communication component implements wireless communication via the MQTT protocol based on the ESP8266 chip; the power management component implements its function through the TPS7A02 voltage regulator and the XL4015 DC-DC converter chip; and the status indication component uses an OLED screen to implement its function.

[0026] According to a preferred embodiment of the present invention, the facial expression acquisition module uses a high-definition camera to detect facial expression image data of the subject and transmits it to the mobile workstation via a wired connection; the high-definition camera is placed above the screen of the mobile workstation.

[0027] In another aspect of the present invention, an integrated multimodal physiological and behavioral signal acquisition method for lie detection scenarios is provided, applied to the integrated multimodal physiological and behavioral signal acquisition system for lie detection scenarios as described above, the method comprising: The brain electrical signals generated by the central nervous system of the subjects were detected using an EEG acquisition module. The peripheral physiological acquisition module was used to detect the physiological signals of the subject's peripheral nervous system regulation, including: using physiological acquisition submodule one to detect the subject's skin conductance, electromyography and pulse signals; and using physiological acquisition submodule two to detect the subject's electrocardiogram and respiratory signals. Macroscopic facial expression image data of subjects were detected using an expression acquisition module; The physiological acquisition submodule includes a second sensor unit and a second signal processing unit, which are communicatively connected. The second sensor unit includes two sets of first surface electrode patches for detecting the subject's electrodermal and electromyographic signals, respectively, and a miniature optical measuring device for detecting the subject's pulse signal. The physiological acquisition sub-module two comprises a third sensor unit and a third signal processing unit, the third sensor unit is in communication connection with the third signal processing unit; the third sensor unit comprises a second surface electrode patch for detecting the ECG signal of the subject, and a respiratory inductance plethysmography band for detecting the respiration signal of the subject; The information collected by the EEG acquisition module, the peripheral physiological acquisition module and the expression acquisition module is uploaded to a mobile workstation, and the mobile workstation realizes human-computer interaction through a visualization system embedded therein.

[0028] Compared with the prior art, the present application has the following advantages: (1) The integrated multi-modal physiological behavior signal acquisition system for a polygraph scene provided by the present application realizes high integration and portability of the system, changes the defects of dispersion between traditional system devices, complex wiring and difficult deployment, improves user experience and detection efficiency, enhances the mobility and environmental adaptability of the device, and provides a feasible technical basis for polygraph application in non-laboratory scenarios.

[0029] (2) The present application constructs a complete signal acquisition system for a polygraph, realizes a better balance between performance and efficiency, synchronously acquires multi-modal physiological and behavior data such as EEG, skin electricity, ECG, EMG, pulse, respiration, facial expression, etc., overcomes the defect of incomplete modal coverage in the prior art, and improves the data integrity and reliability of polygraph judgment. In addition, the 10-20 standard lead system is optimized, the number of electrodes is significantly reduced under the premise of ensuring the capture of signals of key brain areas such as frontal lobe and parietal lobe, and the power consumption and construction cost of the system are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a hardware structure schematic diagram of the integrated multi-modal physiological behavior signal acquisition system for a polygraph scene described in the present application; Figure 2 is a device wearing form schematic diagram of the integrated multi-modal physiological behavior signal acquisition system for a polygraph scene described in embodiment 1 of the present application; Figure 3 is a component architecture diagram of the integrated multi-modal physiological behavior signal acquisition system for a polygraph scene described in embodiment 1 of the present application; Figure 4 is a component architecture diagram of the EEG and peripheral physiological module of the integrated multi-modal physiological behavior signal acquisition system for a polygraph scene described in embodiment 1 of the present application; Figure 5is a data visualization schematic diagram of the integrated multi-modal physiological behavior signal acquisition system for a polygraph scene in Embodiment 1 of the present application. Figure 6 is a flow chart of the integrated multi-modal physiological behavior signal acquisition method for a polygraph scene in the present application. In the figure: 1, electroencephalogram acquisition module; 11, first sensor unit; 12, first signal processing unit; 2, peripheral physiological acquisition module; 21, physiological acquisition submodule one; 211, second sensor unit; 2111, first surface electrode patch; 2112, first lead wire; 2113, miniature optical measuring device; 212, second signal processing unit; 22, physiological acquisition submodule two; 221, third sensor unit; 2211, second lead wire; 2212, second surface electrode patch; 2213, respiratory inductive plethysmography belt; 222, third signal processing unit; 3, expression acquisition module; 4, mobile workstation; 5, visualization system; 6, suitcase; 61, suitcase combination lock; 62, suitcase handle. DETAILED DESCRIPTION

[0031] The present application will be further described below in conjunction with the accompanying drawings and embodiments.

[0032] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0033] It should be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a reference to the presence of a feature, step, operation, device, component, and / or combinations thereof.

[0034] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0035] Embodiment 1, Referring to Figure 1 , Figure 3 The present embodiment provides an integrated multi-modal physiological behavior signal acquisition system for a polygraph scene, which comprises: The electroencephalogram acquisition module 1 is used to detect the brain electrical signals generated by the central nervous system; The peripheral physiological acquisition module 2 is used to detect the physiological signals regulated by the peripheral nervous system; The expression acquisition module 3 is used to detect the macro facial expression image data of the subject; The EEG acquisition module 1, the peripheral physiological acquisition module 2 and the expression acquisition module 3 are in communication connection with the mobile workstation 4, the mobile workstation 4 is embedded with a visualization system 5, and the EEG acquisition module 1, the peripheral physiological acquisition module 2, the expression acquisition module 3 and the mobile workstation 4 are integrated in a suitcase 6.

[0036] Specifically, the EEG acquisition module 1 comprises a first sensor unit 11 and a first signal processing unit 12, and the first sensor unit 11 is in electrical connection with the first signal processing unit 12.

[0037] The first sensor unit 11 is a set of 8-channel dry electrode group, which is convenient to wear; and the dry electrode group follows the 10-20 international standard electrode system, only contains the frontal lobe and parietal lobe partial electrodes with higher degree of relevance to the lie detection task, and is placed on the head of the subject during testing, which can be referred to Figure 2 .

[0038] Further, during the acquisition of the brain electrical signals of the subject, the 8-channel dry electrode group can select silver chloride rigid dry electrodes to ensure good electrical conductivity and biocompatibility; and the matching cap body can select elastic mesh cloth to ensure comfort, air permeability and stable adhesion to the scalp.

[0039] Among them, the 8-channel dry electrode group can select Fz, Cz, F3, F4, C3, C4, P3 and P4 sites respectively, and the electrode layout is optimized for brain cognitive activities in the lie detection scene, which significantly reduces the number of electrodes under the premise of ensuring the capture of signals of key brain areas such as frontal lobe and parietal lobe, eliminates redundant information in the lie detection task, and reduces system power consumption and construction cost compared with more lead electrode layout.

[0040] Among them, the Fz, F3 and F4 sites are located in the frontal lobe, used for acquiring brain electrical signals associated with cognitive conflict, executive control and inhibition of real reaction in the process of lie construction; the P3 and P4 sites are located in the parietal lobe, used for acquiring brain electrical signals associated with attention allocation and memory extraction; the Cz, C3 and C4 sites are located in the central region, used for acquiring brain electrical signals associated with emotional fluctuations and psychological stress while providing signal reference.

[0041] The first signal processing unit 12 comprises a microcontroller component, an analog front-end component, a communication component, a power management component and a state indication component in electrical connection with the microcontroller component, and the architecture thereof is referred to Figure 4 .

[0042] Among them, the analog front-end component is used for lead detection, amplification, analog-to-digital conversion, filtering and common-mode rejection operation on the weak scalp signals received by it, and transmits the processed standard digital signals to the microcontroller component.

[0043] The microcontroller component serves as a module control center, receives digital signals from the analog front-end component, and coordinates and controls the working states of the communication component, the power management component, and the state indication component to complete data acquisition, processing, and transmission tasks.

[0044] The communication component is used to establish communication between the microcontroller component and the mobile workstation 4, and under the instruction of the microcontroller component, it receives instructions from the mobile workstation 4 through wireless communication and transmits the collected data output by the microcontroller.

[0045] The power management component provides stable power supply for the analog front-end component, the microcontroller component, the communication component, and the state indication component, ensuring the normal operation of each component.

[0046] The state indication component is used to obtain key parameters of the microcontroller component, the communication component, and the power management component, and displays the working state of the module, the data connectivity with the mobile workstation 4, and the remaining power information in real time through its embedded display screen.

[0047] As a further preferred embodiment of the present embodiment, in the first signal processing unit 12: The core chip of the microcontroller component can be selected from the STM32F429 series chip to efficiently coordinate the operation of each component; the core chip of the analog front-end component can be selected from the ADS1299 chip, which has high-precision signal acquisition capability; the communication component can realize wireless communication based on the ESP8266 chip through the MQTT protocol, ensuring low-latency data transmission; the core chip of the power management component can be selected from the SGM2082 voltage stabilizer and the MAX865 direct current conversion chip, ensuring the stability of the endurance; the state indication component has an embedded display screen, which can be selected from an OLED screen, which has clear display and low power consumption.

[0048] It can be understood that each of the above components is integrated and packaged in a one-piece black square shell and electrically connected to the first sensor unit 11 through a standardized interface. In addition, the shell structure can be made of lightweight ABS+PC alloy material, which has the characteristics of durability and lightness.

[0049] The peripheral physiological acquisition module 2 includes physiological acquisition sub-module one 21 and physiological acquisition sub-module two 22. The physiological acquisition sub-module one 21 is used to detect skin electricity signals, muscle electricity signals, and pulse signals; the physiological acquisition sub-module two 22 is used to detect electrocardiogram signals and respiratory signals.

[0050] In the present embodiment, the physiological acquisition sub-module 21 comprises a second sensor unit 211 and a second signal processing unit 212, which are electrically connected through a first lead 2112.

[0051] The second sensor unit 211 comprises two groups of first surface electrode patches 2111 for detecting the skin conductance signal and the muscle conductance signal of the subject, respectively, and a miniature optical measurement device 2113 for detecting the pulse signal of the subject.

[0052] Further, when collecting the skin conductance signal of the subject, one group of the first surface electrode patches 2111 of the second sensor unit 211 is placed on the middle finger and the ring finger of the subject; meanwhile, when collecting the muscle conductance signal of the subject, the other group of the first surface electrode patches 2111 of the second sensor unit 211 is placed on the biceps of the upper arm and the inner side of the lower arm of the subject.

[0053] When collecting the pulse signal of the subject, the miniature optical measurement device 2113 of the second sensor unit 211 is fixed on the tip of the index finger of the subject through an adjustable band; the miniature optical measurement device 2113 is internally provided with a light source and a photodetector, and detects the periodic fluctuation of light intensity caused by the change of blood volume through the photoplethysmography method to obtain the pulse signal.

[0054] During the signal acquisition process, the first lead 2112 can ensure the reliability of the data transmission from the second sensor unit 211 to the second signal processing unit 212.

[0055] The second signal processing unit 212 comprises a microcontroller assembly, an analog front-end assembly, a communication assembly, a power management assembly and a state indication assembly electrically connected with the microcontroller assembly, the architecture of which is shown in Figure 4 .

[0056] Among them, the analog front-end assembly is used for lead detection, amplification, analog-to-digital conversion, filtering and common-mode rejection operation on the skin conductance, pulse and muscle conductance signals received thereby, and transmits the standardized digital signals obtained by processing to the microcontroller assembly.

[0057] The microcontroller assembly serves as the control center of the module, receives the digital signals from the analog front-end assembly, and coordinates and controls the working states of the communication assembly, the power management assembly and the state indication assembly to complete the data acquisition, processing and transmission tasks.

[0058] The communication assembly is used for establishing communication between the microcontroller assembly and the mobile workstation 4, and receives instructions from the mobile workstation 4 through wireless communication under the instructions of the microcontroller assembly, and transmits the collected data output by the microcontroller back.

[0059] The power management component provides stable power supply for the analog front-end component, the microcontroller component, the communication component and the state indication component, ensuring normal operation of each component.

[0060] The state indication component is used to acquire key parameters of the microcontroller component, the communication component and the power management component, and display the working state of the sub-module, data connectivity with the mobile workstation 4 and residual power information and the like in real time through the embedded display screen thereof.

[0061] As a further preferred embodiment of the present embodiment, in the second signal processing unit 212: The core chip of the microcontroller component can be selected from an STM32F429 series chip, to realize efficient coordination of operation of each component; the core chip of the analog front-end component can be selected from a MAX30102 pulse blood oxygen chip and an ADS1292 electric signal chip to realize the functions thereof; the communication component can realize wireless communication based on an ESP8266 chip through an MQTT protocol; the core chip of the power management component can be selected from a TPS7A02 voltage stabilizer and an XL4015 direct current conversion chip to realize the functions thereof; the state indication component has an embedded display screen, which can be selected from an OLED screen, with clear display and low power consumption.

[0062] It can be understood that each component described above is integrated and encapsulated in an integrated black square shell, and is electrically connected with the second sensor unit 211 through a standardized interface. The encapsulation structure is integrated on the wristband and fixed on the wrist, which is beneficial to the convenience and comfort of wearing, as shown in Figure 2 Similarly, the encapsulation structure can be selected from a lightweight ABS+PC alloy material, which has the characteristics of durability and lightness.

[0063] In the present embodiment, the physiological acquisition sub-module two 22 comprises a third sensor unit 221 and a third signal processing unit 222, and the third sensor unit 221 and the third signal processing unit 222 can be electrically connected through a second lead 2211.

[0064] The third sensor unit 221 comprises a second surface electrode patch 2212 for detecting the ECG signal of the subject, and a respiratory inductance plethysmography belt 2213 for detecting the respiratory signal of the subject.

[0065] Further, when collecting the ECG signal of the subject, the second surface electrode patch 2212 of the third sensor unit 221 is placed on the chest and back of the subject to detect the tiny potential difference on the skin surface caused by the beating of the heart, thereby recording the heart activity; meanwhile, when collecting the respiration signal of the subject, the respiration inductance plethysmography belt 2213 of the third sensor unit 221 is wrapped around the chest of the subject, and the flexible tension sensing element, such as a flexible strain sensor, is embedded in the belt to detect the belt tension change caused by the chest fluctuation in the respiration process of the subject, and convert the belt tension change into a corresponding electrical signal.

[0066] During the signal collection process, the second lead wire 2211 can ensure the reliability of the data transmission from the third sensor unit 221 to the third signal processing unit 222.

[0067] The third signal processing unit 222 includes a microcontroller assembly, an analog front-end assembly, a communication assembly, a power management assembly and a state indication assembly electrically connected to the microcontroller assembly, and the architecture thereof is shown in Figure 4 .

[0068] The analog front-end assembly is used to perform lead detection, amplification, analog-to-digital conversion, filtering and common-mode rejection operation on the respiration and ECG signals received thereby, and transmit the processed standard digital signals to the microcontroller assembly.

[0069] The microcontroller assembly serves as the module control center, receives the digital signals from the analog front-end assembly, and coordinates and controls the working states of the communication assembly, the power management assembly and the state indication assembly to complete the data collection, processing and transmission tasks.

[0070] The communication assembly is used to establish the communication between the microcontroller assembly and the mobile workstation 4, and under the instruction of the microcontroller assembly, receives the instruction from the mobile workstation 4 through wireless communication and transmits the collected data output by the microcontroller back.

[0071] The power management assembly provides stable power supply for the analog front-end assembly, the microcontroller assembly, the communication assembly and the state indication assembly, and ensures the normal operation of each component.

[0072] The state indication assembly is used to obtain the key parameters of the microcontroller assembly, the communication assembly and the power management assembly, and display the working state of the sub-module, the data connectivity with the mobile workstation 4 and the remaining power information, etc. in real time through the embedded display screen thereof.

[0073] As a further preferred embodiment of the present embodiment, in the third signal processing unit 222: The microcontroller assembly core chip can be selected from an STM32F429 series chip to realize efficient coordination of operation of each component; the analog front-end assembly is selected from an ADS1298R chip to realize electrocardio collection function, and an ADS124S08 chip to realize respiration collection function; the communication assembly can realize wireless communication based on an ESP8266 chip through an MQTT protocol; the power management assembly core chip can be selected from a TPS7A02 voltage stabilizer and an XL4015 direct current conversion chip to realize its function; the state indication assembly has an embedded display screen, which can be selected from an OLED screen, and has clear display and low power consumption.

[0074] It can be understood that each component is integrated and encapsulated in a one-piece black square shell, and is electrically connected with the third sensor unit 221 through a standardized interface. The encapsulation structure is attached to the outside of the breathing belt and fixed to the chest, which is beneficial to the convenience and comfort of wearing, as shown in Figure 2 Similarly, the encapsulation structure can be made of light ABS+PC alloy material, which has the characteristics of durability and light weight.

[0075] In the embodiment, the expression collection module 3 can use a high-definition camera to detect the facial expression image data of the subject, and transmit it to the mobile workstation 4 through a wired connection; the high-definition camera is placed above the screen of the mobile workstation 4, as shown in Figure 2 .

[0076] Preferably, the high-definition camera of the expression collection module 3 supports 1080p or above high-definition resolution, which can clearly capture facial details; the lens is made of multi-layer coated optical glass material to reduce light reflection; the connection mode is selected from a USB interface to ensure high-speed transmission of image data.

[0077] Further, as shown in Figure 5 , the visualization system 5 is used for the subject to control the operation of the collection module, receive and store related multi-modal physiological behavior signals, and display them through the display screen. Specifically, the visualization system 5 displays the running state of the multi-modal physiological behavior signal collection system on the display screen of the mobile workstation 4, including the connection condition with the collection module, the start, pause and stop of collection, data storage and export options, and displays the basic information of the subject, electroencephalogram signal, peripheral physiological signal, facial expression video, experimental stimulation picture, and preliminary real-time reliable level during the collection process.

[0078] Preferably, the mobile workstation 4 on which the visualization system 5 relies has no less than 16GB memory and 512GB storage space, multiple USB interfaces, and Bluetooth, WiFi, etc.

[0079] The suitcase 6 is used to carry the integrated multi-modal physiological behavior signal acquisition system for polygraph scene, the shell is made of high-strength engineering plastic material, and the inner lining is provided with a fixed clamping groove matched with the shape of the multi-modal physiological behavior signal acquisition system component, which is used to stably support and isolate each component device; The suitcase 6 is equipped with a password lock 61 for device security protection; The suitcase 6 is equipped with a handle 62 for carrying.

[0080] Preferably, the shell material of the suitcase 6 can be selected from ABS+PC alloy material to have high structural strength and impact resistance; The inner lining material can be selected from EVA hard closed-cell foam to provide effective buffering and shock protection; The password lock 61 can be selected from a mechanical password lock.

[0081] Embodiment 2, The embodiment provides an integrated multi-modal physiological behavior signal acquisition method for polygraph scene, which is applied to the integrated multi-modal physiological behavior signal acquisition system for polygraph scene as described in embodiment 1, and the method comprises the following steps: The EEG acquisition module 1 is used to detect the brain electrical signals generated by the central nervous system of the subject; The peripheral physiological acquisition module 2 is used to detect the physiological signals regulated by the peripheral nervous system of the subject, including: the physiological acquisition sub-module one 21 is used to detect the skin electrical signals, electromyographic signals and pulse signals of the subject; The physiological acquisition sub-module two 22 is used to detect the electrocardiographic signals and respiratory signals of the subject; The expression acquisition module 3 is used to detect the macro facial expression image data of the subject; The physiological acquisition sub-module one 21 comprises a second sensor unit 211 and a second signal processing unit 212, and the second sensor unit 211 is in communication connection with the second signal processing unit 212; The second sensor unit 211 comprises two groups of first surface electrode patches 2111 for detecting the skin electrical signals and electromyographic signals of the subject, and a micro optical measuring device 2113 for detecting the pulse signals of the subject; The physiological acquisition sub-module two 22 comprises a third sensor unit 221 and a third signal processing unit 222, and the third sensor unit 221 is in communication connection with the third signal processing unit 222; The third sensor unit 221 comprises a second surface electrode patch 2212 for detecting the electrocardiographic signals of the subject, and a respiratory inductive plethysmography belt 2213 for detecting the respiratory signals of the subject; The information collected by the EEG acquisition module 1, the peripheral physiological acquisition module 2 and the expression acquisition module 3 is uploaded to the mobile workstation 4, and the mobile workstation 4 realizes man-machine interaction through the embedded visualization system 5.

[0082] Further, the method comprises the following steps: Figure 6 The collection process of the method is as follows: step1. Start-up: Take the multi-modal physiological behavior signal acquisition system out of the suitcase, turn on the system power, and confirm that each acquisition module is running normally and has sufficient power; step2. Personnel registration: Inform the test subjects of the experimental purpose and experimental criteria, fill out the informed consent form, and register the test subjects' basic information including name, age, height, and gender; step3. Equipment wearing: Correctly wear or install the relevant sensor units of the EEG acquisition module, peripheral physiological acquisition module, and expression acquisition module on the test subject or the corresponding position on the desktop; step4. Parameter setting: Establish a data connection with the mobile workstation and set the correct acquisition parameters to ensure that each modal data can be effectively acquired; step5. Pre-experiment: Perform a pre-experiment to ensure that the experimental procedure is running normally and to familiarize the test subjects with the experimental process; step6. Formal experiment: Send recording instructions to each acquisition module through the mobile workstation to detect the test subjects' EEG, skin electricity, electromyography, ECG, respiration, pulse, and facial expression information during the experiment; step7. Data transmission: The relevant data is transmitted to the mobile workstation through wired or wireless means and is displayed in real time through the visualization system; step8. Archiving: Remove the relevant sensor units of each acquisition module from the test subject, turn off the system power, and place them in the suitcase. In the mobile workstation, organize, store, and backup the relevant data for subsequent analysis.

[0083] In summary, the present application achieves high integration and portability of the system. By integrating the multi-modal acquisition module, mobile workstation, and all supporting components into a suitcase and using wireless communication management, the present application overcomes the defects of traditional systems, such as dispersion between devices, complex wiring, and difficult deployment, improves user experience and detection efficiency, enhances the mobility and environmental adaptability of the equipment, and provides a feasible technical foundation for polygraph applications in non-laboratory scenarios.

[0084] On the other hand, the present application constructs a complete signal acquisition system for polygraph, achieving a better balance between performance and efficiency. It synchronously acquires multi-modal physiological and behavioral data such as EEG, skin electricity, ECG, electromyography, pulse, respiration, and facial expression, overcoming the defect of incomplete modal coverage in the prior art, and improving the data integrity and reliability of polygraph judgment. In addition, the 10-20 standard lead system is optimized, significantly reducing the number of electrodes, and reducing the power consumption and construction cost of the system, while ensuring the capture of signals from key brain regions such as the frontal lobe and parietal lobe.

[0085] Obviously, the above embodiments of the present application are only examples for clearly illustrating the technical solutions of the present application, and are not intended to limit the specific implementation of the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A polyphasic multi-modal physiological behavior signal acquisition system for a lie detection scenario, characterized in that, The system comprises: The brain electrical acquisition module (1) is used for detecting the brain electrical signals generated by the central nervous system; The peripheral physiological acquisition module (2) is used for detecting the physiological signals regulated by the peripheral nervous system, which comprises a physiological acquisition submodule one (21) and a physiological acquisition submodule two (22), the physiological acquisition submodule one (21) is used for detecting the skin electrical signal, the electromyogram and the pulse signal; the physiological acquisition submodule two (22) is used for detecting the electrocardiogram and the respiratory signal; The expression acquisition module (3) is used for detecting the macro facial expression image data of the subject; The physiological acquisition submodule one (21) comprises a second sensor unit (211) and a second signal processing unit (212), the second sensor unit (211) is in communication connection with the second signal processing unit (212); the second sensor unit (211) comprises two groups of first surface electrode patches (2111) for detecting the skin electrical signal and the electromyogram of the subject respectively, and a micro optical measuring device (2113) for detecting the pulse signal of the subject; The physiological acquisition submodule two (22) comprises a third sensor unit (221) and a third signal processing unit (222), the third sensor unit (221) is in communication connection with the third signal processing unit (222); the third sensor unit (221) comprises a second surface electrode patch (2212) for detecting the electrocardiogram of the subject, and a respiratory inductive plethysmography band (2213) for detecting the respiratory signal of the subject; The brain electrical acquisition module (1), the peripheral physiological acquisition module (2) and the expression acquisition module (3) are in communication connection with the mobile workstation (4), the mobile workstation (4) is embedded with a visualization system (5), and the brain electrical acquisition module (1), the peripheral physiological acquisition module (2), the expression acquisition module (3) and the mobile workstation (4) are integrated in a suitcase (6).

2. The polygraph lie detection scenario oriented integrated multi-modal physiological behavior signal acquisition system as claimed in claim 1, wherein, The brain electrical acquisition module (1) comprises a first sensor unit (11) and a first signal processing unit (12), the first sensor unit (11) is in communication connection with the first signal processing unit (12); the first sensor unit (11) is an 8-lead dry electrode group, and when the brain electrical signal of the subject is collected, the 8-lead dry electrode group selects Fz, Cz, F3, F4, C3, C4, P3 and P4 sites respectively.

3. The polygraph scene-oriented integrated multi-modal physiological behavioral signal acquisition system according to claim 1, wherein, When the skin electrical signal of the subject is collected, one group of first surface electrode patches (2111) of the second sensor unit (211) is placed on the middle finger and the ring finger of the subject; when the electromyogram of the subject is collected, the other group of first surface electrode patches (2111) of the second sensor unit (211) is placed on the biceps brachii and the medial epicondyle of the subject.

4. The polygraph scene-oriented integrated multi-modal physiological behavioral signal acquisition system according to claim 1, wherein, When the pulse signal of the subject is collected, the micro optical measuring device (2113) of the second sensor unit (211) is fixed on the fingertip of the index finger of the subject through an adjustable band; the micro optical measuring device (2113) is built-in with a light source and a photodetector, and detects the periodic fluctuation of light intensity caused by the change of blood volume through the photoplethysmography method to obtain the pulse signal.

5. The polygraph scene-oriented integrated multi-modal physiological behavioral signal acquisition system as claimed in claim 1, wherein, When collecting the ECG signal of the subject, the second surface electrode patch (2212) of the third sensor unit (221) is placed on the chest and back of the subject; when collecting the respiration signal of the subject, the respiration plethysmography band (2213) of the third sensor unit (221) is wrapped around the chest of the subject, and a flexible strain sensor is embedded in the band to detect the tension change of the band caused by the chest fluctuation during the respiration of the subject, and convert the tension change of the band into a corresponding electrical signal.

6. The polygraph lie detection scenario oriented integrated multi-modal physiological behavior signal acquisition system as claimed in claim 2, wherein, The first signal processing unit (12), the second signal processing unit (212) and the third signal processing unit (222) each include a microcontroller component, an analog front-end component, a communication component, a power management component and a state indication component electrically connected with the microcontroller component; wherein: The analog front-end component is configured to perform lead detection, amplification, analog-to-digital conversion, filtering and common-mode rejection operation on the signals received thereby, and transmit the processed signals to the microcontroller component; The communication component is configured to establish communication between the microcontroller component and a mobile workstation; The power management component is configured to supply power to the first signal processing unit (12), the second signal processing unit (212) or the third signal processing unit (222) as a whole; The microcontroller component is configured to coordinate and control the analog front-end component, the communication component, the power management component and the state indication component.

7. The integrated multi-modal polygraphic signal acquisition system for polygraphy scene according to claim 6, wherein, In the first signal processing unit (12), the microcontroller component is implemented based on an STM32F429 chip, the analog front-end component is implemented based on an ADS1299 chip, the communication component is implemented based on an ESP8266 chip through an MQTT protocol for wireless communication, the power management component is implemented through an SGM2082 voltage stabilizer and a MAX865 direct current conversion chip, and the state indication component is implemented by using an OLED screen; and / or, in the second signal processing unit (212), the microcontroller component is implemented based on an STM32F429 chip, the analog front-end component is implemented based on a MAX30102 pulse blood oxygen chip and an ADS1292 electrical signal chip, the communication component is implemented based on an ESP8266 chip through an MQTT protocol for wireless communication, the power management component is implemented through a TPS7A02 voltage stabilizer and an XL4015 direct current conversion chip, and the state indication component is implemented by using an OLED screen; and / or, in the third signal processing unit (222), the microcontroller component is implemented based on an STM32F429 chip, the analog front-end component is implemented based on an ADS1298R chip for ECG collection and an ADS124S08 chip for respiration collection, the communication component is implemented based on an ESP8266 chip through an MQTT protocol for wireless communication, the power management component is implemented through a TPS7A02 voltage stabilizer and an XL4015 direct current conversion chip, and the state indication component is implemented by using an OLED screen.

8. The polygraph scene-oriented integrated multi-modal physiological behavioral signal acquisition system as claimed in claim 1, wherein, The expression acquisition module (3) uses a high-definition camera to detect the facial expression image data of the subject, and transmits the data to the mobile workstation (4) through a wired connection; the high-definition camera is placed above the screen of the mobile workstation (4).

9. The polygraph method for polygraph scene-oriented integrated multi-modal physiological behavior signal acquisition system according to any one of claims 1 to 8, characterized in that, The method comprises: The brain electrical acquisition module (1) detects the brain electrical signals generated by the central nervous system of the subject; The peripheral physiological acquisition module (2) detects the physiological signals regulated by the peripheral nervous system of the subject, including: the physiological acquisition sub-module one (21) detects the skin electrical signal, the electromyographic signal and the pulse signal of the subject; the physiological acquisition sub-module two (22) detects the electrocardiographic signal and the respiratory signal of the subject; The expression acquisition module (3) detects the macro facial expression image data of the subject; The physiological acquisition sub-module one (21) comprises a second sensor unit (211) and a second signal processing unit (212), and the second sensor unit (211) is in communication connection with the second signal processing unit (212); the second sensor unit (211) comprises two groups of first surface electrode patches (2111) for detecting the skin electrical signal and the electromyographic signal of the subject, and a miniature optical measuring device (2113) for detecting the pulse signal of the subject; The physiological acquisition sub-module two (22) comprises a third sensor unit (221) and a third signal processing unit (222), and the third sensor unit (221) is in communication connection with the third signal processing unit (222); the third sensor unit (221) comprises a second surface electrode patch (2212) for detecting the electrocardiographic signal of the subject, and a respiratory inductive plethysmography belt (2213) for detecting the respiratory signal of the subject; The information collected by the brain electrical acquisition module (1), the peripheral physiological acquisition module (2) and the expression acquisition module (3) is uploaded to the mobile workstation (4), and the mobile workstation (4) realizes human-computer interaction through the visualization system (5) embedded therein.

Images