An integrated multi-modal physiological behavior signal collection system and method for a polygraph scene

Abstract

The application belongs to the technical field of brain-computer interface and physiological and psychological measurement, and particularly relates to an integrated multi-modal physiological behavior signal acquisition system and method for a lie detection scene. The system comprises: an electroencephalogram acquisition module for detecting brain electrical signals generated by the central nervous system; a peripheral physiological acquisition module comprising a physiological acquisition sub-module one and a physiological acquisition sub-module two, the second sensor unit of the physiological acquisition sub-module one comprising two groups of first surface electrode patches for detecting skin electricity signals and electromyography signals and a micro optical measuring device for detecting pulse signals; the third sensor unit of the physiological acquisition sub-module two comprising second surface electrode patches for detecting electrocardiogram signals and a respiratory inductive plethysmography belt for detecting respiratory signals; and an expression acquisition module for detecting subject macro facial expression image data. The application solves the problems of low system integration and portability and insufficient task specificity of the prior art.

Description

Technical Field

[0001] This invention belongs to the technical field of brain-computer interface and physiological and psychological measurement, specifically relating to an integrated multimodal physiological behavior signal acquisition system and method for lie detection scenarios. Background Technology

[0002] Lie detection technology is of great significance in fields such as judicial interrogation and public safety. Its core objective is to identify the veracity of testimonies, thereby providing objective evidence for key tasks such as criminal investigation and intelligence assessment. This technological field is undergoing profound changes, from peripheral indirect measurement to central direct decoding, and from single-indicator judgment to the fusion of multiple information sources.

[0003] Traditional lie detection primarily relies on monitoring and analyzing peripheral physiological parameters such as respiration, heart rate, and skin conductance, or non-contact behaviors such as facial expressions. While peripheral physiological parameters reflect the activity of the peripheral nervous system, and facial expressions, though regulated by the central nervous system, are still indirect behavioral manifestations, both can reflect the subject's emotional stress and cognitive load to some extent when lying. However, their inherent limitations—low generalization, susceptibility to subjective control, and interference from unrelated emotions and abnormal physiological states—significantly restrict the accuracy and reliability of the test results. For example, subjects can intentionally influence these signals through professional psychological training, leading to recognition failure; traditional lie detection also cannot determine whether the measured signals are triggered by other issues such as anxiety, fear, or hypoglycemia in the subject.

[0004] To overcome these limitations, research in this field has further shifted its focus to the direct decoding of central nervous system activity. Neuroimaging techniques such as electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) can directly capture neurophysiological activities closely related to the cognitive processing of lies, such as neural activation related to the identification of concealed information and the inhibition of genuine responses. These neural signals originate from higher cognitive functional areas of the brain and are difficult to fully control by the subject's consciousness. Therefore, they provide a more direct and reliable physiological basis for achieving highly specific lie detection, effectively compensating for the deficiencies of traditional peripheral physiological indicators or facial expressions.

[0005] Currently, multimodal methods integrating multiple signals have become a key focus of development in the field. A single signal source, whether facial expressions, peripheral physiological signals, or brain signals, is insufficient to fully depict the complex psychophysiological processes involved in lying. Therefore, integrating central nervous system signals (such as EEG) with peripheral physiological signals (such as ECG, EMG, and TENS), and combining this with facial expression and behavioral data for comprehensive analysis, has become an essential path to improve system performance. This multimodal architecture can effectively reduce the impact of individual differences through information complementarity and significantly enhance the system's anti-interference and anti-countermeasure capabilities.

[0006] Currently, some technical solutions involve the construction of multimodal signal acquisition systems and attempts are being made to apply them to tasks such as lie detection or cognitive state investigation:

[0007] Chinese patent document CN115316943A discloses a head-mounted multi-physiological parameter detection system and its detection method. This technical solution aims to conveniently and non-invasively acquire multiple channels of physiological parameters such as impedance, blood oxygenation, pulse rate, and body temperature of a subject through an integrated head-mounted device.

[0008] Chinese patent document CN116965830A discloses an occupant comfort evaluation system and method based on multimodal physiological data. The system integrates a set of human physiological feedback acquisition devices, which can collect multiple signals including electroencephalogram (EEG), electrocardiogram (ECG), electromyography (EMG), electrodermal conductance (EDC), eye movement, and skin temperature, and quantitatively evaluate the occupant's comfort level based on these signals.

[0009] Chinese patent document CN119679411A discloses a multimodal portable cognitive screening device. This device can record multimodal behavioral and physiological data of users during testing and training, including eye movement trajectories, electroencephalogram signals, scale data, facial expressions, and gait characteristics.

[0010] Chinese patent document CN115299947A discloses a method and system for assessing the confidence of psychological scales based on multimodal physiological data. By integrating multimodal data such as micro-expressions, electrocardiograms, electroencephalograms, eye movements, and electrodermal conductance, it can assess the confidence of whether a subject is lying.

[0011] Chinese patent document CN119523439A discloses a wearable multi-channel physiological parameter acquisition system. This system can meet the needs of stable acquisition of physiological data such as electroencephalogram (EEG), blood oxygen saturation, skin conductance, pulse, body temperature, and eye movement of the pilot in special environments such as air flight or ground driving.

[0012] Chinese patent document CN113951886A discloses a brain magnetic signature generation system and a lie detection decision system. This generation and decision system comprehensively collects brain magnetic signature, heart rate, respiratory rate, skin conductance, and micro-expression data of the subject, and quantifies the probability of truthfulness of the answers through statistical analysis of multimodal information.

[0013] However, these existing technologies still have many limitations in terms of system integration, portability, and specialization for lie detection tasks, specifically:

[0014] 1) Insufficient system integration and portability.

[0015] Some existing systems (such as patent document CN115299947A) employ a collaborative working mode of multiple independent devices, which not only increases the overall system complexity but also directly impacts the user experience due to the cumbersome donning process, potentially causing discomfort to test subjects and affecting the stability of signal acquisition and data quality. Furthermore, existing devices (such as patent documents CN116965830A, CN115299947A, and CN113951886A) suffer from large size, relatively dispersed components, or high environmental requirements, making them difficult to integrate into portable carriers. This severely limits deployment to fixed scenarios such as laboratories, hindering their suitability for changing scenarios requiring rapid response, such as on-site investigations and mobile law enforcement.

[0016] 2) Lack of adaptability to lie detection scenarios.

[0017] On the one hand, existing lie detection systems lack comprehensive coverage of relevant indicators, failing to fully collect key physiological and behavioral signals such as EEG, skin conductance, pulse, electromyography, electrocardiography, respiration, and facial expressions (e.g., patent documents CN115316943A, CN116965830A, CN119679411A, CN115299947A, CN119523439A). This makes it difficult to support systematic lie detection judgments; for example, the lack of EEG and electromyography signals may lead to distorted lie detection conclusions in adversarial scenarios. On the other hand, existing devices lack specific optimization designs for lie detection scenarios, especially in the EEG acquisition module. Electrode configuration fails to balance acquisition effectiveness with system power consumption (e.g., patent documents CN116965830A, CN119679411A, CN115299947A, CN119523439A). Redundant electrodes increase system complexity and cost, while insufficient electrodes can cause the loss of signals in key brain regions, directly affecting detection accuracy.

[0018] Based on this, the present invention designs an integrated multimodal physiological behavior signal acquisition system and method for lie detection scenarios, in order to overcome the limitations of existing technologies in terms of system integration, portability, and specialization of lie detection tasks. Summary of the Invention

[0019] This invention aims to overcome at least one of the defects of the prior art and provide an integrated multimodal physiological and behavioral signal acquisition system for lie detection scenarios. By improving module integration and wearability, it constructs an integrated acquisition system that can be integrated into a suitcase carrier. At the same time, in response to the special needs of lie detection applications, it fully covers key physiological indicators such as EEG, skin conductance, pulse, electromyography, electrocardiography, and respiration, as well as behavioral indicators such as facial expressions. Furthermore, it optimizes the configuration of EEG electrodes to achieve a balance between acquisition effect and power consumption cost.

[0020] The present invention also discloses a method for an integrated multimodal physiological behavior signal acquisition system applied to the aforementioned lie detection scenario.

[0021] The detailed technical solution of this invention is as follows:

[0022] An integrated multimodal physiological and behavioral signal acquisition system for lie detection scenarios, comprising:

[0023] The EEG acquisition module is used to detect the electrical signals generated by the central nervous system in the brain;

[0024] The peripheral physiological acquisition module is used to detect physiological signals regulated by the peripheral nervous system. It includes a physiological acquisition submodule one and a physiological acquisition submodule two. The physiological acquisition submodule one is used to detect skin conductance signals, electromyography signals, and pulse signals; the physiological acquisition submodule two is used to detect electrocardiogram signals and respiratory signals.

[0025] The facial expression acquisition module is used to detect macroscopic facial expression image data of the subjects;

[0026] The physiological acquisition submodule includes a second sensor unit and a second signal processing unit, which are communicatively connected. The second sensor unit includes two sets of first surface electrode patches for detecting the subject's electrodermal and electromyographic signals, respectively, and a miniature optical measuring device for detecting the subject's pulse signal.

[0027] The physiological acquisition submodule 2 includes a third sensor unit and a third signal processing unit, and the third sensor unit is communicatively connected to the third signal processing unit; the third sensor unit includes a second surface electrode patch for detecting the subject's electrocardiogram signal and a respiratory induction plethysmography band for detecting the subject's respiratory signal.

[0028] The EEG acquisition module, peripheral physiological acquisition module, and facial expression acquisition module are all communicatively connected to the mobile workstation. The mobile workstation has an embedded visualization system, and the EEG acquisition module, peripheral physiological acquisition module, facial expression acquisition module, and mobile workstation are all integrated into a carrying case.

[0029] According to a preferred embodiment of the present invention, the EEG acquisition module includes a first sensor unit and a first signal processing unit, wherein the first sensor unit is communicatively connected to the first signal processing unit; the first sensor unit is an 8-channel electrode group, and during the acquisition of the subject's EEG signal, the 8-channel electrode group selects Fz, Cz, F3, F4, C3, C4, P3, and P4 sites respectively.

[0030] According to a preferred embodiment of the present invention, when collecting electrodermal signals from the subject, a set of first surface electrode patches of the second sensor unit are placed on the fingertips of the subject's middle and ring fingers; when collecting electromyographic signals from the subject, another set of first surface electrode patches of the second sensor unit are placed on the biceps brachii muscle and inner side of the forearm of the subject's upper arm.

[0031] According to a preferred embodiment of the present invention, during the acquisition of the subject's pulse signal, the miniature optical measuring device of the second sensor unit is fixed to the fingertip of the subject's index finger by an adjustable strap; the miniature optical measuring device has a built-in light source and photodetector, and obtains the pulse signal by detecting the periodic fluctuation of light intensity caused by changes in blood volume through photoplethysmography.

[0032] According to a preferred embodiment of the present invention, when the electrocardiogram signal of the subject is collected, the second surface electrode patch of the third sensor unit is placed on the chest, abdomen and back of the subject; when the respiratory signal of the subject is collected, the respiratory plethysmography band of the third sensor unit is wrapped around the chest of the subject, and a flexible strain sensor is embedded in the band to detect the change in band tension caused by the chest rise and fall during the subject's breathing, and convert this change in band tension into a corresponding electrical signal.

[0033] According to a preferred embodiment of the present invention, the first signal processing unit, the second signal processing unit, and the third signal processing unit each include a microcontroller component, and an analog front-end component, a communication component, a power management component, and a status indication component electrically connected to the microcontroller component; wherein:

[0034] The analog front-end component is used to perform lead detection, amplification, analog-to-digital conversion, filtering, and common-mode rejection operations on the received signal, and transmit the processed signal to the microcontroller component.

[0035] The communication component is used to establish communication between the microcontroller component and the mobile workstation;

[0036] The power management component is used to supply power to the first signal processing unit, the second signal processing unit, or the third signal processing unit corresponding to it.

[0037] The microcontroller component is used to coordinate and control the operation of the analog front-end component, communication component, power management component, and status indication component.

[0038] According to a preferred embodiment of the present invention, in the first signal processing unit: the microcontroller component implements its function based on an STM32F429 chip; the analog front-end component implements its function based on an ADS1299 chip; the communication component implements wireless communication via the MQTT protocol based on an ESP8266 chip; the power management component implements its function through an SGM2082 voltage regulator and a MAX865 DC-DC converter chip; and the status indication component uses an OLED screen to implement its function.

[0039] And / or, in the second signal processing unit: the microcontroller component implements its function based on the STM32F429 chip; the analog front-end component implements its function based on the MAX30102 pulse oximetry chip and the ADS1292 electrical signal chip; the communication component implements wireless communication via the MQTT protocol based on the ESP8266 chip; the power management component implements its function through the TPS7A02 voltage regulator and the XL4015 DC-DC converter chip; and the status indication component uses an OLED screen to implement its function.

[0040] And / or, in the third signal processing unit: the microcontroller component implements its function based on the STM32F429 chip; the analog front-end component implements the ECG acquisition function based on the ADS1298R chip and the respiration acquisition function based on the ADS124S08 chip; the communication component implements wireless communication via the MQTT protocol based on the ESP8266 chip; the power management component implements its function through the TPS7A02 voltage regulator and the XL4015 DC-DC converter chip; and the status indication component uses an OLED screen to implement its function.

[0041] According to a preferred embodiment of the present invention, the facial expression acquisition module uses a high-definition camera to detect facial expression image data of the subject and transmits it to the mobile workstation via a wired connection; the high-definition camera is placed above the screen of the mobile workstation.

[0042] In another aspect of the present invention, an integrated multimodal physiological and behavioral signal acquisition method for lie detection scenarios is provided, applied to the integrated multimodal physiological and behavioral signal acquisition system for lie detection scenarios as described above, the method comprising:

[0043] The brain electrical signals generated by the central nervous system of the subjects were detected using an EEG acquisition module.

[0044] The peripheral physiological acquisition module was used to detect the physiological signals of the subject's peripheral nervous system regulation, including: using physiological acquisition submodule one to detect the subject's skin conductance, electromyography and pulse signals; and using physiological acquisition submodule two to detect the subject's electrocardiogram and respiratory signals.

[0045] Macroscopic facial expression image data of subjects were detected using an expression acquisition module;

[0046] The physiological acquisition submodule includes a second sensor unit and a second signal processing unit, which are communicatively connected. The second sensor unit includes two sets of first surface electrode patches for detecting the subject's electrodermal and electromyographic signals, respectively, and a miniature optical measuring device for detecting the subject's pulse signal.

[0047] The physiological acquisition submodule 2 includes a third sensor unit and a third signal processing unit, and the third sensor unit is communicatively connected to the third signal processing unit; the third sensor unit includes a second surface electrode patch for detecting the subject's electrocardiogram signal and a respiratory induction plethysmography band for detecting the subject's respiratory signal.

[0048] The information collected by the EEG acquisition module, peripheral physiological acquisition module, and facial expression acquisition module is uploaded to the mobile workstation, which realizes human-computer interaction through its embedded visualization system.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] (1) The present invention provides an integrated multimodal physiological behavior signal acquisition system for lie detection scenarios, which realizes the high integration and portability of the system. By integrating the multimodal acquisition module, mobile workstation and all supporting components into a suitcase and adopting wireless communication management on a large scale, it changes the defects of traditional system equipment being scattered, wiring being complicated and deployment being difficult, improves user experience and detection efficiency, enhances the mobility and environmental adaptability of the equipment, and provides a feasible technical foundation for lie detection applications in non-laboratory scenarios.

[0051] (2) This invention constructs a complete signal acquisition system for lie detection, achieving a better balance between performance and efficiency. It simultaneously acquires multimodal physiological and behavioral data such as EEG, electrodermal conductance, electrocardiogram, electromyography, pulse, respiration, and facial expressions, overcoming the shortcomings of incomplete modality coverage in existing technologies and improving the data integrity and reliability of lie detection judgment. In addition, the 10-20 standard lead system has been specifically optimized, significantly reducing the number of electrodes and lowering system power consumption and construction costs while ensuring the capture of signals from key brain regions such as the prefrontal and parietal lobes. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the hardware structure of the integrated multimodal physiological behavior signal acquisition system for lie detection scenarios described in this invention.

[0053] Figure 2This is a schematic diagram of the wearable form of the integrated multimodal physiological behavior signal acquisition system for lie detection scenarios described in Embodiment 1 of the present invention;

[0054] Figure 3 This is a diagram illustrating the architecture of the integrated multimodal physiological behavior signal acquisition system for lie detection scenarios described in Embodiment 1 of the present invention.

[0055] Figure 4 This is a diagram showing the composition architecture of the EEG and peripheral physiological modules of the integrated multimodal physiological behavior signal acquisition system for lie detection scenarios described in Embodiment 1 of the present invention.

[0056] Figure 5 This is a data visualization diagram of the integrated multimodal physiological behavior signal acquisition system for lie detection scenarios described in Embodiment 1 of the present invention;

[0057] Figure 6 This is a flowchart of the integrated multimodal physiological behavior signal acquisition method for lie detection scenarios described in this invention;

[0058] In the diagram: 1. EEG acquisition module; 11. First sensor unit; 12. First signal processing unit; 2. Peripheral physiological acquisition module; 21. Physiological acquisition sub-module one; 211. Second sensor unit; 2111. First surface electrode patch; 2112. First lead wire; 2113. Miniature optical measuring device; 212. Second signal processing unit; 22. Physiological acquisition sub-module two; 221. Third sensor unit; 2211. Second lead wire; 2212. Second surface electrode patch; 2213. Respiratory plethysmography tape; 222. Third signal processing unit; 3. Facial expression acquisition module; 4. Mobile workstation; 5. Visualization system; 6. Suitcase; 61. Suitcase combination lock; 62. Suitcase handle. Detailed Implementation

[0059] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0060] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0061] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0062] Without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0063] Embodiment 1

[0064] Refer Figure 1 、 Figure 3 , this embodiment provides an integrated multi-modal physiological behavior signal acquisition system for the lie detection scenario, which includes:

[0065] An electroencephalogram acquisition module 1, which is used to detect the brain electrical signals generated by the central nervous system;

[0066] A peripheral physiology acquisition module 2, which is used to detect the physiological signals regulated by the peripheral nervous system; [[ID=...]]

[0067] An expression acquisition module 3, which is used to detect the macroscopic facial expression image data of the subject;

[0068] The electroencephalogram acquisition module 1, the peripheral physiology acquisition module 2, and the expression acquisition module 3 are all communicatively connected to a mobile workstation 4. The mobile workstation 4 is embedded with a visualization system 5, and the electroencephalogram acquisition module 1, the peripheral physiology acquisition module 2, the expression acquisition module 3, and the mobile workstation 4 are all integrated in a suitcase ⑥.

[0069] Specifically, the electroencephalogram acquisition module 1 includes a first sensor unit 11 and a first signal processing unit 12, and the first sensor unit 11 is electrically connected to the first signal processing unit 12.

[0070] The first sensor unit 11 is a set of 8-channel dry electrode group, which is convenient to wear; and this dry electrode group follows the 10-20 international standard lead system, and only includes the frontal and parietal electrodes with a relatively high degree of relevance to the lie detection task. When testing, it is placed on the head of the subject, and can be referred to Figure 2 as shown.

[0071] [[ID=...]] Furthermore, when collecting the brain electrical signals of the subject, the 8-channel dry electrode group can select silver chloride-plated rigid dry electrodes to ensure good electrical conductivity and biocompatibility; the supporting cap body can select elastic mesh cloth to ensure comfort, breathability and stable fitting to the scalp.

[0072] Among them, the 8-channel dry electrode group can respectively select the Fz, Cz, F3, F4, C3, C4, P3, and P4 sites. This electrode layout is specifically optimized for the brain cognitive activities in the lie detection scenario. On the premise of ensuring the capture of signals in key brain regions such as the prefrontal lobe and parietal lobe, the number of electrodes is significantly reduced. Compared with the electrode layout with more leads, the redundant information in the lie detection task is eliminated, and the system power consumption and construction cost are reduced.

[0073] Specifically, the Fz, F3, and F4 sites are located in the prefrontal cortex and are used to collect EEG signals associated with cognitive conflict, executive control, and inhibition of real responses during the lie-construction process; the P3 and P4 sites are located in the parietal cortex and are used to collect EEG signals associated with attention allocation and memory retrieval; the Cz, C3, and C4 sites are located in the central region and, while providing a signal reference benchmark, are used to collect EEG signals associated with emotional fluctuations and psychological stress.

[0074] The first signal processing unit 12 includes a microcontroller component, and an analog front-end component, a communication component, a power management component, and a status indication component electrically connected to the microcontroller component, the architecture of which is as follows: Figure 4 As shown.

[0075] The analog front-end component is used to perform lead detection, amplification, analog-to-digital conversion, filtering, and common-mode rejection on the weak scalp signal it receives, and transmits the processed standard digital signal to the microcontroller component.

[0076] The microcontroller component serves as the module control center, receiving digital signals from the analog front-end component and coordinating the working status of the control communication component, power management component, and status indication component to complete data acquisition, processing, and transmission tasks.

[0077] The communication component is used to establish communication between the microcontroller component and the mobile workstation 4. Under the instruction of the microcontroller component, it receives instructions from the mobile workstation 4 via wireless communication and transmits back the collected data output by the microcontroller.

[0078] The power management component provides a stable power supply to the analog front-end component, microcontroller component, communication component, and status indicator component, ensuring the normal operation of each component.

[0079] The status indicator component is used to acquire key parameters of the microcontroller component, communication component and power management component, and to display the module's working status, data connectivity with the mobile workstation 4 and remaining power information in real time through its embedded display screen.

[0080] As a further preferred embodiment, in the first signal processing unit 12:

[0081] The core chip of the microcontroller component can be an STM32F429 series chip, which enables efficient coordination of the operation of various components; the core chip of the analog front-end component can be an ADS1299 chip, which has high-precision signal acquisition capabilities; the communication component can achieve wireless communication based on the ESP8266 chip through the MQTT protocol, ensuring low-latency data transmission; the core chips of the power management component can be an SGM2082 voltage regulator and a MAX865 DC-DC converter chip, which ensures battery life and stability; the status indicator component has an embedded display screen, which can be an OLED screen, providing clear display and low power consumption.

[0082] Understandably, all the aforementioned components are integrated and encapsulated within a single, black, cube-shaped housing, and are electrically connected to the first sensor unit 11 via a standardized interface. Furthermore, the housing structure can be made of lightweight ABS+PC alloy, combining durability and lightness.

[0083] The peripheral physiological acquisition module 2 includes a physiological acquisition submodule 1 21 and a physiological acquisition submodule 22. The physiological acquisition submodule 1 21 is used to detect skin conductance signals, electromyography signals and pulse signals; the physiological acquisition submodule 22 is used to detect electrocardiogram signals and respiratory signals.

[0084] In this embodiment, the physiological acquisition submodule 21 includes a second sensor unit 211 and a second signal processing unit 212, which can be electrically connected through a first wire 2112.

[0085] The second sensor unit 211 includes two sets of first surface electrode patches 2111 for detecting the subject's electrodermal signal and electromyographic signal, respectively, and a miniature optical measuring device 2113 for detecting the subject's pulse signal.

[0086] Furthermore, during the acquisition of the subject's electrodermal signals, a set of first surface electrode patches 2111 of the second sensor unit 211 is placed on the fingertips of the subject's middle and ring fingers; simultaneously, during the acquisition of the subject's electromyographic signals, another set of first surface electrode patches 2111 of the second sensor unit 211 is placed on the subject's biceps brachii and inner side of the forearm.

[0087] During the acquisition of the subject's pulse signal, the miniature optical measuring device 2113 of the second sensor unit 211 is fixed to the fingertip of the subject's index finger by an adjustable strap; the miniature optical measuring device 2113 has a built-in light source and photodetector, and obtains the pulse signal by detecting the periodic fluctuation of light intensity caused by changes in blood volume through photoplethysmography.

[0088] During signal acquisition, the first wire 2112 can ensure the reliability of data transmission from the second sensor unit 211 to the second signal processing unit 212.

[0089] The second signal processing unit 212 includes a microcontroller component, and an analog front-end component, a communication component, a power management component, and a status indication component electrically connected to the microcontroller component, the architecture of which is as follows: Figure 4 As shown.

[0090] The analog front-end component is used to perform lead detection, amplification, analog-to-digital conversion, filtering, and common-mode rejection operations on the received electrodermal, pulse, and electromyographic signals, and transmits the processed standardized digital signals to the microcontroller component.

[0091] The microcontroller component serves as the module control center, receiving digital signals from the analog front-end component and coordinating the working status of the control communication component, power management component, and status indication component to complete data acquisition, processing, and transmission tasks.

[0092] The communication component is used to establish communication between the microcontroller component and the mobile workstation 4. Under the instruction of the microcontroller component, it receives instructions from the mobile workstation 4 via wireless communication and transmits back the collected data output by the microcontroller.

[0093] The power management component provides a stable power supply to the analog front-end component, microcontroller component, communication component, and status indicator component, ensuring the normal operation of each component.

[0094] The status indicator component is used to acquire key parameters of the microcontroller component, communication component and power management component, and to display the sub-module working status, data connectivity with the mobile workstation 4 and remaining power information in real time through its embedded display screen.

[0095] As a further preferred embodiment, in the second signal processing unit 212:

[0096] The core chip of the microcontroller component can be an STM32F429 series chip to achieve efficient coordination of the operation of various components; the core chip of the analog front-end component can be a MAX30102 pulse oximeter chip and an ADS1292 electrical signal chip to realize its function; the communication component can realize wireless communication through the MQTT protocol based on the ESP8266 chip; the core chip of the power management component can be a TPS7A02 voltage regulator and an XL4015 DC-DC converter chip to realize its function; the status indicator component has an embedded display screen, which can be an OLED screen, for clear display and low power consumption.

[0097] Understandably, all the aforementioned components are integrated and encapsulated within a single, black, cube-shaped housing, and electrically connected to the second sensor unit 211 via a standardized interface. This encapsulation structure is integrated onto the wristband and secured to the wrist, a design that promotes ease and comfort of wear, see reference. Figure 2 As shown. Similarly, this packaging structure can be made of lightweight ABS+PC alloy material, combining durability and lightness.

[0098] In this embodiment, the physiological acquisition submodule 22 includes a third sensor unit 221 and a third signal processing unit 222, which can be electrically connected via a second wire 2211.

[0099] The third sensor unit 221 includes a second surface electrode patch 2212 for detecting the subject's electrocardiogram signal and a respiratory plethysmography band 2213 for detecting the subject's respiratory signal.

[0100] Furthermore, during the acquisition of the subject's electrocardiogram signal, the second surface electrode patch 2212 of the third sensor unit 221 is placed on the subject's chest, abdomen, and back to detect the minute potential difference on the skin surface caused by the heartbeat, thereby recording cardiac activity; simultaneously, during the acquisition of the subject's respiratory signal, the respiratory plethysmography band 2213 of the third sensor unit 221 is wrapped around the subject's chest, and the band is embedded with a flexible tension sensing element, such as a flexible strain sensor, to detect the change in band tension caused by the rise and fall of the chest during the subject's breathing, and convert this change in band tension into a corresponding electrical signal.

[0101] During signal acquisition, the second wire 2211 can ensure the reliability of data transmission from the third sensor unit 221 to the third signal processing unit 222.

[0102] The third signal processing unit 222 includes a microcontroller component, and an analog front-end component, a communication component, a power management component, and a status indication component electrically connected to the microcontroller component. Its architecture is as follows: Figure 4 As shown.

[0103] The analog front-end component is used to perform lead detection, amplification, analog-to-digital conversion, filtering, and common-mode suppression operations on the received respiratory and electrocardiogram signals, and transmits the processed standardized digital signals to the microcontroller component.

[0104] The microcontroller component serves as the module control center, receiving digital signals from the analog front-end component and coordinating the working status of the control communication component, power management component, and status indication component to complete data acquisition, processing, and transmission tasks.

[0105] The communication component is used to establish communication between the microcontroller component and the mobile workstation 4. Under the instruction of the microcontroller component, it receives instructions from the mobile workstation 4 via wireless communication and transmits back the collected data output by the microcontroller.

[0106] The power management component provides a stable power supply to the analog front-end component, microcontroller component, communication component, and status indicator component, ensuring the normal operation of each component.

[0107] The status indicator component is used to acquire key parameters of the microcontroller component, communication component and power management component, and to display the sub-module working status, data connectivity with the mobile workstation 4 and remaining power information in real time through its embedded display screen.

[0108] As a further preferred embodiment, in the third signal processing unit 222:

[0109] The core chip of the microcontroller component can be an STM32F429 series chip to achieve efficient coordination of the operation of various components; the analog front-end component uses an ADS1298R chip to realize the ECG acquisition function and an ADS124S08 chip to realize the respiration acquisition function; the communication component can realize wireless communication through the MQTT protocol based on the ESP8266 chip; the core chip of the power management component can be a TPS7A02 voltage regulator and an XL4015 DC-DC converter chip to realize its function; the status indicator component has an embedded display screen, which can be an OLED screen, for clear display and low power consumption.

[0110] Understandably, all the aforementioned components are integrated and encapsulated within a single, black, cube-shaped housing, and electrically connected to the third sensor unit 221 via a standardized interface. This encapsulation structure is attached to the outside of the breathing strap and secured to the chest; this design facilitates ease and comfort of wear. (Refer to...) Figure 2 As shown. Similarly, this packaging structure can be made of lightweight ABS+PC alloy material, combining durability and lightness.

[0111] In this embodiment, the facial expression acquisition module 3 can use a high-definition camera to detect the facial expression image data of the subject and transmit it to the mobile workstation 4 via a wired connection; the high-definition camera is placed above the screen of the mobile workstation 4, for reference. Figure 2 As shown.

[0112] Preferably, the high-definition camera of the expression acquisition module 3 supports 1080p or higher high-definition resolution, which can clearly capture facial details; the lens is made of multi-layer coated optical glass to reduce light reflection; and the connection method is a USB interface to ensure high-speed transmission of image data.

[0113] Furthermore, participants Figure 5 The visualization system 5 is used by the experimenter to control the operation of the acquisition module, receive and store relevant multimodal physiological and behavioral signals, and display them on the screen. Specifically, the visualization system 5 displays the operating status of the multimodal physiological and behavioral signal acquisition system on the screen of the mobile workstation 4, including the connection status with the acquisition module, the start, pause and stop of acquisition, data storage and export options, and displays the subject's basic information, EEG signals, peripheral physiological signals, facial expression videos, experimental stimulus images, and preliminary real-time reliability level during the acquisition process.

[0114] Preferably, the mobile workstation 4 on which the visualization system 5 is based is equipped with no less than 16GB of memory and 512GB of storage space, multiple USB ports, and Bluetooth, WiFi, etc.

[0115] The carrying case 6 is used to carry the integrated multimodal physiological and behavioral signal acquisition system for lie detection scenarios. Its outer shell is made of high-strength engineering plastic, and the inner lining is provided with fixing slots that match the shape of the components of the multimodal physiological and behavioral signal acquisition system to stably support and isolate the components. The carrying case 6 is equipped with a combination lock 61 for equipment security protection. The carrying case 6 is also equipped with a handle 62 for gripping when carrying.

[0116] Preferably, the outer shell of the suitcase 6 can be made of ABS+PC alloy material to provide high structural strength and impact resistance; the inner lining material can be made of rigid EVA closed-cell foam to provide effective cushioning and shock protection; and the combination lock 61 can be a mechanical combination lock.

[0117] Example 2

[0118] This embodiment provides an integrated multimodal physiological and behavioral signal acquisition method for lie detection scenarios, applied to the integrated multimodal physiological and behavioral signal acquisition system for lie detection scenarios described in Embodiment 1. The method includes:

[0119] The brain electrical signals generated by the central nervous system of the subject were detected using the EEG acquisition module 1.

[0120] The peripheral physiological acquisition module 2 is used to detect the physiological signals of the subject's peripheral nervous system regulation, including: using physiological acquisition submodule 1 21 to detect the subject's skin conductance signal, electromyography signal and pulse signal; and using physiological acquisition submodule 2 22 to detect the subject's electrocardiogram signal and respiratory signal.

[0121] Macroscopic facial expression image data of the subjects were detected using expression acquisition module 3;

[0122] The physiological acquisition submodule 21 includes a second sensor unit 211 and a second signal processing unit 212, which are communicatively connected. The second sensor unit 211 includes two sets of first surface electrode patches 2111 for detecting the subject's electrodermal signal and electromyographic signal, respectively, and a miniature optical measuring device 2113 for detecting the subject's pulse signal.

[0123] The physiological acquisition submodule 22 includes a third sensor unit 221 and a third signal processing unit 222. The third sensor unit 221 is communicatively connected to the third signal processing unit 222. The third sensor unit 221 includes a second surface electrode patch 2212 for detecting the subject's electrocardiogram signal and a respiratory induction plethysmography band 2213 for detecting the subject's respiratory signal.

[0124] The information collected by the EEG acquisition module 1, the peripheral physiological acquisition module 2, and the facial expression acquisition module 3 is uploaded to the mobile workstation 4, and the mobile workstation 4 realizes human-computer interaction through its embedded visualization system 5.

[0125] Furthermore, participants Figure 6 The data acquisition process of the method is as follows:

[0126] Step 1. Power on and start up: Take the multimodal physiological behavior signal acquisition system out of the suitcase, turn on the system power, and confirm that each acquisition module is working properly and has sufficient power.

[0127] Step 2. Personnel Registration: Inform the subjects of the purpose and principles of the experiment, have them fill out an informed consent form, and register the subjects' basic information, including name, age, height, and gender.

[0128] Step 3. Device Wearing: Correctly wear or install the relevant sensor units of the EEG acquisition module, peripheral physiological acquisition module, and facial expression acquisition module in the corresponding positions on the subject or the table;

[0129] Step 4. Parameter settings: Establish a data connection with the mobile workstation and set the correct acquisition parameters to ensure that data from each modality can be effectively acquired;

[0130] Step 5. Preliminary Experiment: Conduct a preliminary experiment to ensure the experimental procedure runs normally and to familiarize the subjects with the experimental process;

[0131] Step 6. Formal Experiment: Send recording instructions to each acquisition module via the mobile workstation to detect the subject's EEG, TESC, EMG, ECG, respiration, pulse and facial expression information during the experiment;

[0132] Step 7. Data transmission: Relevant data is transmitted to the mobile workstation via wired or wireless means and displayed in real time through a visualization system;

[0133] Step 8. Organize and archive: Remove the relevant sensor units of each acquisition module from the subject, turn off the system power, and place them in a carrying case. Organize, store, and back up the relevant data in the mobile workstation for subsequent analysis.

[0134] In summary, this invention achieves a high degree of system integration and portability. By integrating the multimodal acquisition module, mobile workstation, and all supporting components into a carrying case and extensively employing wireless communication management, it overcomes the shortcomings of traditional systems, such as dispersed equipment, complex wiring, and difficult deployment. This improves user experience and detection efficiency, enhances the mobility and environmental adaptability of the equipment, and provides a feasible technical foundation for lie detection applications in non-laboratory scenarios.

[0135] On the other hand, this invention constructs a complete signal acquisition system for lie detection, achieving a better balance between performance and efficiency. It simultaneously acquires multimodal physiological and behavioral data, including EEG, electrodermal data, ECG, electromyography, pulse, respiration, and facial expressions, overcoming the shortcomings of incomplete modality coverage in existing technologies and improving the data integrity and reliability of lie detection judgments. Furthermore, the 10-20 standard lead system has been specifically optimized, significantly reducing the number of electrodes and lowering system power consumption and construction costs while ensuring the capture of signals from key brain regions such as the prefrontal and parietal lobes.

[0136] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A polyphasic multi-modal physiological behavior signal acquisition system for a lie detection scenario, characterized in that, The system comprises: An electroencephalogram acquisition module (1) for detecting brain electrical signals generated by the central nervous system; The electroencephalogram acquisition module (1) comprises a first sensor unit (11) and a first signal processing unit (12), and the first sensor unit (11) is in communication connection with the first signal processing unit (12); the first sensor unit (11) is an 8-lead dry electrode set, and when collecting the brain electrical signals of the subject, the 8-lead dry electrode set selects Fz, Cz, F3, F4, C3, C4, P3 and P4 sites respectively; A peripheral physiological acquisition module (2) for detecting physiological signals regulated by the peripheral nervous system, which comprises a physiological acquisition submodule one (21) and a physiological acquisition submodule two (22); the physiological acquisition submodule one (21) is used for detecting skin electrical signals, electromyographic signals and pulse signals; the physiological acquisition submodule two (22) is used for detecting electrocardiographic signals and respiratory signals; An expression acquisition module (3) for detecting macro facial expression image data of the subject; The physiological acquisition submodule one (21) comprises a second sensor unit (211) and a second signal processing unit (212), and the second sensor unit (211) is in communication connection with the second signal processing unit (212); the second sensor unit (211) comprises two groups of first surface electrode patches (2111) for detecting skin electrical signals and electromyographic signals of the subject respectively, and a miniature optical measuring device (2113) for detecting pulse signals of the subject; The physiological acquisition submodule two (22) comprises a third sensor unit (221) and a third signal processing unit (222), and the third sensor unit (221) is in communication connection with the third signal processing unit (222); the third sensor unit (221) comprises a second surface electrode patch (2212) for detecting electrocardiographic signals of the subject, and a respiratory inductive plethysmography belt (2213) for detecting respiratory signals of the subject; The first signal processing unit (12), the second signal processing unit (212) and the third signal processing unit (222) all comprise a communication component, and the communication component realizes wireless communication through an ESP8266 chip based on an MQTT protocol; The electroencephalogram acquisition module (1), the peripheral physiological acquisition module (2) and the expression acquisition module (3) are all in communication connection with a mobile workstation (4), the mobile workstation (4) is embedded with a visualization system (5), and the electroencephalogram acquisition module (1), the peripheral physiological acquisition module (2), the expression acquisition module (3) and the mobile workstation (4) are all integrated in a suitcase (6); When collecting the skin electrical signals of the subject, one group of first surface electrode patches (2111) of the second sensor unit (211) is placed on the middle finger and the middle finger pulp of the subject; when collecting the electromyographic signals of the subject, the other group of first surface electrode patches (2111) of the second sensor unit (211) is placed on the biceps brachii and the medial epicondyle of the subject; In the process of collecting the pulse signal of the subject, the micro optical measuring device (2113) of the second sensor unit (211) is fixed on the fingertip of the subject's index finger through an adjustable bandage; the micro optical measuring device (2113) is internally provided with a light source and a photodetector, and detects the periodic fluctuation of light intensity caused by the change of blood volume through photoplethysmography to obtain the pulse signal; In the process of collecting the ECG signal of the subject, the second surface electrode patch (2212) of the third sensor unit (221) is placed on the chest and back of the subject; in the process of collecting the respiration signal of the subject, the respiration inductive plethysmography band (2213) of the third sensor unit (221) is wrapped around the chest of the subject, and a flexible strain sensor is embedded in the bandage to detect the change of bandage tension caused by the fluctuation of the chest during the respiration of the subject, and convert the change of bandage tension into corresponding electrical signals.

2. The polygraph lie detection scenario oriented integrated multi-modal physiological behavior signal acquisition system as claimed in claim 1, wherein, The first signal processing unit (12), the second signal processing unit (212) and the third signal processing unit (222) each include a microcontroller component, an analog front-end component, a communication component, a power management component and a state indication component electrically connected to the microcontroller component; wherein: The analog front-end component is used for lead detection, amplification, analog-to-digital conversion, filtering and common-mode rejection operation on the signals received thereby, and transmits the processed signals to the microcontroller component; The communication component is used for establishing communication between the microcontroller component and the mobile workstation; The power management component is used for powering the entire first signal processing unit (12), second signal processing unit (212) or third signal processing unit (222) corresponding thereto; The microcontroller component is used for coordinating and controlling the analog front-end component, communication component, power management component and state indication component.

3. The integrated multi-modal polygraphic signal acquisition system for polygraphy scene according to claim 2, wherein, In the first signal processing unit (12): the microcontroller component realizes its function based on an STM32F429 chip; the analog front-end component realizes its function based on an ADS1299 chip; the communication component realizes wireless communication through an ESP8266 chip based on the MQTT protocol; the power management component realizes its function through an SGM2082 voltage stabilizer and a MAX865 direct current conversion chip; and the state indication component realizes its function by using an OLED screen; And / or, in the second signal processing unit (212): the microcontroller component realizes its function based on an STM32F429 chip; the analog front-end component realizes its function based on a MAX30102 pulse blood oxygen chip and an ADS1292 electrical signal chip; the communication component realizes wireless communication through an ESP8266 chip based on the MQTT protocol; the power management component realizes its function through a TPS7A02 voltage stabilizer and an XL4015 direct current conversion chip; and the state indication component realizes its function by using an OLED screen. And / or, in the third signal processing unit (222): the microcontroller component realizes its function based on STM32F429 chip; the analog front-end component realizes electrocardio collection function based on ADS1298R chip, and realizes breath collection function based on ADS124S08 chip; the communication component realizes wireless communication through MQTT protocol based on ESP8266 chip; the power management component realizes its function through TPS7A02 voltage stabilizer and XL4015 direct current conversion chip; the state indication component selects OLED screen to realize its function.

4. The polygraph scene-oriented integrated multi-modal physiological behavioral signal acquisition system as claimed in claim 1, wherein, The expression acquisition module (3) uses a high-definition camera to detect the facial expression image data of the subject and transmits the data to the mobile workstation (4) through a wired connection; the high-definition camera is placed above the screen of the mobile workstation (4).

5. The polygraph test scene-oriented integrated multi-modal physiological behavior signal acquisition method applied to the polygraph test scene-oriented integrated multi-modal physiological behavior signal acquisition system according to any one of claims 1 to 4, characterized in that, The method comprises: The brain electrical signal generated by the central nervous system of the subject is detected by the electroencephalogram acquisition module (1); The physiological signal regulated by the peripheral nervous system of the subject is detected by the peripheral physiological acquisition module (2), including: the skin electrical signal, the electromyogram signal and the pulse signal of the subject are detected by the physiological acquisition sub-module one (21); the electrocardio signal and the breath signal of the subject are detected by the physiological acquisition sub-module two (22); The macro facial expression image data of the subject is detected by the expression acquisition module (3); The physiological acquisition sub-module one (21) comprises a second sensor unit (211) and a second signal processing unit (212), and the second sensor unit (211) is in communication connection with the second signal processing unit (212); the second sensor unit (211) comprises two groups of first surface electrode patches (2111) for detecting the skin electrical signal and the electromyogram signal of the subject respectively, and a micro optical measuring device (2113) for detecting the pulse signal of the subject; The physiological acquisition sub-module two (22) comprises a third sensor unit (221) and a third signal processing unit (222), and the third sensor unit (221) is in communication connection with the third signal processing unit (222); the third sensor unit (221) comprises a second surface electrode patch (2212) for detecting the electrocardio signal of the subject, and a breath inductive plethysmography belt (2213) for detecting the breath signal of the subject; The information collected by the electroencephalogram acquisition module (1), the peripheral physiological acquisition module (2) and the expression acquisition module (3) is uploaded to the mobile workstation (4), and the mobile workstation (4) realizes human-computer interaction through the visualization system (5) embedded therein.

Images