Signal acquisition and display system and method of bimodal sensor array

Abstract

The invention relates to a flexible electronic sensing technology, in particular to a signal acquisition and display system and method for a bimodal sensor array, and the system comprises the bimodal sensor array, each sensing unit of the bimodal sensor array is made of a flexible polypyrrole composite conductive material, and the bimodal sensor array can respond to piezoresistive and thermoelectric response signals in the same structure at the same time; the array signal gating module is used for realizing polling acquisition switching of multi-channel temperature signals and pressure signals; an operational amplification module; the signal reading module is used for sequentially gating each sensing unit according to the control and sequential logic of the array signal gating module in the polling acquisition switching process to obtain a temperature signal or a pressure signal of the sensing unit; the upper computer receives and analyzes the temperature signal and the pressure signal in real time; and the display module is used for displaying and outputting an analysis result of the upper computer on the temperature signal and the pressure signal. The system realizes efficient decoupling and dynamic display of temperature and pressure signals, and is suitable for application scenes such as electronic skin, intelligent monitoring, virtual tactile feedback and the like.

Description

Technical Field

[0001] This invention relates to flexible electronics and multimodal signal sensing technology, specifically to a signal acquisition and display system and method for a dual-modal sensor array. Background Technology

[0002] In recent years, the rapid development of flexible electronics technology has driven the widespread application of multimodal intelligent sensing systems in fields such as human-computer interaction, health monitoring, and intelligent bionics. In particular, the synchronous sensing capability of temperature and pressure, two of the most common environmental physical quantities, is crucial for achieving sensor fusion and enhanced interaction in real-world scenarios. However, most current research still focuses on material modification and sensing performance optimization of the sensor unit itself, lacking overall system-level integration and functional synergy, making it difficult to construct stable, high-precision, multi-channel array sensing systems for practical applications.

[0003] Currently, research on realizing temperature-pressure dual-modal functional sensors mainly follows three technical paths: (1) layered composite structure method: thermistor and pressure-sensitive material layers are constructed in a vertical stacked form to realize the division of sensing functions; (2) interface control and isolation method: signal mode decoupling and crosstalk suppression are realized by constructing ion or charge barrier interfaces; (3) multifunctional material integration method: the fusion of temperature and pressure responses is realized in the same material system. Although the above methods have their own advantages in terms of device function realization, they all face different degrees of difficulties when further expanding to arrayed, programmable acquisition and host computer display systems. For example, the layered composite structure method has a complex structure and insufficient flexibility; the interface control and isolation method has high requirements for environmental stability and device fabrication process control, and is easily affected by environmental fluctuations, resulting in problems such as signal instability and response lag; while the multifunctional material integration method realizes dual response to temperature and pressure signals through intrinsic material properties, realizes modal fusion from the material level, and has advantages such as simple structure, fast response, and good flexibility. However, traditional designs based on multifunctional materials still have insufficient research in system modal decoupling, circuit matching, and array response consistency calibration, which limits their further practical application.

[0004] Furthermore, despite some progress in temperature-pressure dual-modal sensing technology in recent years, significant technical bottlenecks remain in the integrated construction of multimodal sensing systems. Specifically, current systems often face the following challenges:

[0005] (1) The problem of signal cross-interference is prominent: temperature and pressure signals often have physical coupling or overlapping sensing paths, which leads to confusion between different modes and makes it difficult to achieve stable separation and accurate identification.

[0006] (2) Insufficient display and real-time processing of host computer: Existing multimodal array systems lack efficient data reception, parsing and visualization platforms, making it difficult to support high channel number and real-time response of multi-point information synchronous display and interaction.

[0007] (3) Low system-level integration: While domestic and international research has made initial progress in temperature-pressure dual-modal material design and sensing mechanisms, most of these efforts focus on local innovations in sensing units or material structures, lacking integrated solutions for complete system applications. Currently, an integrated framework for collaborative design at all levels—from material design to circuit conditioning, signal acquisition, and human-machine interface—has not yet been formed, resulting in significant shortcomings in functional integration, wiring complexity, and scalability of existing systems.

[0008] In summary, most current flexible sensor array systems only possess single-modal functionality, making it difficult to simultaneously sense temperature and pressure signals in dynamic and complex environments. This severely limits their information representation capabilities and functional richness in practical applications. To achieve multimodal sensing that more closely resembles the functions of real skin, dual-modal sensor arrays have gradually become a research hotspot. However, constructing a dual-modal signal readout system with high sensitivity, high stability, and good decoupling capabilities while ensuring sensing accuracy, flexible wearable comfort, and system integration remains a significant challenge. Summary of the Invention

[0009] This invention proposes a signal acquisition and display system and method for a dual-modal sensor array, which achieves low crosstalk decoupling and consistent response of temperature and pressure signals. It has a highly integrated circuit structure and real-time visualization data display capability, and solves the problems of difficult decoupling of multimodal signals and poor response consistency in the prior art.

[0010] In a first aspect, embodiments of the present invention provide a signal acquisition and display system for a dual-modal sensor array, comprising:

[0011] The dual-modal sensor array includes several independent sensing units; each sensing unit is constructed using a flexible polypyrrole composite conductive material, which can simultaneously respond to piezoresistive and thermoelectric response signals in the same structure, and is used to sense changes in external pressure and temperature gradients, respectively.

[0012] The array signal gating module is used to realize the polling acquisition and switching of multi-channel temperature signals and pressure signals in the dual-mode sensor array;

[0013] The operational amplifier module is used to differentially amplify and boost the level of temperature signals from microvolts to millivolts, and to divide the pressure signal to form a medium-level signal.

[0014] The signal reading module, based on the control and timing logic of the array signal selection module during the polling acquisition switching process, sequentially selects each sensing unit, acquires the temperature or pressure signal corresponding to each sensing unit, and performs scheduling and preliminary processing.

[0015] The host computer is used to receive and parse the temperature and pressure signals acquired by the signal reading module in real time; and

[0016] The display module is used to display the analysis results of the temperature and pressure signals from the host computer.

[0017] Preferably, the array signal gating module includes a main control MCU, a multiplex analog switch, and an inverter; the main control MCU outputs several GPIO channel selection signals and one enable signal to form a channel control command, which is used for time-division multiplexing activation and sampling of multiple channel temperature signals and multiple channel pressure signals; wherein the channel selection signal is output to the multiplex analog switch to realize channel address switching; the enable signal is converted by the inverter to control the enable pin of the multiplex analog switch, so that only one channel is activated at any given time.

[0018] Preferably, the signal reading module sequentially acquires signals from each channel through two ADC channels in conjunction with GPIO control logic. After signal acquisition, it combines a linear interpolation algorithm to perform inverse decomposition of temperature-pressure values ​​and response consistency calibration.

[0019] Secondly, embodiments of the present invention also provide a signal acquisition and display method for a dual-modal sensor array, implemented using the above-described system, the method comprising the following steps:

[0020] S1. Based on the multi-mode channel decoupling and polling method controlled by gating signals, temperature and pressure signals are acquired.

[0021] S2. Perform consistency calibration on the temperature and pressure signals output by the dual-mode sensor array;

[0022] S3. Real-time decoding and visualization output of temperature and pressure signals after consistency calibration.

[0023] Furthermore, step S1 includes:

[0024] S11. The main control MCU chip of the array signal gating module generates a total of 4 level signals, of which 3 are channel selection signals and 1 is an enable signal, which are used to form a 4-bit binary encoded channel control command.

[0025] S12. The enable signal is transformed by the inverter of the array signal gating module to generate a pair of high and low level signals, which are respectively input to the enable pin of the multiplex analog switch to control it to enter the mutually exclusive working state.

[0026] S13. By sequentially adjusting the combination of channel selection signals, the first group of 8 channel signals are polled and acquired one by one; the level of the enable signal is switched, and the combination of channel selection signals is repeatedly adjusted to complete the reading of the remaining 8 channel signals.

[0027] S14. After completing one round of pressure signal acquisition, the control array signal gating module switches the signal acquisition channel, so that the gating signal enters the temperature signal acquisition channel through the operational amplifier circuit module. Then, repeat steps S12 to S13 to read the 16 temperature signals in sequence.

[0028] S15. Initialize two 16-element arrays to store temperature and pressure signal data, respectively.

[0029] Furthermore, step S2 includes:

[0030] S21. Under standard conditions, data is collected and tables are constructed. The temperature and pressure signals collected in step S1 are used to query the constructed tables, and the corresponding temperature and pressure values ​​are obtained through linear interpolation.

[0031] S22. Construct a piecewise linear interpolation model, store the collected temperature and pressure signals into corresponding arrays, and construct a first-order linear interpolation function based on the data of each channel to describe the correspondence between its voltage signal and temperature / pressure.

[0032] Compared with the prior art, the beneficial effects achieved by the present invention include:

[0033] 1. Synchronous acquisition and decoupled display of dual-modal information

[0034] The signal acquisition and display system of this invention is based on a dual-modal sensor unit constructed from polypyrrole material, which can simultaneously respond to temperature and pressure signals. Through hardware gating and software channel-based acquisition strategies, it effectively achieves decoupled acquisition and independent analysis of thermoelectric and piezoresistive response signals, solving the problem of insufficient information dimensions in traditional single-modal systems.

[0035] 2. Integrated array structure design improves space utilization and response consistency.

[0036] The sensor array employs a double-sided FPCB package and a flexible flat signal output line structure, achieving high-density distribution and flexible wiring, supporting simultaneous management of more than 16 channels of signals. A linear interpolation algorithm is used to calibrate the output signal, significantly improving the signal response consistency of the array under the same external excitation.

[0037] 3. Scalable signal polling reading method and multi-channel signal sampling mechanism

[0038] The STM32F103ZET6-based read module supports multi-channel signal polling sampling and fast channel switching. Combined with gating signal control, it can achieve stable reading and data buffering of multi-modal sensor arrays under the condition of limited microcontroller resources.

[0039] 4. High-sensitivity signal conditioning and weak thermoelectric signal amplification capability

[0040] The operational amplifier module uses AD8429 and OPA177 to form a two-stage amplification structure, which performs low-noise, high-gain signal conditioning on μV-level thermoelectric signals, solving the distortion or interference problems caused by the small signal amplitude in conventional thermoelectric acquisition systems.

[0041] 5. A visualization platform that supports mode switching and heatmap display.

[0042] The host computer display system supports switching between independent (temperature signal reading only / pressure signal reading only) and fusion mode (temperature and pressure signals reading simultaneously) display, and intuitively displays the real-time signal response of each sensing unit in the array in the form of a heat map, enhancing the intuitiveness of the system's human-computer interaction.

[0043] 6. It has good structural versatility and scene adaptability.

[0044] The system boasts a high degree of modularity and a compact structure, overcoming technical bottlenecks in traditional solutions such as difficulties in decoupling dual-modal signals, poor signal response consistency, and fixed display methods. It is applicable to various scenarios including flexible electronic skin, intelligent medical monitoring, and virtual interactive terminals, offering advantages such as real-time sensing, strong portability, flexible adaptation, information fusion, and visual interaction. Specifically, it is suitable for the simultaneous acquisition, separation, conditioning, and fusion display of temperature and pressure signals in scenarios such as electronic skin, virtual interaction, and intelligent health monitoring, exhibiting technical advantages such as high integration, good response consistency, and fast response speed.

[0045] 7. The system has controllable costs and is easy to implement in engineering projects.

[0046] Using conventional commercial chips (such as 74HC4051D, AD8429, STM32F1 series, etc.) and easily processed materials (such as polypyrrole), the system has a simple structure, which is convenient for mass production and integration, and can reduce system cost and development threshold while ensuring performance. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the signal acquisition and display system in an embodiment of the present invention;

[0048] Figure 2 This is a flowchart of signal acquisition and processing in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the dual-mode array signal gating module in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the operational amplifier module in an embodiment of the present invention;

[0051] Figure 5 This is a flowchart of the signal acquisition and display method in an embodiment of the present invention. Detailed Implementation

[0052] The dual-modal sensor array signal acquisition and display system proposed in this invention is constructed based on polypyrrole multifunctional conductive material. Following a system integration path of "material-structure-circuit-algorithm," a complete multi-point sensing architecture is built. This system not only achieves time-division multiplexing acquisition of temperature and pressure signals with hardware circuit isolation, but also utilizes an MCU to realize multi-channel polling, high-gain thermal voltage amplification, direct piezoresistive voltage reading, signal calibration, and modal decoupling. Simultaneously, a host computer thermal map visualization interface has been developed, enabling efficient synchronous presentation of multi-channel data and user interaction. This system features high response sensitivity, good response consistency, and compact size, making it suitable for applications such as flexible array expansion and providing effective support for next-generation multimodal human-machine sensing interfaces.

[0053] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments, but the implementation of the present invention is not limited thereto.

[0054] Example

[0055] This embodiment provides a signal acquisition and display system for a dual-modal sensor array. By constructing a flexible array structure with a 4×4 arrangement, the temperature response signal and the pressure response signal are output through gating and operational amplification, thereby realizing real-time sensing of dual-modal signals of temperature and pressure in the external environment.

[0056] See Figure 1 The signal acquisition and display system for the dual-modal sensor array in this embodiment comprises a sensing front-end, a signal conditioning and acquisition section, and a data processing and visualization section. This system can achieve separate acquisition, dynamic encoding, and wireless transmission of piezoresistive and thermoelectric dual-modal signals, and supports real-time signal display and processing. Specifically, the sensing front-end includes a dual-modal sensor array and an array signal gating module; the signal conditioning and acquisition section includes an operational amplifier module and a signal reading module; and the data processing and visualization section includes a host computer and a display module. The dual-modal sensor array, array signal gating module, operational amplifier module, signal reading module, host computer, and display module are connected sequentially.

[0057] The dual-modal sensor array is arranged in a 4×4 matrix and includes 16 independent sensing units. Each sensing unit is constructed using flexible polypyrrole composite conductive material, which can simultaneously respond to piezoresistive and thermoelectric response signals in the same structure, and is used to sense changes in external pressure and temperature gradients, respectively. The sensor array as a whole is packaged on a flexible substrate, which has good flexibility and integrability, and significantly simplifies the wiring structure and physical packaging of the dual-modal system.

[0058] In this embodiment, since the polypyrrole composite conductive material has both thermoelectric and piezoresistive response characteristics, by regulating the thermoelectric and piezoresistive characteristics of the polypyrrole composite conductive material, the coordinated sensing of temperature and pressure in the same sensitive layer is realized, so that each sensing unit can independently generate two types of signal output at the same location, avoiding the structural complexity and signal crosstalk problems caused by multi-layer stacking.

[0059] The dual-modal sensor array adopts a double-sided FPCB integrated layout scheme, which is cleaner and more aesthetically pleasing, and has good flexibility, stability and mechanical compliance. It is connected to the array signal gating module in the later stage through FPC flexible flat signal output lines, realizing high-density wiring and flexible connection of the sensing channel.

[0060] The array signal gating module is used to realize the polling acquisition and switching of multi-channel temperature and pressure signals in a dual-modal sensor array. This module is equipped with two 74HC4051 multiplexer analog switches and one 74HC04 inverter chip. Through three GPIO channel selection signals and one enable signal output from the main control MCU, channel control commands are formed to complete the time-division multiplexing activation and sampling of 16 temperature signals and 16 pressure signals. The channel selection signals are output to the address pins A, B, and C of the multiplexer analog switches to realize channel address switching. The enable signal, after being converted by the inverter, controls the enable pins of the two multiplexer analog switch chips, thus activating only one channel at any given time and avoiding cross-interference and aliasing between different channel signals. The dynamic channel gating and control network structure adopted by the array signal gating module not only reduces wiring complexity but also reduces the occupation of ADC channels. Furthermore, the sampling strategy can be flexibly configured through external GPIO control logic, laying a hardware foundation for subsequent high-precision sampling.

[0061] In this embodiment, the array signal gating module adopts a time-division multiplexing structure based on a polling mechanism, supporting decoupled acquisition of 32 channels of signals under a dual-channel ADC architecture. The sampling logic includes: sequentially adjusting the channel selection signal combination (000 to 111) to achieve polling acquisition of the first group of 8 channels, and then switching the level of the enable signal to complete the acquisition of the other group of 8 channels. In other words, the acquisition process first completes the reading of all piezoresistive response signals (i.e., pressure signals), then switches the signal path to access the thermoelectric response signal (i.e., temperature signal) channel, and then repeats the above sampling process to complete dual-modal data acquisition.

[0062] To achieve high-sensitivity amplification and baseline adjustment of the thermoelectric response signal, this system incorporates a dedicated operational amplifier module. This module includes a high common-mode rejection ratio (CMRR) operational amplifier chip, used to differentially amplify and level-up the microvolt to millivolt level thermoelectric response signal to meet the sampling range requirements of the subsequent ADC. Specifically, it processes the thermoelectric response signal in the dual-mode signal. The weak thermoelectric response signal undergoes high CMRR differential amplification via an AD8429 instrumentation amplifier and is then level-up via an OPA177 precision operational amplifier to ensure the output signal falls within the ADC sampling range. The amplification path is matched to the microvolt level thermoelectric response signal, guaranteeing high-fidelity signal transmission. More specifically, regarding signal amplitude processing, the system distinguishes between two intrinsic voltage levels for different modes: the thermoelectric response signal, typically in the microvolt (μV) to millivolt (mV) range, is a low-level, easily interfered differential signal. Therefore, an AD8429 instrumentation amplifier is used for differential amplification, followed by an OPA177 DC bias amplifier to boost the signal to near the 1.5V median value recognizable by the STM32 ADC, with adjustable gain to adapt to different heat source response intensities. The piezoresistive response signal, on the other hand, is generated into a medium-level signal (0.3V-2.5V) through a voltage divider circuit, falling within the microcontroller's ADC input range and can be directly read without amplification. The piezoresistive response signal is then connected to the microcontroller's ADC port via the voltage divider circuit.

[0063] The signal acquisition module sequentially acquires signals from each channel via two ADC channels and GPIO control logic. After acquisition, it uses a linear interpolation algorithm to perform inverse temperature-pressure value determination and response consistency calibration. During signal acquisition, the system employs a dual-channel ADC sampling mechanism, combined with the control logic of a dual-modal array signal gating module, to achieve polling switching and sequential acquisition of each sensing unit. The signal acquisition module uses an architecture based on the STM32F103ZET6 microcontroller and is responsible for the acquisition, scheduling, and preliminary processing of signals from each channel in the dual-modal sensor array. Relying on the control and timing logic of the array signal gating module during the polling acquisition process, the signal acquisition module can sequentially select each sensing unit and accurately obtain its corresponding thermoelectric or piezoresistive response value.

[0064] In addition, to address the response offset issue caused by individual differences in sensing units, this system introduces a linear interpolation calibration algorithm. Based on reference data under stress-free and standard stress conditions, as well as under normal and high temperature conditions, the system performs one-to-one voltage correction on each sensor, making their outputs more consistent under the same temperature or pressure, thus significantly improving the overall response consistency and distinguishability of the array.

[0065] The host computer communicates with the signal reading module to receive and parse thermoelectric and piezoresistive response signal data in real time. The system supports displaying thermoelectric and piezoresistive response signals as two-dimensional heat maps, supports dynamic updates, and provides an operation interface for switching display modes, allowing switching between split-modal display and fused display to achieve diverse visual expressions.

[0066] The host computer includes:

[0067] (1) Data management module, used to receive and cache temperature and pressure signal data collected by dual-modal sensor array in real time. Pressure value can be obtained by converting pressure through formula. It supports frame data management and historical data backtracking within a set time window. It also has data export function, which can output cached data in CSV format for subsequent analysis and tracing.

[0068] (2) Display control module, displaying temperature thermogram, pressure thermogram, and fusion thermogram. Figure 3 The system provides a display mode to display the parsing results, a parameter call interface for switching display modes, and a reserved graphical interaction support interface to meet future functional expansion needs.

[0069] (3) The graphics rendering module uses fixed coordinate mapping and color gradient function to visualize the output of the calibrated acquisition data and displays the array response status in an intuitive way in the form of images. The module is implemented based on Python language and supports interactive image display and data traceability functions.

[0070] In this embodiment, the system also integrates a low-power BLE Bluetooth module for wireless communication between the host computer and the signal reading module. The BLE Bluetooth module connects to the STM32 main control board via a serial port and can upload encoded and compressed dual-mode signal data at a fixed rate, combining low power consumption and high transmission stability.

[0071] Figure 2 The overall signal acquisition and processing flowchart of this system is presented. This flowchart covers the complete data path from signal acquisition by the dual-modal sensor array, gating control channel switching, thermoelectric response signal conditioning and amplification, ADC data reading and calibration processing, to final visualization display on the host computer. It demonstrates the key nodes and inter-module interaction logic in the data processing chain of this invention, and provides a relatively detailed illustration of the overall signal acquisition and processing flow of the system, mainly including the following five core stages:

[0072] (1) Signal generation and sensing stage: The dual-mode sensor array synchronously responds to changes in external temperature and pressure, generating microvolt-level thermoelectric response signals and piezoresistive response signals that can be read by the ADC respectively;

[0073] (2) Signal channel selection stage: The 74HC4051 multiplexing chip is used to complete the dynamic polling of each channel to ensure that the dual-mode signal is controllable output on a unified bus;

[0074] (3) Signal conditioning and amplification stage: The thermoelectric response signal is differentially amplified and biased by a low-noise dual-stage amplifier circuit composed of AD8429+OPA177, while the piezoresistive response signal is directly output to the ADC by a voltage divider structure.

[0075] (4) Signal reading and calibration stage: The STM32 main control chip reads the ADC values ​​of all channels according to the time-division multiplexing mechanism and performs consistency calibration through the embedded linear interpolation algorithm;

[0076] (5) Data processing and visualization stage: The host computer receives data frames through the serial port and decodes them. According to the configuration, it displays temperature heat map, pressure heat map or fusion mode heat map to realize intuitive real-time feedback on the array status.

[0077] Each rectangular sensing unit is precisely positioned on the FPCB surface using silver paste and is surface-protected and insulated with a polyimide film. Pressure sensing is achieved through the material's inherent response mechanism: under temperature gradient changes, the polypyrrole-based material generates a thermal voltage; under pressure, the polypyrrole conductive network is compressed, resulting in a change in resistance.

[0078] Each sensor unit has an independent electrode channel, and the FPCB design uses a multi-channel ribbon cable lead-out port to connect to the subsequent signal processing module in the form of a flat signal output line. The lead-out line consists of a 16-pin port, ensuring that the thermoelectric response signal and the piezoresistive response signal can be read separately through time-division gating and circuit switching, realizing dual-mode signal decoupling at the hardware and software level.

[0079] From a system perspective, sufficient spacing is reserved at the boundaries of each sensing unit in the array to ensure signal independence between sensing units and facilitate subsequent maintenance and expansion. For ease of debugging and numbering management, each unit is identified by a number on its outer packaging layer, arranged sequentially from top left to bottom right, which facilitates channel mapping when the system reads data.

[0080] Combination Figure 3 and Figure 4 This embodiment further demonstrates the core structural design and functional implementation path of the signal gating and multi-mode signal conditioning circuit in the system.

[0081] like Figure 3 As shown, this embodiment designs a gating control module consisting of two 74HC4051D 8-to-1 analog multiplexers and one 74HC04 hex inverter, realizing dynamic channel switching and reading of 16 dual-mode sensor units. The sensor array is connected to the PCB via FPC flexible cables. The signal first enters the channel gating module, and the address lines (S0, S1, S2) and enable line (E0) within the module are controlled by the STM32F1 series main control chip, which sequentially polls and selects the target channel.

[0082] In practical applications, the piezoresistive response signal does not require additional amplification and can be directly output through a voltage divider circuit and sampled by the ADC channel; while the thermoelectric signal, due to its weak signal voltage, needs to enter the next stage of operational amplifier circuit for signal conditioning.

[0083] like Figure 4 As shown, this system employs a two-stage signal conditioning and amplification structure to meet the high-precision acquisition requirements of microvolt-level thermoelectric signals. This structure consists of an AD8429 high-performance instrumentation amplifier and an OPA177 precision DC operational amplifier, responsible for differential signal amplification and output level adjustment, respectively. The thermoelectric signal is first input to the differential input terminals (IN+ and IN-) of the AD8429. Leveraging its ultra-low noise and high common-mode rejection ratio differential amplification capabilities, the first stage of high-gain signal amplification is achieved. The gain is controlled by an external resistor RG, a precision potentiometer that can be flexibly adjusted according to the output characteristics of different sensors to meet dynamic range matching requirements. The AD8429 operates at ±6V, effectively expanding the output voltage swing and improving the dynamic range of the entire amplification chain. The bias voltage is connected to V... refThe pin is used to set the reference level for the amplified output; the op-amp output port is V. out It is used to output the final amplified thermoelectric response signal.

[0084] The reference voltage V ref A DC bias module, built from an OPA177 operational amplifier, is used to bias V. out The overall signal level is raised to a range acceptable to the STM32's internal ADC (e.g., 0–3.3V). The specific implementation is as follows: The desired voltage V is generated using a precision potentiometer Rs for voltage division. c As the input signal to the OPA177 input terminal (IN+), it is connected to the output terminal V through the inverting input terminal (IN-). ref This forms a feedback loop, allowing the OPA177 to operate in voltage follower mode, thereby ensuring V ref =V c This structure has the advantages of stable bias voltage, continuous adjustment, and strong output drive capability, which can effectively ensure the zero-point adjustment accuracy of the amplified thermoelectric signal.

[0085] Figure 5 This is a flowchart of the signal acquisition and display method for the dual-modal sensor array in this embodiment. It shows the complete processing flow of the system from reading the original sensor output, polling scheduling, data interpolation, Bluetooth transmission to the upper computer visualization display. It focuses on the processing strategy for the array to achieve response consistency, and specifically includes the following steps:

[0086] S1. Based on the multi-mode channel decoupling and polling method controlled by gating signals, temperature and pressure signals are acquired.

[0087] In this embodiment, signal acquisition specifically includes the following steps:

[0088] S11. The main control MCU chip of the array signal gating module generates a total of 4 level signals, of which 3 are channel selection signals and 1 is an enable signal, which are used to form a 4-bit binary encoded channel control command.

[0089] S12. The enable signal is transformed by the inverter of the array signal gating module to generate a pair of high and low level signals, which are respectively input to the enable pin of the multiplex analog switch to control it to enter the mutually exclusive working state; where 0 represents low level and 1 represents high level.

[0090] S13. By sequentially adjusting the three-bit combination of the channel selection signal (from 000 to 111), the first group of 8 channel signals are polled and acquired one by one; further, the level of the enable signal is switched, and the combination of the channel selection signal is repeatedly adjusted to complete the reading of the remaining 8 channel signals.

[0091] S14. After completing one round of pressure signal acquisition, the control array signal gating module switches the signal acquisition channel, so that the gating signal enters the temperature signal acquisition channel through the operational amplifier circuit module. Then, steps S12 to S13 are repeated to read the 16 temperature signals in sequence.

[0092] S15. Initialize two 16-element arrays to store temperature and pressure signal data, respectively. During the acquisition process, each set of data is marked and buffered according to the channel order, and modal differentiation and decoupling processing are performed to complete the independent processing of multimodal signals.

[0093] Specifically, for the data interpolation process, the system first performs benchmark calibration on each 4×4 sensor array unit under three typical conditions (no stress / normal temperature, compressive stress / normal temperature, and compressive stress / high temperature), recording the corresponding voltage output for subsequent output signal inverse decoding and interpolation reference. This operation enables initial compensation of the sensor array under manufacturing differences and environmental disturbances, improving overall measurement accuracy and response consistency.

[0094] Subsequently, the STM32 control chip polls each sensor node via a control signal gating module. For piezoresistive response signals, the STM32 directly acquires the voltage output from the voltage divider circuit; for thermoelectric signals, it acquires the differential voltage amplified by the AD8429+OPA177. During sampling, a channel-by-channel ADC reading method is used, and a full array data poll is completed at fixed time intervals.

[0095] S2. Perform consistency calibration on the temperature and pressure signals output by the dual-mode sensor array.

[0096] After sampling, in order to achieve spatial response consistency and data standardization of the dual-modal sensor array, this embodiment designs an output consistency calibration method based on calibration table building and linear interpolation, which specifically includes the following steps:

[0097] S21. Under standard conditions, data acquisition and table construction are performed. After querying the constructed table using the temperature and pressure signals acquired in step S1, the corresponding temperature and pressure values ​​are obtained through linear interpolation.

[0098] The microcontroller acquires the response voltage of each sensing unit in the 4×4 sensor array at multiple preset standard temperature points (e.g., room temperature, 40℃, 50℃) and standard pressure points (e.g., 0N, 0.5N, 1.0N). During the experiment, weights of different masses are used to simulate pressure input, and a heating stage is used to heat the weights to simulate temperature changes, which are then applied sequentially to the surface of each channel sensor. The voltage output of each channel under both room temperature / zero pressure and stimulated conditions is recorded and stored in six arrays of length 16. Three arrays are used to store the piezoresistive mode voltage value, and the other three are used to store the thermoelectric mode voltage value.

[0099] S22. Construct a piecewise linear interpolation model

[0100] The acquired temperature and pressure signals are stored in their respective arrays, and a first-order linear interpolation function is constructed based on the data from each channel to describe the correspondence between the voltage signal and the temperature / pressure.

[0101] Based on the calibration data recorded for each channel, a first-order linear interpolation function is established to fit the mapping relationship between the voltage output and the corresponding physical quantity (temperature or pressure). In the linear model, the interpolation decoding formula is as follows:

[0102]

[0103] Among them, V cur X is the voltage value collected in real time. cur For the temperature or pressure values ​​obtained by inverse interpolation, V1 and V2 are the boundary points of the voltage range that the channel falls on, and X1 and X2 are the corresponding calibrated temperature or pressure values. During system operation, the real-time voltage values ​​of each channel of the dual-mode sensor array are input into the corresponding first-order linear interpolation function in the microcontroller for inverse interpolation to obtain the corresponding temperature and pressure values.

[0104] S3. Real-time decoding and visualization output of temperature and pressure signals after consistency calibration.

[0105] After obtaining the actual temperature and pressure values, in order to achieve real-time display and feedback of the dual-modal thermal map on the host computer, this embodiment designs a real-time decoding and visualization output method, which specifically includes the following steps:

[0106] The microcontroller encodes the acquired signals sequentially according to their signal type and sensor number, and then wirelessly transmits them to the host computer via a BLE module at a fixed frequency. The host computer receives the data stream through another paired BLE module and performs real-time unpacking and decoding of the thermoelectric and piezoresistive response signals according to the preset data type and sequence.

[0107] The decoded data, based on the standard temperature and pressure points set in step S21, divides the temperature and pressure values ​​into three response intervals, using these as the benchmark for the color mapping threshold to create a dual-modal heatmap that effectively suppresses intermodal crosstalk. Furthermore, after real-time decoding and processing of the data, the host computer instantly generates and outputs clear temperature and pressure heatmaps. The generated heatmaps intuitively reflect the temperature and pressure response distribution of each sensing channel, thus achieving dynamic real-time visualization of multimodal signals for intuitive system feedback and user interface interaction.

[0108] In this embodiment, the sensing unit of the dual-modal sensor array is constructed using polypyrrole composite conductive material. Combined with a high-gain differential amplifier circuit for thermoelectric signals and a piezoresistive signal reading path, it effectively solves the technical bottlenecks in traditional thermo-pressure dual-modal systems, such as weak thermoelectric signals that cannot be read, difficulties in signal decoupling, signal cross-interference, and limited ADC resources. It greatly simplifies the signal reading structure, improves the overall stability and scalability of the sensor array, and has better integration and engineering practicality.

[0109] In this embodiment, the flexible polypyrrole foam sensing material of the sensor array uses cellulose as a base and incorporates lignin to form a first covalent cross-linked network, enhancing the material's mechanical strength. A second network is then constructed using sodium alginate and calcium ions for coordination cross-linking, strengthening elasticity and fatigue resistance. In-situ loading of polypyrrole is achieved through the active functional groups in the material, forming a stable conductive network. The anisotropic porous structure significantly reduces thermal conductivity, improving the material's thermoelectric sensitivity and response performance, thereby achieving dual-modal temperature and pressure sensing and stable sensing output within a single material system.

[0110] In addition, a linear interpolation calibration algorithm for the response consistency among array sensing units was designed, which effectively solves the problem of large individual differences and insufficient response consistency of traditional sensor arrays, and achieves a high degree of consistency in the response of temperature and pressure signals when different channels and different sensing units are subjected to the same temperature and pressure.

[0111] This embodiment constructs a host computer display interface and heat map visualization function for a dual-modal sensor array. A Python-based dual-modal heat map display platform was developed to realize real-time visualization, wireless transmission, and data export of temperature and pressure data from the sensor array, greatly enhancing the system's practical value in real-time sensing, human-computer interaction, and feedback. The interface is written in Python and uses the PyQt5 framework to build a graphical user interface (GUI), implementing functions such as window creation, interactive buttons, layout management, and real-time serial port data reception. Simultaneously, it utilizes the plotting functions provided by the matplotlib graphics library to perform independent heat map visualization processing on the pressure and temperature signals in the array. In "pressure mode," the host computer receives and analyzes the piezoresistive response signal of the 4×4 dual-modal sensor array in real time through the ADC sampling and data transmission mechanism of the STM32 main control chip, generating a two-dimensional heat map in the GUI to intuitively represent the pressure distribution on the sensor array. The pressure can be obtained by converting pressure using a formula, and the magnitude of the pressure is represented by the intensity of the color, significantly improving data analysis efficiency and user interaction experience.

[0112] Furthermore, the host computer performs real-time visualization processing of the thermoelectric signals in "temperature mode" and presents the temperature response of the sensor array in the form of a heat map. When a heat source (such as a heated cup of water) is placed in the central region of the array, multiple sensor units generate thermoelectric voltage outputs of varying amplitudes, resulting in a clear spatial temperature gradient distribution in the heat map. The results show that, except for two sensing units at the array edges which exhibit relatively weak responses due to their smaller contact area with the heat source, the remaining sensing units all demonstrate strong thermoelectric responses, reflecting sufficient heat absorption and sensitive device response in the central region. The overall heat map shows good uniformity, fully demonstrating that the system possesses excellent temperature sensing sensitivity and spatial response consistency, enabling high-resolution sensing and effective imaging of localized heat sources.

[0113] Furthermore, this embodiment experimentally tested the system's response performance under two typical combined operating conditions: high temperature and low pressure, and low temperature and high pressure, demonstrating the mode resolution capability and single-channel sensitive response characteristics of the sensor array. In the experiment, a small weight was heated to approximately 45°C and placed on the surface of the sensor array to simulate an input scenario primarily driven by heat conduction and secondarily by pressure. The results show that the temperature thermogram exhibits a red response in the central area, while other areas maintain a cool background, demonstrating the array's excellent ability to recognize high-temperature heat transfer inputs and the system's low crosstalk characteristics. Simultaneously, the pressure thermogram shows a distinct yellow response block in this area, indicating that even with extremely low applied pressure, it was accurately detected by the system, demonstrating high sensitivity and resolution to pressure.

[0114] This embodiment provides a signal acquisition and display system for a dual-modal sensor array. While achieving basic temperature-pressure sensing using a 4×4 array configuration, it has good scalability and module compatibility, and can be applied to multi-point flexible sensing scenarios with larger area, greater stress, and higher resolution.

[0115] The system employs a modular signal gating structure, utilizing multiple 74HC4051 multiplexers for channel polling control, supporting cascading expansion to larger arrays such as 8×8 and 16×16. The STM32F1ZET6 main control chip boasts abundant resources, featuring a multi-channel ADC and a timer interrupt mechanism, providing hardware support for large-scale parallel reading. The system reserves DMA sampling paths, a ring buffer, and a data framing interface, facilitating subsequent high-frame-rate, high-data-volume signal processing.

[0116] In terms of power consumption, the system maintains low power consumption overall, with core power consumption concentrated in the operational amplifiers and gating modules. Further improvements can be made by incorporating low-power instrumentation amplifiers, dynamic power management chips (PMICs), and sleep / wake mechanisms to further reduce power consumption and meet the needs of wearable electronics and other applications.

[0117] In summary, this invention achieves synergistic optimization at the material, structure, circuit, and algorithm levels, with specific advantages in the following three aspects: First, by employing multifunctional polypyrrole foam material, pressure and temperature are simultaneously sensed within a single sensing unit, avoiding modal crosstalk and structural complexity inherent in traditional layered structures. Second, by combining multimodal channel decoupling and polling methods with differential amplification-bias signal conditioning circuitry, accurate decoupling and stable extraction of dual-modal signals are achieved. Third, by integrating a first-order linear interpolation sensor array output consistency calibration algorithm and a Python heatmap visualization interface, the consistency of array output response and real-time interactive experience are effectively improved. Actual testing shows that the system can stably identify modal responses under complex conditions such as high temperature and low pressure and low temperature and high pressure, exhibiting strong thermo-pressure resolution and intuitive, clear interface feedback, demonstrating excellent multimodal signal sensing performance.

[0118] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A signal acquisition and display system for a dual modality sensor array, comprising: The system comprises: a dual-mode sensor array comprising a plurality of independent sensing units; each sensing unit is constructed with flexible polypyrrole composite conductive material and can simultaneously respond to piezoresistive and thermoelectric response signals in the same structure, which are used to sense external pressure changes and temperature gradients, respectively; an array signal gating module for realizing polling collection switching of multi-channel temperature signals and pressure signals in the dual-mode sensor array; an operational amplifier module for differentially amplifying and level lifting the microvolt to millivolt temperature signals and performing voltage division on the pressure signals to form medium-level signals; a signal reading module for sequentially gating each sensing unit according to the control and timing logic of the array signal gating module in the polling collection switching process, obtaining the corresponding temperature signals or pressure signals of each sensing unit, and performing scheduling and preliminary processing; a host computer for receiving and analyzing the temperature signals and pressure signals obtained by the signal reading module in real time; and a display module for displaying and outputting the analysis results of the temperature signals and pressure signals by the host computer.

2. The signal acquisition and display system of claim 1, wherein, The dual-mode sensor array is packaged with a flexible substrate and connected to the array signal gating module through a FPC flexible flat signal output line.

3. The signal acquisition and display system of claim 1, wherein, The array signal gating module comprises a main control MCU, a multi-channel analog switch and an inverter; a plurality of GPIO channel selection signals and an enable signal output by the main control MCU form a channel control instruction, which is used for time-sharing polling activation and sampling of multi-channel temperature signals and multi-channel pressure signals; wherein the channel selection signal is output to the multi-channel analog switch for realizing channel address switching; the enable signal is transformed by the inverter to control the enable pin of the multi-channel analog switch, so that only one channel is activated at any time.

4. The signal acquisition and display system of claim 1, wherein, The operational amplifier module comprises: an operational amplifier chip for differentially amplifying and level lifting the microvolt to millivolt temperature signals; and a voltage division circuit for voltage division on the pressure signals.

5. The signal acquisition and display system of claim 1, wherein, The signal reading module sequentially collects channel signals through two ADC channels in cooperation with the GPIO control logic, and after signal collection, the temperature-pressure value inverse solution and response consistency calibration are performed by combining the linear interpolation algorithm.

6. The signal acquisition and display system of claim 5, wherein, The signal reading module adopts a microcontroller-based architecture design.

7. The signal acquisition and display system of claim 1, wherein, The host computer comprises: a data management module for receiving and buffering the temperature signals and pressure signals collected by the dual-mode sensor array in real time; a display control module for displaying the analysis results in three display modes of temperature heat map, pressure heat map and fusion heat map; a graphic rendering module for visualizing the calibrated collected data output by using fixed coordinate mapping and color gradient function, and intuitively displaying the array response state in the form of image.

8. A method of signal acquisition and display for a dual modality sensor array, comprising: The method is implemented by using the system of any one of claims 1-7, and comprises the following steps: S1, collecting temperature signals and pressure signals based on the multi-modal channel decoupling and polling mode controlled by the gating signal; S2, performing consistency calibration processing on the temperature signals and pressure signals output by the dual-mode sensor array; S3, performing real-time decoding and visual output on the temperature signals and pressure signals after consistency calibration processing.

9. The signal acquisition and display method of claim 8, wherein, Step S1 comprises: S11, the main control MCU chip of the array signal gating module generates 4 paths of level signals, 3 of which are channel selection signals, and 1 is an enable signal, which is used to form a 4-bit binary coded channel control instruction; S12, the inverter of the array signal gating module is used to transform the enable signal to generate a pair of high and low level signals, which are input into the enable pins of the multi-channel analog switch to control it to enter the mutually exclusive working state; S13, by adjusting the combination of the channel selection signals in turn, the first 8-channel signal is polled and collected; the level state of the enable signal is switched, and the combination of the channel selection signals is adjusted repeatedly to complete the reading of the remaining 8-channel signals; S14, after completing a round of pressure signal collection, the array signal gating module is controlled to switch the signal collection channel, so that the gating signal enters the temperature signal collection channel through the operational amplifier circuit module, and then steps S12 to S13 are repeated to read the 16-channel temperature signals in turn; S15, two 16-element arrays are initialized to store the temperature signal and pressure signal data respectively.

10. The signal acquisition and display method of claim 8, wherein, Step S2 includes: S21, data collection and table construction are performed under standard conditions, the temperature signal and pressure signal collected in step S1 are used to query the constructed table, and the corresponding temperature and pressure values are obtained by linear interpolation method; S22, a piecewise linear interpolation model is constructed, the collected temperature signal and pressure signal values are stored in the corresponding arrays, and a first-order linear interpolation function is constructed based on the data of each channel to describe the corresponding relationship between the voltage signal and temperature / pressure.

Images