Deep learning-based home sub-health auxiliary diagnosis body temperature thermal imaging system and method

Abstract

The present application relates to the field of body temperature thermal imaging technology, and discloses a kind of based on deep learning's home sub-health auxiliary diagnosis body temperature thermal imaging system and method, thermal imaging system includes infrared acquisition module, visible light acquisition module, main control module, data transmission module and power module, main control module is configured I2C peripheral interface connection infrared acquisition module, main control module is configured USB high-speed interface visible light acquisition module is connected, main control module is configured USB full-speed interface data transmission module is connected, data transmission module is connected through USB interface on the Android system display body temperature information of Android mobile phone, power module is powered for infrared acquisition module, visible light acquisition module, main control module, data transmission module.The present application is low in cost, can greatly reduce the volume of real object, improve the portability of the whole system, can improve the effect of sub-health discrimination and body temperature prediction simultaneously, ensure that actual body temperature calculation result is reliable.

Description

Technical Field

[0001] This invention relates to the field of thermal imaging technology, and in particular to a home-use thermal imaging system and method for assisting in the diagnosis of sub-health based on deep learning. Background Technology

[0002] Sub-health refers to a "third state" between health and disease, characterized by non-specific symptoms such as chronic fatigue, sleep disorders, and decreased immunity. Early detection of sub-health can help take timely intervention measures to prevent the occurrence of diseases and maintain overall health.

[0003] Body temperature directly reflects a person's physical condition and is one of the important indicators for diagnosing sub-health conditions. Body temperature measurement can serve as the "first line of defense" for early warning of individual diseases (such as quickly identifying abnormalities such as fever and infection), and it can also be a key barrier for public health security (such as contactless screening in epidemic prevention and control). It is used in multiple scenarios such as medical diagnosis, chronic disease management, sports protection, and care for special populations.

[0004] Current body temperature measurement technologies are mainly divided into "contact" and "non-contact" methods, with significant differences in application scenarios and accuracy. Traditional contact measurement methods primarily use mercury thermometers and electronic thermometers, obtaining temperature by contact with areas such as the mouth, armpit, or rectum. These methods offer high accuracy (error approximately ±0.1℃), but because they require prolonged, fixed measurements (3-5 minutes) and pose a risk of cross-infection, they have been gradually replaced—currently mainly used in homes or medical settings where high accuracy is required. Non-contact body temperature measurement methods primarily use infrared forehead thermometers, ear thermometers, and thermal imagers, calculating temperature by receiving infrared energy radiated by the human body. These methods offer short measurement times and do not involve direct contact with the body, making them suitable for rapid screening in public places (such as airports and hospital entrances). However, non-contact temperature measurement is susceptible to interference from ambient temperature, measurement distance, and sweat, resulting in relatively lower accuracy (error approximately ±0.3℃).

[0005] With the development of technology and people's pursuit of a high-quality life, body temperature measurement has gradually developed with "non-contact" and "high accuracy" as its main goals. Currently, home body temperature measurement suffers from the problem of expensive and inconvenient-to-carry high-precision devices. Therefore, researching a low-cost, distributed, compact, and portable system that combines home body temperature measurement with sub-health diagnosis is of great significance for home-based disease prevention, improving people's quality of life, and promoting the dissemination of traditional medicine. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, the present invention aims to provide a home-use sub-health auxiliary diagnosis thermal imaging system and method based on deep learning. It is low in cost, can significantly reduce the physical size, improve the portability of the entire system, and can simultaneously improve the sub-health identification and body surface temperature prediction effects, ensuring the reliability of actual body temperature measurement results.

[0007] To address the aforementioned technical problems, the present invention adopts the following technical solution: a home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning, comprising an infrared acquisition module, a visible light acquisition module, a main control module, a data transmission module, and a power supply module. The main control module is configured with an I2C peripheral interface to connect to the infrared acquisition module, a USB high-speed interface to connect to the visible light acquisition module, and a USB full-speed interface to connect to the data transmission module. The data transmission module connects to the Android system on an Android phone via a USB interface to display body temperature information. The power supply module provides power to the infrared acquisition module, the visible light acquisition module, the main control module, and the data transmission module.

[0008] Furthermore, the main control module includes a main control chip U1, power supply pins, a high-speed external crystal oscillator branch, a low-speed 32K crystal oscillator branch, a reset / boot configuration branch, and peripheral output pins. The power supply pins include pins 24 (VDD), 36 (VDD), and 48 (VDD) power input terminals, all of which are connected to a 3.3V power supply voltage. Pins 35 (VSS), 23 (VSS), and 47 (VSS) are power ground pins, all of which are electrically connected to GND. C1, C2, and C3 are power decoupling capacitors, with one end connected to 3.3V and the other end grounded, and are connected in parallel between each VDD-VSS group to achieve power filtering. The high-speed external crystal oscillator branch includes a high-speed main crystal oscillator CRYSTAL1, capacitor C6 and capacitor C7. The high-speed main crystal oscillator CRYSTAL1 is connected between pin 5 PH0-OSC_IN and pin 6 PH1-OSC_OUT of the main control chip U1. C6 and C7 are the starting load capacitors at both ends of the crystal oscillator, and the other end of the capacitors is grounded. The low-speed 32K crystal oscillator branch includes a low-speed 32K crystal oscillator pin, which is a 3-pin PC14 / OSC32IN and a 4-pin PC15 / OSC32OUT. A crystal oscillator pad is reserved, and the PC13 pin is brought out as a general-purpose IO pin. The reset and BOOT configuration branch includes a 7-pin NRST, which is the hardware reset pin of the main control chip U1. It consists of R3, R4, C9, and C11 forming a power-on reset circuit. The upper end of R3 is connected to 3.3V, and the lower end is connected in series with C9 to ground. R4 is connected across the NRST pin and the 44-pin BOOT0. The BOOT0 pin is grounded through capacitor C11. The 22-pin VCAP1 is connected to ground by an external filter capacitor C11. The 1-pin VBAT, 8-pin VSSA, and 9-pin VDDA are analog power supply pins. VBAT is connected to 3.3V, VSSA is grounded, and VDDA is connected to 3.3V and grounded with a matching decoupling capacitor. The peripheral output pins include: pin 43 PB7 and pin 42 PB6, with PB7 serving as the SDA signal terminal and PB6 serving as the SCL signal terminal, leading out the I2C bus; pin 13 PA3 is UART2_RX and pin 12 PA2 is UART2_TX, leading out serial port signals; pin 37 PA14 is SWCLK and pin 34 PA13 is SWDIO, with SWDIO and SWCLK pins used for debugging and downloading; pin 33 PA12 is CHIP_DP and pin 32 PA11 is CHIP_DM.

[0009] Furthermore, the infrared acquisition module includes an infrared sensor U2 and an infrared interface network. The infrared interface network includes an infrared power supply branch and an I2C pull-up bus branch. The infrared power supply branch includes capacitors C4 and C5. Pin 2 VDD of the infrared sensor U2 is the power input terminal, and two filter capacitors C4 and C5 are connected in parallel. The other end of the two capacitors is grounded, and pin 2 VDD is connected to a 3.3V power supply. Pin 3 GND of the infrared sensor U2 is the grounding pin and is directly electrically connected to the common ground. The I2C pull-up bus branch includes resistors R1 and R2. Pin 1 SDA of U2 is connected to a 3.3V power supply after being connected in series with pull-up resistor R1. The SDA pin is electrically interconnected with PB7 (SDA) of the main control chip U1. Pin 4 SCL of U2 is connected to a 3.3V power supply after being connected in series with pull-up resistor R2. The SCL pin is electrically interconnected with PB6 (SCL) of the main control chip. R1 and R2 are I2C bus pull-up resistors, which realize the level is pulled up to 3.3V when the bus is idle.

[0010] Furthermore, the visible light acquisition module includes a visible light camera, a USB hub U4, and camera peripheral circuitry. The camera peripheral circuitry includes a camera crystal oscillator start-up branch, a camera power supply filtering branch, and a USB differential signal output terminal. The camera crystal oscillator starting branch includes crystal oscillator X2 and inductor U3. Pin 15 XOUT and pin 16 XIN of USB hub U4 are connected across the surface mount crystal oscillator X2. One end of inductor U3 is grounded and the other end is connected to pin 16 XIN. The camera power supply filtering branch includes capacitors C8 and C10. Pins 14 (VDD18), 13 (VDD33), and 11 (VDD5) of the USB hub U4 are the core power supply pins, respectively. Two 10μF filter capacitors, C8 and C10, are connected in parallel to pin VDD33. The other ends of capacitors C8 and C10 are grounded. Pin 12 (GND) is the common ground pin, connected to the GND of the thermal imaging system. Pins 10 (DP) and 9 (DM) are the uplink USB differential D+ / D- of hub U4, interconnected with the D+ and D- of the subsequent Type-C connector. The USB differential signal output terminals include pin 1 (DM4), pin 2 (DP4), pin 3 (DM3), and pin 4 (DP3). Pins 1 (DM4) and 2 (DP4) are one differential signal, CHIP_DM and CHIP_DP, which are connected to PA11 and PA12 of the main control chip U1. Pins 3 (DM3) and 4 (DP3) are the second differential signal, CAM_DM and CAM_DP, which are connected to an external visible light camera. The DM1 / DP1 and DM2 / DP2 pins are reserved for extended USB channels.

[0011] Furthermore, the data transmission module includes a Type-C connector USB1. The A9 pin VBUS of connector USB1 is the USB input power pin, which is electrically interconnected with the VBUS terminal of the power module. The A7 pin D- and A6 pin D+ of connector USB1 are USB 2.0 differential data lines, which are directly interconnected with the uplink DP / DM pin of hub U4; Pin CC1 (A5) of connector USB1 is the configuration pin for the Type-C connector. It is connected to GND after being connected in series with current-limiting resistor R5 to achieve CC pull-down configuration for power drawing from USB. The A1 pin of connector USB1 is the GND pin of the connector's metal housing, which is directly connected to the system common ground; The A2 pin TX1+, A3 pin TX1-, B10 pin RX1+, and B11 pin RX1- of connector USB1 are high-speed differential pins and are left floating. All pins EH1 to EH4 of connector USB1 are shorted to GND.

[0012] Furthermore, the power module includes an LDO regulator chip, LDO1. The voltage regulator chip LDO1 has a voltage input terminal, pin 1 VIN, which is connected to the 5V input voltage from VBUS in the data transmission module. The input filter capacitor C12 is connected in parallel to the front end of pin 1 VIN. One end of capacitor C12 is connected to VBUS and the other end is grounded. The voltage regulator chip LDO1 has a ground pin (pin 2 VSS) and an enable pin (pin 3 CE). When pin 2 VSS and pin 3 CE are shorted, they are connected to the GND of the thermal imaging system. The 5-pin VOUT of the voltage regulator chip LDO1 is a 3.3V regulated output terminal. Two 100nF output filter capacitors C13 and C14 are connected in parallel at the output node. The other ends of capacitors C13 and C14 are grounded together. The 5-pin VOUT output provides a unified 3.3V network power supply for the entire system. The NC pin of the LDO1 voltage regulator chip is a no-connect pin and is left floating without electrical connection.

[0013] A deep learning-based home-use sub-health auxiliary diagnosis thermal imaging method, using the aforementioned sub-health auxiliary diagnosis thermal imaging system, includes the following steps: Step 1: System initialization and hardware parameter configuration, completing infrared thermal imaging data acquisition and preprocessing; The specific steps for initialization include: First, the LWR CameraAPP software on the Android system starts and performs system initialization, configures thermal imaging acquisition parameters, and acquires a 640×480 pixel single-channel human thermal imaging temperature matrix through infrared acquisition module 1. The thermal imaging temperature matrix is ​​stored in CSV document format. Raw body surface temperature data is read from the CSV document, and the raw temperature values ​​are normalized. The normalized temperature calculation formula is as follows:

[0014] The data is stored in the temps list to eliminate differences in data units; Step 2: Extract basic thermal imaging features based on the ResNet50 backbone network; Specifically, the input thermal imaging temperature matrix is ​​used. The first layer uses a 7×7 large convolutional kernel to extract global temperature features in a coarse-grained manner. Subsequently, a 3×3 sliding convolutional kernel is used to traverse the pixel region and calculate the temperature gradient features. The convolution calculation formula is as follows:

[0015] In the formula, I is the input feature map and K is the convolution kernel; the convolution result is enhanced by the ReLU activation function to enhance the nonlinear feature expression and capture the edge features formed by the slight temperature difference of 0.3℃. Step 3: Multi-scale feature extraction and cross-scale gated feature fusion; At the Conv5 layer, dilated convolution (dilation=2) and spatial pyramid pooling (SPP) structures are fused, and multi-scale feature extraction is completed in parallel using three grid max pooling methods: 6×6, 4×4, and 2×2; During the feature pyramid fusion stage, an Adam gating mechanism is introduced. When the autonomic nervous system dysfunction index > 0.7, the learning rate is reduced from 10... -3 Adaptive boost to 3×10 -3 The Adam parameter update formula relies on the Adam exponential decay parameters β1 and β2 to smooth batch training fluctuations:

[0016]

[0017] = , In the formula, For the current training gradient, m t v t These are first-order and second-order momentum, respectively; Step 4: Build a classification-temperature regression dual-head network structure and construct dual-task branches; Specifically, a feature extraction subnetwork is built after global average pooling in the ResNet backbone. The features are reduced to 512-dimensional feature vectors through two fully connected layers, and then two output heads are set: The classification head consists of a single-layer fully connected layer and Softmax activation, and uses cross-entropy loss to achieve binary classification of health / wind-cold sub-health. Temperature regression head: It consists of a single fully connected layer + sigmoid activation, and relies on mean squared error loss (MSELoss) to predict body surface temperature; The joint loss synchronously optimizes the shared features of the network's underlying layer. It adopts the ReduceLROnPlateau dynamic learning rate reduction strategy and the Early Stopping strategy to reduce the learning rate when the regression loss on the validation set stagnates, thereby suppressing model overfitting. Step 5: Inverse normalize the ResNet50 model prediction results to calculate the actual body temperature; In the specific testing phase, the ResNet50 model outputs a normalized temperature value in the range of 0 to 1. The true body surface temperature is then reconstructed using an inverse normalization formula. The normalized predicted temperature output by the ResNet50 model is as follows:

[0018] At the same time, the output temperature is adjusted to upper and lower limits, limiting it to the reasonable human body temperature range; Step 6: The Qt visualization platform loads the image and outputs the sub-health diagnosis results; The thermal image to be tested is loaded into QFileDialog, OpenCV performs image preprocessing, and after model analysis, abnormal temperature areas such as hands, head, and torso are marked on the interface. Diagnostic text is generated with the help of QTextDocument, and a PDF diagnostic report can be exported through QPrinter.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The home-use sub-health auxiliary diagnosis thermal imaging system based on deep learning of the present invention combines circuit stacking technology in hardware, which has a compact structure and low cost, and can significantly reduce the physical size and improve the portability of the entire system.

[0020] 2. The deep learning-based home-use sub-health auxiliary diagnosis thermal imaging method of this invention uses the human body thermal imaging temperature matrix acquired by an infrared device as input. It integrates dilated convolution, SPP spatial pyramid pooling, and a cross-scale gated fusion structure into a ResNet network to sequentially complete the extraction of local subtle temperature difference features, capture of multi-scale abnormal hot spots, and modeling of whole-body temperature field features, reducing the loss of subtle thermal signals. By setting up a dual-branch network structure for classification and temperature regression, and using a joint loss function of cross-entropy and mean squared error for collaborative training, supplemented by dynamic learning rate adjustment and early stopping optimization, the method improves the model overfitting problem and simultaneously enhances the sub-health detection and body surface temperature prediction effects.

[0021] 3. The home-use sub-health auxiliary diagnosis thermal imaging method based on deep learning of the present invention relies on the Adam optimization module to adaptively change the learning rate according to the autonomic nervous system dysfunction indicators, smooth the fluctuations of training batch data, correct the network weight parameters, and establish a corresponding mapping relationship between body surface temperature characteristics and sub-health state. By normalizing the original temperature data and using inverse normalization to restore the true temperature of the prediction results, the data units are unified, the data errors caused by hardware acquisition are reduced, and the reliability of the actual body temperature measurement results is ensured. Attached Figure Description

[0022] Figure 1 This is a hardware system block diagram of the home-use sub-health auxiliary diagnosis thermal imaging system based on deep learning according to the present invention. Figure 2 This is the overall circuit diagram of the home-use sub-health auxiliary diagnosis thermal imaging system based on deep learning of the present invention; Figure 3 This is a circuit diagram of the main control module in this invention; Figure 4 This is a circuit diagram of the infrared acquisition module in this invention; Figure 5 This is a circuit diagram of the visible light acquisition module in this invention; Figure 6 This is a circuit diagram of the data transmission module in this invention; Figure 7 This is a circuit diagram of the power supply module in this invention; Figure 8 This is a physical image of the home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning, which is based on the present invention. Figure 9 This is a packaged diagram of the home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning according to the present invention after the outer casing is installed; Figure 10 This is a flowchart of the home-based sub-health auxiliary diagnosis thermal imaging method based on deep learning according to the present invention; Figure 11 This is a schematic diagram of the pooling operation of the home-use sub-health auxiliary diagnosis thermal imaging method based on deep learning of the present invention. Figure 12 This is a logic flowchart of the home-use sub-health auxiliary diagnosis thermal imaging method based on deep learning of the present invention; Figure 13 This is a diagnostic output image of the patient's upper body frontal thermal imaging Qt interface. Detailed Implementation

[0023] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0024] The purpose of this invention is to address the shortcomings of existing technologies by providing a home-use sub-health auxiliary diagnostic thermal imaging system and method based on deep learning.

[0025] Example 1 The present invention relates to a home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning, with reference to... Figure 1 As shown, the system includes an infrared acquisition module 1, a visible light acquisition module 2, a main control module 3, a data transmission module 4, and a power supply module 5. The main control module 3 is equipped with an I2C peripheral interface to connect to the infrared acquisition module 1. The main control module 3 is equipped with a USB high-speed interface to connect to the visible light acquisition module 2. The main control module 3 is equipped with a USB full-speed interface to connect to the data transmission module 4. The data transmission module 4 connects to the Android system on an Android phone via a USB interface to display body temperature information. The power supply module 5 provides power to the infrared acquisition module 1, the visible light acquisition module 2, the main control module 3, and the data transmission module 4.

[0026] Reference Figures 2-3 As shown, the main control module 3 includes a main control chip U1, power supply pins, a high-speed external crystal oscillator branch, a low-speed 32K crystal oscillator branch, a reset BOOT configuration branch, and peripheral lead-out pins. In this embodiment, the main control chip U1 adopts an STM32F411CEU6 microcontroller. The main control chip U1 has a built-in hardware FPU floating-point arithmetic unit and a USB physical layer, and has multi-task scheduling, temperature floating-point calculation, and image preprocessing control functions.

[0027] The power supply pins include pins 24 (VDD), 36 (VDD), and 48 (VDD) power input terminals, all of which are connected to a 3.3V power supply voltage. Pins 35 (VSS), 23 (VSS), and 47 (VSS) are power ground pins, all of which are electrically connected to GND. C1, C2, and C3 are power decoupling capacitors, with one end connected to 3.3V and the other end grounded, connected in parallel between each VDD-VSS group to achieve power filtering. The high-speed external crystal oscillator branch includes the high-speed main crystal oscillator CRYSTAL1, capacitor C6 and capacitor C7. The high-speed main crystal oscillator CRYSTAL1 is connected between pin 5 PH0-OSC_IN and pin 6 PH1-OSC_OUT of the main control chip U1. C6 and C7 are the starting load capacitors at both ends of the crystal oscillator, and the other end of the capacitors is grounded. The low-speed 32K crystal oscillator branch includes a low-speed 32K crystal oscillator pin, which is pin 3 PC14 / OSC32IN and pin 4 PC15 / OSC32OUT. A crystal oscillator pad is reserved, and pin PC13 is brought out as a general-purpose IO pin. The reset and BOOT configuration branch includes pin 7 NRST, which is the hardware reset pin of the main control chip U1. It consists of R3, R4, C9, and C11 forming a power-on reset circuit. The upper end of R3 is connected to 3.3V, and the lower end is connected in series with C9 to ground. R4 is connected across the NRST pin and pin 44 BOOT0. The BOOT0 pin is grounded through capacitor C11. Pin 22 VCAP1 is connected to ground by an external filter capacitor C11. Pin 1 VBAT, pin 8 VSSA, and pin 9 VDDA are analog power supply pins. VBAT is connected to 3.3V, VSSA is grounded, and VDDA is connected to 3.3V and grounded with a matching decoupling capacitor. The peripheral output pins include: pin 43 PB7 and pin 42 PB6, with PB7 serving as the SDA signal terminal and PB6 as the SCL signal terminal, leading out the I2C bus; pin 13 PA3 is UART2_RX and pin 12 PA2 is UART2_TX, leading out serial port signals; pin 37 PA14 is SWCLK and pin 34 PA13 is SWDIO, with SWDIO and SWCLK pins used for debugging and downloading; pin 33 PA12 is CHIP_DP and pin 32 PA11 is CHIP_DM.

[0028] The main control module 3 is used for: (1) Thermal imaging data processing: Real-time analysis of the raw data of infrared sensor U2, and completion of radiation calculation and temperature field reconstruction through floating point unit (FPU) (accuracy ≤ ±0.5℃).

[0029] (2) Multimodal fusion control: Execute a pixel-level fusion algorithm for visible light image (640×480@30Hz) and infrared data (32×24@8Hz).

[0030] (3) Data interaction: The integrated USB FS interface is used to output composite video streams with temperature mapping to the Android mobile terminal using batch transmission mode.

[0031] (4) Main control chip U1: CortexM4 core, supports single precision FPU, built-in USB2.0FSPHY, meets the requirements of high precision calculation and real time.

[0032] Reference Figures 2-4 As shown, the infrared acquisition module 1 includes an infrared sensor U2 and an infrared interface network. The infrared interface network includes an infrared power supply branch and an I2C pull-up bus branch. In this embodiment, the infrared sensor U2 uses an MLX90640-BAB thermopile infrared array detector as the infrared detection element. Compared with the long-range detection version of the same model BAA, this BAB device optimizes the near-range pixel sampling density, with a pixel specification of 32×24, totaling 768 temperature measurement sampling points. The device operates in a temperature range of -40℃ to 85℃, and the measured temperature range of the target is -40℃ to 300℃. The frame rate is 8Hz, and the power consumption is ≤70mW. The device outputs data through an I2C serial bus, and the hardware uses the average replacement method of adjacent pixels to compensate and correct failed sampling points.

[0033] The infrared power supply branch includes capacitors C4 and C5. Pin 2 VDD of the infrared sensor U2 is the power input terminal, with two filter capacitors C4 (100nF) and C5 (10μF) connected in parallel. The other ends of the two capacitors are grounded, and pin 2 VDD is connected to a 3.3V power supply. Pin 3 GND of the infrared sensor U2 is the grounding pin and is directly electrically connected to the common ground. The I2C pull-up bus branch includes resistors R1 and R2. Pin 1 (SDA) of U2 is connected to a 3.3V power supply after being connected in series with pull-up resistor R1. The SDA pin is electrically interconnected with PB7 (SDA) of the main control chip U1. Pin 4 (SCL) of U2 is connected to a 3.3V power supply after being connected in series with pull-up resistor R2. The SCL pin is electrically interconnected with PB6 (SCL) of the main control chip. R1 and R2 are I2C bus pull-up resistors, which pull the level up to 3.3V when the bus is idle.

[0034] Infrared acquisition module 1 is used to acquire target thermal radiation data in real time and output a 768-point temperature matrix to the main control module 3 to provide thermal feature input for the fusion algorithm; it has a data calibration function, which can compensate for abnormal data (such as zero value or over-limit points) by the average value of adjacent effective pixels to ensure the reliability of the temperature matrix.

[0035] Reference Figures 2-5As shown, the visible light acquisition module 2 includes a visible light camera, a USB hub U4, and camera peripheral circuitry. The camera peripheral circuitry includes a camera crystal oscillator start-up branch, a camera power supply filtering branch, and a USB differential signal output terminal. In this embodiment, the visible light camera is an HBV-1319 UVC camera with an output resolution of 640×480 and a sampling frame rate of 30Hz, following the USB video transmission protocol. The infrared acquisition module 1 is used to supplement the missing texture details in the infrared image, achieving complementary information between high and low resolution images.

[0036] The camera crystal oscillator starting branch includes crystal oscillator X2 and inductor U3. Pin 15 XOUT and pin 16 XIN of USB hub U4 are connected across the surface-mount crystal oscillator X2. One end of inductor U3 is grounded and the other end is connected to pin 16 XIN. The camera power supply filtering branch includes capacitors C8 and C10. Pins 14 (VDD18), 13 (VDD33), and 11 (VDD5) of the USB hub U4 are the core power supply pins, respectively. Two 10μF filter capacitors, C8 and C10, are connected in parallel to pin VDD33. The other ends of capacitors C8 and C10 are grounded. Pin 12 (GND) is the common ground pin, connected to the GND of the thermal imaging system. Pins 10 (DP) and 9 (DM) are the uplink USB differential D+ / D- of hub U4, interconnected with the D+ and D- of the subsequent Type-C connector. The USB differential signal output terminals include pin 1 (DM4), pin 2 (DP4), pin 3 (DM3), and pin 4 (DP3). Pins 1 (DM4) and 2 (DP4) are one differential signal, CHIP_DM and CHIP_DP, which are connected to PA11 and PA12 of the main control chip U1. Pins 3 (DM3) and 4 (DP3) are the second differential signal, CAM_DM and CAM_DP, which are connected to an external visible light camera. The DM1 / DP1 and DM2 / DP2 pins are reserved for extended USB channels.

[0037] The visible light acquisition module 2 is used to capture high-resolution visible light images. Through edge enhancement and texture extraction algorithms, it provides spatial detail compensation for low-resolution infrared data. After spatiotemporal alignment with the infrared data, it is input into the fusion algorithm module to optimize the visual interpretability of the thermal image.

[0038] Reference Figures 2-6 As shown, the data transmission module 4 includes a Type-C connector USB1. The A9 pin VBUS of connector USB1 is the USB input power pin, which is electrically interconnected with the VBUS terminal of power module 5. The A7 pin D- and A6 pin D+ of connector USB1 are USB 2.0 differential data lines, which are directly interconnected with the uplink DP / DM pin of hub U4; Pin CC1 (A5) of connector USB1 is the configuration pin for the Type-C connector. It is connected to GND after being connected in series with current-limiting resistor R5 to achieve CC pull-down configuration for power drawing from USB. The A1 pin of connector USB1 is the GND pin of the connector's metal housing, which is directly connected to the system common ground; The A2 pin TX1+, A3 pin TX1-, B10 pin RX1+, and B11 pin RX1- of connector USB1 are high-speed differential pins and are left floating. Figure 7 (The middle is marked with an "X" and left empty). All pins EH1 to EH4 of connector USB1 are shorted to GND.

[0039] The data transmission module 4 is based on the SL2.1A chip to build a USB 2.0 Hub, supporting HS (480Mbps) and FS (12Mbps) dual modes; among them, USB Type C (HS) is used to transmit visible light video streams; USB MicroB (FS) is used to transmit infrared data and control commands; the data transmission module 4 is used to realize high-speed data interaction between the main control module 3 and the mobile terminal, and supports real-time video stream push.

[0040] Reference Figures 2-7 As shown, power module 5 includes an LDO regulator chip LDO1. The voltage regulator chip LDO1 has a voltage input terminal, pin 1 VIN, which is connected to the 5V input voltage from VBUS in the data transmission module 4. The input filter capacitor C12 is connected in parallel to the front end of pin 1 VIN. One end of capacitor C12 is connected to VBUS and the other end is grounded. The voltage regulator chip LDO1 has a ground pin (pin 2 VSS) and an enable pin (pin 3 CE). When pin 2 VSS and pin 3 CE are shorted, they are connected to the GND of the thermal imaging system. The 5-pin VOUT of the voltage regulator chip LDO1 is a 3.3V regulated output terminal. Two 100nF output filter capacitors C13 and C14 are connected in parallel at the output node. The other ends of capacitors C13 and C14 are grounded together. The 5-pin VOUT output provides a unified 3.3V network power supply for the entire system. The NC pin of the LDO1 voltage regulator chip is a no-connect pin and is left floating without electrical connection.

[0041] The power module 5 has a 5V input and a 3.3V@500mA output; it is equipped with filter capacitors (10μF+0.1μF) to suppress high-frequency noise, with a ripple coefficient ≤1%; it is used to provide a stable 3.3V power supply for the entire system modules such as the main control chip U1 and the infrared sensor U2, and supports a wide operating temperature range of 20℃~60℃.

[0042] Reference Figure 8The image shown is a circuit diagram of the home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning, as presented in this invention. A 3D-printed shell is fixed to the top of the circuit diagram. The actual circuit diagram is shown below. Figure 9 As shown.

[0043] The home-use sub-health auxiliary diagnosis thermal imaging system based on deep learning of the present invention combines circuit stacking technology in hardware, which is low cost, can greatly reduce the physical size, and improve the portability of the entire system.

[0044] This embodiment also provides a home-based sub-health auxiliary diagnostic body temperature thermal imaging method based on deep learning, using the aforementioned sub-health auxiliary diagnostic body temperature thermal imaging system, referring to... Figure 12 As shown, it includes the following steps: Step 1: System initialization and hardware parameter configuration, completing infrared thermal imaging data acquisition and preprocessing; The specific steps for initialization include: First, the LWR CameraAPP software on the Android system starts and performs system initialization, initializing the infrared sensor U2 and the visible light camera, configuring thermal imaging acquisition parameters, reading the factory calibration coefficients of the infrared sensor U2 and constructing a temperature inversion model, acquiring a 640×480 pixel single-channel human thermal imaging temperature matrix through the infrared acquisition module 1, and storing the thermal imaging temperature matrix in CSV document format; reading the raw body surface temperature data from the CSV document, and performing normalization conversion on the raw temperature values. The normalized temperature calculation formula is as follows:

[0045] The data is stored in the temps list to eliminate differences in data units; Step 2: Refer to Figure 10 As shown, basic thermal imaging features are extracted based on the ResNet50 model backbone network; Specifically, the input thermal imaging temperature matrix is ​​used. The first layer uses a 7×7 large convolutional kernel to extract global temperature features in a coarse-grained manner. Subsequently, a 3×3 sliding convolutional kernel is used to traverse the pixel region and calculate the temperature gradient features. The convolution calculation formula is as follows:

[0046] In the formula, I is the input feature map and K is the convolution kernel; the convolution result is enhanced by the ReLU activation function to enhance the nonlinear feature expression and capture the edge features formed by the slight temperature difference of 0.3℃. Step 3: Multi-scale feature extraction and cross-scale gated feature fusion; refer to Figure 11As shown, a dilated convolution (dilation=2) and a spatial pyramid pooling (SPP) structure are fused in the Conv5 layer, and multi-scale feature extraction is completed using three grid max pooling methods: 6×6, 4×4, and 2×2. An Adam gating mechanism is introduced in the feature pyramid fusion stage; when the autonomic nervous system dysfunction index > 0.7, the learning rate is reduced from 10... -3 Adaptive boost to 3×10 -3 The Adam parameter update formula relies on the Adam exponential decay parameters β1 and β2 to smooth batch training fluctuations:

[0047]

[0048] = ,

[0049] In the formula, For the current training gradient, m t v t These are first-order and second-order momentum, respectively; Step 4: Build a classification-temperature regression dual-head network structure and construct dual-task branches; Specifically, a feature extraction subnetwork is built after global average pooling in the ResNet backbone. The features are reduced to 512-dimensional feature vectors through two fully connected layers, and then two output heads are set: The classification head consists of a single-layer fully connected layer and Softmax activation, and uses cross-entropy loss to achieve binary classification of health / wind-cold sub-health. Temperature regression head: It consists of a single fully connected layer + sigmoid activation, and relies on mean squared error loss (MSELoss) to predict body surface temperature; The joint loss synchronously optimizes the shared features of the underlying network. The ReduceLROnPlateau dynamic learning rate reduction strategy and the Early Stopping strategy are adopted to reduce the learning rate when the regression loss of the validation set stagnates, thereby suppressing the overfitting of the ResNet50 model. Step 5: Inverse normalize the ResNet50 model prediction results to calculate the actual body temperature; In the specific testing phase, the ResNet50 model outputs a normalized temperature value in the range of 0 to 1. The true body surface temperature is then reconstructed using an inverse normalization formula. The normalized predicted temperature output by the ResNet50 model is as follows:

[0050] At the same time, the output temperature is adjusted to upper and lower limits, limiting it to the reasonable human body temperature range; Step 6: The Qt visualization platform loads the image and outputs the sub-health diagnosis results; A full-process interactive interface for thermal imaging image input, analysis, and diagnostic result output is built using the Qt visualization platform. Through Qt's QGraphicsView framework and custom drawing interfaces (such as QPainter), thermal imaging images can be efficiently loaded and annotated, and diagnostic results (such as marking abnormal temperature areas) can be overlaid in real time.

[0051] When setting up the platform, QtDesigner was used to design the main interface. QLabel components were added to display thermal imaging images, QPushButton to initiate image analysis, QTextBrowser to display TCM auxiliary diagnostic results, and QProgressBar to display the analysis progress. Finally, the designed UI file was saved and imported into the project.

[0052] The steps include: Step 61: Use QFileDialog to load the file selected by the user and display the selected image in the QLabel control for easy viewing by the user.

[0053] Step 62: The header file definition includes the class definition for the main window and the user interface definition, as well as the main functions of the OpenCV library. Define a function that accepts a file path as an argument to read and process images. Use OpenCV's `imread` function to read the image file and convert it to grayscale. Save the processed image as a JPEG file. Then add the path to the OpenCV library to the project's `.pro` file so that the compiler can find the OpenCV header and library files.

[0054] Step 63: Use Qt's QPixmap class to load the previously saved processed image, resize the loaded image to fit the size of the QLabel, and maintain the image's aspect ratio.

[0055] Step 64: Use QTextDocument to generate the diagnostic report content and export it as a PDF file using QPrinter. The diagnostic content can be dynamically generated based on the analysis results of a deep learning model. The final diagnostic results are as follows: Figure 13 As shown.

[0056] This invention presents a home-use sub-health auxiliary diagnostic thermal imaging method based on deep learning. It utilizes noise reduction algorithms and Gaussian filtering to further improve the visualization of thermal radiation images, facilitating user observation of patient body temperature and image processing by the diagnostic system. Employing a multi-scale fusion mechanism combined with a dual-head architecture, it achieves high-precision distributed thermal radiation temperature measurement and display. Its advantage lies in effectively solving problems such as missed detection of small targets, loss of detail, and sensitivity to scale changes at a single scale by integrating feature maps of different levels or resolutions. This further enhances the AI ​​model's ability to perceive multi-scale targets and effectively improves the system's temperature and spatial resolution.

[0057] In this embodiment, a comparative verification experiment is set up to verify the effectiveness of the home-based sub-health auxiliary diagnosis thermal imaging method based on deep learning of the present invention.

[0058] Validation dataset: 20,053 thermal images of front and back of healthy individuals with sub-health conditions due to wind-cold and 2,000 thermal images of healthy individuals were used. Each sample was matched with the corresponding body surface temperature label in a CSV document. Label 0 represents healthy and label 1 represents sub-health conditions due to wind-cold.

[0059] Evaluation metrics for validation experiments: F1-score for classification and R-value for temperature prediction were used. 2 As an evaluation metric, the classification F1-score of this invention can reach 0.91, and the thermal asymmetry feature fits metabolic risk with an R-value. 2 =0.92.

[0060] Validation experimental environment and training parameters: The model was built on the PyTorch framework, using pre-trained ResNet50 model weight transfer learning; image preprocessing included resizing, random flipping, and random rotation transformations to achieve data augmentation; the overall optimizer was Adam, with a base learning rate range of 10. -5 ~10 -3 The cross-entropy and mean squared error joint loss function is adopted; the training process is divided into training set and validation set, and the learning rate is dynamically adjusted according to the validation set loss and the early stopping strategy is enabled to save the best model best_mode.pth.

[0061] Analysis of all collected data showed that the probability of misdiagnosis (i.e., inaccurate output of disease information) for patients with wind-cold syndrome was 5%, while the probability of misdiagnosis for healthy individuals was 4%. This demonstrates that the system has high reliability in terms of diagnostic accuracy.

[0062] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning, characterized in that: It includes an infrared acquisition module (1), a visible light acquisition module (2), a main control module (3), a data transmission module (4), and a power supply module (5). The main control module (3) is configured with an I2C peripheral interface to connect to the infrared acquisition module (1). The main control module (3) is configured with a USB high-speed interface to connect to an external visible light acquisition module (2). The main control module (3) is configured with a USB full-speed interface for external data transmission module (4). The data transmission module (4) connects to the Android system via a USB interface to display body temperature information. The power supply module (5) provides power to the infrared acquisition module (1), the visible light acquisition module (2), the main control module (3), and the data transmission module (4).

2. The home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning according to claim 1, characterized in that: The main control module (3) includes a main control chip U1, power supply pins, a high-speed external crystal oscillator branch, a low-speed 32K crystal oscillator branch, a reset BOOT configuration branch, and peripheral lead-out pins. The power supply pins include pins 24 (VDD), 36 (VDD), and 48 (VDD) power input terminals, all of which are connected to a 3.3V power supply voltage. Pins 35 (VSS), 23 (VSS), and 47 (VSS) are power ground pins, all of which are electrically connected to GND. C1, C2, and C3 are power decoupling capacitors, with one end connected to 3.3V and the other end grounded, and are connected in parallel between each VDD-VSS group to achieve power filtering. The high-speed external crystal oscillator branch includes a high-speed main crystal oscillator CRYSTAL1, capacitor C6 and capacitor C7. The high-speed main crystal oscillator CRYSTAL1 is connected between pin 5 PH0-OSC_IN and pin 6 PH1-OSC_OUT of the main control chip U1. C6 and C7 are the starting load capacitors at both ends of the crystal oscillator, and the other end of the capacitors is grounded. The low-speed 32K crystal oscillator branch includes a low-speed 32K crystal oscillator pin, which is a 3-pin PC14 / OSC32IN and a 4-pin PC15 / OSC32OUT. A crystal oscillator pad is reserved, and the PC13 pin is brought out as a general-purpose IO pin. The reset and BOOT configuration branch includes a 7-pin NRST, which is the hardware reset pin of the main control chip U1. It consists of R3, R4, C9, and C11 forming a power-on reset circuit. The upper end of R3 is connected to 3.3V, and the lower end is connected in series with C9 to ground. R4 is connected across the NRST pin and the 44-pin BOOT0. The BOOT0 pin is grounded through capacitor C11. The 22-pin VCAP1 is connected to ground by an external filter capacitor C11. The 1-pin VBAT, 8-pin VSSA, and 9-pin VDDA are analog power supply pins. VBAT is connected to 3.3V, VSSA is grounded, and VDDA is connected to 3.3V and grounded with a matching decoupling capacitor. The peripheral output pins include: pin 43 PB7 and pin 42 PB6, with PB7 serving as the SDA signal terminal and PB6 serving as the SCL signal terminal, leading out the I2C bus; pin 13 PA3 is UART2_RX and pin 12 PA2 is UART2_TX, leading out serial port signals; pin 37 PA14 is SWCLK and pin 34 PA13 is SWDIO, with SWDIO and SWCLK pins used for debugging and downloading; pin 33 PA12 is CHIP_DP and pin 32 PA11 is CHIP_DM.

3. The home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning according to claim 2, characterized in that: The infrared acquisition module (1) includes an infrared sensor U2 and an infrared interface network. The infrared interface network includes an infrared power supply branch and an I2C pull-up bus branch. The infrared power supply branch includes capacitors C4 and C5. Pin 2 VDD of the infrared sensor U2 is the power input terminal, and two filter capacitors C4 and C5 are connected in parallel. The other end of the two capacitors is grounded, and pin 2 VDD is connected to a 3.3V power supply. Pin 3 GND of the infrared sensor U2 is the grounding pin and is directly electrically connected to the common ground. The I2C pull-up bus branch includes resistors R1 and R2. Pin 1 SDA of U2 is connected to a 3.3V power supply after being connected in series with pull-up resistor R1. The SDA pin is electrically interconnected with PB7 of the main control chip U1. Pin 4 SCL of U2 is connected to a 3.3V power supply after being connected in series with pull-up resistor R2. The SCL pin is electrically interconnected with PB6 of the main control chip. R1 and R2 are I2C bus pull-up resistors, which realize the level is pulled up to 3.3V when the bus is idle.

4. The home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning according to claim 3, characterized in that: The visible light acquisition module (2) includes a visible light camera, a USB hub U4, and camera peripheral circuitry. The camera peripheral circuitry includes a camera crystal oscillator start-up branch, a camera power supply filtering branch, and a USB differential signal output terminal. The camera crystal oscillator starting branch includes crystal oscillator X2 and inductor U3. Pin 15 XOUT and pin 16 XIN of USB hub U4 are connected across the surface mount crystal oscillator X2. One end of inductor U3 is grounded and the other end is connected to pin 16 XIN. The camera power supply filtering branch includes capacitors C8 and C10. Pins 14 (VDD18), 13 (VDD33), and 11 (VDD5) of the USB hub U4 are the core power supply pins, respectively. Two 10μF filter capacitors, C8 and C10, are connected in parallel to pin VDD33. The other ends of capacitors C8 and C10 are grounded. Pin 12 (GND) is the common ground pin, connected to the GND of the thermal imaging system. Pins 10 (DP) and 9 (DM) are the uplink USB differential D+ / D- of hub U4, interconnected with the D+ and D- of the subsequent Type-C connector. The USB differential signal output terminals include pin 1 (DM4), pin 2 (DP4), pin 3 (DM3), and pin 4 (DP3). Pins 1 (DM4) and 2 (DP4) are one differential signal, CHIP_DM and CHIP_DP, which are connected to PA11 and PA12 of the main control chip U1. Pins 3 (DM3) and 4 (DP3) are the second differential signal, CAM_DM and CAM_DP, which are connected to an external visible light camera. The DM1 / DP1 and DM2 / DP2 pins are reserved for extended USB channels.

5. The home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning according to claim 4, characterized in that: The data transmission module (4) includes a Type-C connector USB1. The A9 pin VBUS of connector USB1 is the USB input power pin, which is electrically interconnected with the VBUS terminal of power module (5). The A7 pin D- and A6 pin D+ of connector USB1 are USB 2.0 differential data lines, which are directly interconnected with the uplink DP / DM pin of hub U4; Pin CC1 (A5) of connector USB1 is the configuration pin for the Type-C connector. It is connected to GND after being connected in series with current-limiting resistor R5 to achieve CC pull-down configuration for power drawing from USB. The A1 pin of connector USB1 is the GND pin of the connector's metal housing, which is directly connected to the system common ground; The A2 pin TX1+, A3 pin TX1-, B10 pin RX1+, and B11 pin RX1- of connector USB1 are high-speed differential pins and are left floating. All pins EH1 to EH4 of connector USB1 are shorted to GND.

6. The home-use sub-health auxiliary diagnostic thermal imaging system based on deep learning according to claim 5, characterized in that: The power module (5) includes an LDO regulator chip LDO1. The voltage regulator chip LDO1 has a voltage input terminal, VIN, which is connected to the 5V input voltage from VBUS in the data transmission module (4). The input filter capacitor C12 is connected in parallel at the front end of the VIN terminal. One end of the capacitor C12 is connected to VBUS and the other end is grounded. The voltage regulator chip LDO1 has a ground pin (pin 2 VSS) and an enable pin (pin 3 CE). When pin 2 VSS and pin 3 CE are shorted, they are connected to the GND of the thermal imaging system. The 5-pin VOUT of the voltage regulator chip LDO1 is a 3.3V regulated output terminal. Two output filter capacitors C13 and C14 are connected in parallel at the output node. The other ends of capacitors C13 and C14 are grounded together. The 5-pin VOUT output provides a unified 3.3V network power supply for the entire system. The NC pin of the LDO1 voltage regulator chip is a no-connect pin and is left floating without electrical connection.

7. A home-based sub-health auxiliary diagnostic thermal imaging method based on deep learning, using the aforementioned sub-health auxiliary diagnostic thermal imaging system, includes the following steps: Step 1: System initialization and hardware parameter configuration, completing infrared thermal imaging data acquisition and preprocessing; The specific steps for initialization include: First, the LWR CameraAPP software on the Android system starts and performs system initialization, configures thermal imaging acquisition parameters, and acquires a 640×480 pixel single-channel human thermal imaging temperature matrix through the infrared acquisition module (1). The thermal imaging temperature matrix is ​​stored in the form of a CSV document. The original surface temperature data is read from the CSV document, and the original temperature values ​​are normalized. The normalized temperature calculation formula is: ； The data is stored in the temps list to eliminate differences in data units; Step 2: Extract basic thermal imaging features based on the ResNet50 backbone network; Given a thermal imaging temperature matrix, the first layer uses a 7×7 large convolutional kernel to extract global temperature features in a coarse-grained manner. Subsequent layers use a 3×3 sliding convolutional kernel to traverse the pixel region and calculate temperature gradient features. The convolution calculation formula is as follows: ； In the formula, I is the input feature map and K is the convolution kernel; the convolution result is enhanced by the ReLU activation function to enhance the nonlinear feature expression and capture the edge features formed by subtle temperature differences; Step 3: Multi-scale feature extraction and cross-scale gated feature fusion; At the Conv5 layer, dilated convolution and spatial pyramid pooling (SPP) structures are fused, and multi-scale feature extraction is completed in parallel using three grid max pooling methods: 6×6, 4×4, and 2×2; During the feature pyramid fusion stage, an Adam gating mechanism is introduced. When the autonomic nervous system dysfunction index > 0.7, the learning rate is reduced from 10... -3 Adaptive boost to 3×10 -3 The Adam parameter update formula relies on the Adam exponential decay parameters β1 and β2 to smooth batch training fluctuations: ； ； = ， ； In the formula, For the current training gradient, m t v t These are first-order and second-order momentum, respectively; Step 4: Build a classification-temperature regression dual-head network structure and construct dual-task branches; A feature extraction subnetwork is built after global average pooling on the ResNet backbone. The features are reduced to 512-dimensional feature vectors through two fully connected layers, and then two output heads are set: The classification head consists of a single-layer fully connected layer and Softmax activation, and relies on cross-entropy loss to achieve binary classification of health / wind-cold sub-health. Temperature regression head: It consists of a single-layer fully connected layer + sigmoid activation, and relies on mean squared error loss to predict body surface temperature; The joint loss synchronously optimizes the shared features of the network's underlying layer. It adopts the ReduceLROnPlateau dynamic learning rate reduction strategy and the Early Stopping strategy to reduce the learning rate when the regression loss on the validation set stagnates, thereby suppressing model overfitting. Step 5: Inverse normalize the ResNet50 model prediction results to calculate the actual body temperature; During the testing phase, the ResNet50 model outputs a normalized temperature value in the range of 0 to 1. The true body surface temperature is then reconstructed using an inverse normalization formula. The normalized predicted temperature output by the ResNet50 model is as follows: ； At the same time, the output temperature is adjusted to upper and lower limits, limiting it to the reasonable human body temperature range; Step 6: The Qt visualization platform loads the image and outputs the sub-health diagnosis results; The thermal image to be tested is loaded into QFileDialog, image preprocessing is performed by OpenCV, and after analysis by the ResNet50 model, abnormal temperature areas such as hands, head, and torso are marked on the interface. Diagnostic text is generated with the help of QTextDocument, and a PDF diagnostic report can be exported through QPrinter.

Images