Auxiliary diagnosis and treatment system and method based on thermal imaging equipment

Abstract

The invention particularly relates to an auxiliary diagnosis and treatment system and method based on thermal imaging equipment, and the system comprises an acquisition module which is used for collecting a body surface temperature distribution image of a patient and symptom information of the patient; the processing module is used for preprocessing the voice information, the pulse condition information, the inquiry information and the patient body surface temperature distribution image to obtain fused patient symptom information and a structured temperature distribution image, encrypting the fused patient symptom information and the structured temperature distribution image and uploading the fused patient symptom information and the structured temperature distribution image to the diagnosis module; the diagnosis module is used for encoding the fused patient symptom information and the structured temperature distribution image based on a preset cloud diagnosis model to obtain encoded fusion data, and generating a preliminary diagnosis result and / or suggesting supplementary inspection items based on the encoded fusion data; and the output module is used for acquiring confirmation information and / or supplementary information generated based on the preliminary diagnosis result and / or suggested supplementary inspection items, and generating a diagnosis report according to the information. Therefore, the problems of insufficient experience, high misdiagnosis rate and the like of doctors in remote areas are solved, the diagnosis result is optimized, and the misdiagnosis risk is reduced.

Description

Technical Field

[0001] This application relates to the field of medical technology, and in particular to an auxiliary diagnosis and treatment system and method based on thermal imaging equipment. Background Technology

[0002] In remote or resource-scarce areas, general practitioners, limited by their experience, often face the challenge of accurately diagnosing certain complex diseases (such as rheumatic and immune diseases, early inflammatory lesions, and autonomic nervous system dysfunction), significantly increasing the risk of misdiagnosis and missed diagnosis. In cities, traditional imaging equipment (such as computed tomography, CT) is widely used for examinations, offering high accuracy, but its high cost and complex operation limit its widespread application in primary healthcare institutions and remote areas. Meanwhile, traditional Chinese medicine (TCM) diagnosis, relying on observation, auscultation, inquiry, and palpation, possesses the advantage of holistic diagnosis, emphasizing the functional state and dynamic balance of the body's organs and meridians, offering a unique advantage in early disease detection and treatment. However, current TCM diagnosis lacks quantitative and objective auxiliary diagnostic methods, and diagnostic results rely heavily on the physician's subjective experience, making standardization and widespread application difficult.

[0003] Therefore, there is an urgent need for a new auxiliary diagnostic and treatment method that combines modern medical functional imaging with traditional Chinese medicine syndrome differentiation theory. This method can not only objectively reflect the early functional changes of diseases, but also give full play to the advantages of traditional Chinese medicine holistic syndrome differentiation, providing doctors with more accurate and comprehensive diagnostic references.

[0004] Infrared thermal imaging, a functional medical imaging technology based on the theory of human life thermodynamics, dynamically, continuously, and comprehensively acquires high-resolution temperature distribution maps of the human body surface in a non-contact manner, reflecting the body's energy metabolism and the state of imbalance between cold and heat. Combined with the theory of Zang-Xiang (organ manifestation) and meridian zoning in traditional Chinese medicine, infrared thermal imaging can realize the correspondence between body surface temperature and the functional state of internal organs, assisting in the objective quantification and visualization analysis of syndromes such as cold and heat.

[0005] Compared to traditional morphological imaging techniques, infrared thermal imaging offers advantages such as being non-invasive, non-destructive, easy to operate, and low-cost, making it suitable for widespread application in grassroots and remote areas. Leveraging artificial intelligence and cloud-based large language model-assisted diagnostic technologies, it enables intelligent preprocessing on edge devices and precise cloud-based analysis, effectively addressing diagnostic challenges arising from insufficient physician experience and limited medical resources, and promoting the intelligentization and widespread adoption of integrated traditional Chinese and Western medicine treatment models. Summary of the Invention

[0006] This application provides an auxiliary diagnosis and treatment system and method based on thermal imaging equipment to solve the problems of insufficient experience of doctors in remote areas and high misdiagnosis rate, optimize diagnostic results, and reduce the risk of misdiagnosis.

[0007] A first aspect of this application provides an auxiliary diagnostic and treatment system based on thermal imaging equipment, comprising: an acquisition module, a processing module, a diagnostic module, and an output module, wherein the diagnostic module is deployed in the cloud.

[0008] The acquisition module acquires images of the patient's body surface temperature distribution based on an infrared thermal imaging device, and acquires patient symptom information based on a voice acquisition terminal and a physiological signal interface. The patient symptom information includes voice information, pulse information, and consultation information. The image acquisition is based on an infrared thermal imaging device, and the symptom information acquisition includes a voice acquisition terminal and a physiological signal interface.

[0009] The processing module is used to preprocess the patient's body surface temperature distribution image on the terminal side using an integrated lightweight AI model, and to structure the temperature information of different parts into body surface temperature feature data according to the first processing result; at the same time, it performs semantic recognition and structured processing on the voice information, pulse information and consultation information, and fuses the second processing result with the body surface temperature feature data to obtain fused structured multimodal feature data, and encrypts the fused structured multimodal feature data before uploading it to the diagnostic module;

[0010] The diagnostic module is used to receive the fused structured multimodal feature data, and to perform reasoning analysis on the fused structured multimodal feature data based on the medical big language model and the fused knowledge graph to obtain encoded fused data, and to generate preliminary diagnostic results and / or suggestions for supplementary examination items based on the encoded fused data.

[0011] The output module is used to acquire confirmation information and / or supplementary information generated based on the preliminary diagnosis results and / or the suggested supplementary examination items, and generate a diagnostic report based on the doctor's review and confirmation results and the preliminary diagnosis results and / or the suggested supplementary examination items and / or supplementary information.

[0012] Optionally, the acquisition module includes:

[0013] The thermal imaging acquisition unit is used to acquire images of the patient's body surface temperature distribution.

[0014] A multimodal information acquisition unit is used to acquire the voice information, the pulse information, and the consultation information.

[0015] Optionally, the processing module is specifically used for:

[0016] The first processing unit is used to convert the voice information, pulse information and consultation information into text information through natural language processing based on the patient symptom information, perform semantic parsing, extract patient symptom features for structured representation, and obtain fused patient symptom information based on the semantic parsing results.

[0017] The second processing unit is used to perform preliminary feature extraction based on the patient's body surface temperature distribution image through an end-side neural network, and to identify, classify and locate abnormal areas in the data extracted by the preliminary feature extraction based on a convolutional neural network. It also uses a site recognition model to label the temperature values ​​of the corresponding anatomical regions and generate a structured temperature distribution image as an image feature representation.

[0018] Optionally, the thermal imaging-based assisted diagnosis and treatment system further includes:

[0019] An optimization module is used to continuously optimize and update the preset cloud-based diagnostic model based on the preliminary diagnostic results and / or the results of the suggested supplementary examinations. The optimization includes calibrating the model inference path based on the doctor's confirmation results, fine-tuning the weights based on newly collected sample data, and improving the generalization ability of the preset cloud-based diagnostic model for rare symptom combinations through a meta-learning mechanism. Furthermore, the module dynamically adjusts the parameter configuration of the diagnostic model based on the clinical characteristics, disease spectrum differences, and lifestyle habits of different regions and populations.

[0020] Optionally, the processing module is specifically used for:

[0021] Based on a preset data security strategy, the fused patient symptom information and structured temperature distribution image are locally encrypted. The encryption strategy includes using symmetric encryption and heterogeneous feature masking methods for multimodal data respectively. The fused patient symptom information and structured temperature distribution image are then encrypted and uploaded to the preset cloud diagnostic model.

[0022] Optionally, the diagnostic module is further configured to:

[0023] After generating the diagnostic results, the causes of the patient's symptoms are comprehensively assessed based on multimodal fusion data, combining the TCM syndrome differentiation and treatment system with modern medical clinical standards, to obtain a personalized rehabilitation plan that integrates TCM and Western medicine.

[0024] A second aspect of this application provides an auxiliary diagnosis and treatment method based on a thermal imaging device, employing the aforementioned auxiliary diagnosis and treatment system based on a thermal imaging device, comprising the following steps:

[0025] Collect images of the patient's body surface temperature distribution and patient symptom information, wherein the patient symptom information includes voice information, pulse information and consultation information;

[0026] The voice information, pulse information, and consultation information are preprocessed to obtain fused patient symptom information. The patient's body surface temperature distribution image is preprocessed to obtain a structured temperature distribution image. The fused patient symptom information and the structured temperature distribution image are encrypted and uploaded to a preset cloud diagnostic model.

[0027] Based on the preset cloud-based diagnostic model, the fused patient symptom information and the structured temperature distribution image are encoded to obtain encoded fused data. Based on the encoded fused data, a preliminary diagnostic result and / or suggestions for supplementary examinations are generated as a draft treatment recommendation.

[0028] Obtain confirmation and / or supplementary information generated based on the preliminary diagnosis results and / or the suggested supplementary examination items, and generate a final diagnosis report based on the above diagnosis results and information after the doctor's review and confirmation.

[0029] Optionally, the acquisition of the patient's body surface temperature distribution image and patient symptom information includes:

[0030] Acquire images of the patient's body surface temperature distribution;

[0031] Collect the voice information, pulse information, and consultation information.

[0032] Optionally, obtaining confirmation information and / or supplementary information generated based on the preliminary diagnostic results and / or the recommended supplementary examination items, and generating a diagnostic report based on the preliminary diagnostic results and / or the recommended supplementary examination items and / or supplementary information, includes:

[0033] Based on the patient's symptom information, the speech information, pulse information, and consultation information are converted into text information through natural language processing, and then semantic parsing is performed to extract the patient's symptom features for structured representation. The fused patient symptom information is obtained based on the semantic parsing results.

[0034] Based on the patient's body surface temperature distribution image, preliminary feature extraction is performed using an edge-side neural network, and abnormal region detection is performed on the data extracted by the preliminary feature extraction using a convolutional neural network. Abnormal regions with temperature deviation characteristics in the patient's body surface temperature distribution image are automatically labeled, forming a structured temperature distribution image and its location semantic labels.

[0035] Optionally, the process of encoding the fused patient symptom information and the structured temperature distribution image based on the preset cloud diagnostic model to obtain encoded fused data, and generating preliminary diagnostic results and / or suggested supplementary examination items based on the encoded fused data as a draft treatment recommendation, further includes:

[0036] Based on the preliminary diagnosis results and / or the results of the suggested supplementary examinations, the preset cloud-based diagnostic model is continuously optimized and updated. The optimization includes calibrating the model inference path based on the doctor's confirmation results, fine-tuning the weights based on newly collected sample data, and improving the generalization ability of the preset cloud-based diagnostic model for rare symptom combinations through a meta-learning mechanism. Furthermore, the parameter configuration of the diagnostic model is dynamically adjusted based on the clinical characteristics, disease spectrum differences, and lifestyle habits of different regions and populations.

[0037] Optionally, encrypting the fused patient symptom information and the structured temperature distribution image and uploading them to a preset cloud diagnostic model includes:

[0038] Based on a preset data security strategy, the fused patient symptom information and structured temperature distribution image are locally encrypted. The encryption strategy includes using symmetric encryption and heterogeneous feature masking methods for multimodal data respectively. The fused patient symptom information and structured temperature distribution image are then encrypted and uploaded to the preset cloud diagnostic model.

[0039] Optionally, the step of obtaining confirmation information and / or supplementary information generated based on the preliminary diagnostic results and / or the suggested supplementary examination items, and generating a final diagnostic report based on the above diagnostic results and information after review and confirmation by a doctor, further includes:

[0040] After generating the diagnostic results, the causes of the patient's symptoms are comprehensively assessed based on multimodal fusion data, combining the TCM syndrome differentiation and treatment system with modern medical clinical standards, to obtain a personalized rehabilitation plan that integrates TCM and Western medicine.

[0041] Therefore, the acquisition module collects images of the patient's body surface temperature distribution and patient symptom information; the processing module preprocesses the voice information, pulse information, consultation information, and patient body surface temperature distribution images to obtain fused patient symptom information and a structured temperature distribution image, encrypts them, and uploads them to the diagnosis module; the diagnosis module encodes the fused patient symptom information and structured temperature distribution image based on a preset cloud-based diagnosis model to obtain encoded fused data, and generates a preliminary diagnosis result and / or suggested supplementary examination items based on this; the output module obtains confirmation information and / or supplementary information generated based on the preliminary diagnosis result and / or suggested supplementary examination items, and generates a diagnostic report based on this information. This addresses the problems of insufficient doctor experience and high misdiagnosis rates in remote areas, optimizes diagnostic results, and reduces the risk of misdiagnosis.

[0042] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0043] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0044] Figure 1 This is a block diagram of an auxiliary diagnostic and treatment system based on a thermal imaging device according to an embodiment of this application;

[0045] Figure 2 This is a schematic diagram of the overall system architecture of an auxiliary diagnosis and treatment system based on thermal imaging equipment according to an embodiment of this application;

[0046] Figure 3 This is a schematic diagram illustrating an embodiment of an auxiliary diagnostic and treatment system based on a thermal imaging device according to one embodiment of this application;

[0047] Figure 4 This is an operation flowchart of an auxiliary diagnosis and treatment system based on a thermal imaging device according to an embodiment of this application;

[0048] Figure 5 This is a flowchart of an auxiliary diagnosis and treatment method based on a thermal imaging device provided according to an embodiment of this application. Detailed Implementation

[0049] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0050] The following describes an embodiment of the auxiliary diagnosis and treatment system and method based on thermal imaging equipment according to the present application, with reference to the accompanying drawings. Addressing the problems of insufficient experience and high misdiagnosis rates among doctors in remote areas mentioned in the background art, this application provides an auxiliary diagnosis and treatment system based on thermal imaging equipment. The system involves: acquiring patient surface temperature images via an acquisition module; acquiring voice information, pulse information, and consultation information to obtain patient symptom information; preprocessing multimodal data via a processing module to obtain fused patient symptom information; preprocessing the patient surface temperature distribution image to obtain a structured temperature distribution image; encrypting the fused patient symptom information and the structured temperature distribution image and uploading them to a diagnosis module; encoding the fused patient symptom information and the structured temperature distribution image based on a preset cloud-based diagnosis model to obtain encoded fused data, generating a preliminary diagnosis result and / or suggested supplementary examination items; and obtaining confirmation information and / or supplementary information generated based on the preliminary diagnosis result and / or suggested supplementary examination items via an output module, and generating a diagnostic report. This solves the problems of insufficient experience and high misdiagnosis rates among doctors in remote areas, optimizes diagnostic results, and reduces the risk of misdiagnosis.

[0051] Specifically, Figure 1 This is a block diagram of an auxiliary diagnosis and treatment system based on thermal imaging equipment provided in an embodiment of this application.

[0052] like Figure 1 As shown, the thermal imaging-based assisted diagnosis and treatment system 10 includes: an acquisition module 100, a processing module 200, a diagnostic module 300, and an output module 400, wherein...

[0053] The acquisition module 100 is used to acquire images of the patient's body surface temperature distribution based on an infrared thermal imaging device, and to acquire patient symptom information based on a voice acquisition terminal and a physiological signal interface. The patient symptom information includes voice information, pulse information, and consultation information. The image acquisition is based on an infrared thermal imaging device, and the symptom information acquisition includes a voice acquisition terminal and a physiological signal interface.

[0054] The processing module 200 is used to preprocess the patient's body surface temperature distribution image on the terminal side using an integrated lightweight AI model, and to structure the temperature information of different parts into body surface temperature feature data according to the first processing result; at the same time, it performs semantic recognition and structured processing on voice information, pulse information and consultation information, and fuses the second processing result with the body surface temperature feature data to obtain fused structured multimodal feature data, and encrypts the fused structured multimodal feature data before uploading it to the diagnostic module 300;

[0055] The diagnostic module 300 is used to receive the fused structured multimodal feature data, and to perform reasoning analysis on the fused structured multimodal feature data based on the medical big language model and the fused knowledge graph to obtain the encoded fused data, and to generate preliminary diagnostic results and / or suggestions for supplementary examination items based on the encoded fused data.

[0056] Output module 400 is used to acquire confirmation information and / or supplementary information generated based on preliminary diagnostic results and / or suggested supplementary examination items, and to generate a diagnostic report based on the doctor's review and confirmation results, according to the preliminary diagnostic results and / or suggested supplementary examination items and / or supplementary information.

[0057] Specifically, this embodiment utilizes a low-cost thermal imager to acquire images of relevant patient areas and incorporates an intelligent algorithm to perform preliminary detection of areas with abnormal temperatures (such as inflammation, abnormal local blood circulation, etc.). Simultaneously, it acquires patient complaints and TCM diagnostic information through various methods, including voice, text, tongue imaging, pulse diagnosis, and TCM consultation. The acquired data is encrypted and then uploaded to a pre-set cloud-based diagnostic model. This model integrates images, structured text, and TCM data to generate a preliminary diagnostic result and recommend further examinations. After confirmation by a doctor, this embodiment automatically generates a formatted diagnostic report and subsequent treatment suggestions. This embodiment fully leverages the complementary advantages of TCM and Western medicine diagnostic methods, possessing high accuracy, data security, and self-learning optimization capabilities, making it suitable for auxiliary diagnosis of pain-related diseases in remote and primary healthcare institutions.

[0058] Optionally, in some embodiments, the acquisition module 100 includes: a thermal imaging acquisition unit for acquiring images of the patient's body surface temperature distribution; and a multimodal information acquisition unit for acquiring voice information, pulse information, and consultation information.

[0059] Understandably, the thermal imaging acquisition unit utilizes a low-cost, high-sensitivity thermal imager to acquire images of the target area and capture information on the temperature distribution of the human body surface. The multimodal information acquisition unit includes voice input, which converts the patient's described pain symptoms and discomfort into text data through an integrated voice recognition module; text and consultation input, allowing doctors to supplement patient medical history, chief complaints, and other information through typing or structured consultation input, further refining the patient's data file; and a traditional Chinese medicine data acquisition submodule, which includes, but is not limited to, tongue imaging, pulse diagnosis, and recording of traditional Chinese medicine consultation information, used to obtain the basic data required for traditional Chinese medicine diagnosis.

[0060] Optionally, in some embodiments, the processing module 200 is specifically used for: a first processing unit, which is used to convert speech information, pulse information and consultation information into text information through natural language processing based on patient symptom information, perform semantic parsing, extract patient symptom features for structured representation, and obtain fused patient symptom information based on the semantic parsing results; and a second processing unit, which is used to perform preliminary feature extraction based on the patient's body surface temperature distribution image through an edge-side neural network, and perform abnormal region identification, classification and localization on the data of the preliminary feature extraction based on a convolutional neural network, and label the temperature values ​​of the corresponding anatomical regions based on the location recognition model to generate a structured temperature distribution image as an image feature representation.

[0061] Understandably, Natural Language Processing (NLP) is used to semantically analyze the speech-to-text data, extract keywords, and convert it into structured data to aid image diagnosis. The patient's body surface temperature distribution image undergoes preliminary feature extraction using an edge-side neural network. These features may include information such as average temperature and temperature gradient in different regions. Edge-side processing helps reduce the amount of data transmitted to the cloud, improving processing speed and privacy protection. Anomaly detection is performed on the data from the preliminary feature extraction using a Convolutional Neural Network (CNN). Abnormal areas are circled or highlighted on the original image, and additional information, such as estimated severity or type, may be added. Finally, all extracted information and annotations are integrated to generate a structured temperature distribution image, which not only includes the original temperature data but also overlays the location, size, and other relevant information of abnormal areas, facilitating doctors' rapid understanding of the patient's condition.

[0062] Optionally, in some embodiments, the system further includes: an optimization module, used to continuously optimize and update a preset cloud-based diagnostic model based on preliminary diagnostic results and / or the results of suggested supplementary examinations. The optimization includes calibrating the model inference path based on the doctor's confirmation results, fine-tuning the weights based on newly collected sample data, and improving the generalization ability of the preset cloud-based diagnostic model for rare symptom combinations through a meta-learning mechanism. Furthermore, the system dynamically adjusts the parameter configuration of the diagnostic model based on the clinical characteristics, disease spectrum differences, and lifestyle habits of different regions and populations.

[0063] It is understood that this application includes optimization and self-learning mechanisms, allowing doctors to provide feedback on diagnostic results and regularly update the cloud database to continuously optimize the model's accuracy and adaptability. Embodiments of this application can also self-adjust and personalize the parameters of the preset cloud diagnostic model based on the clinical characteristics of different regions and populations. Doctor confirmation results are used as a supervisory signal to calibrate the model's inference path, ensuring consistency with clinical logic. Simultaneously, newly collected sample data is used to fine-tune the model weights online. For the challenge of rare symptom identification, embodiments of this application introduce a meta-learning mechanism, significantly improving the model's generalization ability to sparse symptom combinations by simulating multi-task learning scenarios. Embodiments of this application also possess regional adaptive functionality, dynamically adjusting model parameter configurations by analyzing population characteristics, disease spectrum distribution, and lifestyle differences in different regions, achieving regionalized and population-sensitive personalized inference path selection.

[0064] It should be noted that the big data and deep learning integrated diagnostic model deployed on the cloud server integrates preprocessed multimodal data from various endpoints, including thermal imaging image information that has been preliminarily judged by the built-in neural network, text data obtained through speech recognition and natural language processing, and tongue image, pulse, and other information obtained by the TCM data acquisition module 100. The core of the preset cloud diagnostic model lies in constructing a knowledge graph based on professional pain guidelines and other data. It utilizes authoritative pain diagnosis guidelines and standardized diagnostic SOP processes, and extracts and constructs a knowledge graph based on a large amount of desensitized clinical data, providing a structured knowledge foundation for comprehensive diagnosis. Based on the constructed knowledge graph, the preset cloud diagnostic model is fine-tuned to better meet the diagnostic needs of pain-related diseases. Furthermore, a large expert model (combining expert rules and machine learning models) performs multi-dimensional analysis of the comprehensive data, generating preliminary diagnostic results and recommending subsequent supplementary examinations (such as blood tests, ultrasound, etc.) based on these results. After preprocessing, all data is uniformly encoded in the preset cloud diagnostic model using an adaptive data fusion algorithm, providing the large model with a high-quality, cross-modal input automatic diagnostic report and feedback module.

[0065] Optionally, in some embodiments, the processing module 200 is specifically used to: locally encrypt the fused patient symptom information and structured temperature distribution image based on a preset data security strategy. The encryption strategy includes using symmetric encryption and heterogeneous feature masking methods for multimodal data respectively, and then encrypting the fused patient symptom information and structured temperature distribution image before uploading it to a preset cloud diagnostic model.

[0066] Understandably, all collected data is encrypted and stored using technologies that comply with relevant medical data protection standards before transmission. Encryption is performed based on a preset encryption strategy, with optimal encryption methods deployed for different modalities of data. Specifically, a highly efficient symmetric encryption algorithm is used for structured temperature distribution images to ensure data processing performance, while heterogeneous feature masking technology is applied to desensitize sensitive information in patient symptom text information. After encryption, the processing module 200 synchronously transmits the encrypted patient symptom data and temperature image data to the designated cloud-based diagnostic analysis model via a secure communication channel, ensuring patient privacy and information security, and complying with medical data security standards.

[0067] Optionally, in some embodiments, the diagnostic module 300 is further configured to: after generating a diagnostic result, combine the TCM syndrome differentiation and treatment system with modern medical clinical standards, and comprehensively analyze the causes of the patient's symptoms based on multimodal fusion data to obtain a personalized rehabilitation plan combining TCM and Western medicine.

[0068] Understandably, after the doctor reviews and confirms the preliminary diagnosis, the system extracts core symptoms and diagnostic tags through structured analysis results and combines them with the theory of Zang-Xiang in Traditional Chinese Medicine, the distribution patterns of meridians, and the modern disease classification system to recommend Chinese medicine prescriptions, meridian intervention programs (such as acupuncture and moxibustion), and corresponding modern treatments or rehabilitation methods (such as drug therapy, physical rehabilitation training, and lifestyle interventions). This results in a personalized rehabilitation plan that combines Chinese and Western medicine, and automatically generates a detailed report containing diagnostic conclusions, examination suggestions, treatment plans, and follow-up plans. It supports multiple output formats (such as PDF, electronic medical record interface, etc.). This embodiment of the application not only provides feedback on possible disease diagnoses but also provides targeted treatment suggestions based on individual patient differences and large sample data to assist doctors in making final decisions.

[0069] To facilitate those skilled in the art to further understand the thermal imaging-based assisted diagnosis and treatment system of this application, the following is combined with... Figures 2 to 4 The embodiments shown will be described in detail.

[0070] Specifically, such as Figure 2 As shown, Figure 2This diagram illustrates the overall system architecture of an auxiliary diagnostic and treatment system based on thermal imaging equipment, provided in one embodiment of this application. It mainly includes an edge data acquisition layer, a cloud processing layer, and an application output layer. The diagram shows the complete process from thermal imaging data acquisition, multimodal input, cloud-based comprehensive diagnosis to final report generation, as well as the data interaction relationships between modules. The deep learning model in the diagram is a preset cloud diagnostic model for this embodiment. In the edge data acquisition layer, this embodiment simultaneously acquires various diagnostic and treatment data through a multimodal input device; the thermal imaging device acquires body surface temperature distribution data, the voice input module records the patient's chief complaint, and the traditional Chinese medicine acquisition device also acquires TCM diagnostic data including pulse and tongue appearance. After preliminary processing, these raw data form structured abnormal region feature codes and preprocessed sample data, laying the foundation for subsequent analysis. The cloud processing layer is the intelligent core of the entire system. The knowledge graph module integrates TCM and Western medicine diagnostic rules and clinical databases; the multimodal fusion engine performs correlation analysis on different types of data using a deep learning large model; the diagnostic decision module generates preliminary diagnostic conclusions based on deep learning algorithms; and the secure storage cluster ensures the security and privacy protection of medical data through a distributed encrypted storage module. The application output layer provides intelligent diagnosis and treatment services; the intelligent report generator automatically generates dynamic reports containing diagnostic results, treatment suggestions, and follow-up plans; doctors can correct the diagnostic results, and this correction information will be used to continuously optimize the system algorithm, forming a closed-loop learning mechanism. This application embodiment provides an intuitive and easy-to-use user interface, supporting touch, voice, and text input, enabling general practitioners to quickly master the operation process; it displays real-time thermal imaging images and preliminary abnormality prompts, allowing doctors to instantly view and adjust the acquisition angle to ensure data acquisition quality; it preprocesses and formats image data and text information for subsequent unified diagnostic analysis in the cloud. Therefore, this application embodiment uses advanced image processing and artificial intelligence technologies to achieve rapid assisted screening of patients' pain sites, and combines traditional Chinese medicine tongue images, pulse patterns, and consultation information to optimize diagnostic results and reduce the risk of misdiagnosis.

[0071] Furthermore, such as Figure 3 As shown, Figure 3This illustration shows an implementation diagram of an auxiliary diagnostic system based on thermal imaging equipment, provided in one embodiment of this application. The thermal imaging acquisition module acquires images of the patient's body surface temperature distribution using a thermal imager, a commercially available and mature product. The imager has a built-in dedicated embedded processor running a pre-trained neural network. The device is designed for portability and ease of use, making it particularly suitable for mobile medical scenarios in remote areas. The image processing algorithm uses a deep learning model suitable for temperature anomaly detection to segment, extract features, and locate abnormal regions in the thermal image. Natural language processing performs semantic analysis on the speech-to-text data, extracting keywords to assist in image diagnosis. A pre-set cloud-based diagnostic model integrates multi-layer deep learning and expert rules to achieve data cross-validation and comprehensive judgment, improving diagnostic accuracy. After review by the doctor, the diagnostic results are labeled and used as data for retraining the pre-set cloud-based diagnostic model, forming a closed-loop learning mechanism.

[0072] Furthermore, such as Figure 4 As shown, Figure 4 This application provides an operational flowchart of an auxiliary diagnostic and treatment system based on thermal imaging equipment, according to one embodiment of the present application. After the doctor activates the equipment, a thermal imaging image of the target area is first acquired. Simultaneously, the patient describes their symptoms via voice or text. The data is preprocessed and then fused with the image data. The data is uploaded to a cloud server through a secure channel. The server performs comprehensive analysis of the data based on a large model, providing preliminary diagnostic results and suggestions for supplementary examinations. The doctor confirms or provides supplementary explanations based on the feedback results. In the final embodiment of this application, a diagnostic report and treatment recommendations are automatically generated, and the data is archived in the electronic medical record system.

[0073] Therefore, this application embodiment, through the deep integration of low-cost thermal imaging equipment and artificial intelligence, can quickly and accurately assist in the diagnosis of pain-related diseases in remote and primary healthcare institutions; the application of multimodal data fusion and cloud-based large models greatly improves the accuracy and objectivity of diagnosis and reduces the misdiagnosis rate of general practitioners; the automatic report generation and feedback mechanism realizes closed-loop data management, providing a guarantee for continuous optimization of the diagnostic model; comprehensive data security measures ensure patient privacy and information security, meeting the requirements of modern medical information management; this application embodiment provides a reasonably structured, easy-to-operate, efficient and reliable auxiliary diagnostic system, effectively solving the problems of insufficient medical resources and inaccurate diagnosis in remote areas, and has significant social and economic benefits.

[0074] According to the thermal imaging-based assisted diagnosis and treatment system proposed in this application, the acquisition module is used to acquire images of the patient's body surface temperature distribution and patient symptom information; the processing module is used to preprocess the voice information, pulse information, consultation information, and patient body surface temperature distribution image to obtain fused patient symptom information and structured temperature distribution image, encrypt them, and upload them to the diagnosis module; the diagnosis module is used to encode the fused patient symptom information and structured temperature distribution image based on a preset cloud diagnosis model to obtain encoded fused data, and generate preliminary diagnostic results and / or suggested supplementary examination items based on this; the output module is used to obtain confirmation information and / or supplementary information generated based on the preliminary diagnostic results and / or suggested supplementary examination items, and generate a diagnostic report based on this information. This solves the problems of insufficient doctor experience and high misdiagnosis rates in remote areas, optimizes diagnostic results, and reduces the risk of misdiagnosis.

[0075] Next, referring to the accompanying drawings, the auxiliary diagnosis and treatment method based on thermal imaging equipment proposed in the embodiments of this application is described, employing... Figure 1 The embodiment shows an auxiliary diagnosis and treatment system based on thermal imaging equipment.

[0076] Figure 5 This is a flowchart of an assisted diagnosis and treatment method based on thermal imaging equipment according to an embodiment of this application.

[0077] like Figure 5 As shown, the thermal imaging-based assisted diagnosis and treatment method includes the following steps:

[0078] In step S501, the patient's body surface temperature distribution image and patient symptom information are acquired, including voice information, pulse information and consultation information.

[0079] In step S502, the voice information, pulse information and consultation information are preprocessed to obtain fused patient symptom information, the patient's body surface temperature distribution image is preprocessed to obtain a structured temperature distribution image, and the fused patient symptom information and structured temperature distribution image are encrypted and uploaded to a preset cloud diagnostic model.

[0080] In step S503, the fused patient symptom information and structured temperature distribution image are encoded based on a preset cloud diagnostic model to obtain encoded fused data. Based on the encoded fused data, a preliminary diagnostic result and / or suggestions for supplementary examinations are generated as a draft treatment recommendation.

[0081] In step S504, confirmation information and / or supplementary information generated based on the preliminary diagnosis results and / or suggested supplementary examination items are obtained, and a final diagnosis report is generated based on the above diagnosis results and information after the doctor's review and confirmation.

[0082] Optionally, in some embodiments, acquiring images of the patient's body surface temperature distribution and patient symptom information includes: acquiring images of the patient's body surface temperature distribution; acquiring voice information, pulse information, and consultation information.

[0083] Optionally, in some embodiments, obtaining confirmation information and / or supplementary information generated based on preliminary diagnostic results and / or suggested supplementary examination items, and generating a diagnostic report based on the preliminary diagnostic results and / or suggested supplementary examination items and / or supplementary information, includes: based on patient symptom information, converting speech information, pulse information, and consultation information into text information through natural language processing, performing semantic parsing, extracting patient symptom features for structured representation, and obtaining fused patient symptom information based on the semantic parsing results; based on the patient's body surface temperature distribution image, performing preliminary feature extraction through an edge neural network, and performing abnormal region detection on the data of the preliminary feature extraction based on a convolutional neural network, automatically labeling abnormal regions with temperature deviation features in the patient's body surface temperature distribution image, forming a structured temperature distribution image and its location semantic labels.

[0084] Optionally, in some embodiments, the fused patient symptom information and structured temperature distribution image are encoded based on a preset cloud-based diagnostic model to obtain encoded fused data. A preliminary diagnostic result and / or suggested supplementary examination items are generated based on the encoded fused data as a draft diagnosis and treatment recommendation. The method further includes: continuously optimizing and updating the preset cloud-based diagnostic model based on the preliminary diagnostic result and / or the examination results of the suggested supplementary examination items. The optimization includes calibrating the model inference path based on the doctor's confirmation result, fine-tuning the weights based on newly collected sample data, and improving the generalization ability of the preset cloud-based diagnostic model for rare symptom combinations through a meta-learning mechanism. Furthermore, the optimization module dynamically adjusts the parameter configuration of the diagnostic model based on the clinical characteristics, disease spectrum differences, and lifestyle habits of different regions and populations.

[0085] Optionally, in some embodiments, encrypting the fused patient symptom information and structured temperature distribution image and uploading it to a preset cloud diagnostic model includes: locally encrypting the fused patient symptom information and structured temperature distribution image based on a preset data security strategy, wherein the encryption strategy includes using symmetric encryption and heterogeneous feature masking methods for multimodal data respectively, and uploading the fused patient symptom information and structured temperature distribution image to the preset cloud diagnostic model after encryption.

[0086] Optionally, in some embodiments, obtaining confirmation information and / or supplementary information generated based on preliminary diagnostic results and / or suggested supplementary examination items, and generating a final diagnostic report based on the above diagnostic results and information after review and confirmation by a doctor, further includes: after generating diagnostic results, combining the TCM syndrome differentiation and treatment system with modern medical clinical standards, comprehensively judging the causes of the patient's symptoms based on multimodal fusion data, and obtaining a personalized rehabilitation plan combining TCM and Western medicine.

[0087] It should be noted that the foregoing explanation of the embodiment of the thermal imaging-based auxiliary diagnosis and treatment system also applies to the thermal imaging-based auxiliary diagnosis and treatment method of this embodiment, and will not be repeated here.

[0088] According to the thermal imaging-based assisted diagnosis and treatment method proposed in this application, the following steps are taken: First, a patient's body surface temperature distribution image and patient symptom information, including voice information, pulse information, and consultation information, are acquired. The voice information, pulse information, and consultation information are preprocessed to obtain fused patient symptom information. The patient's body surface temperature distribution image is preprocessed to obtain a structured temperature distribution image. The fused patient symptom information and structured temperature distribution image are encrypted and uploaded to a preset cloud-based diagnostic model. Based on the preset cloud-based diagnostic model, the fused patient symptom information and structured temperature distribution image are encoded to obtain encoded fused data. A preliminary diagnostic result and / or suggested supplementary examination items are generated based on the encoded fused data. Confirmation information and / or supplementary information generated based on the preliminary diagnostic result and / or suggested supplementary examination items are obtained. A diagnostic report is generated based on the preliminary diagnostic result and / or suggested supplementary examination items and / or supplementary information. This method solves the problems of insufficient doctor experience and high misdiagnosis rates in remote areas, optimizes diagnostic results, and reduces the risk of misdiagnosis.

[0089] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0090] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0091] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0092] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0093] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. An auxiliary diagnostic and treatment system based on thermal imaging equipment, characterized in that, It includes a data acquisition module, a processing module, a diagnostic module, and an output module. The diagnostic module is deployed in the cloud. The acquisition module is used to acquire images of the patient's body surface temperature distribution based on an infrared thermal imaging device, and to acquire patient symptom information based on a voice acquisition terminal and a physiological signal interface. The patient symptom information includes voice information, pulse information, and consultation information. The image acquisition is based on an infrared thermal imaging device, and the symptom information acquisition includes a voice acquisition terminal and a physiological signal interface. The processing module is used to preprocess the patient's body surface temperature distribution image on the terminal side using an integrated lightweight AI model, and to structure the temperature information of different parts into body surface temperature feature data according to the first processing result; at the same time, it performs semantic recognition and structured processing on the voice information, pulse information and consultation information, and fuses the second processing result with the body surface temperature feature data to obtain fused structured multimodal feature data, and encrypts the fused structured multimodal feature data before uploading it to the diagnostic module; The diagnostic module is used to receive the fused structured multimodal feature data, and to perform reasoning analysis on the fused structured multimodal feature data based on the medical big language model and the fused knowledge graph to obtain encoded fused data, and to generate preliminary diagnostic results and / or suggestions for supplementary examination items based on the encoded fused data. The output module is used to acquire confirmation information and / or supplementary information generated based on the preliminary diagnosis results and / or the suggested supplementary examination items, and to generate a diagnostic report based on the doctor's review and confirmation results, according to the preliminary diagnosis results and / or the suggested supplementary examination items and / or supplementary information.

2. The system according to claim 1, characterized in that, The acquisition module includes: A thermal imaging acquisition unit is used to acquire images of the patient's body surface temperature distribution. A multimodal information acquisition unit is used to acquire the voice information, the pulse information, and the consultation information.

3. The system according to claim 1, characterized in that, The processing module is specifically used for: The first processing unit is used to convert the voice information, pulse information and consultation information into text information through natural language processing based on the patient symptom information, perform semantic parsing, extract patient symptom features for structured representation, and obtain fused patient symptom information based on the semantic parsing results. The second processing unit is used to perform preliminary feature extraction based on the patient's body surface temperature distribution image through an end-side neural network, and to identify, classify and locate abnormal areas in the data extracted by the preliminary feature extraction based on a convolutional neural network. It also uses a site recognition model to label the temperature values ​​of the corresponding anatomical regions and generate a structured temperature distribution image as an image feature representation.

4. The system according to claim 1, characterized in that, Also includes: The optimization module is used to continuously optimize and update the preset cloud-based diagnostic model based on the preliminary diagnostic results and / or the results of the suggested supplementary examinations. The optimization includes calibrating the model inference path based on the doctor's confirmation results, fine-tuning the weights based on newly collected sample data, and improving the generalization ability of the preset cloud-based diagnostic model for rare symptom combinations through a meta-learning mechanism. Furthermore, the module dynamically adjusts the parameter configuration of the diagnostic model based on the clinical characteristics, disease spectrum differences, and lifestyle habits of different regions and populations.

5. The system according to claim 1, characterized in that, The processing module is specifically used for: Based on a preset data security strategy, the fused patient symptom information and structured temperature distribution image are locally encrypted. The encryption strategy includes using symmetric encryption and heterogeneous feature masking methods for multimodal data respectively. The fused patient symptom information and structured temperature distribution image are then encrypted and uploaded to the preset cloud diagnostic model.

6. The system according to claim 1, characterized in that, The diagnostic module is also used for: After generating the diagnostic results, the causes of the patient's symptoms are comprehensively assessed based on multimodal fusion data, combining the TCM syndrome differentiation and treatment system with modern medical clinical standards, to obtain a personalized rehabilitation plan that integrates TCM and Western medicine.

7. A diagnostic and treatment method based on thermal imaging equipment, characterized in that, The auxiliary diagnostic and treatment system based on thermal imaging equipment as described in any one of claims 1-6, wherein the method includes the following steps: Collect images of the patient's body surface temperature distribution and patient symptom information, wherein the patient symptom information includes voice information, pulse information and consultation information; The voice information, pulse information, and consultation information are preprocessed to obtain fused patient symptom information. The patient's body surface temperature distribution image is preprocessed to obtain a structured temperature distribution image. The fused patient symptom information and the structured temperature distribution image are encrypted and uploaded to a preset cloud diagnostic model. Based on the preset cloud-based diagnostic model, the fused patient symptom information and the structured temperature distribution image are encoded to obtain encoded fused data. Based on the encoded fused data, a preliminary diagnostic result and / or suggestions for supplementary examinations are generated as a draft treatment recommendation. Obtain confirmation and / or supplementary information generated based on the preliminary diagnosis results and / or the suggested supplementary examination items, and generate a final diagnosis report based on the above diagnosis results and information after the doctor's review and confirmation.

8. The method according to claim 7, characterized in that, The acquisition of images of the patient's body surface temperature distribution and patient symptom information includes: Acquire images of the patient's body surface temperature distribution; Collect the voice information, pulse information, and consultation information.

9. The method according to claim 7, characterized in that, The step of obtaining confirmation information and / or supplementary information generated based on the preliminary diagnostic results and / or the recommended supplementary examination items, and generating a diagnostic report based on the preliminary diagnostic results and / or the recommended supplementary examination items and / or supplementary information, includes: Based on the patient's symptom information, the speech information, pulse information, and consultation information are converted into text information through natural language processing, and then semantic parsing is performed to extract the patient's symptom features for structured representation. The fused patient symptom information is obtained based on the semantic parsing results. Based on the patient's body surface temperature distribution image, preliminary feature extraction is performed using an edge-side neural network, and abnormal region detection is performed on the data extracted by the preliminary feature extraction using a convolutional neural network. Abnormal regions with temperature deviation characteristics in the patient's body surface temperature distribution image are automatically labeled, forming a structured temperature distribution image and its location semantic labels.

10. The method according to claim 7, characterized in that, The process of encoding the fused patient symptom information and the structured temperature distribution image based on the preset cloud diagnostic model to obtain encoded fused data, and generating preliminary diagnostic results and / or suggestions for supplementary examinations based on the encoded fused data, further includes: Based on the preliminary diagnostic results and / or the results of the suggested supplementary examinations, update the preset cloud diagnostic model, and / or adjust the parameters of the preset cloud diagnostic model according to the clinical characteristics of different regions and populations.

Images