Traditional Chinese medicine acupoint auxiliary diagnosis and treatment system integrating deep learning and applet

Abstract

The invention discloses a deep learning and applet integrated traditional Chinese medicine acupoint auxiliary diagnosis and treatment system, which comprises a mobile terminal and a cloud server, the mobile terminal is in communication connection with the cloud server, the mobile terminal is provided with an applet, and the cloud server is configured with a target detection model and a key point prediction model for classified storage of parts. The cloud server is connected with a cloud database, the corresponding relation between diseases and acupuncture points and treatment scheme information are stored in the cloud database, accurate positioning of the acupuncture points of different parts of the body is achieved through target detection and a key point prediction model, and a simple and visual user interface of an applet is combined; according to the system, the user can quickly know the associated information of the disease and the acupoint, check the prediction result and obtain the treatment suggestion, the scientificity and the accuracy of acupoint positioning are improved, and meanwhile, the dependence of traditional Chinese medicine treatment on professional skills is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of TCM acupoint positioning, and in particular to a TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs. Background Art

[0002] With the rapid development of artificial intelligence technology and the continuous innovation and integration of TCM concepts, traditional TCM is gradually moving towards intelligence in disease prevention, health management and precision treatment. In TCM practice, accurate positioning of human acupoints is the key to the successful implementation of acupuncture, massage and other therapies. However, traditional acupoint positioning methods mainly rely on the experience of physicians. This method often leads to insufficient accuracy and consistency in acupoint positioning due to differences in patient body shape, anatomical characteristics and physician level, thus affecting treatment effects and efficacy evaluation.

[0003] In recent years, advances in computer vision technology have provided new possibilities for the precise positioning of acupoints. By training large-scale human image datasets with deep learning algorithms, acupoints in specific parts can be automatically identified and accurately located, thus providing a scientific basis for traditional Chinese medicine treatment. However, the performance of existing deep learning models in complex application scenarios still faces many challenges. For example, in situations with complex lighting conditions, severe background interference, and large differences in the back morphology of patients, the robustness and adaptability of the model are still insufficient, affecting the stability and accuracy of acupoint positioning. In addition, current solutions are still insufficient in user experience and convenience, making it difficult to widely promote them among the general population.

[0004] The patent document with publication number CN114373190B discloses an intelligent recognition and automatic positioning system for human acupuncture points; when the system is in use, it is not possible for the user to intuitively understand the correlation information between diseases and acupuncture points, and it is not possible to be well applied to auxiliary diagnosis and treatment scenarios.

[0005] The patent document with publication number CN112184705B discloses a human acupoint recognition, positioning and application system based on computer vision technology. It helps users find acupoints easily by building an acupoint and disease information database, an acupoint recognition and positioning algorithm module, and an interactive processing program. However, when the system is in use, it is easily affected by lighting conditions, background and posture, resulting in inaccurate acupoint positioning. Summary of the invention

[0006] The purpose of the present invention is to solve the problems in the prior art and propose a TCM acupoint auxiliary diagnosis and treatment system that integrates deep learning and mini-programs. It combines target detection and key point prediction models to achieve automatic recognition and precise positioning of acupoints throughout the body. At the same time, it provides detailed acupoint information query and common disease treatment plan support through mini-program interaction, and is widely used in TCM acupuncture, massage and other auxiliary diagnosis and treatment scenarios.

[0007] To achieve the above-mentioned purpose, the present invention proposes a TCM acupoint auxiliary diagnosis and treatment system that integrates deep learning and mini-programs, including a mobile terminal and a cloud server. The mobile terminal is communicatively connected to the cloud server, and the mini-program is installed on the mobile terminal. The cloud server is configured with a target detection model and a key point prediction model for part classification storage. The cloud server is connected to a cloud database, and the cloud database stores the correspondence between diseases and acupoints and treatment plan information.

[0008] Preferably, the method for using the TCM acupoint auxiliary diagnosis and treatment system comprises the following steps: S1. Typical disease information query; S2, whitelist access; S3. User selects the predicted part: The user selects the body part to be predicted through the mini program and uploads the image of the corresponding part; S4. The server dynamically loads the target detection model and key point prediction model stored by part classification according to the user's selection, and completes the target area detection, key point prediction and result calibration; S5. Prediction result calibration: Calibrate the positions of the predicted key points; S6. Result return and display: After the calibration is completed, the calibrated acupoint annotation image is generated and the result is returned to the mini program interface through the API. The mini program overlays the calibrated acupoint points and related reference line annotations on the image uploaded by the user to intuitively display the automated prediction results. S7, Acupoint details view: The user clicks on any acupuncture point in the prediction result image to view its detailed information.

[0009] Preferably, in S1, after the user enters the mini program, he can browse the disease list in the common disease interface, and click on a specific disease to enter the details page; the details page displays the associated acupoints, acupuncture, massage efficacy descriptions and recommended treatment plans for the disease; these data are stored in a structured manner in the cloud development database, including acupoint distribution and efficacy information recorded in classical Chinese medicine literature, and combined with supplementary descriptions of efficacy based on modern medical research.

[0010] Preferably, in S2, a cloud development database is used to implement a whitelist access mechanism. When a user logs in to the mini program, the system will obtain his / her WeChat OpenID and verify in the cloud database whether he / she is in the whitelist. Only after the verification is passed can the user enter the mini program function interface, otherwise a prompt will be displayed indicating that the user does not have access rights. The administrator dynamically manages whitelist users through cloud development tools, including adding or removing permissions.

[0011] Preferably, the S4 specifically comprises the following steps: (1) Construction and annotation of data sets Diversified data collection: Collect images of different body parts under various lighting conditions, different backgrounds, and different postures; (2) Standardized annotation: After data collection, under the guidance of professional medical institutions, images are annotated using professional annotation tools, including target area contour annotation and acupoint key point annotation, and the location and corresponding relationship of key acupoints are determined with reference to traditional Chinese medicine standards; (3) Data division: The data set is divided into training set, validation set and test set; (4) Model training: We adopt a "top-down" approach to model construction and training. First, we use the target detection model to locate the various regions in the whole body image, and then apply the key point detection model to accurately annotate the acupoints based on the positioning results. During the training process, we use the constructed large-scale annotated dataset and combine it with a variety of data enhancement techniques (such as flipping, rotation, scaling, etc.) to improve the generalization ability of the model. After multiple iterations of optimization, the model gradually converges, and pre-trained models for target detection and key point detection are generated for different parts to ensure the adaptability and stability of the model in complex scenarios. (5) Target detection: Use the target detection model to detect the target area in the uploaded image and identify the contour of the body part selected by the user; (6) Key point prediction: Use the key point detection model to predict the key points of acupuncture points in the target area and generate the initial coordinates of the key points.

[0012] Preferably, the S5 comprises: Baseline fitting calibration: Generate a reference line through linear fitting of the acupuncture points on the baseline, and adjust the predicted points to the reference line position; Bone measurement calibration: determine the relative length of "one inch", adjust the relative distance of the predicted points according to the standard acupoint distribution, and allow the error range to be ±0.35 inch; Regional symmetry calibration: adjust the positions of acupuncture points that are symmetrically distributed on the left and right sides to ensure their symmetry and consistency.

[0013] Preferably, in S7, the information of each acupoint is pre-stored in the cloud development database, including the following contents: Acupoint name: standardized TCM acupoint naming; Acupuncture and massage effects: Detailed description of the mechanism of action and indications of acupoints; Locations recorded in ancient books: Refer to the theoretical description and positioning basis of acupoints in classical Chinese medical literature; Modern medical background: Combined with the research results of modern medicine, explain the auxiliary therapeutic effect of this acupoint in relieving specific diseases or symptoms; After the user clicks on an acupoint, the mini program queries the corresponding information in the database through the back-end interface, and displays the query results in a pop-up window or an independent details page; the system supports dynamic updating of acupoint information in the database to ensure consistency with the latest research results in traditional Chinese medicine.

[0014] Beneficial effects of the invention: The invention realizes accurate positioning of acupuncture points in different parts of the body through target detection and key point prediction models, and combined with the simple and intuitive user interface of the applet, enables users to quickly understand the association information between diseases and acupuncture points, view prediction results and obtain treatment suggestions; The system not only improves the scientificity and accuracy of acupoint positioning, but also reduces the dependence of TCM treatment on professional skills, promotes the modernization and development of TCM treatment, and provides a new intelligent direction for TCM health management and auxiliary treatment. Based on the images uploaded by users, the mini program overlays the calibrated acupoints and related reference line annotations to intuitively display the automatic prediction results. This function achieves high-precision acupoint positioning through dynamic model calling and calibration optimization, and significantly improves the practicality and accuracy of acupoint identification throughout the body. The present invention designs an acupoint details viewing function in the WeChat applet. Users can click on any acupoint in the prediction result image to view its detailed information. This function not only provides users with a convenient learning approach, but also improves the scientificity and practicality of TCM acupoint therapy on the basis of combining TCM theory with modern technology, while enriching the interactivity and user experience of the applet.

[0015] The features and advantages of the present invention will be described in detail through embodiments in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram of the applet interface of the present invention; Figure 2 It is a finishing flow chart of the present invention; Figure 3 is a detailed flow chart of the present invention; Figure 4 is the model architecture of the target detection model of the present invention; Figure 5 It is the model architecture of the key point detection model of the present invention. DETAILED DESCRIPTION

[0017] See also Figures 1 to 5A TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs includes a mobile terminal and a cloud server. The mobile terminal is communicatively connected to the cloud server, and the mini-program is installed on the mobile terminal. The cloud server is configured with a target detection model and a key point prediction model for part classification storage. The cloud server is connected to a cloud database, and the cloud database stores the correspondence between diseases and acupoints and treatment plan information.

[0018] The method for using the TCM acupoint auxiliary diagnosis and treatment system comprises the following steps: S1. Typical disease information query: The present invention designs a common disease query function in the WeChat applet, and stores the correspondence between diseases and acupoints and treatment plan information through the cloud development database; after entering the applet, the user can browse the disease list on the common disease interface, such as diabetes, insomnia, cervical spondylosis, etc., and click on a specific disease to enter the detail page; the detail page displays the associated acupoints of the disease, acupuncture and massage efficacy descriptions, and recommended treatment plans; these data are stored in a structured manner in the cloud development database, including acupoint distribution and efficacy information recorded in classical Chinese medicine literature, and combined with a supplementary description of efficacy based on modern medical research; S2. Whitelist access: The cloud development database is used to implement a whitelist access mechanism to ensure the security and privacy of user data. When a user logs in to the mini program, the system will obtain their WeChat OpenID and verify whether it is in the whitelist in the cloud database. Only after the verification is passed can they enter the mini program function interface, otherwise they will be prompted that they do not have access rights. Administrators can dynamically manage whitelist users through cloud development tools, including adding or removing permissions; This mechanism ensures the privacy of user data while limiting unauthorized access, and is suitable for medical mini-program scenarios that require privacy protection; S3. User selects the predicted part: The user selects the body part to be predicted (such as the head, back or limbs) through the mini program and uploads the image of the corresponding part; S4. The server dynamically loads the target detection model and key point prediction model stored by part classification according to the user's selection, and completes the target area detection, key point prediction and result calibration; The specific steps include: (1) Construction and annotation of the dataset Diversified data collection: High-quality images are collected for different body parts (such as head, back, limbs) under various lighting conditions (strong light, weak light), different backgrounds (simple, complex), and different postures (front, side); the design of the dataset focuses on diversity to enhance the model's generalization ability in complex environments; (2) Standardized annotation: After data collection, under the guidance of professional medical institutions, images are annotated using professional annotation tools (such as LabelMe), including: target area contour annotation (such as head boundary, back area, etc.); acupoint key point annotation, referring to traditional Chinese medicine standards to determine the location and corresponding relationship of key acupoints; (3) Data partitioning: The dataset is divided into training set, validation set, and test set to ensure a comprehensive evaluation of the model performance during the training process; (4) Model training: We adopt a "top-down" approach to model construction and training. First, we use the target detection model to locate the various regions in the whole body image, and then apply the key point detection model to accurately annotate the acupoints based on the positioning results. During the training process, we use the constructed large-scale annotated dataset and combine it with a variety of data enhancement techniques (such as flipping, rotation, scaling, etc.) to improve the generalization ability of the model. After multiple iterations of optimization, the model gradually converges, and pre-trained models for target detection and key point detection are generated for different parts to ensure the adaptability and stability of the model in complex scenarios. (5) Target detection: Use the target detection model to detect the target area in the uploaded image and identify the contour of the body part selected by the user; (6) Key point prediction: Use the key point detection model to predict the key points of acupuncture points in the target area and generate the initial coordinates of the key points; S5. Prediction result calibration: Calibrate the predicted key points; including: Baseline fitting calibration: Generate a reference line through linear fitting for the acupuncture points on the baseline (such as the acupuncture points on the posterior midline of the back), and adjust the predicted points to the reference line position; Bone measurement calibration: determine the relative length of "one inch", adjust the relative distance of the predicted points according to the standard acupoint distribution, and allow the error range to be ±0.35 inch; Regional symmetry calibration: adjust the positions of acupuncture points that are symmetrically distributed on the left and right sides to ensure their symmetry and consistency; S6. Result return and display: After the calibration is completed, the calibrated acupoint annotation image is generated and the result is returned to the mini program interface through the API. The mini program overlays the calibrated acupoint points and related reference line annotations on the image uploaded by the user to intuitively display the automated prediction results. S7, Acupoint details view: The present invention designs an acupoint detail viewing function in the WeChat applet. The user can click on any acupoint in the prediction result image to view its detailed information; the information of each acupoint is pre-stored in the cloud development database, including the following contents: Acupoint name: standardized TCM acupoint names, such as "Zusanli" and "Hegu", etc. Acupuncture and massage effects: Detailed description of the mechanism of action and indications of acupuncture points, such as pain relief, improved digestive function, etc.; Locations recorded in ancient books: Refer to the theoretical description and positioning basis of acupoints in classical Chinese medical literature; Modern medical background: Combined with the research results of modern medicine, explain the auxiliary therapeutic effect of this acupoint in relieving specific diseases or symptoms; After the user clicks on an acupoint, the mini program queries the corresponding information in the database through the back-end interface, and displays the query results in a pop-up window or an independent details page, allowing the user to intuitively understand the background and efficacy of the acupoint; in addition, the system supports dynamic updating of acupoint information in the database to ensure consistency with the latest research results in traditional Chinese medicine.

[0019] The above embodiments are intended to illustrate the present invention, not to limit the present invention. Any solution that is a simple transformation of the present invention belongs to the protection scope of the present invention.

Claims

1. A TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs, characterized by: It includes a mobile terminal and a cloud server. The mobile terminal is connected to the cloud server for communication. A small program is installed in the mobile terminal. The cloud server is configured with a target detection model and a key point prediction model for body classification storage. The cloud server is connected to a cloud database. The cloud database stores the correspondence between diseases and acupoints and treatment plan information.

2. The TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs as claimed in claim 1 is characterized by: The method for using the TCM acupoint auxiliary diagnosis and treatment system comprises the following steps: S1. Typical disease information query; S2, whitelist access; S3. User selects the predicted part: The user selects the body part to be predicted through the mini program and uploads the image of the corresponding part; S4. The server dynamically loads the target detection model and key point prediction model stored by part classification according to the user's selection, and completes the target area detection, key point prediction and result calibration; S5. Prediction result calibration: Calibrate the positions of the predicted key points; S6. Result return and display: After the calibration is completed, the calibrated acupoint annotation image is generated and the result is returned to the mini program interface through the API. The mini program overlays the calibrated acupoint points and related reference line annotations on the image uploaded by the user to intuitively display the automated prediction results. S7, Acupoint details view: The user clicks on any acupuncture point in the prediction result image to view its detailed information.

3. The TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs as claimed in claim 2 is characterized by: In S1, after the user enters the mini program, he can browse the disease list in the common disease interface, and click on a specific disease to enter the details page; the details page displays the associated acupoints, acupuncture, massage efficacy descriptions and recommended treatment plans for the disease; these data are stored in a structured manner in the cloud development database, including acupoint distribution and efficacy information recorded in classical Chinese medicine literature, and combined with supplementary descriptions of efficacy based on modern medical research.

4. The TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs as claimed in claim 2 is characterized by: In S2, a cloud development database is used to implement a whitelist access mechanism. When a user logs in to a mini program, the system obtains their WeChat OpenID and verifies in the cloud database whether they are in the whitelist. Only after the verification is passed can they enter the mini program function interface. Otherwise, they will be prompted that they do not have access rights. Administrators can dynamically manage whitelist users through cloud development tools, including adding or removing permissions.

5. The TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs as claimed in claim 2 is characterized by: The S4 specifically comprises the following steps: (1) Construction and annotation of data sets Diversified data collection: Collect images of different body parts under various lighting conditions, different backgrounds, and different postures; (2) Standardized annotation: After data collection, under the guidance of professional medical institutions, images are annotated using professional annotation tools, including target area contour annotation and acupoint key point annotation, and the location and corresponding relationship of key acupoints are determined with reference to traditional Chinese medicine standards; (3) Data division: The data set is divided into training set, validation set and test set; (4) Model training: We adopt a "top-down" approach to model construction and training. First, we use the target detection model to locate the various regions in the whole body image, and then apply the key point detection model to accurately annotate the acupoints based on the positioning results. During the training process, we use the constructed large-scale annotated dataset and combine it with a variety of data enhancement techniques to improve the generalization ability of the model. After multiple iterations of optimization, the model gradually converges, and pre-trained models for target detection and key point detection are generated for different parts. (5) Target detection: Use the target detection model to detect the target area in the uploaded image and identify the contour of the body part selected by the user; (6) Key point prediction: Use the key point detection model to predict the key points of acupuncture points in the target area and generate the initial coordinates of the key points.

6. The TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs as claimed in claim 2, characterized in that: The S5 includes: Baseline fitting calibration: Generate a reference line through linear fitting of the acupuncture points on the baseline, and adjust the predicted points to the reference line position; Bone measurement calibration: determine the relative length of "one inch", adjust the relative distance of the predicted points according to the standard acupoint distribution, and allow the error range to be ±0.35 inch; Regional symmetry calibration: adjust the positions of acupuncture points that are symmetrically distributed on the left and right sides to ensure their symmetry and consistency.

7. The TCM acupoint auxiliary diagnosis and treatment system integrating deep learning and mini-programs as claimed in claim 2 is characterized by: In S7, the information of each acupuncture point is pre-stored in the cloud development database, including the following contents: Acupoint name: standardized TCM acupoint naming; Acupuncture and massage effects: Detailed description of the mechanism of action and indications of acupoints; Locations recorded in ancient books: Refer to the theoretical description and positioning basis of acupoints in classical Chinese medical literature; Modern medical background: Combined with the research results of modern medicine, explain the auxiliary therapeutic effect of this acupoint in relieving specific diseases or symptoms; After the user clicks on an acupoint, the mini program queries the corresponding information in the database through the back-end interface, and displays the query results in a pop-up window or an independent details page; the system supports dynamic updating of acupoint information in the database to ensure consistency with the latest research results in traditional Chinese medicine.

Images