Acupuncture point recognition system based on multi-mode digital twinning technology

Abstract

The invention discloses an acupoint recognition system based on a multi-modal digital twinning technology, which comprises a multi-modal data acquisition module, a multi-modal data transmission module, a multi-modal data processing module, a digital twinning construction module, an acupoint state and positioning prediction network, a result presentation module and a multi-modal data acquisition module integrated high-performance sensor group, a high-precision clock synchronization system is connected, and acupoint and human body data are acquired in all directions by utilizing a space calibration frame; the data transmission module integrates multiple transmission modes and supports flexible switching and cooperation; the data processing module runs in a special customized RISC-V architecture processor and performs preprocessing, conversion, feature extraction, fusion and dimension reduction on the data; the digital twinning construction module constructs an acupoint physiological and pathological state digital twinning model according to the processed data; the acupuncture point state and positioning prediction network module operates an algorithm in the NPU processor, and accurately predicts the state and position of the acupuncture point according to the human body data; and the result display module displays related information on a human-computer interaction interface.

Description

Technical Field

[0001] The present invention relates to the technical field of acupoint recognition systems, and specifically provides an acupoint recognition system based on multimodal digital twin technology. Background Art

[0002] Digital twin is a method that uses digital technology to create a virtual model of a physical entity, system, or process. This virtual model is highly synchronized and mutually mapped with the real object throughout its entire life cycle. As an emerging digital tool, digital twin technology can construct high-precision virtual models and synchronize the state changes of physical objects in real time. In the medical field, digital twin technology has been used in disease diagnosis, surgical planning, and personalized treatment, demonstrating great application potential. By combining multimodal data fusion and artificial intelligence algorithms to dynamically simulate and real-time track the true state of acupoints, digital twin technology is expected to provide a more accurate, dynamic, and personalized solution for acupoint recognition, break the shackles of traditional recognition methods, create a new situation of accurate and intelligent acupoint recognition, and inject powerful impetus into the inheritance and modern development of traditional Chinese medicine.

[0003] In the traditional diagnosis and treatment system of traditional Chinese medicine, acupoint recognition and positioning have always occupied a crucial position and are the key prerequisites for the accurate implementation of many treatment methods such as acupuncture, massage, and moxibustion. However, traditional acupoint recognition methods mainly rely on the experience and palpation of physicians, which have problems such as strong subjectivity and low accuracy and are difficult to meet the modern medical requirements for precision and standardization. In recent years, with the development of digital technology, technologies such as electrical detection, infrared thermography, and ultrasonic waves have been introduced into the field of acupoint recognition, providing new solutions for acupoint positioning. However, these technologies usually use single-modal data, are difficult to comprehensively reflect acupoint characteristics, lack the ability to provide real-time feedback on the dynamic changes of the human body, and the recognition accuracy is affected by environmental factors and individual differences, making it impossible to meet the strict requirements of modern traditional Chinese medicine for accurate, stable, and real-time acupoint recognition.

[0004] Therefore, an acupoint recognition system based on multimodal digital twin technology is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide an acupoint recognition system based on multimodal digital twin technology to solve the problems raised in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solution: An acupoint recognition system based on multimodal digital twin technology includes a multimodal data acquisition module, a multimodal data transmission module, a multimodal data processing module, a digital twin construction module, an acupoint state and positioning prediction network module, and a result presentation module;

[0007] The multimodal data acquisition module is used to comprehensively acquire acupoint multimodal data and human body data;

[0008] The multimodal data transmission module is used to meet the differentiated requirements of data transmission methods in various application scenarios;

[0009] The multimodal data processing module is used to preprocess, transform, extract features, fuse, and reduce the dimension of the acupoint multimodal data and human body data;

[0010] The digital twin construction module is used to construct a digital twin model of the physiological state of acupoints and a digital twin model of the pathological state of acupoints based on the processed data;

[0011] The acupoint state and location prediction network module is used to accurately predict the acupoint state and acupoint location according to the collected human body data;

[0012] The result presentation module is used to present the digital twin 3D human body model, human body data, acupoint multimodal data, and acupoint annotation information of acupoints on the human-computer interaction interface.

[0013] Preferably, the multimodal data acquisition module includes: a bioelectric signal acquisition unit, a thermal imaging unit, an optical imaging unit, a mechanical signal acquisition unit, an anatomical structure imaging unit, a three-dimensional morphology acquisition unit, and a physiological parameter monitoring unit;

[0014] The bioelectric signal acquisition unit is used to acquire the resistance and electromyogram signals in the acupoint area;

[0015] The thermal imaging unit is used to acquire the temperature and skin optical characteristic data in the acupoint area;

[0016] The optical imaging unit is used to acquire the appearance image of the acupoint area, and the appearance image of the acupoint area includes skin texture and color changes;

[0017] The mechanical signal acquisition unit is used to acquire the tenderness threshold in the acupoint area;

[0018] The anatomical structure imaging unit is used to acquire the information of the muscle, fascia, and blood vessel layers in the local acupoint area, as well as the image information of the overall human body's bones, muscles, organs, blood vessels, and nerves;

[0019] The three-dimensional morphology acquisition unit is used to acquire the three-dimensional coordinate information, volume, and surface area of the human body;

[0020] The physiological parameter monitoring unit is used to acquire the basic vital signs of the human body's respiratory rate, temperature, blood pressure, and heart rate, and is used to provide physiological background data for acupoint state analysis;

[0021] Each acquisition device of the multimodal data acquisition module is connected to the same high-precision clock synchronization system. This system is based on an atomic clock to ensure that each device starts data acquisition synchronously within millisecond-level accuracy. Using a spatial calibration framework, the relative positions and angles of each acquisition device are precisely adjusted during the device installation phase, so that their acquisition fields of view for the same acupoint area are accurately overlapped. Based on the atomic clock and using the spatial calibration framework, high-precision synchronization in the time and space dimensions of data acquisition is guaranteed;

[0022] The data collected by the multimodal data acquisition module includes two aspects: physiological state and pathological state.

[0023] Preferably, the multimodal data transmission module includes wired and wireless transmission methods, supports flexible switching and collaborative operation between multiple data transmission methods, and can dynamically select a transmission mode based on real-time network bandwidth and latency data to ensure that data transmission always remains smooth.

[0024] Preferably, the multimodal data processing module can perform preliminary preprocessing, transformation, feature extraction, and fusion on the data collected by the multimodal data acquisition module and human body data. The multimodal data processing module runs on a dedicated customized RISC-V architecture processor, and cleaning, noise reduction, filtering, and normalization processes are used in the dedicated customized RISC-V architecture processor;

[0025] To achieve efficient quantization and feature fusion of multimodal data, the system uses a dynamic binning strategy to convert multimodal continuous data into discrete categories, unify the data scale, enhance the robustness of the model, and provide structured feature inputs for subsequent deep learning. The specific method is as follows:

[0026] S1: Data standardization, and its formula is as follows:

[0027]

[0028] Where: α norm represents the standardized data; α represents the collected multimodal data; represents the data mean; σ represents the data standard deviation;

[0029] S2: Dynamic binning, and the specific operation is as follows:

[0030] The standardized data α norm is assigned to a symmetric interval centered on the mean (to avoid boundary bias) and the interval is divided into k discrete categories A1, A2, A3…, A k , and a small compensation of the data mean is introduced through γ to avoid zero-value bias;

[0031] Dynamic binning width:

[0032] where represents the equal-width bin width based on the data range; β·σ represents the equal-interval bin width based on the standard deviation; γ represents the global scaling factor used to unify different sensor dimensions (γ = 0.1); β represents the dynamic weight coefficient, which flexibly adapts to different sensor characteristics by weighted fusion of the data range and the standard deviation (β = 1.2 for bioelectrical signal unit data, physiological parameter detection unit data, and mechanical signal acquisition unit data; β = 0.8 for thermal imaging unit data, optical imaging unit data, anatomical structure unit data, and three-dimensional morphology acquisition unit data); i represents the index of the interval (an integer); N represents the total number of data; α max and α min represent the maximum and minimum values of a certain type of data respectively; k e represents the number of bins;

[0033] When maps α norm to the discrete category A i ;

[0034] A i represents the quantized discrete category, and each A i category corresponds to a standardized feature representation, which ultimately constitutes a multi-modal feature vector;

[0035] Input the discrete category A i in the form of one-hot encoding into a deep convolutional neural network and a bidirectional long short-term memory network for key feature extraction, and perform processing such as fusion and dimensionality reduction on these features, and further use them as important features for input to form a multi-modal acupoint feature vector and a human body feature vector;

[0036] The specific formula for the feature vector of the acupoint is as follows:

[0037]

[0038] where, X 生理 refers to the physiological state of the acupoint; X 病理 refers to the pathological state of the acupoint; X 电阻 refers to the quantization of the resistance in the acupoint area; X 肌电 refers to the quantization of the electromyogram signal in the acupoint area; X 温度 refers to the quantization of the temperature in the acupoint area; X 反射率 refers to the quantization of the skin light reflectance in the acupoint area; X 吸收率 refers to the quantization of the skin light absorption rate in the acupoint area; X 颜色 refers to the quantization of the appearance color feature in the acupoint area; X 纹理 , refers to the quantization of the texture feature in the acupoint area; X 压痛阈值 refers to the quantization of the tenderness threshold in the acupoint area; X穴位解剖 Refers to the information of the muscle, fascia, and blood vessels in the acupoint area;

[0039] The specific formula for the feature vector of the human body is as follows:

[0040]

[0041] Y 生理 Refers to the physiological state of the human body; Y 病理 Refers to the pathological state of the human body; Y 体温 Is the quantification of the human body temperature; Y 呼吸 Is the quantification of the human respiratory rate; Y 血压 Is the quantification of the human blood pressure; Y 心率 Is the quantification of the human heart rate; Y 三维 Is the quantification of the three dimensions of the human body; Y 解剖 Refers to the imaging information of the bones, muscles, organs, blood vessels, and nerves of the whole human body.

[0042] Preferably, the digital twin model construction module can form a digital twin model of the acupoints based on the acupoint feature vector data processed by the multi-modal data processing module, the knowledge data sorted out in the early stage, and the human body 3D model constructed from the human body data. The digital twin model of the acupoints covers the digital twin model of the acupoint physiological state and the digital twin model of the acupoint pathological state. According to the collected human body data, using professional 3D modeling software, a highly accurate standard human body 3D model is constructed, and the deeply processed acupoint feature vector data is accurately mapped and associated onto the human body 3D model to ensure that the model can truly and meticulously restore the actual situation of the acupoints and accurately identify the specific positions of each acupoint.

[0043] Preferably, the acupoint state and positioning prediction network module can accurately construct a personalized human body 3D model according to the real-time collected human body data using professional 3D modeling software. On this basis, the real-time collected data is accurately compared with the human body data stored in the system to determine the current state of the human body, and the acupoint state is predicted accordingly. The corresponding digital twin model of the acupoints is mapped and associated to the customized human body 3D model that has been constructed, thereby establishing a customized digital twin model of the acupoints. The acupoint state and positioning prediction network module runs in the NPU processor, and the NPU processor uses deep learning algorithms generated between the deep convolutional neural network, bidirectional long short-term memory network, gated recurrent unit, and multi-layer perceptron, as well as supervised learning algorithms such as support vector machines and decision trees. By analyzing and processing the input human body data, accurate prediction of the acupoint state and positioning is achieved.

[0044] Preferably, the result presentation module is used to present the human 3D model, acupoint multi-modal data, and acupoint annotation information in an intuitive and visual manner on a human-computer interaction platform, and this human-computer interaction platform supports interactive operations.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] Through the multi-modal data acquisition module of the present invention, with the help of multi-element high-performance sensors and high-precision synchronous calibration, the accuracy and integrity of acupoint and human body data acquisition are ensured, overcoming traditional defects and facilitating precise traditional Chinese medicine treatment; through the multi-modal data transmission module, which integrates multiple transmission methods and supports flexible switching, the flexibility and reliability of data transmission are enhanced; through the multi-modal data processing module, which utilizes a customized RISC-V processor and advanced algorithms, the efficiency and effectiveness of data processing are improved; through the digital twin construction module, a refined and personalized acupoint digital twin model covering physiological and pathological states is constructed; through the acupoint state and location prediction network module, which combines multiple algorithms in an NPU processor, precise prediction of acupoint state and location is achieved; the result presentation module provides an intuitive and visual human-computer interaction interface, facilitating user operation, bringing significant advantages to fields such as acupoint-related medical treatment, health monitoring, and scientific research, and having broad application prospects and important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the overall framework flowchart of the multi-modal digital twin system of the present invention;

[0048] Figure 2 It is the schematic diagram of the elements of the multi-modal digital twin acupoint recognition system of the present invention;

[0049] Figure 3 It is the schematic diagram of the multi-modal data acquisition module of the present invention;

[0050] Figure 4 It is the schematic diagram of the multi-modal data processing module of the present invention;

[0051] Figure 5 It is the schematic diagram of the human-computer interaction interface of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0053] Please refer to Figures 1 to 5, the present invention provides a technical solution for an acupoint recognition system based on multi-modal digital twin technology:

[0054] An acupoint recognition system based on multi-modal digital twin technology includes a multi-modal data acquisition module, a multi-modal data transmission module, a multi-modal data processing module, a digital twin construction module, an acupoint state and location prediction network module, and a result presentation module;

[0055] The multi-modal data acquisition module can comprehensively obtain acupoint multi-modal data and human body data;

[0056] The multi-modal data transmission module can meet the differentiated requirements of data transmission methods in various application scenarios;

[0057] The multi-modal data processing module can preprocess, transform, extract features, fuse, and reduce the dimension of the acupoint multi-modal data and human body data;

[0058] The digital twin construction module can construct a digital twin model of acupoint physiological state and a digital twin model of acupoint pathological state based on the processed data;

[0059] The acupoint state and location prediction network module can accurately predict the acupoint state and acupoint location according to the collected human body data;

[0060] The result presentation module can present a digital twin human body 3D model of the acupoint, acupoint multi-modal data, and acupoint annotation information on the human-computer interaction interface.

[0061] As an embodiment of the present invention, as Figure 1 and Figure 5 shown, the multi-modal data acquisition module includes a bioelectric signal acquisition unit, a thermal imaging unit, an optical imaging unit, a mechanical signal acquisition unit, an anatomical structure imaging unit, a three-dimensional morphology acquisition unit, and a physiological parameter monitoring unit.

[0062] The bioelectric signal acquisition unit is used to collect the resistance and electromyogram signals in the acupoint area;

[0063] The thermal imaging unit is used to collect the temperature and skin optical property data in the acupoint area;

[0064] The optical imaging unit is used to collect the appearance image of the acupoint area, including skin texture and color change;

[0065] The mechanical signal acquisition unit is used to collect the tenderness threshold in the acupoint area;

[0066] The anatomical structure imaging unit is used to collect the information of the muscle, fascia, and blood vessel layers in the local acupoint area, as well as the imaging information of the overall human body bones, muscles, organs, blood vessels, and nerves;

[0067] The three-dimensional shape acquisition unit is used to acquire human three-dimensional coordinate information, volume, and surface area;

[0068] The physiological parameter monitoring unit is used to acquire basic vital signs such as human respiratory rate, temperature, blood pressure, and heart rate, providing physiological background data for acupoint state analysis.

[0069] The multi-modal data of the acupoints includes: acupoint area resistance, acupoint area electromyogram signal, acupoint area temperature, acupoint area skin light reflectance, acupoint area skin light absorbance, acupoint area appearance color characteristics (such as average color value, color histogram, etc.), acupoint area texture characteristics (such as gray-level co-occurrence matrix characteristics, local binary pattern characteristics, etc.), acupoint area tenderness threshold, acupoint area muscle thickness, acupoint area fascia elasticity, acupoint area blood vessel diameter, etc. The data collected from the acupoints is not limited to the above content. Human data includes: coordinate information (x, y, z-axis coordinates) of various parts of the human body, human volume, human surface area, human bones, human muscles, human blood vessels, human nerves, human organs, human temperature, respiratory rate, blood pressure, and heart rate, etc.;

[0070] Each acquisition unit of the multi-modal data acquisition module needs to be installed according to a precisely designed layout. The acquisition unit in the acupoint area should be aligned with the key acupoint areas of the human body, and the three-dimensional shape acquisition unit should be placed in a spatial position that can comprehensively scan the human body, etc. Each device is connected to the same high-precision clock synchronization system based on an atomic clock to ensure that each unit starts acquisition synchronously within millisecond-level accuracy. When the acupoint data acquisition instruction is issued, all sensors can capture their corresponding data at the same instant, avoiding data misalignment caused by time differences; in the initial installation process, use a high-precision spatial calibration framework to carefully adjust the relative positions and angles between devices, ensuring that the acquisition fields of view for the same acupoint area are perfectly coincident. Through technologies such as laser ranging and image registration, using fixed anatomical landmark points of the human body (such as the pubic symphysis, the spinous process of the seventh cervical vertebra, etc.) as a reference, perform spatial positioning calibration on each unit to ensure the spatial consistency of different modal data, laying a foundation for subsequent accurate data acquisition. Using the spatial calibration framework based on an atomic clock to ensure high-precision synchronization of data acquisition in the time and space dimensions;

[0071] The data collected by the multi-modal data acquisition module includes two aspects: physiological state and pathological state, and the acupoint data acquisition is based on the accurate acupoint positioning marked by multiple professional and experienced physicians.

[0072] As an embodiment of the present invention, such as Figure 2 and Figure 3As shown, the multimodal data transmission module integrates multiple data transmission methods, aiming to fully meet the differentiated requirements of various application scenarios; the wired transmission method can be applied to scenarios that require high-speed data transmission to ensure reliable data transmission, avoid data loss or delay, and ensure the timely and accurate transmission of key information; the wireless transmission method can be applied to wearable devices and remote medical monitoring. With the help of wireless technologies such as Bluetooth and Wi-Fi, the acupoint data collected can be quickly transmitted to the central server or mobile terminal for subsequent analysis and processing, facilitating users to conduct acupoint monitoring and health management anytime and anywhere; the multimodal data transmission module supports flexible switching and collaborative operation between multiple data transmission methods, and can intelligently judge and automatically select the most suitable data transmission method according to different usage scenarios and real-time network environments to ensure that data transmission always remains smooth.

[0073] As an embodiment of the present invention, as Figures 2 to 5 shown, the multimodal data processing module can perform preliminary preprocessing, transformation, feature extraction, and fusion on the data collected by the multimodal data acquisition module and human body data. The multimodal data processing module runs in a dedicated customized RISC-V architecture processor, in which cleaning, noise reduction, filtering, and normalization processes are used. To achieve efficient quantization and feature fusion of multimodal data, the system uses a dynamic binning strategy to convert multimodal continuous data into discrete categories, unify the data scale, and enhance the robustness of the model, providing structured feature input for subsequent deep learning. The specific method is as follows:

[0074] S1: Data standardization, and its formula is as follows:

[0075]

[0076] where: α norm represents the standardized data; α represents the collected multimodal data; represents the data mean; σ represents the data standard deviation;

[0077] S2: Dynamic binning, and the specific operation is as follows:

[0078] Allocate the standardized data α norm to a symmetric interval centered on the mean (to avoid boundary bias) and divide the interval into k discrete categories A1, A2, A3…, A k , and introduce a small compensation for the data mean through γ to avoid zero-value bias;

[0079] Dynamic binning width:

[0080] where denotes the equal-width bin width based on the data range; β·σ denotes the equal-interval bin width based on the standard deviation; γ denotes the global scaling factor for unifying different sensor dimensions (γ = 0.1); β denotes the dynamic weight coefficient, which flexibly adapts to different sensor characteristics by weighted fusion of the data range and the standard deviation (β = 1.2 for the data of the bioelectric signal unit, the physiological parameter detection unit, and the mechanical signal acquisition unit; β = 0.8 for the data of the thermal imaging unit, the optical imaging unit, the anatomical structure unit, and the three-dimensional morphology acquisition unit); i denotes the index (integer) of the interval; N denotes the total number of data; α max and α min respectively denote the maximum and minimum values of a certain type of data; k e denotes the number of bins;

[0081] When maps α norm to the discrete category A i ;

[0082] A i denotes the quantized discrete category, and each A i category corresponds to a standardized feature representation, which ultimately constitutes a multi-modal feature vector;

[0083] Inputs the discrete category A i in the form of one-hot encoding into a deep convolutional neural network and a bidirectional long short-term memory network for key feature extraction, performs processing such as fusion and dimensionality reduction on these features, and further inputs them as important features to form a multi-modal acupoint feature vector and a human body feature vector;

[0084] The specific formula for the feature vector of the acupoint is as follows:

[0085]

[0086] Among them, X 生理 refers to the physiological state of the acupoint; X 病理 refers to the pathological state of the acupoint; X 电阻 refers to the quantization of the resistance in the acupoint area; X 温度 is the quantization of the temperature in the acupoint area; X 反射率 refers to the quantization of the skin light reflectance in the acupoint area; X 吸收率 refers to the quantization of the skin light absorption rate in the acupoint area; X 颜色 refers to the quantization of the appearance color feature in the acupoint area; X 纹理 , refers to the quantization of the texture feature in the acupoint area; X 压痛阈值 refers to the quantization of the tenderness threshold in the acupoint area; X 穴位解剖 refers to the hierarchical information of muscles, fascia, and blood vessels in the acupoint area;

[0087] The specific formula for the feature vector of the human body is as follows:

[0088]

[0089] Y 生理 Refers to the physiological state of the human body; Y 病理 Refers to the pathological state of the human body; Y 体温 Is the quantification of human body temperature; Y 呼吸 Is the quantification of human respiratory rate; Y 血压 Is the quantification of human blood pressure; Y 心率 Is the quantification of human heart rate; Y 三维 Is the quantification of the three dimensions of the human body; Y 解剖 Refers to the imaging information of the bones, muscles, organs, blood vessels, and nerves of the whole human body;

[0090] By collecting a large number of professionally labeled acupoint data samples, the training model learns the differences and similarity patterns between different acupoints based on quantitative features. For example, through the training of a large number of samples, the model can find that certain acupoints have specific laws in the combination of quantitative categories of pressure and temperature features, and have unique performances in the quantitative categories of image features, so as to accurately identify acupoints.

[0091] As an embodiment of the present invention, as Figures 1 to 5 shown, the digital twin model construction module can form a digital twin model of acupoints based on the acupoint feature vector data processed by the multi-modal data processing module, the knowledge data sorted out in the early stage, and the human body 3D model constructed from human body data. The digital twin model of acupoints covers the digital twin model of acupoint physiological state and the digital twin model of acupoint pathological state. According to the collected human body data, using professional 3D modeling software, a highly accurate standard human body 3D model is constructed, and the deeply processed acupoint feature vector data is accurately mapped and associated onto the human body 3D model to ensure that the model can truly and meticulously restore the actual situation of acupoints and accurately identify the specific positions of each acupoint;

[0092] The key element formula for the establishment process of the multi-modal digital twin acupoint recognition system is as follows:

[0093]

[0094] Among them: D represents the elements related to the physical entity of data acquisition. The symbols corresponding to D in the brackets refer to: bioelectric signal acquisition unit, thermal imaging unit, optical imaging unit, mechanical signal acquisition unit, anatomical structure imaging unit, three-dimensional morphology acquisition unit, and physiological parameter monitoring unit;

[0095] K represents the knowledge data element. The symbols corresponding to K in the brackets refer to: a large number of modern research literatures related to acupoint location, traditional Chinese medicine classic literatures, and acupoint location data labeled by a large number of clinicians;

[0096] T represents a data transmission element, and the symbols corresponding to T in the brackets respectively refer to: wired transmission and wireless transmission;

[0097] H represents a data processing element, and the symbols corresponding to H in the brackets respectively refer to: data preprocessing, data transformation, data feature extraction, and data fusion;

[0098] M represents a digital twin model element, and the symbols corresponding to M in the brackets respectively refer to: the digital twin model of acupoint physiological state and the digital twin model of acupoint pathological state;

[0099] F represents a prediction network element, and the symbols corresponding to F in the brackets respectively refer to: the acupoint state prediction network and the acupoint location prediction network;

[0100] S represents a result presentation element, and the symbols corresponding to S in the brackets respectively refer to: personalized human 3D model, acupoint multimodal data, and acupoint annotation information (the meridian to which it belongs, the main treatment diseases, and acupuncture techniques).

[0101] During operation,

[0102] As an embodiment of the present invention, as Figure 5 shown, the acupoint state and location prediction network module can accurately construct a personalized human 3D model according to the real-time collected human data by using professional 3D modeling software. On this basis, the real-time collected data is accurately compared with the human data stored in the system to determine the current state of the human body, and the acupoint state is predicted accordingly. The corresponding acupoint digital twin model is mapped and associated to the customized human 3D model that has been constructed, thereby establishing a customized acupoint digital twin model. The acupoint state and location prediction network module runs in the NPU processor, and the NPU processor uses deep learning algorithms generated between deep convolutional neural networks, bidirectional long short-term memory networks, gated recurrent units, and multi-layer perceptrons, as well as supervised learning algorithms such as support vector machines and decision trees. By analyzing and processing the input human data, accurate prediction of acupoint state and location is achieved.

[0103] During operation, the acupoint state and location prediction network module can, in actual application scenarios, accurately construct a personalized 3D human model based on real-time collected human data using professional 3D modeling software. On this basis, the real-time collected data is accurately compared with the human data stored in the system to determine whether the current human state is a physiological state or a pathological state. Further, according to the obtained human state judgment result, the corresponding acupoint digital twin model is mapped and associated to the customized 3D human model that has been constructed, thereby establishing a customized acupoint digital twin model. For example, when comparing the real-time collected human data with the human data stored in the system, it is found that the human is in a pathological state, and the acupoint pathological state digital twin model is mapped and associated to the customized 3D human model, and the accurate acupoint positions in the pathological state are marked on the customized 3D human model. The acupoint state and location prediction network can provide strong data support and advanced technical guarantee for many important fields such as precision medicine and health management, helping related fields achieve more efficient and high-quality services and development. The acupoint state and location prediction network module operates in an NPU processor, which uses deep learning algorithms including deep convolutional neural network (DCNN), bidirectional long short-term memory network (Bi-LSTM), gated recurrent unit (GRU), multi-layer perceptron (MLP), etc., and supervised learning algorithms such as support vector machine (SVM) and decision tree. These algorithms cooperate with each other to analyze and process the input human data to achieve accurate prediction of acupoint state and location.

[0104] As an embodiment of the present invention, as Figure 5 shown, the result presentation module is used to present the 3D human model, acupoint multi-modal data, and acupoint annotation information in an intuitive and visual manner on the human-computer interaction interface.

[0105] During operation, there is a touch screen panel on the human-computer interaction interface. A 3D human model is presented on this panel, and all acupoints of the human body are marked on the model. Users only need to click on an acupoint, and an information interface of acupoint multi-modal data, acupoint annotation information (such as the meridian to which it belongs, the main treatment diseases, acupuncture techniques, etc.) will immediately pop up, providing rich and professional references for users.

[0106] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An acupoint recognition system based on multi-modal digital twin technology, characterized in that: It includes a multi-modal data acquisition module, a multi-modal data transmission module, a multi-modal data processing module, a digital twin construction module, an acupoint state and location prediction network module, and a result presentation module; The multi-modal data acquisition module is used to obtain acupoint multi-modal data and human body data in all directions; The multi-modal data transmission module is used to meet the differentiated requirements of data transmission methods in various application scenarios; The multi-modal data processing module is used to preprocess, transform, extract features, fuse, and reduce the dimension of the acupoint multi-modal data and human body data; The digital twin construction module is used to construct a digital twin model of acupoint physiological state and a digital twin model of acupoint pathological state based on the processed data; The acupoint state and location prediction network module is used to accurately predict the acupoint state and acupoint location according to the collected human body data; The result presentation module is used to present the digital twin 3D human body model of acupoints, human body data, acupoint multi-modal data, and acupoint annotation information on the human-computer interaction interface.

2. The acupoint recognition system based on multi-modal digital twin technology according to claim 1, wherein: The multi-modal data acquisition module includes: a bioelectric signal acquisition unit, a thermal imaging unit, an optical imaging unit, a mechanical signal acquisition unit, an anatomical structure imaging unit, a three-dimensional shape acquisition unit, and a physiological parameter monitoring unit; The bioelectric signal acquisition unit is used to acquire the resistance and electromyogram signals in the acupoint area; The thermal imaging unit is used to acquire the temperature and skin optical characteristic data in the acupoint area; The optical imaging unit is used to acquire the appearance image of the acupoint area, and the appearance image of the acupoint area includes skin texture and color change; The mechanical signal acquisition unit is used to acquire the tenderness threshold in the acupoint area; The anatomical structure imaging unit is used to acquire the information of the muscle, fascia, and blood vessel layers in the local acupoint area, as well as the imaging information of the overall human body bones, muscles, organs, blood vessels, and nerves; The three-dimensional shape acquisition unit is used to acquire the three-dimensional coordinate information, volume, and surface area of the human body; The physiological parameter monitoring unit is used to acquire the basic vital signs of the human body's respiratory rate, temperature, blood pressure, and heart rate, and is used to provide physiological background data for acupoint state analysis; Each acquisition device of the multi-modal data acquisition module is connected to the same high-precision clock synchronization system, which is based on an atomic clock to ensure that each device starts acquisition synchronously within the millisecond-level accuracy. Using a spatial calibration framework, the relative positions and angles of each acquisition device are precisely adjusted during the device installation stage, so that their acquisition fields of view for the same acupoint area are accurately overlapped. Based on the atomic clock and using the spatial calibration framework, high-precision synchronization of data acquisition in the time and space dimensions is guaranteed; The data collected by the multi-modal data acquisition module includes two aspects: physiological state and pathological state.

3. The acupoint recognition system based on multi-modal digital twin technology according to claim 1, wherein: The multi-modal data transmission module includes wired and wireless transmission methods, supports flexible switching and collaborative operation between multiple data transmission methods, and can dynamically select the transmission mode based on real-time network bandwidth and latency data to ensure that data transmission always remains smooth.

4. The acupoint recognition system based on multimodal digital twin technology according to claim 1, wherein: The multimodal data processing module can perform preliminary preprocessing, transformation, feature extraction, and fusion on the data collected by the multimodal data acquisition module and human body data. The multimodal data processing module runs on a dedicated customized RISC-V architecture processor, in which cleaning, noise reduction, filtering, and normalization processes are used; To achieve efficient quantization and feature fusion of multimodal data, the system uses a dynamic binning strategy to convert multimodal continuous data into discrete categories, unify the data scale, enhance the robustness of the model, and provide structured feature input for subsequent deep learning. The specific method is as follows: S1: Data standardization, and its formula is as follows: where: α norm represents the standardized data; α represents the collected multi-modal data; represents the data mean; σ represents the data standard deviation; S2: Dynamic binning, and the specific operation is as follows: The standardized data α norm is assigned to a symmetric interval centered on the mean and the interval is divided into k discrete categories A1, A2, A3…, A k , and a small compensation for the data mean is introduced through γ to avoid zero-value bias; Dynamic bin width: Wherein: represents the equal-width bin width based on the data range; β·σ represents the equal-interval bin width based on the standard deviation; γ represents the global scaling factor for unifying the dimensions of different sensors; β represents the dynamic weight coefficient, which flexibly adapts to different sensor characteristics by weighted fusion of the data range and the standard deviation. β = 1.2 is used for the data of the bioelectric signal unit, the data of the physiological parameter detection unit, and the data of the mechanical signal acquisition unit, and β = 0.8 is used for the data of the thermal imaging unit, the data of the optical imaging unit, the data of the anatomical structure unit, and the data of the three-dimensional morphology acquisition unit; i represents the index of the interval; N represents the total number of data; α max and α min respectively represent the maximum value and the minimum value of a certain type of data; k e represents the number of bins; When α norm is mapped to the discrete category A i ; A i represents the quantized discrete category, each i category corresponds to a standardized feature representation, and finally forms a multi-modal feature vector; Input the discrete category A i into a deep convolutional neural network and a bidirectional long short-term memory network in one-hot encoding form for key feature extraction. Process these features through fusion, dimensionality reduction, etc., and further input them as important features to form a multi-modal acupoint feature vector and a human body feature vector; The specific formula for the feature vector of the acupoint is as follows: Among them, X 生理 refers to the physiological state of the acupoint; X 病理 refers to the pathological state of the acupoint; X 电阻 refers to the quantification of the electrical resistance in the acupoint area; X 肌电 refers to the quantification of the electromyogram signal in the acupoint area; X 温度 refers to the quantification of the temperature in the acupoint area; X 反射率 refers to the quantification of the skin light reflectance in the acupoint area; X 吸收率 refers to the quantification of the skin light absorption rate in the acupoint area; X 颜色 refers to the quantification of the appearance color characteristics in the acupoint area; X 纹理 , refers to the quantification of the texture characteristics in the acupoint area; X 压痛阈值 refers to the quantification of the tenderness threshold in the acupoint area; X 穴位解剖 refers to the hierarchical information of muscles, fascia, and blood vessels in the acupoint area; The specific formula for the feature vector of the human body is as follows: Y 生理 Refers to the physiological state of the human body; Y 病理 Refers to the pathological state of the human body; Y 体温 Is the quantification of human body temperature; Y 呼吸 Is the quantification of human respiratory rate; Y 血压 Is the quantification of human blood pressure; Y 心率 Is the quantification of human heart rate; Y 三维 Is the quantification of the three dimensions of the human body; Y 解剖 Refers to the imaging information of the bones, muscles, organs, blood vessels, and nerves of the whole human body.

5. The acupoint recognition system based on multi-modal digital twin technology according to claim 1, characterized in that: The digital twin model construction module can form a digital twin model of the acupoint based on the acupoint feature vector data processed by the multimodal data processing module, the knowledge data sorted out in the early stage, and the human body 3D model constructed from human body data. The digital twin model of the acupoint covers the digital twin model of the acupoint physiological state and the digital twin model of the acupoint pathological state. According to the collected human body data, a highly accurate standard human body 3D model is constructed using professional 3D modeling software. The deeply processed acupoint feature vector data is accurately mapped and associated onto the human body 3D model to ensure that the model can truly and meticulously restore the actual situation of the acupoint and accurately mark the specific position of each acupoint.

6. The acupoint recognition system based on multi-modal digital twin technology according to claim 1, wherein: The acupoint state and positioning prediction network module can accurately construct a personalized human body 3D model using professional 3D modeling software based on the real-time collected human body data. On this basis, the real-time collected data is accurately compared with the human body data stored in the system to determine the current state of the human body, and the acupoint state is predicted accordingly. The corresponding acupoint digital twin model is mapped and associated to the customized human body 3D model that has been constructed, thereby establishing a customized acupoint digital twin model. The acupoint state and positioning prediction network module runs on an NPU processor, and the NPU processor uses deep learning algorithms generated between deep convolutional neural networks, bidirectional long short-term memory networks, gated recurrent units, and multi-layer perceptrons, as well as supervised learning algorithms such as support vector machines and decision trees. By analyzing and processing the input human body data, accurate prediction of the acupoint state and positioning is achieved.

7. The acupoint recognition system based on multi-modal digital twin technology according to claim 1, characterized in that: The result presentation module is used to present the human body 3D model, acupoint multimodal data, and acupoint annotation information in an intuitive and visual manner on a human-computer interaction platform, and this human-computer interaction platform supports interactive operations.

Images