Human body meridian point visualization method and visualization teaching system based on intelligent AI

Abstract

The invention discloses a human body meridian point visualization method based on intelligent AI and a visualization teaching system, and relates to the technical field of human body meridian points. According to the human body meridian point visualization method based on the intelligent AI, a three-dimensional virtual human body model is constructed based on magnetic resonance imaging and computed tomography; acupoint positioning and meridian drawing are carried out on the constructed three-dimensional virtual human body model, and visualization is carried out; relative spatial position relations of human meridians, acupuncture points and human tissues are displayed based on a quantitative mixed reality technology, effective integration of advantage information of two images is realized on an image level, a foundation is laid for obtaining comprehensive and accurate human tissue and organ structure data, and a model is established on the aspect of modeling. The problem that a complex human body structure and a space relation are difficult to present in traditional modeling is solved, accurate labeling and visual display of meridian points on the virtual model are achieved in meridian point processing, and in a teaching system, interaction convenience and naturalness of a user and the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of human meridian acupoints, and specifically to a visualization method and a visualization teaching system of human meridian acupoints based on intelligent AI. Background Art

[0002] In traditional Chinese medicine theory, meridians are channels for running qi and blood, connecting zang-fu organs and the body surface, and are important regulatory systems of the human body functions. The meridian theory is not only the basis of acupuncture and massage in traditional Chinese medicine, but also an important part of traditional Chinese medicine, which has far-reaching significance for ensuring human health. The existing methods for measuring and displaying the positions and related information of human meridians and acupoints mainly rely on recognized standard academic publications, such as "Acupuncture Points and Acupuncture Science" (Yang Jiasan), "Notes and Illustrations of Acupuncture Science", etc.; relevant national standards such as GB / T12346-2021 "Names and Positions of Acupuncture Points"; physical or 3D virtual human meridian acupoint models for confirmation.

[0003] In recent years, with the rapid development of computer technology and artificial intelligence, especially the wide application of technologies such as image recognition and three-dimensional visualization, new possibilities have been provided for the research of meridians and acupoints. However, the existing technologies have the following defects:

[0004] At the image level, the effective integration of the advantageous information of two types of images has been realized, laying a foundation for obtaining comprehensive and accurate data of human tissue and organ structures. In terms of modeling, the problem that traditional modeling is difficult to present complex human structures and spatial relationships has been solved. In the processing of meridians and acupoints, the accurate labeling and visualization display of meridians and acupoints on virtual models have been realized. In the teaching system, the convenience and naturalness of user interaction with the model have been improved. Summary of the Invention

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: A visualization method of human meridian acupoints based on intelligent AI, including the following steps: constructing a three-dimensional virtual human model based on magnetic resonance imaging and computed tomography; performing acupoint positioning and meridian drawing on the constructed three-dimensional virtual human model and visualizing them; displaying the relative spatial position relationship between human meridians, acupoints and human tissues based on quantitative mixed reality technology.

[0006] Furthermore, the construction of the three-dimensional virtual human body model based on magnetic resonance imaging and computed tomography includes the following steps: fusing the MRI images obtained by magnetic resonance imaging and the CT images obtained by computed tomography and then reconstructing to obtain a basic image; using the trained fully convolutional neural network to process the basic image to obtain the required tissue and organ structure data, and after preprocessing the required tissue and organ structure data, obtaining the preprocessed tissue and organ structure data; establishing a human body mixed reality structure model based on three-dimensional modeling technology and the preprocessed tissue and organ structure data; and mapping the human body mixed reality structure model into the three-dimensional virtual human body space coordinate axis system to obtain the three-dimensional virtual human body model.

[0007] Furthermore, the step of fusing the MRI images obtained by magnetic resonance imaging and the CT images obtained by computed tomography and then reconstructing to obtain a basic image includes the following steps: registering the MRI image and the CT image based on feature points to obtain the registered MRI image and the registered CT image; and fusing and reconstructing the registered MRI image and the registered CT image to obtain the basic image.

[0008] Furthermore, the step of registering the MRI image and the CT image based on feature points to obtain the registered MRI image and the registered CT image includes the following steps: detecting the extreme points of the MRI image based on the scale-invariant feature transform algorithm and extracting the feature points with scale invariance, denoted as the SIFT feature point set of the MRI image; detecting the CT image based on the Hessian matrix determinant of the accelerated robust feature algorithm to obtain the SURF feature point set of the CT image; constructing a KD tree for the SIFT feature point set of the MRI image and the SURF feature point set of the CT image, and determining the nearest neighbor relationship based on the Euclidean distance to obtain the effective matching pairs; using the RANSAC algorithm, randomly selecting several pairs of effective matching pairs from the effective matching pairs as the initial sample points, and calculating the preliminary rigid transformation matrix T; transforming all the effective matching pairs based on the preliminary rigid transformation matrix T, and calculating the error between the transformed points and the actual corresponding points: if the error is less than the set threshold, the point is considered an inlier, otherwise it is an outlier; after experiencing the set number of iterations, randomly selecting several pairs of sample points each time to calculate the transformation matrix and counting the number of inliers, and finally selecting the transformation matrix with the largest number of inliers as the final registration transformation matrix to align the MRI image and the CT image in the spatial position to obtain the registered MRI image and the registered CT image.

[0009] Further, the step of reconstructing a basic image after fusing the registered MRI image and the registered CT image includes the following steps: performing Laplacian pyramid decomposition algorithm processing on the registered MRI image and the registered CT image to obtain an MRI Laplacian pyramid image and a CT Laplacian pyramid image, where the number of layers of the MRI Laplacian pyramid image and the CT Laplacian pyramid image is the same; on the low-frequency components, fusing the MRI Laplacian pyramid image and the CT Laplacian pyramid image according to the energy distribution of each level; on the high-frequency components, fusing the MRI Laplacian pyramid image and the CT Laplacian pyramid image according to the regional characteristics; reconstructing the fused Laplacian pyramid, starting from the Laplacian image of the highest level, through upsampling, and adding it to the low-frequency component of the next layer, gradually restoring the fused image to obtain the basic image.

[0010] Further, the step of performing acupoint location and meridian drawing on the constructed three-dimensional virtual human model and visualizing it includes the following steps: based on real-time registration technology of optical tracking or inertial tracking, performing real-time registration of the three-dimensional virtual human model with the real human body; marking the positions of each acupoint on the registered three-dimensional virtual human model; according to the positions of the acupoints, drawing the human meridian system on the three-dimensional virtual human model along the running route of the meridians; through visualization technology, visualizing each meridian in the human meridian system in different colors and line styles.

[0011] The intelligent AI-based human meridian acupoint visualization teaching system for the above-mentioned intelligent AI-based human meridian acupoint visualization method includes a data acquisition module, a model establishment and optimization module, a model display module, and a human-computer interaction module, where: the data acquisition module is used to acquire the data required for human meridian acupoint structure modeling; the model establishment and optimization module is used to use 3Dmax software to establish and optimize a human 3D structure model including the human meridian acupoint system according to the data required for human meridian acupoint structure modeling collected; the model display module is used to display the established human 3D structure model in the form of a holographic image through a HoloLens display device; the human-computer interaction module is used to acquire interaction instructions and perform interaction operations on the human 3D structure model based on the interaction instructions.

[0012] Further, the data required for human meridian acupoint structure modeling includes tissue and organ structure data and meridian acupoint data. The tissue and organ structure data includes the shape, size, position, density of each tissue and organ, and the boundary information between different tissues. The meridian acupoint data includes acupoint position, acupoint name, and acupoint efficacy.

[0013] Further, the interaction instructions include gesture instructions and voice instructions, and the interaction operations include splitting, scaling, rotating, moving, and depth cutting.

[0014] Further, performing an interaction operation on the human 3D structure model based on the interaction instruction includes the following steps: comparing the interaction instruction with each reference instruction stored in the database to determine the reference instruction identical to the interaction instruction; obtaining the interaction operation corresponding to the reference instruction from the database, and operating on the human 3D structure model based on the obtained interaction operation.

[0015] The present invention has the following beneficial effects:

[0016] (1) For the visualization method of human meridian acupoints based on intelligent AI, the MRI and CT images are accurately registered and fused, and processed by a fully convolutional neural network to obtain detailed and accurate human tissue and organ structure data, and a highly realistic three-dimensional virtual human model is constructed.

[0017] (2) For the visualization teaching system of human meridian acupoints based on intelligent AI, through gesture and voice interaction technologies, students can interact with the human 3D structure model in a natural and convenient way, perform operations such as splitting, scaling, and rotating, and deeply observe the distribution and trend of meridian acupoints.

[0018] Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of the visualization method of human meridian acupoints based on intelligent AI of the present invention.

[0020] Figure 2 It is a flowchart block diagram of the visualization teaching system of human meridian acupoints based on intelligent AI of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In the embodiments of the present application, through the visualization method and visualization teaching system of human meridian acupoints based on intelligent AI, at the image level, the effective integration of the advantageous information of the two images is realized, laying a foundation for obtaining comprehensive and accurate human tissue and organ structure data. In terms of modeling, the problem that traditional modeling is difficult to present complex human structures and spatial relationships is solved. In the processing of meridian acupoints, the accurate annotation and visualization display of meridian acupoints on the virtual model are realized. In the teaching system, the interaction convenience and naturalness between the user and the model are improved.

[0022] Please refer to Figure 1 , the embodiments of the present invention provide a technical solution: a visualization method of human meridian acupoints based on intelligent AI, including the following steps:

[0023] Construct a three-dimensional virtual human body model based on magnetic resonance imaging and computed tomography;

[0024] Fuse the MRI images obtained by magnetic resonance imaging and the CT images obtained by computed tomography, and then reconstruct to obtain the basic image;

[0025] Register the MRI image and the CT image based on feature points to obtain the registered MRI image and the registered CT image; fuse the registered MRI image and the registered CT image and then reconstruct to obtain the basic image.

[0026] Detect the extreme points of the MRI image based on the scale-invariant feature transform algorithm, and extract the feature points with scale invariance, denoted as the SIFT feature point set of the MRI image;

[0027] For the MRI image I MRI (x1, y1), the Gaussian convolution at the scale parameter σ is I MRI (x1, y1, σ) = G(x1, y1, σ) * I MRI (x1, y1), where is the kernel function, e is the natural constant, π is the circumference ratio, and (x1, y1) is the pixel coordinate of the MRI image. Detect the extreme points through the difference of Gaussian at different scales. If the point is an extreme point, then take it as the SIFT feature point to obtain the SIFT feature point set of the MRI image.

[0028] Detect the CT image based on the accelerated robust feature algorithm by calculating the determinant of the Hessian matrix to obtain the SURF feature point set of the CT image;

[0029] For the CT image I CT (x2, y2), the Hessian matrix at a certain point (x2, y2) in the image where and are the second-order derivatives in the x and y directions of the CT image, which are obtained by convolving the Gaussian convolution kernel with the CT image. When the determinant of the Hessian matrix det(H CT (x2, y2, σ)) is greater than the set threshold, this point is considered as the SURF feature point to obtain the SURF feature point set of the CT image.

[0030] By preprocessing MRI images and CT images, such as denoising and other operations, the noise and interference factors in the images are removed, the image quality is improved, and a good foundation is laid for subsequent feature extraction and registration. Feature extraction can highlight the important structures and information in the images, such as SURF feature points and SIFT feature points. These feature points are unique and stable, and can effectively represent the local features of the images, facilitating subsequent matching and registration between images.

[0031] Construct a KD tree for the SIFT feature point set of the MRI image and the SURF feature point set of the CT image, and determine the nearest neighbor relationship based on the Euclidean distance to obtain valid matching pairs;

[0032] For the feature point P MRI in the SIFT feature point set S MRI (x MRI , y MRI ) and the feature point CT in the SURF feature point set S The Euclidean distance between them is Through the fast search of the KD tree, for each feature point in S MRI find the nearest neighbor point in S CT to obtain a set of valid matching pairs.

[0033] Using the RANSAC algorithm, randomly select several pairs of valid matching pairs from the valid matching pairs as the initial sample points, and calculate the preliminary rigid transformation matrix T; based on this preliminary rigid transformation matrix T, transform all valid matching pairs, and calculate the error between the transformed points and the actual corresponding points: if the error is less than the set threshold, then the point is considered an inlier, otherwise it is an outlier; after experiencing the set number of iterations, each iteration randomly selects several pairs of sample points to calculate the transformation matrix, and counts the number of inliers. Finally, select the transformation matrix with the largest number of inliers as the final registration transformation matrix to align the MRI image and the CT image in the spatial position, and obtain the registered MRI image and the registered CT image.

[0034] Realize the alignment of the MRI image and the CT image in the spatial position, and eliminate the image misalignment and differences caused by different imaging devices and imaging methods. This enables subsequent operations such as image fusion and analysis to be carried out in the same spatial coordinate system, avoiding incorrect conclusions due to position deviations, providing a prerequisite for accurately fusing two different modality images, and improving the accuracy and reliability of the fusion results.

[0035] Specifically, randomly select n pairs of sample points (x i,MRI , y i,MRI ) and (x i,CT , y i,CT), where i is the sample point number. For rigid transformation, its transformation model is where R 11 , R 12 , R 21 and R 22 are the elements of the rotation matrix, and t x and t y are the elements of the translation vector.

[0036] Minimize the reprojection error to obtain the preliminary rigid transformation matrix T.

[0037] Based on the preliminary rigid transformation matrix T, transform all valid matching pairs, calculate the error A. If A is less than the set error threshold, then the point is an inlier; otherwise it is an outlier. After a set number of iterations, each iteration randomly selects n pairs of sample points to calculate the transformation matrix and counts the number of inliers. Finally, select the transformation matrix with the largest number of inliers as the final registration transformation matrix to align the MRI image and the CT image in the spatial position to obtain the registered MRI image and the registered CT image

[0038] Perform the Laplacian pyramid decomposition algorithm on the registered MRI image and the registered CT image to obtain the MRI Laplacian pyramid image and the CT Laplacian pyramid image, and the number of layers of the MRI Laplacian pyramid image and the CT Laplacian pyramid image is the same;

[0039] For the registered MRI image Downsample through a Gaussian low-pass filter to obtain a low-resolution image Laplacian image U is the upsampling operation. Repeat this process to obtain the MRI Laplacian pyramid image where l is the pyramid layer number. Similarly, for the registered CT image Obtain the CT Laplacian pyramid image

[0040] On the low-frequency components, fuse the MRI Laplacian pyramid image and the CT Laplacian pyramid image according to the energy distribution of each level;

[0041] For the low-frequency component of the l-th layer, calculate the energy of each pixel point of the MRI Laplacian pyramid image and the energy of each pixel point of the CT Laplacian pyramid image where ω(m,n) is the Gaussian weighted window. If then the fused low-frequency component otherwise

[0042] On the high-frequency components, the MRI Laplacian pyramid image and the CT Laplacian pyramid image are fused according to the regional characteristics;

[0043] The image is divided into multiple small regions. For each region R, the MRI variance and the CT variance are calculated, and the fusion weight is determined according to the difference in regional characteristics. The fused high-frequency component ω MRI (R) is the MRI fusion weight, is the pixel value of the pixel point in the h-th layer high-frequency layer of the MRI Laplacian pyramid image, ω CT (R) is the CT fusion weight, is the pixel value of the pixel point in the h-th layer high-frequency layer of the CT Laplacian pyramid image.

[0044] The fused Laplacian pyramid is reconstructed. Starting from the highest-level Laplacian image, through upsampling and adding to the low-frequency component of the next layer, the fused image is gradually restored to obtain the base image.

[0045] The registered MRI image and CT image are fused to make full use of the respective advantageous information of the two images. The MRI image has an advantage in soft tissue imaging and can clearly show the structures of soft tissues such as the brain and spinal cord; the CT image is excellent in bone and hard tissue imaging. The obtained base image after fusion contains both the detailed information of soft tissues and the clear contours of hard tissues such as bones, providing more complete and accurate information for subsequent in-depth tissue and organ structure analysis and three-dimensional modeling operations based on this base image.

[0046] The trained fully convolutional neural network is used to process the base image to obtain the required tissue and organ structure data. After preprocessing the required tissue and organ structure data, the preprocessed tissue and organ structure data is obtained;

[0047] Let the trained fully convolutional neural network be CNN. The base image is input into CNN and, after being processed through a series of convolutional layers, pooling layers, and deconvolutional layers, the predicted tissue and organ segmentation map is output. Post-processing is performed on the tissue and organ segmentation map, such as morphological operations (erosion, dilation, etc.), to remove noise and small connected regions to obtain the required tissue and organ structure data. Preprocessing is performed on the required tissue and organ structure data, including normalization operations, to obtain the preprocessed tissue and organ structure data.

[0048] The fully convolutional neural network can automatically learn the feature patterns of different tissues and organs in the basic image. Through a series of convolutional, pooling, and deconvolution operations, it performs fine feature extraction and reconstruction on the basic image. It can effectively capture features at different levels in the image, from local features to global features, thus being able to more accurately predict the segmentation map of tissues and organs. This helps to precisely determine the boundaries and contours of tissues and organs, providing a reliable basis for subsequent processing of tissue and organ structure data and avoiding the subjectivity and errors that may be brought about by manual segmentation.

[0049] Preprocessing operations (such as normalization) standardize the tissue and organ structure data, making its numerical range within a specific interval and eliminating the dimensional differences and numerical magnitude differences between different features. This enables each feature to have the same weight and comparability during subsequent modeling and analysis, which is conducive to improving the training effect and prediction accuracy of the model and avoiding situations where the model training deviates or does not converge due to uneven data distribution or excessive numerical differences.

[0050] Based on three-dimensional modeling technology and the preprocessed tissue and organ structure data, a human body mixed reality structure model is established; the human body mixed reality structure model is mapped into a three-dimensional virtual human body space coordinate axis system to obtain a three-dimensional virtual human body model.

[0051] Using three-dimensional modeling technologies such as polygon modeling or surface modeling, according to the shape, size, position, etc. of each tissue and organ in the preprocessed tissue and organ structure data, a human body mixed reality structure model is established.

[0052] Using three-dimensional modeling technology to establish a human body mixed reality structure model based on the preprocessed tissue and organ structure data can visually display the tissue and organ structure inside the human body and their interrelationships. This provides a very realistic visualization model for doctors and researchers, facilitating intuitive observation and analysis, helping them better understand the physiological structure and pathological changes of the human body, and having important significance in aspects such as teaching, clinical diagnosis, and research.

[0053] Through the mapping operation, the human body mixed reality structure model is accurately placed in the three-dimensional virtual human body space coordinate axis system, establishing a corresponding relationship with the standard three-dimensional space coordinate system. This enables the model to be combined with the unified space coordinate system in subsequent acupoint location, meridian drawing, and spatial position relationship analysis operations, facilitating precise quantitative analysis and visual display, and providing an accurate coordinate basis for various subsequent operations and research based on spatial position relationships.

[0054] Perform acupoint location and meridian drawing on the constructed three-dimensional virtual human body model and perform visualization;

[0055] The real-time registration technology based on optical tracking or inertial tracking is used to perform real-time registration of a three-dimensional virtual human body model with a real human body; mark the positions of each acupoint on the registered three-dimensional virtual human body model; draw the human meridian system on the three-dimensional virtual human body model according to the running routes of the meridians; and visually display each meridian in the human meridian system with different colors and line styles through visualization technology. Among them, the running route can use the Dijkstra algorithm to determine the shortest path of the meridian in the model for drawing. Different colors are assigned to each meridian, and at the same time, different line styles are assigned to each meridian, such as line width and line type. During the visual rendering process of the three-dimensional virtual human body model, for each line segment on the meridian, it is drawn using the corresponding color and line style.

[0056] The mixed reality device is used to fuse and display the human meridian and acupoint system with the human mixed reality structure model, providing users with an immersive experience. Users can intuitively see the scene where virtual meridians, acupoints and human tissues are fused with each other in the real environment, as if they are in a real human anatomy environment. This way of fused display can greatly enhance users' perception and understanding of the spatial position relationship between human meridians, acupoints and tissues, and improve the teaching and learning effect.

[0057] Based on the quantitative mixed reality technology, the relative spatial position relationship between human meridians, acupoints and human tissues is displayed. In the mixed reality environment, distance measurement is realized based on the principle of triangulation, and angle measurement is carried out based on the principle of vector operation.

[0058] By realizing distance measurement based on the principle of triangulation, the actual distance information between acupoints and different points on human tissues can be accurately obtained. This is of great significance for studying the spatial position relationship between acupoints and surrounding tissues, evaluating the distance correlation between acupoints and diseased tissues, etc., providing quantitative data support for clinical diagnosis and treatment, helping doctors more accurately judge the relative position between acupoints and diseased parts, and formulating more precise treatment plans.

[0059] By performing angle measurement based on the principle of vector operation, the angle relationship between acupoints and relevant meridian line segments or characteristic positions of human tissues can be accurately determined. Angle information is very crucial for analyzing the running direction of meridians, the spatial angle between acupoints and meridians, etc., helping to deeply understand the three-dimensional structure and functional characteristics of the meridian system, and providing important quantitative indicators for studying the physiological functions and pathological changes of meridians.

[0060] Integrating and statistically analyzing distance and angle data, calculating the mean and standard deviation, and analyzing correlations can comprehensively understand the overall characteristics and distribution patterns of the spatial position relationships among human meridians, acupoints, and tissues. This helps to discover the commonalities and differences in spatial positions among different acupoints and different tissues, providing data support for establishing a more scientific theoretical model of human meridians and acupoints.

[0061] A visualization teaching system for human meridians and acupoints based on intelligent AI is used for the above-mentioned visualization method of human meridians and acupoints based on intelligent AI, such as Figure 2 shown, including a data acquisition module, a model establishment and optimization module, a model display module, and a human-computer interaction module, where:

[0062] The data acquisition module is used to acquire the data required for modeling the structure of human meridians and acupoints;

[0063] Acquiring tissue and organ structure data and meridian and acupoint data from multiple channels (such as magnetic resonance imaging, computed tomography images, as well as traditional Chinese medicine ancient books, clinical research databases, etc.) ensures the comprehensiveness and accuracy of the data. These data are the basis for constructing the visualization teaching system of human meridians and acupoints, providing reliable data support for subsequent model establishment and teaching applications, enabling the system to truly reflect the structural and functional characteristics of human meridians and acupoints.

[0064] The model establishment and optimization module is used to use 3Dmax software to establish and optimize a 3D structure model of the human body containing the human meridian and acupoint system according to the data required for modeling the structure of human meridians and acupoints collected;

[0065] Using 3Dmax software to establish a 3D structure model of the human body containing the human meridian and acupoint system can construct a highly realistic three-dimensional model based on detailed data. This allows users to clearly see the distribution and morphology of human meridians and acupoints in a virtual environment, intuitively feel the three-dimensional structure of the human body, helps users better understand the spatial positions and mutual relationships of meridians and acupoints, and provides a highly realistic visualization platform for teaching and research.

[0066] Simplifying the number of polygons can reduce the complexity of the model, reduce the computational burden of the system, improve the rendering speed and smoothness of the model on different devices, and ensure smooth display and interactive operations in various hardware environments. Adjusting the material and lighting effects can make the model more realistic and vivid, with different parts having different material properties and lighting effects, enhancing the visual effect of the model, enabling students to obtain a more realistic visual experience during the learning process and better understand the characteristics of different human tissues and organs.

[0067] The model display module is used to display the established human body 3D structure model in the form of a holographic image through the HoloLens display device; the human-computer interaction module is used to obtain interaction instructions and perform interactive operations on the human body 3D structure model based on the interaction instructions.

[0068] The HoloLens display device is selected as the hardware platform for model display, and its high resolution, large field of view and real-time tracking are used to provide users with an immersive mixed reality experience. The 3D structure model of the human body is displayed in a mixed reality environment, realizing the fusion of virtual and real, making users feel as if they are in a real human body world, greatly enhancing users' perception and understanding of the human meridian and acupoint system, and improving the interest and effectiveness of teaching.

[0069] Specifically, the data required for modeling the human meridian acupoint structure include tissue and organ structure data and meridian acupoint data. The tissue and organ structure data include the shape, size, position, density of each tissue and organ, and the boundary information between different tissues. The meridian acupoint data includes the acupoint location, acupoint name and acupoint efficacy. Interactive commands include gesture commands and voice commands, and interactive operations include splitting, scaling, rotating, moving and deep cutting.

[0070] In this implementation, the gesture recognition function of the HoloLens device is used to allow users to interact with the 3D structure model of the human body through simple gesture operations, such as clicking, sliding, zooming, etc. A method based on the histogram of oriented gradients (HOG) features and a support vector machine (SVM) classifier is used for gesture recognition.

[0071] Specifically, interactive operations are performed on a 3D structural model of the human body based on interactive instructions, including the following steps: comparing the interactive instructions with reference instructions stored in a database to determine a reference instruction that is identical to the interactive instruction; obtaining the interactive operation corresponding to the reference instruction from the database, and operating the 3D structural model of the human body based on the obtained interactive operation.

[0072] For the input gesture image, the image is first normalized and then divided into multiple small cells. In each cell, the gradient direction histogram is calculated. The sum of the gradient amplitudes in each interval is counted to obtain the gradient direction histogram of the cell. Then the adjacent cells are combined into blocks, and the gradient direction histogram of the cells in each block is normalized to obtain the feature vector of the block. The feature vectors of all blocks are connected to obtain the HOG feature vector of the entire gesture image.

[0073] In the training phase, a large number of labeled gesture image samples are used to extract their HOG feature vectors, which are then input into an SVM classifier for training. The goal of the SVM classifier is to find an optimal hyperplane to separate gesture feature vectors of different classes.

[0074] When the user makes a gesture of clenching both fists and pulling them apart to the sides, after the system recognizes this gesture, it determines the part of the human 3D structure model that the user is currently focusing on. Assume the model consists of multiple sub-models, and each sub-model is represented by a set of vertices and a set of faces. The system finds the part that needs to be split by looking for the connection relationships between sub-models, such as the relationships of shared vertices or faces. Then the connection relationships are broken, and the vertex and face information is updated to achieve the splitting of the model. For example, if we want to split the arm model, find the set of vertices shared by the arm model and the shoulder model, break the references of these vertices in the arm model and the shoulder model, and update the vertex and face information of the two models respectively.

[0075] Integrating speech recognition technology, users can interact with the system through voice commands. A speech recognition model based on deep learning is adopted, such as a deep neural network (DNN)-hidden Markov model (HMM) hybrid model. For the input speech signal, preprocessing operations are first performed, such as pre-emphasis, framing, windowing, etc. Then speech features are extracted, and commonly used features include Mel-frequency cepstral coefficients (MFCC). For each frame of the speech signal, calculate its power spectrum, obtain the Mel-spectrum through a Mel filter bank for the power spectrum, take the logarithm of the Mel-spectrum and perform discrete cosine transform (DCT) to obtain the MFCC feature vector.

[0076] In the training phase, the extracted MFCC feature vectors are input into the DNN for feature transformation. The DNN consists of multiple hidden layers, and the output of each hidden layer is, where is the weight matrix, is the bias term, and is the activation function (such as the sigmoid function or ReLU function). The output of the DNN serves as the observation probability distribution of the HMM. The HMM consists of a state set, a state transition probability matrix, and an observation probability distribution. The parameters of the DNN-HMM model are trained by methods such as maximum likelihood estimation. In the recognition phase, for the input speech signal, MFCC feature vectors are extracted, and after being transformed by the DNN, the Viterbi algorithm is used to find the optimal state sequence in the HMM to identify the speech content.

[0077] Users can command the system to display specific acupoints or meridians through voice, or perform specific operations on the model. For example, when the user says "display Zusanli acupoint", after the system recognizes the command through the voice recognition function, it highlights the location of the Zusanli acupoint on the human 3D structure model; when the user says "rotate the model", the system performs a rotation operation on the model according to the preset rotation rules.

[0078] After the user issues an interaction instruction (gesture instruction or voice instruction), the system needs to compare it with each reference instruction stored in the database to determine the corresponding interaction operation. Once the reference instruction matching the user's interaction instruction is determined, the interaction operation corresponding to the reference instruction is obtained from the database. The interaction operations stored in the database can be represented by an operation type (such as splitting, scaling, etc.) and operation parameters (such as scaling ratio, rotation angle, etc.). For example, for a scaling operation, the operation parameter may be a scaling factor; for a rotation operation, the operation parameters may be a rotation axis and a rotation angle. According to the obtained interaction operation, the corresponding model operation function is called to operate on the human 3D structure model.

[0079] An electronic device, comprising: a processor; and a memory in which computer program instructions are stored, and when the computer program instructions are run by the processor, the processor is caused to execute the above-mentioned method for visualizing human meridian acupoints based on intelligent AI.

[0080] A computer-readable storage medium for storing a program, and when the program is executed by a processor, it implements the above-mentioned method for visualizing human meridian acupoints based on intelligent AI.

[0081] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0082] The present invention is described with reference to the flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for the functions specified in one block or multiple blocks.

[0083] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements in the processFigure 1 one process or multiple processes and / or blocks Figure 1 the functions specified in one block or multiple blocks.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process or multiple processes and / or blocks Figure 1 one process or multiple processes and / or blocks Figure 1 the steps of the functions specified in one block or multiple blocks.

[0085] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0086] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A visualization method for human meridian acupoints based on intelligent AI, characterized in that, Including the following steps: Construct a three-dimensional virtual human body model based on magnetic resonance imaging and computed tomography; Perform acupoint location and meridian mapping on the constructed three-dimensional virtual human body model and visualize it; Based on quantitative mixed reality technology, display the relative spatial position relationship between human meridians, acupoints and human tissues.

2. The visualization method of human meridian acupoints based on intelligent AI according to claim 1, characterized in that, The constructing of the three-dimensional virtual human body model based on magnetic resonance imaging and computed tomography includes the following steps: Fuse the MRI images obtained by magnetic resonance imaging and the CT images obtained by computed tomography and then reconstruct to obtain a basic image; Process the basic image using a trained fully convolutional neural network to obtain the required tissue and organ structure data, and after preprocessing the required tissue and organ structure data, obtain the preprocessed tissue and organ structure data; Establish a human mixed reality structure model based on three-dimensional modeling technology and the preprocessed tissue and organ structure data; Map the human mixed reality structure model into the three-dimensional virtual human body space coordinate axis system to obtain a three-dimensional virtual human body model.

3. The method for visualizing human meridian acupoints based on intelligent AI according to claim 2, wherein, The fusing the MRI images obtained by magnetic resonance imaging and the CT images obtained by computed tomography and then reconstruct to obtain a basic image includes the following steps: Register the MRI image and the CT image based on feature points to obtain the registered MRI image and the registered CT image; Fuse and then reconstruct the registered MRI image and the registered CT image to obtain a basic image.

4. The visualization method of human meridian acupoints based on intelligent AI according to claim 3, characterized in that The registering the MRI image and the CT image based on feature points to obtain the registered MRI image and the registered CT image includes the following steps: Detect extreme points of the MRI image based on the scale-invariant feature transform algorithm, extract feature points with scale invariance, denoted as the SIFT feature point set of the MRI image; Detect the CT image based on the accelerated robust feature algorithm using the determinant of the Hessian matrix to obtain the SURF feature point set of the CT image; Construct a KD tree for the SIFT feature point set of the MRI image and the SURF feature point set of the CT image, and determine the nearest neighbor relationship based on the Euclidean distance to obtain valid matching pairs; Using the RANSAC algorithm, randomly select several pairs of valid matching pairs from the valid matching pairs as initial sample points, and calculate the preliminary rigid transformation matrix T; Based on the preliminary rigid transformation matrix T, transform all valid matching pairs, and calculate the error between the transformed points and the actual corresponding points: If the error is less than the set threshold, then consider this point as an inlier, otherwise as an outlier; After experiencing the set number of iterations, each iteration randomly selects several pairs of sample points again to calculate the transformation matrix, and counts the number of inliers. Finally, select the transformation matrix with the largest number of inliers as the final registration transformation matrix to align the MRI image and the CT image in spatial position, and obtain the registered MRI image and the registered CT image.

5. The method for visualizing human meridian acupoints based on intelligent AI according to claim 3, wherein The fusing and then reconstructing the registered MRI image and the registered CT image to obtain a basic image includes the following steps: The Laplacian pyramid decomposition algorithm is applied to the registered MRI image and the registered CT image to obtain the MRI Laplacian pyramid image and the CT Laplacian pyramid image, and the number of layers of the MRI Laplacian pyramid image and the CT Laplacian pyramid image is the same; On the low-frequency components, the MRI Laplacian pyramid image and the CT Laplacian pyramid image are fused according to the energy distribution of each level; On the high-frequency components, the MRI Laplacian pyramid image and the CT Laplacian pyramid image are fused according to the regional characteristics; The fused Laplacian pyramid is reconstructed. Starting from the Laplacian image of the highest level, through upsampling and adding the low-frequency components of the next layer, the fused image is gradually restored to obtain the basic image.

6. The visualization method of human meridian acupoints based on intelligent AI according to claim 1, characterized in that The acupoint location and meridian drawing of the constructed three-dimensional virtual human model and visualization thereof include the following steps: Based on the real-time registration technology of optical tracking or inertial tracking, the three-dimensional virtual human model is registered with the real human body in real time; The positions of each acupoint are marked on the registered three-dimensional virtual human model; According to the positions of the acupoints and along the running routes of the meridians, the human meridian system is drawn on the three-dimensional virtual human model; Through visualization technology, each meridian in the human meridian system is visually displayed in different colors and line styles.

7. An intelligent AI-based visualization teaching system for human meridian points, used for the intelligent AI-based visualization method of human meridian points described in any one of claims 1-6, characterized in that It includes a data acquisition module, a model establishment and optimization module, a model display module, and a human-computer interaction module, where: The data acquisition module is used to acquire the data required for the modeling of the human meridian acupoint structure; The model establishment and optimization module is used to use 3Dmax software to establish and optimize a 3D human structure model including the human meridian acupoint system according to the data required for the modeling of the human meridian acupoint structure collected; The model display module is used to display the established 3D human structure model in the form of a holographic image through a HoloLens display device; The human-computer interaction module is used to obtain interaction instructions and perform interaction operations on the 3D human structure model based on the interaction instructions.

8. The visualization teaching system of human meridian acupoints based on intelligent AI according to claim 7, characterized in that, The data required for the modeling of the human meridian acupoint structure includes tissue and organ structure data and meridian acupoint data. The tissue and organ structure data includes the shape, size, position, density of each tissue and organ, and the boundary information between different tissues. The meridian acupoint data includes acupoint positions, acupoint names, and acupoint effects.

9. The visualization teaching system of human meridian points based on intelligent AI according to claim 7, characterized in that, The interaction instructions include gesture instructions and voice instructions, and the interaction operations include splitting, scaling, rotating, moving, and depth cutting.

10. The visualization teaching system of human meridian acupoints based on intelligent AI according to claim 8, characterized in that, Performing interaction operations on the 3D human structure model based on the interaction instructions includes the following steps: Comparing the interaction instructions with each reference instruction stored in the database to determine the reference instruction identical to the interaction instructions; Obtaining the interaction operation corresponding to the reference instruction from the database and operating on the 3D human structure model based on the obtained interaction operation.

Images