Hand acupuncture point recognition method and system, equipment and storage medium

Abstract

The invention relates to a hand acupuncture point recognition method and system, equipment and a storage medium, and belongs to the technical field of acupuncture point data management application. The method comprises the following steps: preprocessing training data containing hand acupuncture point information; carrying out model training on the training data set obtained by preprocessing by adopting an attitude estimation model; based on a hand image obtained by the trained attitude estimation model, performing attitude estimation recognition through the trained attitude estimation model; and outputting a hand image calibrated with acupuncture points according to a posture estimation recognition result. According to the method, a large number of human body acupoint pictures are recognized, an exclusive database is established, and links between finger and hand key point data and hand acupoint information data are established, so that the finger and hand key point data and the hand acupoint information data can mutually access and query data, and real-time interaction is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of acupoint data management application, and in particular relates to a hand acupoint recognition method and system, equipment, and storage medium. Background Art

[0002] Acupoints, as an important concept and treatment method in traditional Chinese medicine, have a history of thousands of years. With the increasing recognition and application of traditional medicine worldwide, the demand for research and application of acupoints is also growing. However, the collection and annotation of hand acupoint data is a complex and time-consuming task that requires the participation of professional doctors or field experts. The difficulty of data collection and the accuracy of annotation may affect the performance and application effect of hand acupoint recognition algorithms. In addition, the positions of hand acupoints of different people may have slight differences in diversity and individual differences, which poses a challenge to the robustness and generalization ability of hand acupoint recognition algorithms. Therefore, algorithms need to be designed and optimized to adapt to the individual differences and diversity of different populations. At the same time, hand acupoint recognition usually requires the use of specific equipment and sensors to obtain information about hand acupoints. However, expensive equipment and sensors may limit the promotion and application scope of hand acupoint recognition technology. With the improvement of people's health awareness, more and more people are beginning to pay attention to physical health and health preservation. Traditional therapies such as acupoint massage and acupuncture are believed to have the effect of regulating the body and alleviating diseases, so the demand for accurate identification and positioning of acupoints is also increasing. The problem of inaccuracy and inconvenience of manual identification of acupoints. Traditional human acupoint recognition relies on professional doctors or masseurs, who determine the location of acupoints through hand feel and experience. However, due to differences in human body structure and the different technical levels of operators, there is a certain error in the accuracy of recognition. Intelligent human acupoint recognition technology can accurately identify the location of human acupoints by combining computer vision and machine learning algorithms.

[0003] Intelligent human acupoint recognition technology analyzes and processes human body images, extracts key feature points, and then uses training models to identify the acupoints represented by these feature points. This technology can not only accurately identify the location of acupoints, but also monitor and track them in real time, providing real-time feedback and guidance. In this way, it can not only improve the accuracy of acupoint recognition, but also save manpower and time costs and improve work efficiency.

[0004] Intelligent human acupoint recognition technology can also be applied to medical research and clinical treatment. By analyzing and mining a large amount of human acupoint data, the correlation between acupoints and diseases can be discovered, and further research can be conducted on the physiological mechanisms of the human body and the mechanisms of disease occurrence. In clinical treatment, intelligent human acupoint recognition technology can help doctors locate acupoints more accurately and improve the accuracy of treatment effects and efficacy evaluation. Summary of the invention

[0005] In view of the above problems, the present invention provides a hand acupoint recognition method and system, device, and storage medium.

[0006] The first object of the present invention is to provide a method for identifying acupoints on a hand, comprising:

[0007] Preprocessing the training data containing the hand acupoint information;

[0008] The posture estimation model is used to train the model on the training data set obtained by preprocessing;

[0009] Based on the hand image obtained by the trained posture estimation model, posture estimation and recognition are performed through the trained posture estimation model;

[0010] According to the results of posture estimation and recognition, a hand image with marked acupoints is output.

[0011] Furthermore, the hand acupoint information includes different hand posture images and acupoint labeling information corresponding to the hand.

[0012] Furthermore, the different hand posture images are captured by multiple single-chip microcomputer cameras with depth sensors.

[0013] Furthermore, the different hand posture images include different palm postures and different palm back postures, and the acupoint marking information corresponding to the hand includes acupoint marking information corresponding to the palm and acupoint marking information corresponding to the palm back.

[0014] Furthermore, the preprocessing includes image denoising, image enhancement and data enhancement.

[0015] Furthermore, the hand image obtained based on the trained posture estimation model is subjected to posture estimation and recognition through the trained posture estimation model, including:

[0016] Obtain hand images through the trained pose estimation model;

[0017] The trained posture estimation model is used to detect and identify the key points of the acquired hand image and mark the key points.

[0018] Returns a tuple of hand image pose estimation solutions containing keypoint location information through the trained pose estimation model;

[0019] The trained posture estimation model locates and labels acupoints on the acquired hand image according to the tuple of the hand image posture estimation solution containing key point position information, and obtains the result of posture estimation and recognition.

[0020] Furthermore, the trained posture estimation model locates and labels acupoints on the acquired hand image according to the tuple of the hand image posture estimation solution containing key point position information, including:

[0021] Using the trained posture estimation model, the tuple of hand image posture estimation solutions containing the key point position information is used to compare the key point positions with the acupoint positions.

[0022] The trained posture estimation model annotates the acupoint names on the acquired hand image according to the comparison result of the position relationship, and returns the image annotated with the acupoint names.

[0023] Furthermore, the trained posture estimation model marks the acupoint names on the acquired hand image according to the comparison result of the position relationship, including:

[0024] If the comparison result of the position relationship is that the key point position coincides with the acupoint position on or within the hand skeleton, the key point is located and the corresponding acupoint name is marked through the trained posture estimation model;

[0025] or,

[0026] If the comparison result of the position relationship is that the key point position is adjacent to the acupoint position outside the hand skeleton, the trained posture estimation model will locate multiple key points adjacent to the acupoint position outside the hand skeleton and mark the corresponding acupoint names.

[0027] The second object of the present invention is to provide a hand acupoint recognition method and a hand acupoint recognition system, comprising:

[0028] Preprocessing module: used to preprocess the training data containing hand acupoint information;

[0029] Training module: used to use the posture estimation model to perform model training on the training data set obtained by preprocessing;

[0030] Posture estimation and recognition module: used to obtain hand images based on the trained posture estimation model, and perform posture estimation and recognition through the trained posture estimation model;

[0031] Output module: used to output hand images with acupoints marked based on the results of posture estimation and recognition.

[0032] A third object of the present invention is to provide an electronic device, comprising: a processor, wherein the processor is coupled to a memory;

[0033] The memory is used to store computer programs;

[0034] The processor is used to execute the computer program stored in the memory so that the electronic device performs the above method.

[0035] The third object of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a program or instruction, and when the program or instruction is executed on a computer, the computer executes the method as described above.

[0036] Beneficial effects of the present invention:

[0037] The present invention provides a hand acupoint recognition method, system, device, and storage medium. Through the key point technology of recognizing the human hand of the posture estimation model, the posture estimation model is trained and learned in combination with training data containing hand acupoint information, and a posture estimation model capable of accurately detecting and identifying key points related to hand acupoints is obtained. The recognition of hand acupoints in images is completed according to the specific positional relationship between the key points and the hand acupoints. In addition, the present invention recognizes a large number of human acupoint images, establishes an exclusive database, and establishes a link between two data (referring to hand key point data and hand acupoint information data) so that they can access and query data with each other, thereby realizing real-time interaction.

[0038] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flow chart of a hand acupoint recognition method according to an embodiment of the present invention is shown;

[0041] Figure 2 A schematic diagram of 3D hand shape correction by multiple cameras according to an embodiment of the present invention is shown;

[0042] Figure 3 A framework diagram of a hand acupoint recognition system according to an embodiment of the present invention is shown;

[0043] Figure 4 A framework diagram of an electronic device according to an embodiment of the present invention is shown;

[0044] In the figure: pre-processing module 100; training module 200; posture estimation and recognition module 300; output module 400; electronic device 500; processor 501; memory 502. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] like Figure 1 As shown, a hand acupoint recognition method according to an embodiment of the present invention includes:

[0047] S1, preprocessing the training data containing hand acupoint information;

[0048] S2, using the posture estimation model to perform model training on the training data set obtained by preprocessing;

[0049] S3, based on the hand image obtained by the trained posture estimation model, performing posture estimation and recognition through the trained posture estimation model;

[0050] S4. Outputting a hand image with acupoints marked thereon according to the result of posture estimation and recognition.

[0051] In certain embodiments of the present invention, in step S1, the hand acupoint information includes different hand posture images and acupoint marking information corresponding to the hand; the different hand posture images include different hand palm postures and different hand back postures, and the hand acupoint marking information corresponding to the hand includes acupoint marking information corresponding to the palm and acupoint marking information corresponding to the back of the palm.

[0052] In an embodiment of the present invention, the acupoint marking information corresponding to the hand is based on the hand acupoint information generated by the same body inch positioning method in traditional Chinese medicine, wherein the same body inch positioning method is a method of converting the width of fingers into inches to calculate the positions of human acupoints.

[0053] In certain embodiments of the present invention, the different hand posture images are captured by multiple single-chip microcomputer cameras with depth sensors. In the example of the present invention, multiple single-chip microcomputer cameras with depth sensors are used to replace computer cameras, and these sensors are calibrated in practice. Compared with local cameras or tensor space, the images obtained by multiple single-chip microcomputer cameras with depth sensors can achieve more accurate 3D world space detection, and each camera needs to detect hand feature points. The position of the camera and the screen direction are different, so a higher hand visible frequency can be achieved, because from the perspective of one camera, the hand may be blocked by another hand, but from the perspective of another camera, there may be no occlusion. At the same time, the single-chip microcomputer is small in size, highly integrated, and light in weight. Various functional components are integrated on the crystal chip. The chip itself is designed according to the requirements of industrial measurement and control environment, the internal wiring is very short, and its anti-industrial noise performance is better than that of general CPUs. The single-chip microcomputer program description, constants and tables are solidified in ROM and are not easy to be damaged. Many signal channels are in the chip, so the reliability is high. The control function is strong. In order to meet the control requirements of the object, the instruction system conditions of the single-chip microcomputer are very rich: branch transfer capability, logical operation of the I / O port and position processing capability are very suitable for special control functions. The high cost performance of the single-chip microcomputer camera can save costs while ensuring the realization of project functions. This greatly improves the accuracy and completeness of the image recognition of the present invention, and on the other hand, it also ensures the accuracy of the two-way translation of the method of the present invention. Figure 2 A schematic diagram of 3D hand shape correction using multiple cameras of the present invention is shown.

[0054] For example, the different hand posture images are captured by a RealSense stereoscopic depth camera, which integrates two infrared sensors (IR StereoCamera), an infrared laser emitter (IR Projector) and a color camera (ColorCamera). The maximum capture distance can reach 10 meters, and it can also be used outdoors in the sun, greatly improving the accuracy and robustness of visual recognition.

[0055] In some embodiments of the present invention, the preprocessing includes image denoising, image enhancement and data enhancement. Exemplarily, the present invention uses OpenCV image processing technology in this embodiment to perform operations such as image denoising, image enhancement and data enhancement. OpenCV is a cross-platform computer vision and machine learning software library that provides a rich variety of image processing and computer vision algorithms. The following is a detailed introduction to the OpenCV image processing module:

[0056] (1) HighGUI: HighGUI is a graphical interface library of OpenCV, which is used to create and display images. It provides various functions such as window management, image display, mouse and keyboard event processing, so that users can interact with images.

[0057] (2) Core: The Core module is the basic module of OpenCV. It contains a variety of basic data structures and algorithms, including matrices, vectors, vectorized operations, mathematical operations, statistical functions, tracking algorithms, etc. This module provides support for other modules and is the basis for other modules of OpenCV.

[0058] (3) Imgcodecs: The Imgcodecs module is used to read and write image files. It supports a variety of image formats, including JPEG, PNG, BMP, GIF, etc. This module provides simple image reading and writing functions, which can read, save and convert images.

[0059] (4) Imgproc: The Imgproc module is an image processing module that contains a variety of image processing algorithms, such as image smoothing, image enhancement, image transformation, image segmentation, etc. This module provides a series of image processing functions that allow developers to perform various operations and processing on images.

[0060] (5) Features2d: The Features2d module is a feature extraction module that contains a variety of feature extraction algorithms, such as SIFT, SURF, ORB, etc. This module provides a series of feature extraction functions that allow developers to extract useful feature points from images.

[0061] (6) Calib3d: The Calib3d module is a stereo vision and 3D reconstruction module that contains a variety of camera calibration and stereo vision algorithms, such as camera calibration, stereo matching, and 3D reconstruction. This module provides a series of stereo vision functions that can perform operations such as camera calibration, stereo matching, and 3D reconstruction.

[0062] (7) Dnn: The Dnn module is a deep learning module that contains a variety of deep learning algorithms, such as convolutional neural networks, recurrent neural networks, etc. This module provides a series of deep learning functions that allow developers to use deep learning algorithms for image analysis and recognition.

[0063] (8) Flann: The Flann module is a high-dimensional approximate nearest neighbor fast search algorithm library, which contains a variety of fast approximate nearest neighbor search algorithms and clustering algorithms. This module provides a series of functions that can perform fast approximate nearest neighbor search and clustering operations.

[0064] (9) Gapi: The Gapi module is an acceleration framework that is used to accelerate conventional image processing operations. It is based on OpenCL and Vulkan and can make full use of multi-core CPU and GPU resources to improve image processing speed and efficiency.

[0065] (10) Ml: The Ml module is a machine learning module that contains a variety of machine learning algorithms, such as statistical classification, regression, clustering, etc. This module provides a series of machine learning functions that can perform various machine learning operations and training.

[0066] (11) Objdetect: The Objdetect module is an object detection module that contains a variety of object detection algorithms, such as Haar cascade classifier, HOG+SVM, etc. This module provides a series of object detection functions that can perform object detection and recognition operations.

[0067] (12) Photo: The Photo module is a computational photography module that contains a variety of image restoration and denoising algorithms, such as image restoration, denoising, color correction, etc. This module provides a series of image restoration and denoising functions that can perform various image restoration and color correction operations.

[0068] (13) Stitching: The Stitching module is an image stitching module that contains a variety of image stitching algorithms, such as feature point finding, image stitching, panoramic stitching, etc. This module provides a series of image stitching functions that can stitch multiple images into a large image.

[0069] (14) Video: The Video module is a video processing module that contains a variety of video processing algorithms, such as motion estimation, background separation, object tracking, etc. This module provides a series of video processing functions that can perform various video processing operations.

[0070] In the embodiment of the present invention, the image detection and molding-related functions of OpenCV are used to load all the classifiers of itself, read the image to be detected, read the picture or video, and judge the extracted picture features through the classifier, and then bring the identified information into the library. OpenCV can be used to detect and recognize faces, recognize objects, classify human behaviors in videos, track camera movements, track moving objects, extract 3D models of objects, generate 3D point clouds from stereo cameras, stitch images together to produce high-resolution images of the entire scene, find similar images from image databases, eliminate red eyes from images taken with flash, follow eye movements, recognize scenery and create tags to overlay them with augmented reality, etc. Under normal relevant circumstances, we need to read, write, display and compile images, which can be implemented in OpenCV through Core, Improc, Highgui, Feature2d, Objectect and other functions. The embodiment of the present invention uses OpenCV technology to realize the reading, display and compilation of images.

[0071] In certain embodiments of the present invention, the posture estimation model in step S2 is a posture estimation model in the MediaPipe hand recognition framework, wherein the MediaPipe hand recognition framework is a real-time hand recognition and processing pipeline that provides a set of powerful tools and algorithms for detecting and tracking hand key points, gestures, and movements, thereby realizing hand recognition and interaction functions. At the same time, it can provide instance-based tracking, that is, maintaining targets across frames. The image frames in the video or camera video stream can be used to provide time for the starting position of the detection box and indicate the area of ​​interest to be tracked, and then calculate the position of the tracked detection box in each frame without running detection for each frame. This allows us to run a higher load but more accurate detection model. The Mediapipe hand recognition framework includes the following technologies:

[0072] (1) Computer vision technology: MediaPipe uses feature detection and descriptor algorithms in computer vision technology, such as SIFT, SURF, HOG, etc., to detect and describe hand key points. By using these algorithms, MediaPipe can detect and track the position and posture of each joint of the hand, such as fingers, wrists, palms, etc. MediaPipe also uses object detection and tracking algorithms, such as the YOLO algorithm based on deep learning, to detect and track hand postures and movements.

[0073] (2) Deep learning technology: MediaPipe uses convolutional neural networks (CNN) and recurrent neural networks (RNN) in deep learning technology to recognize hand postures and movements. CNN is mainly used to detect and locate key points of the hand, while RNN is used to model and predict hand movements. MediaPipe also provides pre-trained models and datasets to help developers quickly build and train hand recognition models for specific applications.

[0074] (3) Image processing technology: MediaPipe uses image enhancement and denoising algorithms in image processing technology, such as histogram equalization and Gaussian filtering, to improve the quality and clarity of hand images. This technology helps improve the contrast, clarity, and details of hand images, thereby improving the accuracy and stability of hand recognition.

[0075] (4) Interaction technology: MediaPipe provides interaction technologies such as wrist detection and tracking, palm orientation detection, and gesture recognition. This technology can help applications interact with users and implement richer and more customized functions, such as hand motion control and gesture recognition.

[0076] (5) Cross-platform support: MediaPipe supports multiple platforms, including mobile devices, desktop computers, servers, etc. This allows developers to easily integrate hand recognition functions into various applications and achieve cross-platform compatibility.

[0077] In an embodiment of the present invention, Mediapipe is used to provide instance-based tracking, that is, maintaining the target across frames. A general detection frame tracking solution can use image frames in a video or camera video stream to provide a timestamp for the start position of the detection frame and indicate the area of ​​interest to be tracked, and then calculate the position of the tracked detection frame in each frame. There is no need to run detection for each frame. These advantages allow the method of the present invention to run a more load-intensive but more accurate detection model while maintaining the lightweight and real-time performance of the pipeline on mobile devices. With the tracking function, the position of the target can be consistent in time, which means that the jitter of the target observed in different frames will be smaller. Image processing and recognition will be clearer. Hand feature point detection and creation of 3D hand data through a single-chip camera with a depth sensor and correction of the 3D hand shape based on three cameras, the next step is to filter the data and perform data smoothing to replicate the precise measurements provided by the colored glove markers, use the hand model and generate colored markers based on the landmarks to obtain the hand posture. You can use the graphics processing framework of MediaPipe to build a decomposable task model of the obtained hand gestures, train the created gesture decomposition model, and finally use the library and tools of MediaPipe to realize the real-time gesture decomposition function and apply it to the corresponding application scenarios. In the embodiment of the present invention, the processing and recognition of images based on the mediapipe algorithm are utilized to make the read-in images, videos and other information clearer. And the acquisition and decomposition of hand gestures are realized by using the library and tools of mediapipe. It can effectively simplify the process of processing audio and video data and reduce the pressure on developers. It can effectively simplify the process of processing audio and video data and reduce the pressure on developers. Through the training of the posture estimation model in Mediapipe, the key points of the human body can be accurately located, and the acupuncture points of the human hand can be accurately marked.

[0078] In some embodiments of the present invention, in step S2, the present invention performs model training on the training data set obtained by preprocessing. For example, during model training, the parameters batch_size (batch size) = 8 (each time the model reads 8 samples from the training data set for training), learning_rate (learning rate) = 0.3, epochs (number of training rounds) = 50 (the entire training data set is repeatedly input into the model 50 times), and the trained posture estimation model is derived. And the model learning rate is controlled at 0.001.

[0079] In some embodiments of the present invention, step S3 includes:

[0080] S3-1, obtaining a hand image through the trained posture estimation model;

[0081] S3-2, detecting and identifying key point positions of the acquired hand image through the trained posture estimation model, and marking the key points;

[0082] S3-3. The trained posture estimation model returns a tuple of the hand image posture estimation solution containing key point location information, where the tuple consists of three parts, including key point location information, confidence score and data identifier. The key point location information refers to the key point coordinates listed for the hand image, such as the base of the index finger: (150, 250), the tip of the index finger: (180, 210), the base of the middle finger: (140, 250), and the tip of the middle finger: (160, 200). The confidence score refers to the reliability of the key point model's prediction of the position of each key point, such as the confidence score of the base of the index finger is 0.95. The data identifier refers to listing the image ID as img_12345.jpg to facilitate tracking and associating key points with the original image.

[0083] S3-4. The trained posture estimation model locates and labels acupoints on the acquired hand image according to the tuple of the hand image posture estimation solution containing key point position information, thereby obtaining the posture estimation and recognition result.

[0084] In the embodiment of the present invention, exemplarily, the hand image S3-1 is a hand image extracted from an image acquired in real time by OpenCV.

[0085] In certain embodiments of the present invention, exemplarily, in step S3-1, the width when captured by the camera and the height when captured by the camera are set, such as the width when captured by the camera is 960 pixels, and the height when captured by the camera is 540 pixels. The mode of the picture is inferred according to MediaPipe and is not specifically limited.

[0086] In certain embodiments of the present invention, exemplarily, in step S3-2, during detection, the detection confidence threshold and the tracking confidence threshold are set separately, such as the minimum detection confidence is set to 0.5, and the tracking confidence threshold is set to 0.5.

[0087] In some embodiments of the present invention, step S3-4 includes:

[0088] S3-4-1, using the trained posture estimation model to estimate the tuple of the hand image posture solution containing the key point position information, and compare the relationship between the key point position and the acupoint position;

[0089] S3-4-2. The trained posture estimation model labels the acupoint names on the acquired hand image according to the comparison result of the position relationship, and returns the image with the labeled acupoint names.

[0090] In some embodiments of the present invention, step S3-4-2 includes:

[0091] i. If the comparison result of the position relationship is that the key point position coincides with the acupoint position on or within the hand skeleton, the key point is located and the corresponding acupoint name is marked through the trained posture estimation model;

[0092] or,

[0093] ii. If the comparison result of the position relationship is that the key point position is adjacent to the acupoint position outside the hand skeleton, the trained posture estimation model will locate multiple key points adjacent to the acupoint position outside the hand skeleton and mark the corresponding acupoint names.

[0094] In certain embodiments of the present invention, step i specifically comprises:

[0095] The key point positions coincide with the acupoint positions in the hand skeleton. The corresponding key point coordinates can be directly located through the trained posture estimation model and marked as the corresponding acupoint name. For example, the Shixuan acupoint is located at the center of the ten fingertips and coincides with the key points 4, 8, 12, 16, and 20 of the fingertips in Mediapipe. Taking the Shixuan acupoint on the index finger as an example, the relevant code is as follows:

[0096] #Mark the location of Shixuan acupoint

[0097] img=cv2.circle(img,(cx8,cy8),radius,(1,240,255),-1)

[0098] #Call custom function to mark acupoint name

[0099] img=paint_chinese_opencv(img,'Shixuan point',(int(hand_21.landmark[8].x*w),int(hand_21.landmark[8].y*h)),(65,105,225)).

[0100] The key point positions coincide with the acupoint positions on the hand skeleton, and the trained posture estimation model locates the key points based on the positional relationship between the key points and the acupoints on the skeleton. For example, the Yuji acupoint is located at the midpoint of the first metacarpal bone. The relevant code is as follows:

[0101] #Calculation of the coordinates of the Yuji acupoint

[0102] cx21,cy21=(cx2+(cx1-cx2) / / 2),(cy2+(cy1-cy2) / / 2)

[0103] #Mark the location of the Yuji acupoint

[0104] img=cv2.circle(img,(cx21,cy21),radius,(255,0,153),cv2.FILLED)

[0105] #Call custom function to mark acupoint name

[0106] img = paint_chinese_opencv(img,'Yuji point',(cx21,cy21),(153,0,255)).

[0107] In certain embodiments of the present invention, step ii specifically comprises:

[0108] The key point position is adjacent to the acupoint position of the hand skeleton. The posture estimation model after training locates multiple key points adjacent to the acupoint position of the hand skeleton and marks the corresponding acupoint names. For example, the front point is located on the radial side of the first interphalangeal joint of the proximal end of the index finger. The relevant code is as follows:

[0109] #Auxiliary distance value (i.e. positioning between multiple adjacent key points. In this example, the distance between two adjacent key points is selected for positioning)

[0110] x_juli0,y_juli0=(cx13-cx17) / / 3,(cy13-cy17) / / 3

[0111] #Calculate the coordinates of the leading point

[0112] x_R_qiantou,y_R_qiantou=(cx6+x_juli0),cy6

[0113] #Mark the acupuncture point position in front

[0114] img=paint_chinese_opencv(img,'heading point',(x_R_qiantou,y_R_qiantou),(255,240,1))

[0115] #Call custom function to mark acupoint name

[0116] img=cv2.circle(img,(x_R_qiantou,y_R_qiantou),radius,(1,240,255),cv2.FILLED).

[0117] like Figure 3 As shown, a hand acupoint recognition system according to an embodiment of the present invention includes:

[0118] Preprocessing module 100: used for preprocessing training data containing hand acupoint information;

[0119] Training module 200: used to perform model training on the training data set obtained by preprocessing using a posture estimation model;

[0120] Posture estimation and recognition module 300: used to obtain a hand image based on a trained posture estimation model, and perform posture estimation and recognition through the trained posture estimation model;

[0121] Output module 400: used to output a hand image with acupoints marked according to the result of posture estimation and recognition.

[0122] like Figure 4 As shown, some embodiments of the present invention provide an electronic device, the electronic device 500, comprising: a processor 501, the processor 501 is coupled to a memory 502;

[0123] The memory 502 is used to store computer programs;

[0124] The processor 501 is configured to execute the computer program stored in the memory 502 so that the electronic device executes the method described in the above embodiment.

[0125] In certain embodiments of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a program or instruction. When the program or instruction is executed on a computer, the computer executes the method described in the above embodiments.

[0126] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an electronic device or a device.

[0127] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hand acupoint recognition method, characterized in that: include: Preprocessing the training data containing the hand acupoint information; The posture estimation model is used to train the model on the training data set obtained by preprocessing; Based on the hand image obtained by the trained posture estimation model, posture estimation and recognition are performed through the trained posture estimation model; According to the results of posture estimation and recognition, a hand image with marked acupoints is output.

2. A hand acupoint recognition method according to claim 1, characterized in that: The hand acupoint information includes different hand posture images and acupoint marking information corresponding to the hand.

3. A hand acupoint recognition method according to claim 2, characterized in that: The different hand posture images are captured by multiple single-chip cameras with depth sensors.

4. A hand acupoint recognition method according to claim 2, characterized in that: The different hand posture images include different palm postures and different palm back postures, and the acupoint marking information corresponding to the hand includes acupoint marking information corresponding to the palm and acupoint marking information corresponding to the palm back.

5. A hand acupoint recognition method according to claim 1, characterized in that: The preprocessing includes image denoising, image enhancement and data enhancement.

6. A hand acupoint recognition method according to any one of claims 1 to 5, characterized in that: The hand image obtained based on the trained posture estimation model, and posture estimation and recognition performed by the trained posture estimation model, include: Obtain hand images through the trained pose estimation model; The trained posture estimation model is used to detect and identify the key points of the acquired hand image and mark the key points. Returns a tuple of hand image pose estimation solutions containing keypoint location information through the trained pose estimation model; The trained posture estimation model locates and labels acupoints on the acquired hand image according to the tuple of the hand image posture estimation solution containing key point position information, and obtains the result of posture estimation and recognition.

7. A hand acupoint recognition method according to claim 6, characterized in that: The trained posture estimation model locates and labels acupoints on the acquired hand image according to the tuple of the hand image posture estimation solution containing key point position information, including: Using the trained posture estimation model, the tuple of hand image posture estimation solutions containing the key point position information is used to compare the key point positions with the acupoint positions. The trained posture estimation model annotates the acupoint names on the acquired hand image according to the comparison result of the position relationship, and returns the image annotated with the acupoint names.

8. A hand acupoint recognition method according to claim 7, characterized in that: The trained posture estimation model marks the acupoint names on the acquired hand image according to the comparison result of the position relationship, including: If the comparison result of the position relationship is that the key point position coincides with the acupoint position on or within the hand skeleton, the key point is located and the corresponding acupoint name is marked through the trained posture estimation model; or, If the comparison result of the position relationship is that the key point position is adjacent to the acupoint position outside the hand skeleton, the trained posture estimation model will locate multiple key points adjacent to the acupoint position outside the hand skeleton and mark the corresponding acupoint names.

9. A hand acupoint recognition system, characterized in that: include: Preprocessing module: used to preprocess the training data containing hand acupoint information; Training module: used to use the posture estimation model to perform model training on the training data set obtained by preprocessing; Posture estimation and recognition module: used to obtain hand images based on the trained posture estimation model, and perform posture estimation and recognition through the trained posture estimation model; Output module: used to output hand images with acupoints marked based on the results of posture estimation and recognition.

10. An electronic device, characterized in that: include: a processor coupled to the memory; The memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or an instruction, and when the program or the instruction is executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.

Images