Human body acupuncture point positioning training data acquisition and rapid labeling method based on image completion

Abstract

The invention discloses a human body acupoint positioning training data acquisition and rapid labeling method based on image completion, which is characterized in that after markers are pasted at acupoint positions, acupoints represented by the markers and positions of the acupoints are identified by using a detection network, and accurate and automatic acupoint positioning is realized. And generating a marker positioning image capable of completely covering the marker by using coordinate information and a target detection bounding box in the acupuncture point tag, thereby explicitly marking marker areas in the image and removing the areas. And complementing the image from which the marker is removed through an image complementing algorithm, reconstructing vivid and natural skin textures, then carrying out edge smoothing processing, and recovering an image without a digital marker. And finally, associating the acupoint coordinates and category labels obtained by quick labeling with the images without digital markers obtained by image completion, and generating high-quality human body acupoint positioning training data. According to the method disclosed by the invention, only one-time data acquisition is needed, so that the method is rapid and the data labeling is more accurate.

Description

Technical Field

[0001] The present invention relates to the technical fields of deep learning technology and biomedical image processing, and in particular to a method for collecting and quickly labeling human acupoint positioning training data based on image completion. Background Art

[0002] Acupoints in traditional Chinese medicine are the core of treatment methods such as acupuncture and massage, and have important medical value and broad application prospects. With the development of modernization and digitalization of traditional Chinese medicine, standardized acupoint datasets are essential for scientific research, education and clinical applications. By constructing standardized and accurate acupoint data, we can promote the development of intelligent diagnosis and treatment systems for traditional Chinese medicine, realize the recommendation of personalized treatment plans, and improve the effectiveness and operability of traditional Chinese medicine clinical practice. However, the construction of human acupoint recognition datasets currently faces multiple difficulties. First, the annotation of acupoint locations is highly dependent on the experience of traditional Chinese medicine practitioners, and the annotation inconsistency caused by individual differences and subjective judgment is high. Secondly, traditional manual annotation is time-consuming and laborious. It takes about 6 minutes for a professional traditional Chinese medicine practitioner to annotate a human image with 50 acupoints, and about 10 hours to annotate 100 data. Moreover, the current human acupoint recognition data lacks diversity and representativeness, and it is difficult to cover diverse populations of different genders, ages and body shapes, as well as acupoint images in various postures and angles, which limits the generalization ability of artificial intelligence models. Therefore, it is urgent to use advanced artificial intelligence technologies, especially computer vision and deep learning, to provide technical support for the rapid and accurate production of high-quality acupoint data, paving the way for the intelligent development of traditional Chinese medicine.

[0003] With the development of computer vision and artificial intelligence, more and more researchers have tried to apply deep learning technology to acupuncture point recognition and labeling to improve accuracy and efficiency, such as target detection (SSD, FasterR-CNN) and image segmentation (UNet, Mask R-CNN). However, these methods have limited accuracy in detecting small targets, long training time and high computational complexity. In the prior art, a patent (application number 2024113234665) discloses a technical method for acupuncture point labeling by detecting acupuncture point images with labels through YOLO, and then collecting a set of acupuncture point images without labels. However, the data accuracy of this method is easily affected by changes in the body position of the person being collected, and it requires two data collections, which is time-consuming. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for collecting and quickly labeling human acupoint positioning training data based on image completion. After pasting markers at the acupoint positions, a detection network is used to identify the acupoints represented by these markers and their positions, so as to achieve accurate and automatic acupoint positioning. In addition, the image of the acupoint with markers is directly repaired by image completion technology to obtain realistic acupoint images without markers, so as to generate high-quality human acupoint positioning training data. This method only requires one data collection, which is not only fast, but also more accurate in data labeling.

[0005] To achieve the above purpose, the technical solution provided by the present invention is:

[0006] The human acupoint location training data collection and rapid annotation method based on image completion includes:

[0007] The person being collected takes off his clothes according to the instructions of the Chinese medicine practitioner to fully expose the acupuncture points. The professional Chinese medicine practitioner accurately sticks numbered markers on the acupuncture points of the person being collected. The numbers correspond to the acupuncture point categories one by one, and the center point of the marker coincides with the acupuncture point.

[0008] A multi-angle sampling method is used to collect image data to obtain images of human body parts with digital markers attached;

[0009] Create a detection network and train it with the collected images with digital markers. Then use the trained detection network to quickly and automatically annotate the images with digital markers to obtain accurate acupuncture point coordinates and category labels, including the coordinate information of the center point of the marker and the category label of the corresponding acupuncture point.

[0010] The acupuncture point coordinates and category labels generated by the detection network and the corresponding image with digital markers are input into the marker localization module. The marker center coordinates in the acupuncture point label and the length and width of the target bounding box are used to generate a marker localization image that can completely cover the marker, so as to explicitly mark the marker area in the image and remove these areas;

[0011] The image with the removed markers is input into the image completion algorithm for completion, thus reconstructing a realistic and natural skin texture.

[0012] The completed image is optimized using the edge smoothing method of the completed area, so that the edge of the completed texture in the marker area is smoother and the image is more realistic, thereby restoring the human acupuncture point image without digital markers;

[0013] The acupuncture point coordinates and category labels generated by the association detection network and the image without digital circular markers generated by the image completion algorithm are used to obtain the human acupuncture point positioning training data with precise labels.

[0014] Furthermore, a detection network is created and trained with the collected images with digital markers, including:

[0015] Create a detection network;

[0016] A small number of samples are randomly selected from the collected images with digital markers as a training set for the detection network to identify the markers, and the coordinate positions of the markers and their corresponding acupoint categories are marked on the training set;

[0017] The detection network is trained using a labeled marker recognition training set. By detecting markers with obvious features, the detection network can quickly learn how to locate the center coordinates of the markers, accurately identify the numbers on the markers, and mark the corresponding acupuncture point categories.

[0018] Furthermore, the detection network created is a YOLO detection network, which is based on the YOLOv5 network model and includes three parts: Backbone, Neck, and Head;

[0019] in,

[0020] Backbone uses deep learning modules and mechanisms to achieve deep learning and expression of image features through layer-by-layer feature extraction and multi-scale feature fusion;

[0021] Neck adopts the structure of FPN+PAN. FPN uses a top-down architecture with lateral connections to build high-level semantic feature maps at all scales. PAN adds a bottom-up route to supplement and strengthen positioning information.

[0022] The main part of the Head is three Detect detectors, which use grid-based anchors to perform target detection on feature maps of different scales;

[0023] The loss function of the YOLOv5 network model consists of three losses: confidence loss L obj , classification loss L class And the prediction box position loss L box , as shown in formula (1):

[0024] L Total =L obj +L class +L box (1)

[0025] Confidence loss L obj As shown in formula (2); where S 2 Indicates that the image is divided into S×S grids, c is the confidence obtained by the model, a is the ath prediction box in a certain grid, represents the probability that the target exists at the i, jth position, and On the contrary, n and λ o are the confidence loss coefficients without target and with target respectively;

[0026]

[0027] Classification loss L class As shown in formula (3), N represents the total number of categories, x i is the predicted value of the current category, y i is the probability of the current category after the activation function, It is the true value of the current category;

[0028]

[0029] Prediction box position loss function L box As shown in formula (4), CIoU is calculated by taking into account the overlapping area, center point distance, and aspect ratio; CIoU is shown in formula (5), where IoU is the intersection over union ratio of the predicted box and the true box, C and c gt is the center point position of the predicted box and the real box, ρ is the Euclidean distance between the two center points, and d is the diagonal distance of the minimum closure area that can contain both boxes at the same time; a penalty factor αv is added to CIoU, as shown in (6, 7), w, h and w gt ,h gt are the position parameters of the predicted box and the real box;

[0030] L box =1-CIoU (4)

[0031]

[0032] Furthermore, the detection network labels the acupoint category by identifying the numbers on the markers, and detects the center point of the marker to mark the coordinate position of the acupoint.

[0033] Further, the marker with numbers is a circular number marker;

[0034] The marker localization module uses the acupuncture point coordinates and category labels obtained by the detection network to accurately find the circular digital markers in the image, calculates the center and radius of the marker area through the coordinates and bounding box provided by the acupuncture point label, and generates a binary marker localization image that can completely cover the marker. The marker localization image is then used to remove the marker area in the original image.

[0035] Further, the completed image is optimized using the completed region edge smoothing method, including:

[0036] Assume that the size of the completed image φ(x,y) is m×n, and the completed area φ rec The size of (x, y) is a×b, and the other background areas φ back (x,y) is divided into eight parts, namely φ back1 (x,y)~φ back8 (x,y);

[0037] First, construct two matrices J and K:

[0038]

[0039] The matrices J and K are divided into column matrices [J1J2J3] and row matrices [K1K2K3] respectively. T , J1, J2 and J3 columns correspond to φ back1 ,φ back2 and φ back3 The row matrix K1, K2, K3 corresponds to φ back1 ,φ back4 and φ back6 The column matrix of

[0040] So the gradient of φ(x,y) in the x direction is expressed as:

[0041]

[0042] The gradient of v(x,y) in the y direction is expressed as:

[0043]

[0044] Among them, H in formulas (9) and (10) rec and Q rec Details are as follows:

[0045]

[0046] Next, calculate the sum of the gradients of φ(x,y) in the x and y directions:

[0047] TV[φ(x,y)] 2 =Tr[[Jφ(x,y)] T [Jφ(x,y)]]+Tr[[φ(x,y)K][φ(x,y)K] T ],(12)

[0048] Then, by completing the region φ rec The derivative of (x, y) is equal to 0 to solve the minimization problem in formula (13), as shown below:

[0049]

[0050] Formula (13) can be rewritten as follows through formulas (8) to (12):

[0051]

[0052] make Solving the matrix equation, we get φ rec (x,y);

[0053] Finally, using φ rec (x, y) replaces the edge of the completed area originally generated by the image completion algorithm, that is, the smooth edge of the completed area is obtained.

[0054] Compared with the prior art, the principles and advantages of this technical solution are as follows:

[0055] 1. Fast labeling speed: The present invention uses a deep learning detection network for data labeling, which greatly shortens the manpower and time required for traditional manual labeling.

[0056] 2. High labeling accuracy: Automatic labeling through the detection network reduces the subjective errors caused by manual labeling and ensures the consistency and accuracy of the labeling of each acupoint location.

[0057] 3. High data diversity: This process can be repeated on subjects of different genders, ages, and body types to build a more representative and diverse dataset.

[0058] 4. Strong adaptability: The method is suitable for acupoint labeling under different human postures, body shapes and angles. It has strong adaptability and generalization ability and can cover a variety of data scenarios.

[0059] 5. Unified standardization: By matching the names of acupuncture points with digital markers, data standardization is ensured, making it easier to compare and integrate different data.

[0060] 6. High degree of automation: The detection network is used to generate acupuncture point coordinates and category labels, which greatly reduces manual intervention and improves the efficiency of data collection.

[0061] 7. Reduce labeling costs: Compared with the method that completely relies on manual labeling by professionals, this semi-automatic labeling method can significantly reduce time and labor costs.

[0062] 8. Improve data quality: After removing markers through image completion, the image quality is closer to the natural state, which helps the model learn more realistic acupoint features and improves the practical application performance of the model.

[0063] 9. Easy data management and updating: The use of markers and digital management make data easy to expand and update, and new samples and annotation information can be added quickly, improving the flexibility of data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the services required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Figure 1 The figure is a principle flow chart of the method for collecting and quickly labeling human acupoint location training data based on image completion of the present invention;

[0066] Figure 2 It is a schematic diagram of the human body acupoint positioning training data collection and rapid annotation method based on image completion of the present invention;

[0067] Figure 3 It is a structural diagram of the multi-angle image acquisition mobile platform (A1 is a display showing the acquired image with digital graphic marks, B1 is the target object to be acquired, C1 is a depth camera, and D1 is an industrial robot arm);

[0068] Figure 4 Flow chart for acupuncture point labeling using the YOLO detection network;

[0069] Figure 5 It is a marker removal and completion method based on the Large Mark Inpainting (LaMa) image completion algorithm. DETAILED DESCRIPTION

[0070] The present invention will be further described below in conjunction with specific embodiments:

[0071] like Figure 1 As shown, the method for collecting and quickly labeling human acupoint location training data based on image completion described in this embodiment includes the following steps:

[0072] S1: The person being collected takes off his clothes according to the instructions of the Chinese medicine practitioner to fully expose the acupoints. The professional Chinese medicine practitioner will accurately paste numbered markers (such as ①②, etc.) on the acupoints of the person being collected. The numbers and acupoint categories must correspond one to one (such as ① represents Hegu acupoint, ② represents Yemen acupoint), and the center point of the marker must also coincide with the acupoint.

[0073] S1-1: In view of the wide variety of human acupuncture points, the different locations of different acupuncture points, and the difficulty of Chinese medicine in marking images, a "digital marking acupuncture point marking method" is designed, using markers with numbers to represent acupuncture points. Among them, each number corresponds to a specific human acupuncture point, and the center point of the marker corresponds to the acupuncture point (assuming there are K acupuncture points, the numbers are marked as 1, 2, ..., K). This method allows Chinese medicine practitioners to place markers on human acupuncture points, and then use annotation software to mark the markers on the image. This method replaces Chinese medicine practitioners directly marking acupuncture points on the image, which allows Chinese medicine practitioners to mark the accurate acupuncture point locations more intuitively. In the later stage, detection networks (such as YOLO) can also be used to identify these markers, and then identify the corresponding acupuncture point locations and categories, so as to complete the rapid annotation of acupuncture point images.

[0074] S1-2: If Figure 2 As shown, this embodiment realizes acupoint location and recognition by using the YOLO detection network with the help of the digital and center coordinates of the marker, generates a marker location image with the help of the marker coordinates and detection bounding box obtained by YOLO to locate and remove the marker, and restores realistic and natural skin texture through the Large Mark Inpainting (LaMa) image completion algorithm. Finally, the acupoint coordinates and category labels generated by the detection network are associated with the image file generated by the image completion algorithm to obtain labeled human acupoint location training data.

[0075] S2: This embodiment designs a mobile platform for automatic image data acquisition, including components such as an industrial robot arm and a depth camera. The robot arm moves along a preset trajectory. During the movement, the depth camera on the robot arm takes a fixed-point shot at a specified position for each step, and performs multi-view image acquisition of the object to be captured with a marker. The person being captured needs to maintain his or her position during the operation of the device to ensure the quality of the captured image. Images with digital markers are obtained through this multi-view image acquisition method.

[0076] S2-1: Figure 3 As shown, in order to collect diverse data and high-quality images, this embodiment designs a mobile platform, including components such as an industrial robotic arm and a depth camera. The depth camera is fixed on the robotic arm, and the robotic arm drives the depth camera to collect images of human acupoints.

[0077] S2-2: This embodiment plans the motion path of the robot arm on the mobile platform, and designs a set of grid-like stop points to evenly cover the area above the acupoint collector. When collecting acupoints, the robot arm traverses each stop point according to the preset path, and collects a human acupoint image at each stop point. In this way, randomized human acupoint image data containing markers at different heights and angles are collected.

[0078] S3: A small number of samples are randomly selected from the collected image set as the training set for the detection network to identify markers, and the Labelme annotation software is used to annotate the marker coordinate positions (acupoints) and their corresponding acupoint categories on the training set.

[0079] S4: This embodiment uses a labeled marker recognition training set to train the YOLO detection network. By detecting markers with obvious features for training, the YOLO deep learning target detection network can quickly learn how to locate the center point coordinates (acupoints) of the markers, and accurately identify the numbers on the markers and mark the corresponding acupoint categories.

[0080] S4-1: Input a small number of images with digital markers marked in S3 into the YOLO detection network to learn the rapid recognition and positioning of digital markers. Since the automatic sampling device shoots human acupuncture points at multiple angles, the scales of the markers in images from different perspectives are inconsistent. The multi-scale accurate detection characteristics of YOLO and the fast performance of the model detection rate are well suited to the design requirements of the present invention.

[0081] S4-2: The YOLO detection network is based on the YOLOv5 network model, which is a real-time single-stage target detection algorithm with the advantages of high detection accuracy and fast running speed. Its maximum image detection speed can reach 140 frames per second, which can greatly meet the needs of fast and high-precision image detection. Figure 4 As shown, the YOLOv5 network is mainly divided into three parts: Backbone, Neck and Head.

[0082] Backbone in YOLOv5 uses a series of efficient deep learning modules and mechanisms to achieve deep learning and expression of image features through layer-by-layer feature extraction and multi-scale feature fusion. Among them, the C3 structure can reduce computational redundancy while maintaining the diversity of feature information and the lightweight of the model, enabling Backbone to strike a balance between accuracy and efficiency, providing a strong feature foundation for target detection tasks.

[0083] Neck uses the FPN+PAN structure. FPN uses a top-down architecture with lateral connections to build high-level semantic feature maps at all scales, which is a classic structure of feature pyramid. Since FPN passes through multiple layers of the network, the underlying target information is blurred. PAN adds a bottom-up route to make up for and strengthen this part of the positioning information.

[0084] The main part of the head is the three Detect detectors, which are the process of performing target detection on feature maps of different scales using grid-based anchors. When the input is 640×640, the feature maps at three scales are: 80×80, 40×40, and 20×20.

[0085] S4-3: The loss function of YOLOv5 consists of three losses: confidence loss L obj , classification loss L class And the prediction box position loss L box , as shown in formula (1):

[0086] L Total =L obj +L class +L box (1)

[0087] Confidence loss L obj As shown in formula (2); where S 2 Indicates that the image is divided into S×S grids, c is the confidence obtained by the model, a is the ath prediction box in a certain grid, represents the probability that the target exists at the i, jth position, and On the contrary, n and λ o are the confidence loss coefficients without target and with target respectively;

[0088]

[0089] Classification loss L class As shown in formula (3). N represents the total number of categories, x i is the predicted value of the current category, y i is the probability of the current category after the activation function, is the true value of the current category.

[0090]

[0091] Prediction box position loss function L box As shown in formula (4), CIoU (Complete-IoU Loss) is used to calculate the overlap area, center point distance, and aspect ratio, which are important geometric factors in the target box regression loss. CIoU is shown in formula (5), where IoU is the intersection-over-union ratio of the predicted box and the true box, C and c gt is the center point position of the predicted box and the real box, ρ is the Euclidean distance between the two center points, and d is the diagonal distance of the minimum closure area that can contain both boxes at the same time; a penalty factor αv is added to CIoU, as shown in (6, 7), w, h and w gt ,hgt are the position parameters of the predicted box and the real box;

[0092] L box =1-CIoU (4)

[0093]

[0094] S5: The trained YOLO detection network quickly and automatically annotates the remaining large number of images with digital markers to obtain accurate acupuncture point coordinates and category labels, including the coordinate information of the center point of the marker (the location of the acupuncture point) and the category label of the corresponding acupuncture point.

[0095] S5-1: This embodiment combines the YOLO detection network with the "digital marking acupoint labeling method" to mark the acupoint category by identifying the numbers on the marker and to mark the coordinate position of the acupoint by detecting the center point of the marker.

[0096] S6: This embodiment uses a marker positioning module to assist the marker removal process. The acupuncture point coordinates and category labels generated by the detection network and the corresponding image with digital markers need to be input into the marker positioning module, and the marker center point coordinates in the acupuncture point label and the length and width of the target bounding box are used to generate a marker positioning image that can completely cover the marker. In this way, the marker areas in the image are explicitly marked and these marker areas are removed.

[0097] S6-1: The marker positioning module can accurately find the circular digital markers in the image with the help of the acupuncture point coordinates and category labels obtained by the YOLO detection network, calculate the center and radius of the marker area through the coordinates and bounding box provided by the acupuncture point label, and generate a binary marker positioning image that can completely cover the marker. The marker positioning image is then used to remove the marker area in the original image.

[0098] S7: In this embodiment, the Large Mark Inpainting (LaMa) image completion algorithm is used to complete the image after the marker is occluded. The image after the marker is removed is input into the LaMa image completion algorithm to reconstruct a realistic and natural skin texture;

[0099] S7-1: Figure 5 As shown, the present invention inputs the image after removing the marker into the image completion algorithm. The deep learning image completion algorithm can generate realistic and natural skin texture on the marker. The image completion algorithm automatically removes the marker, effectively avoiding the errors and cumbersome operations caused by manual label removal and resampling.

[0100] S7-2: LaMa uses an improved U-Net architecture that can effectively extract local and global features of images. By combining downsampling and upsampling paths, the model can gradually learn information from local to global. The network structure of LaMa can be divided into three main parts: encoder, fast Fourier residual fast, and decoder. The introduction of Fourier transform in the traditional convolutional layer enables the model to capture a wider range of frequency features, especially long-range dependencies. Therefore, in the LaMa network, the Fourier convolution layer can help the model learn the global structure of the image faster, which helps to converge faster.

[0101] S7-3: In order to better handle occluded areas, LaMa uses a composite loss function that combines perceptual loss, adversarial loss, and Fourier loss.

[0102] Among them, the perceptual loss is based on the deep features extracted by the pre-trained convolutional neural network (such as the VGG network), which is used to measure the difference between the generated image and the original image in high-level semantic features. It can help the model generate more natural restoration results and effectively improve the visual quality of the repaired area.

[0103] Adversarial loss introduces the idea of ​​generative adversarial networks (GANs), using a discriminator to evaluate the authenticity of the restored image. The generator is optimized by learning the ability to deceive the discriminator, which improves the naturalness of the generated images.

[0104] Fourier loss uses Fourier transform to calculate the difference between high-frequency and low-frequency parts, which promotes the model to generate natural transitions and delicate textures in the frequency domain. The introduction of Fourier loss helps LaMa maintain a balance between global structure and local details, thereby accelerating convergence.

[0105] The composite loss function that combines the above three losses can not only preserve the structural information and high-frequency details of the image, but also improve the visual quality of the restoration results.

[0106] S8: Use the edge smoothing method of the completed region to optimize the completed image, so that the edge of the completed texture in the marker region is smoother and the image is more realistic, thereby restoring the human acupuncture point image without digital markers;

[0107] The optimization of the completed image using the completed area edge smoothing method includes:

[0108] Assume that the size of the completed image φ(x,y) is m×n, and the completed area φ rec The size of (x, y) is a×b, and the other background areas φ back (x,y) is divided into eight parts, namely φ back1(x,y)~φ back8 (x,y);

[0109] First, construct two matrices J and K:

[0110]

[0111] The matrices J and K are divided into column matrices [J1J2J3] and row matrices [K1K2K3] respectively. T , J1, J2 and J3 columns correspond to φ back1 ,φ back2 and φ back3 The row matrix K1, K2, K3 corresponds to φ back1 ,φ back4 and φ back6 The column matrix of

[0112] So the gradient of φ(x,y) in the x direction is expressed as:

[0113]

[0114] The gradient of φ(x,y) in the y direction is expressed as:

[0115]

[0116] Among them, H in formulas (9) and (10) rec and Q rec Details are as follows:

[0117]

[0118] Next, calculate the sum of the gradients of φ(x,y) in the x and y directions:

[0119] TV[φ(x,y)] 2 =Tr[[Jφ(x,y)] T [Jφ(x,y)]]+Tr[[φ(x,y)K][φ(x,y)K] T ],(12)

[0120] Then, by completing the region φ rec The derivative of (x, y) is equal to 0 to solve the minimization problem in formula (13), as shown below:

[0121]

[0122] Formula (13) can be rewritten as follows through formulas (8) to (12):

[0123]

[0124] make Solving the matrix equation, we get φ rec (x,y);

[0125] Finally, using φ rec (x, y) replaces the edge of the completed area originally generated by the image completion algorithm, that is, the smooth edge of the completed area is obtained.

[0126] S9: Associate the acupuncture point coordinates and category labels containing markers generated by the YOLO detection network with the images without digital markers generated by the LaMa image completion algorithm to obtain human acupuncture point positioning training data with precise labels.

[0127] The difference between the image with digital markers directly obtained by image data acquisition and the image without digital markers generated by the LaMa graph completion algorithm is only the presence or absence of markers. So in theory, the label categories and coordinates corresponding to them are consistent, that is, the acupoint labels corresponding to the two images are consistent. Therefore, by associating the acupoint points and category labels with the image files without markers, we can obtain the human acupoint positioning training data with precise labels.

[0128] The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for collecting and quickly labeling human acupoint location training data based on image completion, characterized in that: include: The person being collected takes off his clothes according to the instructions of the Chinese medicine practitioner to fully expose the acupuncture points. The professional Chinese medicine practitioner accurately sticks numbered markers on the acupuncture points of the person being collected. The numbers correspond to the acupuncture point categories one by one, and the center point of the marker coincides with the acupuncture point. A multi-angle sampling method is used to collect image data to obtain images of human body parts with digital markers attached; Create a detection network and train it with the collected images with digital markers. Then use the trained detection network to quickly and automatically annotate the images with digital markers to obtain accurate acupuncture point coordinates and category labels, including the coordinate information of the center point of the marker and the category label of the corresponding acupuncture point. The acupuncture point coordinates and category labels generated by the detection network and the corresponding image with digital markers are input into the marker localization module. The marker center coordinates in the acupuncture point label and the length and width of the target bounding box are used to generate a marker localization image that can completely cover the marker, so as to explicitly mark the marker area in the image and remove these areas; The image with the removed markers is input into the image completion algorithm for completion, thus reconstructing a realistic and natural skin texture. The completed image is optimized using the edge smoothing method of the completed area, so that the edge of the completed texture in the marker area is smoother and the image is more realistic, thereby restoring the human acupuncture point image without digital markers; The acupuncture point coordinates and category labels generated by the association detection network and the image without digital circular markers generated by the image completion algorithm are used to obtain the human acupuncture point positioning training data with precise labels.

2. The method for collecting and quickly labeling human acupoint location training data based on image completion according to claim 1, characterized in that: Create a detection network and train it on collected images with digital markers, including: Create a detection network; A small number of samples are randomly selected from the collected images with digital markers as a training set for the detection network to identify the markers, and the coordinate positions of the markers and their corresponding acupoint categories are marked on the training set; The detection network is trained using a labeled marker recognition training set. By detecting markers with obvious features, the detection network can quickly learn how to locate the center coordinates of the markers, accurately identify the numbers on the markers, and mark the corresponding acupuncture point categories.

3. The method for collecting and quickly labeling human acupoint location training data based on image completion according to claim 1, characterized in that: The detection network created is the YOLO detection network, which is based on the YOLOv5 network model and consists of three parts: Backbone, Neck, and Head; in, Backbone uses deep learning modules and mechanisms to achieve deep learning and expression of image features through layer-by-layer feature extraction and multi-scale feature fusion; Neck adopts the structure of FPN+PAN. FPN uses a top-down architecture with lateral connections to build high-level semantic feature maps at all scales. PAN adds a bottom-up route to supplement and strengthen positioning information. The main part of the Head is three Detect detectors, which use grid-based anchors to perform target detection on feature maps of different scales; The loss function of the YOLOv5 network model consists of three losses: confidence loss L obj , classification loss L class And the prediction box position loss L box , as shown in formula (1): L Total =L obj +L class +L box (1) Confidence loss L obj As shown in formula (2); where S 2 Indicates that the image is divided into S×S grids, c is the confidence obtained by the model, a is the ath prediction box in a certain grid, represents the probability that the target exists at the i, jth position, and On the contrary, n and λ o are the confidence loss coefficients without target and with target respectively; Classification loss L class As shown in formula (3), N represents the total number of categories, x i is the predicted value of the current category, y i is the probability of the current category after the activation function, It is the true value of the current category; Prediction box position loss function L box As shown in formula (4), CIoU is calculated by taking into account the overlapping area, center point distance, and aspect ratio; CIoU is shown in formula (5), where IoU is the intersection over union ratio of the predicted box and the true box, C and c gt is the center point position of the predicted box and the real box, ρ is the Euclidean distance between the two center points, and d is the diagonal distance of the minimum closure area that can contain both boxes at the same time; a penalty factor αυ is added to CIoU, as shown in (6, 7), w, h and w gt ,h gt Are the position parameters of the predicted box and the real box; L box =1-CIoU (4) 4. According to the method for collecting and quickly labeling human acupoint positioning training data based on image completion in claim 1, the detection network labels the acupoint category by identifying the numbers on the markers, and marks the coordinate position of the acupoint by detecting the center point of the marker.

5. The method for collecting and quickly labeling human acupoint location training data based on image completion according to claim 1, characterized in that: Markers with numbers are circular number markers; The marker localization module uses the acupuncture point coordinates and category labels obtained by the detection network to accurately find the circular digital markers in the image, calculates the center and radius of the marker area through the coordinates and bounding box provided by the acupuncture point label, and generates a binary marker localization image that can completely cover the marker. The marker localization image is then used to remove the marker area in the original image.

6. The method for collecting and quickly labeling human acupoint location training data based on image completion according to claim 1, characterized in that: The optimization of the completed image using the completed area edge smoothing method includes: Assume that the size of the completed image φ(x,y) is m×n, and the completed area φ rec The size of (x, y) is a×b, and the other background areas φ back (x,y) is divided into eight parts, namely φ back1 (x,y)~φ back8 (x,y); First, construct two matrices J and K: The matrices J and K are divided into column matrices [J1J2J3] and row matrices [K1K2K3] respectively. T , J1, J2 and J3 columns correspond to φ back1 ,φ back2 and φ back3 The row matrix K1, K2, K3 corresponds to φ back1 ,φ back4 and φ back6 The column matrix of So the gradient of φ(x,y) in the x direction is expressed as: The gradient of v(x,y) in the y direction is expressed as: Among them, H in formulas (9) and (10) rec and Q rec Details are as follows: Next, calculate the sum of the gradients of φ(x,y) in the x and y directions: TV[φ(x,y)] 2 =Tr[[jφ(x,y)] T [Jφ(x,y)]]+TR[[φ(x,y)K][φ(x,y)K] T ], (12) Then, by completing the region φ rec The derivative of (x, y) is equal to 0 to solve the minimization problem in formula (13), as shown below: Formula (13) can be rewritten as follows through formulas (8) to (12): make Solving the matrix equation, we get φ rec (x,y); Finally, using φ rec (x, y) replaces the edge of the completed area originally generated by the image completion algorithm, that is, the smooth edge of the completed area is obtained.

Images