A visual localization method for human organs and meridians based on deep learning

Abstract

The present invention relates to a method for visually locating human organs and meridians based on deep learning, and belongs to the technical field of computer vision and machine learning. The method comprises: extracting the outline of a treatment bed and setting its center position as the coordinate origin; screening video frames containing characters and meeting posture conditions for key point annotation and constructing a sample set; obtaining the optimal MobileNet network model based on sample set training; using the optimal network model to analyze the measured video to obtain the position of the patient's key points relative to the center of the treatment bed, and then estimating the position of the patient's organs and meridians. The present invention is a real-time method for estimating the position of a patient's organs and meridians based on deep learning, which can quickly, accurately and stably locate the position of the patient's organs and meridians on the treatment bed, thereby improving the treatment effect of the patient's diseased part.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and machine learning, and relates to a visual positioning method of human organs and meridians based on deep learning. Background Art

[0002] At present, the detection of key points of the human body is the basis of many computer vision tasks, such as action classification, abnormal behavior detection, and autonomous driving. In recent years, with the development of deep learning technology, the detection effect of key points of the human body has been continuously improved, and it has been widely used in various fields of computer vision. In medicine, the detection of key points of the human body is mainly used in the field of sports medicine to scientifically understand and analyze the movement process of patients with bone diseases. However, there is no relevant research on estimating the position of human organs and meridians based on key points of the human body. Especially in rehabilitation medicine, the research on estimating the position of human organs and meridians based on key points of the human body is crucial. This research will avoid the excessive reliance of medical staff on the experience of organ position estimation, and take into account the individual differences of patients, which is helpful for accurate treatment of the diseased parts; in addition, during the treatment process, patients can be monitored in real time and the treatment content can be adjusted in time to improve the treatment effect of patients. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a visual positioning method for human organs and meridians based on deep learning, so as to quickly, accurately and stably locate the position of the patient's organs and meridians on the treatment bed through the relative position relationship between the key points of the human body and the organs and meridians, so as to carry out targeted treatment of the patient's diseased parts and improve the treatment effect.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A method for visual positioning of human organs and meridians based on deep learning, specifically comprising the following steps:

[0006] S1: Filter the video frames when the treatment bed is idle from the video, extract the contour information of the treatment bed, calculate the center position of the treatment bed according to the contour information of the treatment bed and set it as the origin of the image coordinates;

[0007] S2: When the patient lies flat on the treatment bed, select the video frames containing people and meeting the posture conditions from the video for annotation and construct a sample set;

[0008] S3: Input the sample set into the MobileNet network for training to obtain an optimal network detection module that can extract the key point positions of the human body;

[0009] S4: Input the video to be analyzed into the optimal network detection module to obtain the key point information of the human body, and use the coordinate origin calculated in step S1 to correct the coordinates of the key points, so as to estimate the position of the human organs and meridians relative to the treatment bed.

[0010] Further, in step S1, extracting the contour information of the treatment bed specifically includes the following steps:

[0011] S11: performing Gaussian filtering on the filtered video frame image to reduce noise on the image and prevent image distortion;

[0012] S12: grayscale and binarization processing of the filtered image;

[0013] S13: dilate the binary image to further eliminate noise and separate independent elements in the image;

[0014] S14: Retrieve the outer contour of the treatment couch from the image.

[0015] Further, in step S2, a sample set is constructed, specifically including: screening out video frames containing people and meeting posture conditions; dividing the video frames into a training set, a test set and a verification set; and marking key points of the human body on the video frames in the training set.

[0016] Further, in step S2, the key points of the human body include: left and right necks, left and right shoulders, left and right armpits, left and right hips, thigh roots, left and right middle thighs, left and right knees, left and right middle calves, and left and right ankles.

[0017] Further, in step S4, the positions of human organs and meridians are estimated, including the heart, left lung, right lung, spleen, liver, the starting and ending points of the main meridians, the starting and ending points of the right leg meridians, and the starting and ending points of the left leg meridians. The specific calculation formula is:

[0018] Heart: x = x B2 -(x B2 -x B1 ) / 2, y=y B2 -(y B2 -y B3 ) / 2, where B1, B2, and B3 correspond to the left side of the neck, the top of the left shoulder, and the left armpit, respectively;

[0019] Right lung: x = x A1 -(x A1 -x A2 ) / 2, y=y A3 , where A1, A2, and A3 correspond to the right side of the neck, the top of the right shoulder, and the right armpit, respectively;

[0020] Left lung: x = x B2 -(x B2 -xB1 ) / 2, y=y B3 ;

[0021] Spleen: x = x B2 -(x B2 -x B1 ) / 2, y=y B3 -(y B3 -y B4 ) / 2, where B4 corresponds to the left hip;

[0022] Liver: x = x A1 –(x A1 -x A2 ) / 2, y=y A3 –(y A3 –y A4 ) / 3, where A4 corresponds to the right hip;

[0023] Main meridian starting point: x = x C , y=y A1 ; End point: x = x C , y=y C , where C corresponds to the thigh root;

[0024] Right leg meridian starting point: x=x A1 -(x A1 -x A2 ) / 2, y=y C ; End point: x = x A11 , y=y A11 , where A11 corresponds to the right ankle;

[0025] Left leg meridian starting point: x=x B2 -(x B2 -x B1 ) / 2, y=y C ; End point: x = x B11 , y=y B11 , where B11 corresponds to the left ankle.

[0026] The beneficial effect of the present invention is that the present invention can quickly, accurately and stably locate the position of the patient's organs and meridians relative to the treatment bed through the relative position relationship between the key points of the human body and the organs and meridians, thereby performing targeted treatment on the patient's diseased parts and improving the treatment effect.

[0027] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0029] Figure 1 This is a flow chart of the visual positioning method of human organs and meridians based on deep learning of the present invention;

[0030] Figure 2 Schematic diagram of key points of the human body. DETAILED DESCRIPTION

[0031] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0032] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0033] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0034] See also Figure 1-2 This embodiment provides a method for visually locating human organs and meridians based on deep learning, which specifically includes the following steps:

[0035] S1: Filter the video frames when the treatment bed is idle from the video, extract the contour information of the treatment bed, calculate the center position of the treatment bed according to the contour information of the treatment bed and set it as the origin of the image coordinates;

[0036] The extraction of the treatment bed contour information specifically includes the following steps:

[0037] S11: performing Gaussian filtering on the screened video frame image to reduce noise on the image and prevent image distortion;

[0038] S12: grayscale and binarization processing of the filtered image;

[0039] S13: dilate the binary image to further eliminate noise and separate independent elements in the image;

[0040] S14: Retrieve the outer contour of the treatment couch from the image.

[0041] S2: When the patient lies flat on the treatment bed, select the video frames containing people and meeting the posture conditions from the video for annotation and construct a sample set;

[0042] Among them, constructing a sample set specifically includes: screening out video frames that contain people and meet posture conditions; dividing the video frames into a training set, a test set, and a verification set; and marking key points of the human body in the video frames in the training set.

[0043] The key points of the human body include: left and right necks, left and right shoulders, left and right armpits, left and right hips, thigh roots, left and right middle thighs, left and right knees, left and right middle calves, and left and right ankles.

[0044] S3: Input the sample set into the MobileNet network for training to obtain an optimal network detection module that can extract the key point positions of the human body;

[0045] S4: Input the video to be analyzed into the optimal network detection module to obtain the key point information of the human body, and use the coordinate origin calculated in step S1 to correct the coordinates of the key points, so as to estimate the position of the human organs and meridians relative to the treatment bed.

[0046] Among them, the estimation of the positions of human organs and meridians includes the estimation of the heart, left lung, right lung, spleen, liver, the starting and ending points of the main meridians, the starting and ending points of the right leg meridians, and the starting and ending points of the left leg meridians. The specific calculation formula is:

[0047] Heart: x = x B2 -(x B2 -x B1 ) / 2, y=y B2 -(y B2 -y B3 ) / 2, where B1, B2, and B3 correspond to the left side of the neck, the top of the left shoulder, and the left armpit, respectively;

[0048] Right lung: x = x A1 -(xA1 -x A2 ) / 2, y=y A3 , where A1, A2, and A3 correspond to the right side of the neck, the top of the right shoulder, and the right armpit, respectively;

[0049] Left lung: x = x B2 -(x B2 -x B1 ) / 2, y=y B3 ;

[0050] Spleen: x = x B2 -(x B2 -x B1 ) / 2, y=y B3 -(y B3 -y B4 ) / 2, where B4 corresponds to the left hip;

[0051] Liver: x = x A1 –(x A1 -x A2 ) / 2, y=y A3 –(y A3 –y A4 ) / 3, where A4 corresponds to the right hip;

[0052] Main meridian starting point: x = x C , y=y A1 ; End point: x = x C , y=y C , where C corresponds to the thigh root;

[0053] Right leg meridian starting point: x=x A1 -(x A1 -x A2 ) / 2, y=y C ; End point: x = x A11 , y=y A11 , where A11 corresponds to the right ankle;

[0054] Left leg meridian starting point: x=x B2 -(x B2 -x B1 ) / 2, y=y C ; End point: x = x B11 , y=y B11 , where B11 corresponds to the left ankle.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A visual positioning method for human organs and meridians based on deep learning, characterized in that: The method specifically comprises the following steps: S1: Filter the video frames when the treatment bed is idle from the video, extract the contour information of the treatment bed, calculate the center position of the treatment bed according to the contour information of the treatment bed and set it as the origin of the image coordinates; S2: When the patient lies flat on the treatment bed, select the video frames containing people and meeting the posture conditions from the video for annotation and construct a sample set; S3: Input the sample set into the MobileNet network for training to obtain an optimal network detection module that can extract the key point positions of the human body; S4: Input the video to be analyzed into the optimal network detection module to obtain the key point information of the human body, and use the coordinate origin calculated in step S1 to correct the coordinates of the key points, so as to estimate the positions of human organs and meridians relative to the treatment bed; Estimate the positions of human organs and meridians, including the heart, left lung, right lung, spleen, liver, the starting and ending points of the main meridians, the starting and ending points of the right leg meridians, and the starting and ending points of the left leg meridians. The specific calculation formula is: Heart: x = x B2 -(x B2 -x B1 ) / 2, y=y B2 -(y B2 -y B3 ) / 2, where B1, B2, and B3 correspond to the left side of the neck, the top of the left shoulder, and the left armpit, respectively; Right lung: x = x A1 -(x A1 -x A2 ) / 2, y=y A3 , where A1, A2, and A3 correspond to the right side of the neck, the top of the right shoulder, and the right armpit, respectively; Left lung: x = x B2 -(x B2 -x B1 ) / 2, y = y B3 ; Spleen: x = x B2 -(x B2 -x B1 ) / 2, y=y B3 -(y B3 -y B4 ) / 2, where B4 corresponds to the left hip; Liver: x = x A1 –(x A1 -x A2 ) / 2, y=y A3 –(y A3 –y A4 ) / 3, where A4 corresponds to the right hip; Main meridian starting point: x=x C , y=y A1 ; End point: x = x C , y=y C , where C corresponds to the thigh root; Right leg meridian starting point: x=x A1 -(x A1 -x A2 ) / 2, y=y C ; End point: x = x A11 , y=y A11 , where A11 corresponds to the right ankle; Left leg meridian starting point: x=x B2 -(x B2 -x B1 ) / 2, y=y C ; End point: x = x B11 , y=y B11 , where B11 corresponds to the left ankle.

2. The visual positioning method of human organs and meridians according to claim 1, characterized in that: In step S1, extracting the contour information of the treatment bed includes the following steps: S11: performing Gaussian filtering on the screened image; S12: grayscale and binarization processing of the filtered image; S13: dilate the binary image and separate independent elements in the image; S14: Retrieve the outer contour of the treatment couch from the image.

3. The visual positioning method of human organs and meridians according to claim 1, characterized in that: In step S2, a sample set is constructed, specifically including: screening out video frames containing people and meeting posture conditions; dividing the video frames into a training set, a test set, and a verification set; and marking key points of the human body on the video frames in the training set.

4. The visual positioning method of human organs and meridians according to claim 3, characterized in that: In step S2, the key points of the human body include: left and right necks, left and right shoulders, left and right armpits, left and right hips, thigh roots, left and right middle thighs, left and right knees, left and right middle calves, and left and right ankles.

Images