Auxiliary X-film shooting method, device and equipment based on key point identification

Abstract

The invention provides an X-film auxiliary shooting method, device and equipment based on key point recognition, and relates to the technical field of medical technology, and the method comprises the steps: collecting a real-time posture image of a target patient based on pre-built image collection equipment; according to a pre-trained human body key point identification model, key point identification is carried out on the real-time posture image of the target patient, and the identified key points are marked on the real-time posture image; based on the marked real-time posture image, determining a key point spatial relationship between the at least two key points; and obtaining posture error information of the target patient based on the spatial relationship of each key point and the positioning standard posture, and determining a posture adjustment suggestion based on the posture error information, so as to shoot an X film for the target patient according to an adjustment result. According to the invention, the patient can be reminded to keep a standard posture in real time without subjective influence of technicians, so that X film effects shot by different technicians can be kept consistent.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and in particular to a method, apparatus and equipment for X-ray assisted imaging based on key point recognition. Background Technology

[0002] Knee osteoarthritis is one of the most common diseases leading to joint dysfunction in clinical practice. Full-length anteroposterior X-rays of both lower extremities are an important imaging tool for the diagnosis and treatment of knee osteoarthritis. With their panoramic imaging view, they can comprehensively present key information such as lower limb alignment, joint spaces, and bone morphology, helping orthopedic surgeons better address the challenges posed by this prevalent disease. During imaging examinations, correct positioning helps obtain clear and accurate images, ensuring full exposure and correct localization of the examined area.

[0003] In existing technologies, when taking full-length anteroposterior X-rays of both lower limbs, radiology technicians need to visually assess whether the patient is in a standard positioning posture based on their experience, and verbally guide the patient to adjust their posture. After the posture is adjusted, the patient is asked to maintain the posture while waiting for the technician to return to the radiography room for exposure and shooting. In addition, there are some fixation devices that can assist patients in positioning.

[0004] However, in existing technologies, the accuracy of patient positioning is influenced by the radiology technician's subjectivity, resulting in varying imaging effects between different technicians. Furthermore, the time required for technicians to return to the imaging room, coupled with patient discomfort due to pain, prevents them from maintaining the technician's instructed positioning, leading to poor imaging results. Additionally, the fixation devices used to assist patient positioning are complex to operate, inefficient, and unsuitable for different patient conditions. Summary of the Invention

[0005] This invention provides a method, device, and equipment for assisting X-ray imaging based on key point recognition, in order to solve the problems in the prior art where the patient's positioning posture is affected by the technician's subjectivity and pain, and the fixing devices for assisting patient positioning are complicated to operate and cannot adapt to different patients, resulting in poor X-ray imaging effects.

[0006] In a first aspect, embodiments of the present invention provide an X-ray assisted imaging method based on key point recognition, comprising:

[0007] Real-time posture images of the target patient are acquired using pre-built image acquisition equipment.

[0008] Based on a pre-trained human key point recognition model, key points are identified in the real-time posture image of the target patient, and the identified key points are marked on the real-time posture image.

[0009] Based on the labeled real-time pose image and the standard pose, determine the key point spatial relationship between at least two key points;

[0010] Based on the spatial relationship of each key point, the posture error information of the target patient is obtained, and posture adjustment suggestions are determined based on the posture error information, so as to take X-rays for the target patient according to the adjustment results.

[0011] Secondly, embodiments of the present invention provide an X-ray assisted imaging device based on key point recognition, comprising:

[0012] The acquisition module is used to acquire real-time posture images of the target patient based on a pre-built image acquisition device;

[0013] The recognition module is used to identify key points in the real-time posture image of the target patient based on a pre-trained human key point recognition model, and to annotate the identified key points on the real-time posture image.

[0014] The determination module is used to determine the key point spatial relationship between at least two key points based on the labeled real-time pose image and the standard pose.

[0015] The suggestion module is used to obtain the posture error information of the target patient based on the spatial relationship of each key point, and to determine posture adjustment suggestions based on the posture error information, so as to take X-rays for the target patient according to the adjustment results.

[0016] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect or any possible implementation thereof.

[0017] In this embodiment of the invention, real-time posture images of the target patient are acquired using a pre-built image acquisition device. A pre-trained human key point recognition model is then used to identify key points in the real-time posture images, making it adaptable to different patients. The identified key points are annotated on the real-time posture images, and the spatial relationship between at least two key points is determined using the annotated images. This spatial relationship is then used to determine the patient's posture error information, further leading to posture adjustment suggestions. By identifying key points through real-time posture images and the pre-trained model, and using the information from these key points to determine posture adjustment suggestions, X-rays are taken for the target patient based on the adjustment results. This provides real-time reminders to the patient to maintain a standard posture and is not influenced by the technician's subjectivity, ensuring consistent X-ray results from different technicians. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the implementation of the X-ray assisted imaging method based on key point recognition provided in this embodiment of the invention.

[0019] Figure 2 This is a schematic diagram of key points and connections in the X-ray assisted imaging method based on key point recognition provided in this embodiment of the invention;

[0020] Figure 3 This is a flowchart illustrating the implementation of step S140 of the X-ray assisted imaging method based on key point recognition provided in an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the X-ray assisted imaging device based on key point recognition provided in an embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0024] See Figure 1 The document illustrates a flowchart of the implementation of the X-ray assisted imaging method based on key point recognition provided in an embodiment of the present invention, which is described in detail below:

[0025] Step S110: Acquire real-time posture images of the target patient based on the pre-built image acquisition equipment.

[0026] In some embodiments, the pre-built image acquisition equipment includes a high-definition camera, an adjustable camera mount, and other devices, primarily used to acquire real-time posture images of the target patient. The target patient refers to a specific individual, such as a patient with knee osteoarthritis requiring full-length anteroposterior X-rays of both lower limbs. Real-time posture images refer to images acquired by the image acquisition equipment in sync with the patient's current posture, reflecting the patient's positioning at every moment.

[0027] Step S120: Based on the pre-trained human key point recognition model, key points are identified in the real-time pose image of the target patient, and the identified key points are marked on the real-time pose image.

[0028] It should be noted that the human body key point recognition model can be the YOLO model.

[0029] In some embodiments, a pre-trained human keypoint recognition model refers to a model that has been trained and optimized with a large amount of relevant image data before being used to recognize keypoints in patient pose images. This model is based on the YOLO model, and after training, it is deployed on a local computer. By inputting a large number of images containing different human poses annotated by physicians, the model learns how to accurately determine the position of various skeletal keypoints of the human body, thus enabling it to quickly and accurately identify keypoints in images during practical applications. Keypoint recognition refers to the process of locating and recognizing skeletal keypoints of a target patient in real-time pose images using the aforementioned pre-trained model.

[0030] Step S130: Based on the annotated real-time pose image and the standard pose, determine the key point spatial relationship between at least two key points.

[0031] In some embodiments, the standard positioning posture is the correct posture for taking a full-length anteroposterior X-ray of both lower limbs. The standard positioning posture includes: shoulders level, arms hanging naturally, feet shoulder-width apart, toes pointing forward, knees fully extended, and center of gravity evenly distributed. Specifically, the angle between the line connecting the left and right shoulders and the Y-axis should be approximately 0°, with an error range of ±5°; the lines connecting the left shoulder, left elbow, and left wrist should be approximately 180°, and the lines connecting the right shoulder, right elbow, and right wrist should also be 180°; the distance between the lines connecting the left and right shoulders should be approximately the same as the distance between the lines connecting the left and right ankles, with an error in the image of less than 0.1cm; the lines connecting the left hip, left knee, and left ankle should be approximately 180°, and the lines connecting the right hip, right knee, and right ankle should also be approximately 180°, with an error range of ±5°; the distance between the left hip and left ankle should be approximately the same as the distance between the right hip and right ankle, with an error in the image of less than 0.1cm.

[0032] In some embodiments, the labeled real-time posture image refers to an image that reflects the target patient's positioning posture in real time after the identified key points (such as the left shoulder, right hip, left knee, etc.) have been labeled in a specific manner. The key point spatial relationship between at least two key points refers to the positional connection between two or more labeled skeletal key points in terms of angles, distances, etc., such as the angle of the line connecting the three key points of left hip, left knee, and left ankle, or the comparison of the distance between the lines connecting the left and right shoulders and the distance between the left and right ankles, etc. Whether these relationships meet the standards directly determines whether the patient's positioning is correct.

[0033] In one possible implementation, the key points include upper body posture key points, lower body posture key points, and standing posture key points; the lower body posture key points include the left hip, right hip, left ankle, and right ankle; the specific implementation of step S130 is as follows: for the upper body posture key points of the target patient, on the annotated real-time posture image, connect two adjacent key points to obtain multiple line segments; for the lower body posture key points of the target patient, on the annotated real-time posture image, connect two adjacent key points, and connect the left ankle and left hip. Connect the right ankle and right hip to obtain multiple line segments; determine the key point spatial relationship between the corresponding multiple key points by at least one of the following: the length of each line segment, the angle between each line segment and the x-axis of the coordinate system, the angle between each line segment and the y-axis of the coordinate system, or the angle between each line segment and the other line segments; wherein, the origin of the coordinate system is located at the midpoint of the line connecting the left toe and the right toe in the standard positioning posture, the x-axis is perpendicular to the plane where the chest X-ray frame of the X-ray machine is located, the y-axis is parallel to the plane where the chest X-ray frame of the X-ray machine is located, and the z-axis is perpendicular to the ground.

[0034] In some embodiments, key points of upper body posture refer to the skeletal key points located in the upper body and related to upper body positioning, such as the left shoulder, right shoulder, left elbow, right elbow, left wrist, and right wrist. Key points of lower body posture refer to the skeletal key points located in the lower body and related to lower body positioning, such as the left hip, right hip, left knee, right knee, left ankle, right ankle, left toe, and right toe. See also Figure 2 , Figure 2 It includes fourteen key points and coordinate systems that need to be labeled on the real-time pose image.

[0035] In some embodiments, two adjacent keypoints refer to two keypoints that are close in location and have continuity in the human skeletal structure. For example, in the upper body, the left shoulder and left elbow are adjacent keypoints, and they are continuous in the arm skeletal structure. A line segment is a straight line segment formed by connecting two keypoints. A coordinate system is a reference frame used to locate and describe the spatial position of keypoints and line segments. It contains an origin, an x-axis, a y-axis, and a z-axis—three mutually perpendicular axes. This frame quantifies the angular and length relationships of line segments. The plane on which the X-ray machine's chest frame is located refers to the plane on which the components of the X-ray machine that carry the film are located. It is usually a vertical plane and serves as a reference for determining the x-axis and y-axis directions. The x-axis is an axis in the coordinate system perpendicular to the plane on which the X-ray machine's chest frame is located. The plane on which the chest frame is located is usually vertical, and the x-axis points horizontally towards the chest frame. It is used to describe the angular relationship of line segments in the direction perpendicular to the plane on which the chest frame is located. The y-axis is an axis in the coordinate system parallel to the plane on which the X-ray machine's chest frame is located. The z-axis is an axis in the coordinate system perpendicular to the ground, i.e., in the vertical direction. The angle between each line segment and the x-axis of the coordinate system refers to the angle formed between the line segment and the x-axis, used to describe the degree of inclination of the line segment in the direction perpendicular to the plane of the chest X-ray frame. The angle between each line segment and the y-axis of the coordinate system refers to the angle formed between the line segment and the y-axis, used to describe the degree of inclination of the line segment in the plane parallel to the chest X-ray frame. The angle between each line segment and the other line segments refers to the angle formed between one line segment and any other line segment, used to describe the relative positional relationship between different limb parts.

[0036] Step S140: Based on the spatial relationship of each key point, obtain the posture error information of the target patient, and determine posture adjustment suggestions based on the posture error information, so as to take X-rays of the target patient according to the adjustment results.

[0037] In some embodiments, posture error information includes upper body posture error information, standing posture error information, and lower body posture error information. Posture error information for the target patient refers to the deviation between the real-time posture reflected in the target patient's real-time posture image and the standard positioning posture. Posture adjustment suggestions are specific and actionable posture correction guidelines provided to the patient based on the obtained posture error information. For example, when the system detects that the patient's right shoulder is low, it will suggest, "Right shoulder is low, please keep both shoulders level"; when it detects that the patient's left toe is internally rotated, it will suggest, "Left foot is internally rotated, please keep the left toe pointing forward," helping the patient adjust to the standard posture. The adjustment result refers to the patient's new posture state conforming to the standard positioning posture after correcting their posture according to the posture adjustment suggestions.

[0038] See Figure 3 The specific implementation of step S140 above includes steps S1401-S1403, as detailed below:

[0039] Step S1401: Determine the upper body posture error information of the target patient based on the spatial relationship between key points of the upper body posture key points of the target patient.

[0040] In some embodiments, the spatial relationship between key points of the upper body posture refers to the positional connection between key points of the upper body in terms of angles, distances, etc., such as the angle between the line connecting the left and right shoulders and the y-axis of the coordinate system. Upper body posture error information is obtained by comparing the actual spatial relationship between key points of the target patient's upper body posture with the spatial relationship of corresponding key points in the standard positioning posture, thus providing information on the deviation between the actual upper body posture and the standard posture.

[0041] In one possible implementation, the key points of upper body posture include the left shoulder, right shoulder, left elbow, right elbow, left wrist, and right wrist; the upper body posture error information includes shoulder posture error information, left arm posture error information, and right arm posture error information; the specific implementation of step S1401 is as follows: the angle between the line segment formed by the left and right shoulders of the target patient and the y-axis is determined as the first angle, and the first angle is compared with the first preset angle to obtain shoulder posture error information; the angle between the line segment formed by the left shoulder and left elbow of the target patient and the line segment formed by the left elbow and left wrist is determined as the second angle, and the second angle is compared with the second preset angle to obtain left arm posture error information; the angle between the line segment formed by the left shoulder and left elbow of the target patient and the line segment formed by the left elbow and left wrist of the target patient is determined as the second angle, and the second angle is compared with the second preset angle to obtain left arm posture error information; the angle between the line segment formed by the left shoulder and right elbow of the target patient and the line segment formed by the left elbow and left wrist of the target patient is determined as the second angle, and the angle between the line segment formed by the left shoulder and left elbow of the target patient and the line segment formed by the left elbow and left wrist of the target patient is determined as the second angle, and the angle between the line segment formed by the left shoulder and right ... The angle between the line segment formed by the elbow and the line segment formed by the right elbow and right wrist is determined as the third angle. The third angle is compared with the second preset angle to obtain the right arm posture error information. Accordingly, posture adjustment suggestions are determined based on the posture error information, including: if the absolute value of the first angle in the shoulder posture error information is greater than the first preset angle, the posture adjustment suggestion is to keep both shoulders level; if the absolute value of the difference between the second angle and the second preset angle in the arm posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the left arm hanging naturally; if the absolute value of the difference between the third angle and the second preset angle in the arm posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the right arm hanging naturally.

[0042] In some embodiments, shoulder posture error information is obtained by comparing the actual angle (a first angle) between the line connecting the left and right shoulders of the target patient and the y-axis with a first preset angle, where the first preset angle is 5°. Left arm posture error information is obtained by comparing the angle formed by the left shoulder, left elbow, and left wrist of the target patient (a second angle) with a second preset angle, where the second preset angle is 180°. Right arm posture error information is obtained by comparing the angle formed by the right shoulder, right elbow, and right wrist of the target patient (a third angle) with the second preset angle. The preset angle difference is 5°. That is, when the absolute value of the difference between the second angle and the second preset angle is greater than 5°, a reminder is needed for the target patient's left arm; when the absolute value of the difference between the third angle and the second preset angle is greater than 5°, a reminder is needed for the target patient's right arm.

[0043] In some embodiments, clockwise direction is defined as positive and counterclockwise direction as negative. When the angle between the line connecting the left and right shoulders and the y-axis is positive and its absolute value is greater than 5°, it indicates that the right shoulder is too low; please keep both shoulders level. When the angle between the line connecting the left and right shoulders and the y-axis is negative and its absolute value is greater than 5°, it indicates that the left shoulder is too low; please keep both shoulders level. When the angle between the left shoulder, elbow, and wrist is greater than 185° or less than 175°, it indicates that the left arm should hang naturally. When the angle between the right shoulder, elbow, and wrist is greater than 185° or less than 175°, it indicates that the right arm should hang naturally.

[0044] Step S1402: Determine the standing posture error information of the target patient based on the spatial relationship between multiple key points of the target patient's standing posture.

[0045] In some embodiments, standing posture key points refer to key points related to the human standing posture, including the left shoulder, right shoulder, left ankle, and right ankle. The spatial relationship between multiple key points in standing posture refers to the distance between the left and right shoulders and the distance between the left and right ankles.

[0046] In one possible implementation, the key points of the standing posture include the left shoulder, right shoulder, left ankle, and right ankle; the specific processing method of step S1402 is as follows: the distance between the left and right shoulders of the target patient is determined as the first distance, and the distance between the left and right ankles of the target patient is determined as the second distance; the first difference between the first distance and the second distance is calculated, and the first difference is compared with a preset difference to obtain the standing posture error information of the target patient; accordingly, a posture adjustment suggestion is determined based on the posture error information, including: if the absolute value of the first difference in the standing posture error information is greater than the preset difference, the posture adjustment suggestion is to adjust the feet so that the feet are shoulder-width apart.

[0047] In some embodiments, the standing posture error information refers to the posture error caused by the inability to maintain shoulder-width apart when standing. This is determined by comparing the first distance between the left and right shoulders and the second distance between the left and right ankles of the target patient. It should be noted that both the first and second distances are the distances shown on the image. The first difference is the difference between the first and second distances as shown on the image, with a preset difference of 0.1 cm. When the absolute value of the first difference exceeds the preset difference, if the first distance is greater than the second distance, the first difference is positive, indicating a reminder to slightly open the feet and maintain shoulder-width apart; if the first distance is less than the second distance, the first difference is negative, indicating a reminder to slightly close the feet and maintain shoulder-width apart.

[0048] Step S1403: Based on the spatial relationship between key points of the target patient's lower body posture, determine the target patient's lower body posture error information.

[0049] In some embodiments, the spatial relationship between key points of lower body posture refers to the positional connections in terms of angles and distances between key points of the lower body, such as the left hip, right hip, left knee, right knee, left ankle, right ankle, left toe, and right toe. Examples include the angle formed by the line connecting the left hip, left knee, and left ankle; the angle between the line connecting the right toe and right ankle and the x-axis of the coordinate system; and a comparison of the lengths of the lines connecting the left hip to the left ankle and the right hip to the right ankle. These relationships collectively reflect the overall positioning of the lower body. Lower body posture error information is obtained by comparing the actual spatial relationship between the key points of the target patient's lower body posture with the spatial relationship of the corresponding key points in the standard positioning posture, thus revealing the deviation between the actual lower body posture and the standard posture.

[0050] In one possible implementation, the key points of the lower body posture include the left knee, right knee, left toe, and right toe; the lower body posture error information includes foot posture error information, left leg posture error information, right leg posture error information, and center of gravity error information; the specific processing method of step S1403 is as follows: the angle between the line segment formed by the left toe and left ankle and the x-axis is determined as the fourth angle, the angle between the line segment formed by the right toe and right ankle and the x-axis is determined as the fifth angle, and the fourth and fifth angles are compared with the third preset angle to obtain the foot posture error information; the angle between the line segment formed by the left hip and left knee and the line segment formed by the left knee and left ankle is determined as the sixth angle, and the sixth angle is compared with the second preset angle to obtain the left leg posture error information; the angle between the line segment formed by the right hip and right knee and the line segment formed by the right knee and right ankle is determined as the seventh angle, and the seventh angle is compared with the second preset angle to obtain the right leg posture error information; the third distance between the left hip and left ankle and the right... The distance between the hip and the right ankle is determined as the fourth distance; the second difference between the third and fourth distances is calculated and compared with a preset difference to obtain the target patient's center of gravity error information; accordingly, posture adjustment suggestions are determined based on the posture error information, including: if the absolute value of the fourth angle in the foot posture error information is greater than the third preset angle, the posture adjustment suggestion is to keep the left toes pointing forward; if the absolute value of the fifth angle in the foot posture error is greater than the third preset angle, the posture adjustment suggestion is to keep the right toes pointing forward; if the absolute value of the difference between the sixth angle and the second preset angle in the left leg posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the left knee fully extended; if the absolute value of the difference between the seventh angle and the second preset angle in the right leg posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the right knee fully extended; if the absolute value of the second difference in the center of gravity error information is greater than the preset difference, the posture adjustment suggestion is to maintain a uniform center of gravity.

[0051] In some embodiments, foot posture error information is obtained by comparing the actual angle (fourth angle) of the line connecting the target patient's left toe and left ankle to the x-axis, and the actual angle (fifth angle) of the line connecting the target patient's right toe and right ankle to the x-axis, with a third preset angle of 2°. Left leg posture error information is obtained by comparing the angle (sixth angle) formed by the target patient's left hip, left knee, and left ankle with a second preset angle. Right leg posture error information is obtained by comparing the angle (seventh angle) formed by the target patient's right hip, right knee, and right ankle with a second preset angle. Center of gravity error information refers to the posture error caused by the shift of the body's center of gravity when standing, and is obtained by comparing the third distance between the target patient's left hip and left ankle, and the fourth distance between the target patient's right hip and right ankle. It should be noted that the third and fourth distances are the distances shown on the image. The second difference is the difference between the third and fourth distances shown on the image, with a preset difference of 0.1 cm.

[0052] In some embodiments, clockwise direction is defined as positive and counterclockwise direction as negative. When the angle between the line connecting the left toe and left ankle and the x-axis is positive and its absolute value is greater than 2°, it indicates that the left foot is externally rotated; please keep the left toe pointing forward. When the angle between the line connecting the left toe and left ankle and the x-axis is negative and its absolute value is greater than 2°, it indicates that the left foot is internally rotated; please keep the left toe pointing forward. When the angle between the line connecting the right toe and right ankle and the x-axis is positive and its absolute value is greater than 2°, it indicates that the right foot is externally rotated; please keep the right toe pointing forward. When the angle between the right toe and right ankle and the x-axis is negative and its absolute value is greater than 2°, it indicates that the right foot is internally rotated; please keep the right toe pointing forward. When the left hip-knee-ankle angle is greater than 185°, it indicates that the left knee is externally rotated; please keep it fully extended. When the left hip-knee-ankle angle is less than 175°, it indicates that the left knee is internally rotated; please keep it fully extended. When the right hip-knee-ankle angle is greater than 185°, it indicates that the right knee is externally rotated; please keep it fully extended. When the right hip-knee-ankle angle is less than 175°, it indicates that the right knee is internally rotated; please keep it fully extended. When the absolute value of the second difference exceeds the preset difference, if the third distance is greater than the fourth distance, it indicates that the center of gravity is shifted to the right; please keep the center of gravity even. If the third distance is less than the fourth distance, it indicates that the center of gravity is shifted to the left; please keep the center of gravity even.

[0053] By using pre-set high-definition cameras, real-time posture images of the target patient are captured. A pre-trained YOLO model is then used as a human keypoint recognition model to identify key points in the real-time posture images, demonstrating high adaptability to different patients. The identified key points are annotated on the real-time posture images. The spatial relationships between multiple key points are determined using the annotated images, and posture error information is determined based on one or more key point spatial relationships. Further posture adjustment suggestions are then made based on this information. The human keypoint recognition model identifies key points on the real-time posture images, and posture adjustment suggestions are determined based on the spatial relationships between these key points. Based on the adjustment results, X-rays are taken of the target patient, providing real-time reminders to maintain standard posture and offering adjustment suggestions, thus ensuring the quality of the X-ray images.

[0054] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0055] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0056] Figure 4 A schematic diagram of the structure of the X-ray assisted imaging device based on key point recognition provided in an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below:

[0057] like Figure 4 As shown, the X-ray assisted imaging device 4 based on key point recognition includes:

[0058] Acquisition module 41 is used to acquire real-time posture images of the target patient based on a pre-built image acquisition device;

[0059] The recognition module 42 is used to identify key points in the real-time posture image of the target patient based on a pre-trained human key point recognition model, and to mark the identified key points on the real-time posture image.

[0060] Module 43 is used to determine the spatial relationship between at least two key points based on the annotated real-time pose image and the standard pose.

[0061] The suggestion module 44 is used to obtain the posture error information of the target patient based on the spatial relationship of each key point, and to determine the posture adjustment suggestion based on the posture error information, so as to take X-rays for the target patient according to the adjustment result.

[0062] In one possible implementation, the standard positioning posture is the correct posture when taking a full-length anteroposterior X-ray of both lower limbs. The standard positioning posture includes shoulders level, arms hanging naturally, feet shoulder-width apart, toes pointing forward, knees fully extended, and center of gravity evenly distributed.

[0063] In one possible implementation, the human body key point recognition model is the YOLO model.

[0064] In one possible implementation, the key points include upper body posture key points, lower body posture key points, and standing posture key points; the lower body posture key points include the left hip, right hip, left ankle, and right ankle; the determination module 43 is specifically used for: for the upper body posture key points of the target patient, connecting two adjacent key points on the labeled real-time posture image to obtain multiple line segments; for the lower body posture key points of the target patient, connecting two adjacent key points on the labeled real-time posture image, and connecting the left ankle and left hip, etc. Connect the right ankle and right hip to obtain multiple line segments; determine the key point spatial relationship between the corresponding multiple key points by at least one of the following: the length of each line segment, the angle between each line segment and the x-axis of the coordinate system, the angle between each line segment and the y-axis of the coordinate system, or the angle between each line segment and the other line segments; wherein, the origin of the coordinate system is located at the midpoint of the line connecting the left toe and the right toe in the standard positioning posture, the x-axis is perpendicular to the plane where the chest X-ray frame of the X-ray machine is located, the y-axis is parallel to the plane where the chest X-ray frame of the X-ray machine is located, and the z-axis is perpendicular to the ground.

[0065] In one possible implementation, the posture error information includes upper body posture error information, standing posture error information, and lower body posture error information; the suggestion module 44 is specifically used to: determine the upper body posture error information of the target patient based on the spatial relationship between key points of the upper body posture key points; determine the standing posture error information of the target patient based on the spatial relationship between multiple key points of the standing posture key points; and determine the lower body posture error information of the target patient based on the spatial relationship between key points of the lower body posture key points.

[0066] In one possible implementation, the key points of upper body posture include the left shoulder, right shoulder, left elbow, right elbow, left wrist, and right wrist; the upper body posture error information includes shoulder posture error information, left arm posture error information, and right arm posture error information; the suggestion module 44 is further used to: determine the angle between the line segment formed by the left and right shoulders of the target patient and the y-axis as a first angle, compare the first angle with a first preset angle to obtain shoulder posture error information; determine the angle between the line segment formed by the left shoulder and left elbow of the target patient and the line segment formed by the left elbow and left wrist as a second angle, compare the second angle with a second preset angle to obtain left arm posture error information; determine the angle between the line segment formed by the left shoulder and left elbow of the target patient and the line segment formed by the left elbow and left wrist as a second angle, compare the second angle with a second preset angle to obtain left arm posture error information; and determine the angle between the line segment formed by the right shoulder and right elbow of the target patient and the line segment formed by the left elbow and left wrist as a second angle, compare the second angle with a second preset angle to obtain left arm posture error information; and determine the angle between the line segment formed by the left ... The angle between the line segment formed by the right elbow and right wrist and the line segment formed by the right elbow and right wrist is determined as the third angle. The third angle is compared with the second preset angle to obtain the right arm posture error information. Accordingly, posture adjustment suggestions are determined based on the posture error information, including: if the absolute value of the first angle in the shoulder posture error information is greater than the first preset angle, the posture adjustment suggestion is to keep both shoulders level; if the absolute value of the difference between the second angle and the second preset angle in the arm posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the left arm hanging naturally; if the absolute value of the difference between the third angle and the second preset angle in the arm posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the right arm hanging naturally.

[0067] In one possible implementation, the key points of the standing posture include the left shoulder, right shoulder, left ankle, and right ankle; the suggestion module 44 is further configured to: determine the distance between the left and right shoulders of the target patient as a first distance, and determine the distance between the left and right ankles of the target patient as a second distance; calculate a first difference between the first distance and the second distance, and compare the first difference with a preset difference to obtain the standing posture error information of the target patient; accordingly, determine a posture adjustment suggestion based on the posture error information, including: if the absolute value of the first difference in the standing posture error information is greater than the preset difference, then the posture adjustment suggestion is to adjust the feet so that the feet are shoulder-width apart.

[0068] In one possible implementation, the key points of the lower body posture include the left knee, right knee, left toe, and right toe; the lower body posture error information includes foot posture error information, left leg posture error information, right leg posture error information, and center of gravity error information; the suggestion module 44 is further used to: determine the angle between the line segment formed by the left toe and left ankle and the x-axis as the fourth angle, and the angle between the line segment formed by the right toe and right ankle and the x-axis as the fifth angle, and compare the fourth angle and the fifth angle with the third preset angle respectively to obtain the foot posture error information; determine the angle between the line segment formed by the left hip and left knee and the line segment formed by the left knee and left ankle as the sixth angle, and compare the sixth angle with the second preset angle to obtain the left leg posture error information; determine the angle between the line segment formed by the right hip and right knee and the line segment formed by the right knee and right ankle as the seventh angle, and compare the seventh angle with the second preset angle to obtain the right leg posture error information; and determine the third distance between the left hip and left ankle and the right hip and right ankle as the fourth angle. The ankle distance is determined as the fourth distance; the second difference between the third and fourth distances is calculated and compared with a preset difference to obtain the target patient's center of gravity error information; accordingly, posture adjustment suggestions are determined based on the posture error information, including: if the absolute value of the fourth angle in the foot posture error information is greater than the third preset angle, the posture adjustment suggestion is to keep the left toes pointing forward; if the absolute value of the fifth angle in the foot posture error is greater than the third preset angle, the posture adjustment suggestion is to keep the right toes pointing forward; if the absolute value of the difference between the sixth angle and the second preset angle in the left leg posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the left knee fully extended; if the absolute value of the difference between the seventh angle and the second preset angle in the right leg posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the right knee fully extended; if the absolute value of the second difference in the center of gravity error information is greater than the preset difference, the posture adjustment suggestion is to maintain a uniform center of gravity.

[0069] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Figure 5 As shown, the electronic device 5 of this embodiment includes a processor 50 and a memory 51. The memory 51 stores a computer program 52. When the processor 50 executes the computer program 52, it implements the steps in the various method embodiments described above. Alternatively, when the processor 50 executes the computer program 52, it implements the functions of each module / unit in the various device embodiments described above.

[0070] For example, computer program 52 may be divided into one or more modules / units, which are stored in memory 51 and executed by processor 50 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 52 in electronic device 5.

[0071] Electronic device 5 may include, but is not limited to, processor 50 and memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 5 may also include input / output devices, network access devices, buses, etc.

[0072] For the sake of simplicity and clarity, only the above-described functional modules / units are used as examples. In practical applications, the functions described above can be assigned to different functional modules / units as needed. These modules / units can be implemented in hardware, software, or a combination of both.

[0073] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0074] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for assisting X-ray imaging based on key point recognition, characterized in that, include: Real-time posture images of the target patient are acquired using pre-built image acquisition equipment. Based on a pre-trained human key point recognition model, key points are identified in the real-time posture image of the target patient, and the identified key points are marked on the real-time posture image. Based on the labeled real-time pose image and the standard pose, determine the key point spatial relationship between at least two key points; Based on the spatial relationship of each key point, the posture error information of the target patient is obtained, and posture adjustment suggestions are determined based on the posture error information, so as to take X-rays of the target patient according to the adjustment results.

2. The X-ray assisted imaging method based on key point recognition according to claim 1, characterized in that, The key points include key points of upper body posture, key points of lower body posture, and key points of standing posture; the key points of lower body posture include the left hip, right hip, left ankle, and right ankle; The determination of key point spatial relationships between at least two key points based on the labeled real-time pose image and the standard positioning pose includes: For the key points of the upper body posture of the target patient, on the labeled real-time posture image, connect two adjacent key points to obtain multiple line segments; For the key points of the lower body posture of the target patient, on the labeled real-time posture image, connect two adjacent key points, connect the left ankle with the left hip, and connect the right ankle with the right hip to obtain multiple line segments; The length of each line segment, the angle between each line segment and the x-axis of the coordinate system, the angle between each line segment and the y-axis of the coordinate system, or the angle between each line segment and the other line segments are determined as the key point spatial relationship between multiple key points; wherein, the origin of the coordinate system is located at the midpoint of the line connecting the left and right toes of the standard positioning posture, the x-axis is perpendicular to the plane where the chest X-ray frame of the X-ray machine is located, the y-axis is parallel to the plane where the chest X-ray frame of the X-ray machine is located, and the z-axis is perpendicular to the ground.

3. The X-ray assisted imaging method based on key point recognition according to claim 2, characterized in that, The posture error information includes upper body posture error information, standing posture error information, and lower body posture error information; The method for obtaining the posture error information of the target patient based on the spatial relationship of each key point includes: Based on the spatial relationship between key points of the target patient's upper body posture, determine the upper body posture error information of the target patient; Based on the spatial relationship between multiple key points of the target patient's standing posture, the standing posture error information of the target patient is determined; Based on the spatial relationship between key points of the target patient's lower body posture, the lower body posture error information of the target patient is determined.

4. The X-ray assisted imaging method based on key point recognition according to claim 3, characterized in that, The key points of the upper body posture include the left shoulder, right shoulder, left elbow, right elbow, left wrist, and right wrist; the upper body posture error information includes shoulder posture error information, left arm posture error information, and right arm posture error information. The step of determining the upper body posture error information of the target patient based on the spatial relationship between key points of the upper body posture includes: The angle between the line segment formed by the left and right shoulders of the target patient and the y-axis is determined as the first angle. The first angle is compared with the first preset angle to obtain the shoulder posture error information. The angle between the line segment formed by the left shoulder and left elbow of the target patient and the line segment formed by the left elbow and left wrist is determined as the second angle. The second angle is compared with the second preset angle to obtain the left arm posture error information. The angle between the line segment formed by the right shoulder and right elbow of the target patient and the line segment formed by the right elbow and right wrist is determined as the third angle. The third angle is compared with the second preset angle to obtain the right arm posture error information. Accordingly, determining the posture adjustment suggestion based on the posture error information includes: If the absolute value of the first angle in the shoulder posture error information is greater than the first preset angle, the posture adjustment suggestion is to keep both shoulders level. If the absolute value of the difference between the second angle and the second preset angle in the arm posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the left arm hanging naturally. If the absolute value of the difference between the third angle and the second preset angle in the arm posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the right arm hanging naturally.

5. The X-ray assisted imaging method based on key point recognition according to claim 3, characterized in that, The key points of the standing posture include the left shoulder, right shoulder, left ankle, and right ankle; the determination of the standing posture error information of the target patient based on the spatial relationship between multiple key points of the target patient's standing posture includes: The distance between the left and right shoulders of the target patient is defined as the first distance, and the distance between the left and right ankles of the target patient is defined as the second distance; Calculate the first difference between the first distance and the second distance, and compare the first difference with a preset difference to obtain the standing posture error information of the target patient; Accordingly, determining the posture adjustment suggestion based on the posture error information includes: If the absolute value of the first difference in the standing posture error information is greater than the preset difference, the posture adjustment suggestion is to adjust the feet so that they are shoulder-width apart.

6. The X-ray assisted imaging method based on key point recognition according to claim 3, characterized in that, The key points of the lower body posture include the left knee, right knee, left toe, and right toe; the lower body posture error information includes foot posture error information, left leg posture error information, right leg posture error information, and center of gravity error information. Based on the spatial relationship between key points in the lower body posture of the target patient, the lower body posture error information of the target patient is determined, including: The angle between the line segment formed by the left toe and left ankle and the x-axis is determined as the fourth angle, and the angle between the line segment formed by the right toe and right ankle and the x-axis is determined as the fifth angle. The fourth angle and the fifth angle are compared with the third preset angle to obtain the foot posture error information. The angle between the line segment formed by the left hip and left knee and the line segment formed by the left knee and left ankle is determined as the sixth angle. The sixth angle is compared with the second preset angle to obtain the left leg posture error information. The angle between the line segment formed by the right hip and right knee and the line segment formed by the right knee and right ankle is determined as the seventh angle. The seventh angle is compared with the second preset angle to obtain the right leg posture error information. The third distance between the left hip and the left ankle and the distance between the right hip and the right ankle are defined as the fourth distance; Calculate the second difference between the third distance and the fourth distance, and compare the second difference with a preset difference to obtain the centroid error information of the target patient; Accordingly, determining the posture adjustment suggestion based on the posture error information includes: If the absolute value of the fourth angle in the foot posture error information is greater than the third preset angle, the posture adjustment suggestion is to keep the left toes pointing forward; if the absolute value of the fifth angle in the foot posture error is greater than the third preset angle, the posture adjustment suggestion is to keep the right toes pointing forward. If the absolute value of the difference between the sixth angle and the second preset angle in the left leg posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the left knee fully extended. If the absolute value of the difference between the seventh angle and the second preset angle in the right leg posture error information is greater than the preset angle difference, the posture adjustment suggestion is to keep the right knee fully extended. If the absolute value of the second difference in the center of gravity error information is greater than the preset difference, the posture adjustment suggestion is to maintain a uniform center of gravity.

7. The X-ray assisted imaging method based on key point recognition according to claim 1, characterized in that, The standard positioning posture is the correct posture when taking a full-length frontal X-ray of both lower limbs. The standard positioning posture includes shoulders level, arms hanging naturally, feet shoulder-width apart, toes pointing forward, knees fully extended, and center of gravity evenly distributed.

8. The X-ray assisted imaging method based on key point recognition according to claim 1, characterized in that, The human body key point recognition model is the YOLO model.

9. An X-ray assisted imaging device based on key point recognition, characterized in that, include: The acquisition module is used to acquire real-time posture images of the target patient based on a pre-built image acquisition device; The recognition module is used to perform key point recognition on the real-time posture image of the target patient based on a pre-trained human key point recognition model, and to mark the recognized key points on the real-time posture image. The determination module is used to determine the key point spatial relationship between at least two key points based on the labeled real-time pose image and the standard pose. The suggestion module is used to obtain the posture error information of the target patient based on the spatial relationship of each key point, and to determine posture adjustment suggestions based on the posture error information, so as to take X-rays of the target patient according to the adjustment results.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 8.

Images