Method, device, medium and product for calculating distance from mandibular feature point to ground

Abstract

The invention provides a method and device for calculating the distance between a mandibular feature point and the ground, a medium and a product, and the method comprises the steps: obtaining a face image, extracting a face bounding box, and extracting a two-dimensional coordinate point of the mandibular feature point from the face bounding box; obtaining a face region point cloud subset according to the face point cloud image; constructing a space ray equation according to the two-dimensional coordinate points of the mandibular feature points and the internal reference of a camera shooting device for shooting a face image; finding a point closest to a ray represented by the space ray equation from the face region point cloud subset as a three-dimensional coordinate point of the lower jaw feature point; acquiring a three-dimensional coordinate point fitting ground plane equation of at least three lower jaw feature points; and converting the three-dimensional coordinate points of the mandible feature points from a camera coordinate system to a world coordinate system, and calculating the distance from the mandible to the ground. The technical problems that in the prior art, a patient needs to cooperate with additional actions or equipment for chest radiography shooting, and the discomfort of the patient and the operation complexity of a doctor are increased are solved.

Description

Technical Field

[0001] The present invention relates to the field of software, and in particular to a method, device, medium and product for calculating the distance from a mandibular feature point to the ground. Background Art

[0002] In the field of medical diagnosis, especially in digital radiography (DR) chest examinations, accurately acquiring the three-dimensional spatial coordinates of specific patient parts and adjusting the patient's position and imaging equipment accordingly is a key step in ensuring image quality and diagnostic accuracy. Existing technologies primarily rely on the doctor's voice guidance (such as "chest up," "abdomen in," "adjust standing position," etc.) or the doctor repeatedly moves back and forth between the operating table and the standing detector assembly, manually helping the patient adjust their posture and body position, while also adjusting parameters such as the height and angle of the standing detector assembly.

[0003] However, the above-mentioned traditional positioning methods have significant defects: 1. Low efficiency of manual intervention: Doctors need to repeatedly observe the patient's posture, operating table image and equipment parameters, which is time-consuming. Especially when the patient's cooperation is low or the body shape is special, the positioning process may need to be corrected multiple times, resulting in a lengthy examination process; 2. High labor intensity for doctors: Frequent travel back and forth between the operating table and the examination area not only increases the doctor's workload, but also may bring health risks due to multiple exposure to X-ray radiation environment; 3. Positioning accuracy depends on experience: The accuracy of patient position adjustment is highly dependent on the doctor's clinical experience and lacks quantitative three-dimensional spatial coordinate data support. It may cause the relative position deviation between the standing detector assembly and the patient's target part, thereby affecting the imaging quality (such as exposure area offset, tissue overlap, etc.) and increasing the probability of repeated examinations; 4. Poor patient experience: Non-intuitive voice guidance may lead to patient misunderstanding deviation, and multiple passive posture adjustments can easily cause tension and reduce examination cooperation. Summary of the Invention

[0004] Based on the above problems, the present invention proposes a method, device, medium and product for calculating the distance from the mandibular feature point to the ground. The present invention solves the technical problem in the prior art that taking chest X-rays requires the patient to cooperate with additional actions or equipment, which increases the patient's discomfort and the complexity of the doctor's operation. The method of the present invention enables the patient to achieve real-time and non-sensing positioning without additional cooperation, improves DR image quality, shooting efficiency and patient comfort, reduces manual intervention, and the method provided by the present invention is simple to deploy and low-cost.

[0005] The present invention proposes a method for calculating the distance from a mandibular feature point to the ground, comprising:

[0006] Obtain a face image and extract a face bounding box, and extract the two-dimensional coordinate points of the mandibular feature points from the face bounding box;

[0007] Obtain a face area point cloud subset based on the face point cloud image;

[0008] Constructing a spatial ray equation based on the two-dimensional coordinates of the mandibular feature points and the internal parameters of the camera device that captures the facial image;

[0009] Find the point closest to the ray represented by the spatial ray equation from the face area point cloud subset as the three-dimensional coordinate point of the mandibular feature point;

[0010] Obtain three-dimensional coordinate points of at least three mandibular feature points and fit them to the ground plane equation;

[0011] The three-dimensional coordinates of the mandibular feature points are converted from the camera coordinate system to the world coordinate system with the ground plane equation as a reference, and the three-dimensional coordinates of the mandibular feature points in the world coordinate system are used to calculate the distance from the mandibular to the ground.

[0012] In addition, obtaining a face area point cloud subset according to the face point cloud image includes:

[0013] A three-dimensional search space is constructed according to the center of the face bounding box, and the points that meet the screening conditions constitute the face area point cloud subset.

[0014] In addition, the construction of the spatial ray equation based on the two-dimensional coordinate points of the mandibular feature points and the internal parameters of the camera device for capturing the facial image includes:

[0015] Input the two-dimensional coordinates of the mandibular feature points and normalize them using the internal parameters of the camera device;

[0016] Determine the ray direction vector;

[0017] The space ray equation is constructed based on the normalized two-dimensional coordinate points and the ray direction vector. The ray direction vector is

[0018]

[0019] Among them, K is the internal parameter of the camera device, x c ,y c is a two-dimensional coordinate point;

[0020] The space ray equation is

[0021]

[0022] Where t represents the distance the ray extends from the optical center along the direction of the ray direction vector.

[0023] In addition, the method of finding the point closest to the ray represented by the spatial ray equation from the face region point cloud subset as the three-dimensional coordinate point of the mandibular feature point includes:

[0024] The calculation formula is:

[0025]

[0026] Among them, o is the origin of the camera coordinate system, is the three-dimensional coordinate point of the mandibular feature point, and P is the nearest point.

[0027] Furthermore, obtaining the three-dimensional coordinate points of at least three mandibular feature points and fitting the ground plane equation includes:

[0028] The least square method is used to fit the ground plane equation Ax+By+Cz+D=0;

[0029] Among them, A, B, C, D are the plane parameters to be determined;

[0030] Input the three-dimensional coordinates of the mandibular feature points into the ground plane equation to solve the optimal plane parameters.

[0031] In addition, the reference measurement value H is introduced ref Make the fitted ground plane equation consistent with the actual ground, H ref The actual measured distance from the mandibular point to the ground. The plane parameters must meet the following conditions:

[0032]

[0033] Among them, X ref , Y ref , Z ref is the three-dimensional coordinate point of the measured mandible.

[0034] In addition, the three-dimensional coordinates of the mandibular feature points are converted from the camera coordinate system to a world coordinate system with the ground plane equation as a reference, and the three-dimensional coordinates of the mandibular feature points in the world coordinate system are used to calculate the distance from the mandibular to the ground, including:

[0035] The coordinate conversion formula for converting the three-dimensional coordinate point of the mandibular feature point from the camera coordinate system to the world coordinate system with the ground plane equation as the reference is:

[0036]

[0037] in, is the three-dimensional coordinate point in the world coordinate system, is the three-dimensional coordinate point in the camera coordinate system, R is the rotation matrix, and T is the translation vector. R and T are determined by the ground plane equation;

[0038] The calculation formula for calculating the distance d from the mandible to the ground using the three-dimensional coordinates of the mandibular feature points in the world coordinate system is:

[0039]

[0040] Among them, x w ,y w , z w is the three-dimensional coordinate point of the mandible in the world coordinate system.

[0041] The present invention further provides an electronic device, comprising:

[0042] at least one processor; and,

[0043] a memory communicatively connected to at least one of the processors; wherein,

[0044] The memory stores instructions that can be executed by at least one of the processors, and the instructions are executed by at least one of the processors to enable the at least one processor to perform the method for calculating the distance from a mandibular feature point to the ground as described in any one of the above items.

[0045] The present invention also proposes a storage medium storing computer instructions. When a computer executes the computer instructions, it is used to execute all steps of any of the above-mentioned methods for calculating the distance from a mandibular feature point to the ground.

[0046] The present invention also proposes a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, it implements any of the above-mentioned methods for calculating the distance from the mandibular feature point to the ground.

[0047] The present invention solves the technical problem in the prior art that chest X-rays require patients to cooperate with additional actions or equipment, which increases the patient's discomfort and the complexity of the doctor's operation. The method of the present invention enables patients to achieve real-time, non-sensing positioning without additional cooperation, improves DR image quality, shooting efficiency and patient comfort, reduces manual intervention, and the method provided by the present invention is simple to deploy and low-cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a method for calculating the distance from a mandibular feature point to the ground provided in one embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a photographing device according to an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of image processing provided by one embodiment of the present invention;

[0051] Figure 4 A schematic diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described in detail below with reference to specific embodiments and accompanying drawings, which are intended only to elaborate on the specific embodiments of the present invention and do not impose any limitation on the present invention. The scope of protection of the present invention shall be subject to the claims.

[0053] Reference Figure 1 The present invention proposes a method for calculating the distance from the mandibular feature point to the ground, comprising:

[0054] Step S001: Acquire a face image and extract a face bounding box, and extract the two-dimensional coordinate points of the mandibular feature point from the face bounding box;

[0055] Step S002, obtaining a face area point cloud subset based on the face point cloud image;

[0056] Step S003, constructing a spatial ray equation based on the two-dimensional coordinates of the mandibular feature points and the internal parameters of the camera device that captures the facial image;

[0057] Step S004, finding the point closest to the ray represented by the spatial ray equation from the face region point cloud subset as the three-dimensional coordinate point of the mandibular feature point;

[0058] Step S005, obtaining the three-dimensional coordinates of at least three mandibular feature points and fitting them to the ground plane equation;

[0059] Step S006: convert the three-dimensional coordinates of the mandibular feature points from the camera coordinate system to a world coordinate system with the ground plane equation as a reference, and calculate the distance from the mandibular feature points to the ground using the three-dimensional coordinates of the mandibular feature points in the world coordinate system.

[0060] Existing 3D facial reconstruction technologies mainly include binocular stereo vision, structured light scanning, and multi-perspective reconstruction based on deep learning. Binocular stereo vision technology simulates the parallax principle of the human eye, uses two cameras to acquire images from different angles, and calculates the parallax to restore the depth information of the scene. However, this method is sensitive to lighting conditions and occlusions, and the equipment installation and calibration are complex. Structured light scanning technology projects a specific grating pattern onto the surface of an object and analyzes the deformation of the pattern to reconstruct the three-dimensional shape. It has high accuracy, but the equipment cost is high and it has strict requirements on ambient lighting. The multi-perspective reconstruction method based on deep learning uses a large amount of training data to learn the mapping relationship from two-dimensional images to three-dimensional shapes. It has good robustness, but requires a large amount of labeled data and has high computing resource requirements.

[0061] Traditional methods for facial landmark detection, such as active shape models (ASMs) and active appearance models (AAMs), use statistical models to describe facial shape variations, but are less adaptable to changes in expression and posture. Non-parametric methods based on cascaded shape regression optimize landmark positions through step-by-step regression, improving detection accuracy. In recent years, advances in deep learning technology have enabled significant progress in facial landmark detection methods based on convolutional neural networks (CNNs), enabling high-precision landmark location in complex environments.

[0062] However, in medical diagnostic scenarios, particularly during DR chest X-rays, patients are required to maintain specific postures. Traditional 3D reconstruction and key point detection methods require additional movements or equipment, increasing patient discomfort and complicating the physician's operation. Therefore, a method is urgently needed to utilize machine vision technology to seamlessly acquire the 3D coordinates of key facial features, such as the mandible, during DR chest X-rays. This approach can improve diagnostic accuracy and efficiency, minimize patient disruption, and enhance the efficiency and quality of DR imaging.

[0063] In step S001, a face image is acquired and a face bounding box is extracted, and the two-dimensional coordinate points of the mandibular feature points are extracted from the face bounding box;

[0064] The process of obtaining a face image and extracting the face bounding box is as follows:

[0065] There are many methods for image edge detection, such as extracting the face outline through Canny operator edge detection and then generating a bounding box. Usually, a set of numerical values ​​is used to represent the position and range of the face in the image.

[0066] For example, the upper left corner + lower right corner coordinate representation (xmin, ymin, xmax, ymax), for example: (100, 150, 300, 400) means the left boundary is in the 100th column, the upper boundary is in the 150th row, the right boundary is in the 300th column, and the lower boundary is in the 400th row.

[0067] The representation of center point + width and height (x_center, y_center, width, height) is as follows:

[0068] The rectangle is represented by the center coordinates (x_center, y_center) plus the width and height. For example, (200,275,200,250) means the center is at (200,275), the width is 200 pixels, and the height is 250 pixels.

[0069] Normalization can be performed later.

[0070] The two-dimensional coordinate points of the mandibular feature points extracted from the face bounding box are as follows:

[0071] Extracting mandibular key points involves filtering out mandibular-related points (such as the mandibular apex and mandibular line points) from the key point list. For example, in a 68-point model, the mandibular key points are located at points 17-26 (a total of 10 points, arranged clockwise from left to right along the mandibular line). Point 17 is the starting point of the left mandible (the leftmost end of the mandibular line), point 26 is the starting point of the right mandible (the rightmost end of the mandibular line), and point 24 is the mandibular apex (the lowest point on the mandibular line and the point with the largest y coordinate). Each point is represented by pixel coordinates (x, y). For example, point 24 has coordinates (x24, y24).

[0072] You can also choose a lightweight model to output 5 points, 10 points, or only the mandibular tip (or mandibular midpoint). For example, the 5th point is the midpoint of the mandibular bottom (x_center, y_bottom).

[0073] The process of extracting mandibular key points can be quickly implemented by directly calling mature image processing libraries (such as Dlib and OpenCV) without manually calculating coordinates.

[0074] Face images can be acquired through RGB cameras.

[0075] In step S002, a face area point cloud subset is obtained based on the face point cloud image;

[0076] The facial point cloud image can be acquired by a depth point cloud camera. A 3D search space is constructed based on the center of the face bounding box, and points that meet the screening criteria constitute the face area point cloud subset.

[0077] The specific method is to traverse each point (x, y, z) in the original point cloud (face bounding box), retaining the points that meet the preset conditions to form the face area point cloud subset. The process from raw data to point cloud subset is the process of filtering out useful points, retaining only the necessary points. Therefore, the input data is the original image data, and the output data is the filtered image data.

[0078] In step S003, a spatial ray equation is constructed based on the two-dimensional coordinates of the mandibular feature points and the internal parameters of the camera device that captures the facial image;

[0079] Input the two-dimensional coordinates of the mandibular feature points and normalize them using the internal parameters of the camera device;

[0080] Determine the ray direction vector;

[0081] The space ray equation is constructed based on the normalized two-dimensional coordinate points and the ray direction vector. The ray direction vector is

[0082]

[0083] Among them, K is the internal parameter of the camera device, x c ,y c is a two-dimensional coordinate point;

[0084] The space ray equation is

[0085]

[0086] Where t represents the distance the ray extends from the optical center along the direction of the ray direction vector.

[0087] The spatial ray equation represents a three-dimensional ray starting from the optical center of the camera and passing through a point on the image plane. The ray direction vector points from the optical center to the unit vector of the target point on the image plane.

[0088] The two-dimensional coordinate point is converted into the normalized coordinate in the camera coordinate system through the camera intrinsic parameter matrix K.

[0089] In the camera coordinate system, the ray starts from the optical center (0,0,0) and passes through the normalized coordinate point (x,y,1), so the direction vector is:

[0090]

[0091] Substituting the starting point (0,0,0) and the direction vector, we get the spatial ray equation:

[0092]

[0093] In step S004, the point closest to the ray represented by the spatial ray equation is found from the face area point cloud subset as the three-dimensional coordinate point of the mandibular feature point;

[0094] The calculation formula is:

[0095]

[0096] Among them, o is the origin of the camera coordinate system, is the three-dimensional coordinate point of the mandibular feature point, and P is the nearest point.

[0097] In step S005, the three-dimensional coordinates of at least three mandibular feature points are obtained and fitted to the ground plane equation;

[0098] The least square method is used to fit the ground plane equation Ax+By+Cz+D=0;

[0099] Among them, A, B, C, D are the plane parameters to be determined;

[0100] Input the three-dimensional coordinates of the mandibular feature points into the ground plane equation to solve the optimal plane parameters.

[0101] When solving for the optimal plane parameters, the singular value decomposition (SVD) method and the normal equation formula solving method can be used to solve for the optimal plane parameters.

[0102] For example, the process of solving the normal equation formula is as follows:

[0103]

[0104] Solution:

[0105] In this case, C is the inverse of the coefficient of Z.

[0106] M is the design matrix, Z is the observation vector, M T is the transpose of M, (M T M) -1 is the matrix M T The inverse of M.

[0107] The SVD solution is to decompose the original matrix M and use its pseudo-inverse M + To directly calculate the solution. The two methods are mature algorithms and will not be described in detail here.

[0108] In step S006, the three-dimensional coordinates of the mandibular feature points are converted from the camera coordinate system to the world coordinate system with the ground plane equation as a reference, and the three-dimensional coordinates of the mandibular feature points in the world coordinate system are used to calculate the distance from the mandibular to the ground.

[0109] The coordinate conversion formula for converting the three-dimensional coordinate point of the mandibular feature point from the camera coordinate system to the world coordinate system with the ground plane equation as the reference is:

[0110]

[0111] in, is the three-dimensional coordinate point in the world coordinate system, is the three-dimensional coordinate point in the camera coordinate system, R is the rotation matrix, and T is the translation vector. R and T are determined by the ground plane equation;

[0112] The calculation formula for calculating the distance d from the mandible to the ground using the three-dimensional coordinates of the mandibular feature points in the world coordinate system is:

[0113]

[0114] Among them, x w ,y w , z w is the three-dimensional coordinate point of the mandible in the world coordinate system.

[0115] The shooting device in the present invention is an integration of an RGB camera and a point cloud depth camera.

[0116] The present invention provides a real-time, non-sensing three-dimensional positioning method for human body feature points based on the fusion of RGB camera and point cloud depth camera, which is particularly suitable for medical imaging DR imaging scenes. The method includes: real-time acquisition of patient RGB images and depth images; using face detection and key point recognition algorithms based on deep learning to accurately determine the position of the patient's mandibular feature points from the two-dimensional image; establishing a local search area in three-dimensional space, combining the camera's internal reference model to construct spatial rays, and quickly and accurately obtaining the three-dimensional coordinates of the mandible; by collecting the three-dimensional coordinate data of the mandible at multiple positions, fitting the ground reference plane, performing coordinate system conversion, calculating and displaying in real time the vertical distance from the mandibular feature point to the ground. The method of the present invention is simple to deploy and low-cost, and can achieve real-time non-sensing positioning without the need for additional cooperation from the patient, effectively improving the DR image quality, shooting efficiency and patient comfort, and has important clinical application value.

[0117] The present invention solves the technical problem in the prior art that chest X-rays require patients to cooperate with additional actions or equipment, which increases the patient's discomfort and the complexity of the doctor's operation. The method of the present invention enables patients to achieve real-time, non-sensing positioning without additional cooperation, improves DR image quality, shooting efficiency and patient comfort, reduces manual intervention, and the method provided by the present invention is simple to deploy and low-cost.

[0118] In one embodiment, obtaining a face region point cloud subset based on a face point cloud image includes:

[0119] A three-dimensional search space is constructed according to the center of the face bounding box, and the points that meet the screening conditions constitute the face area point cloud subset.

[0120] By establishing a subset of the face area point cloud, the processed data is reduced and the subsequent processing speed is increased.

[0121] In one embodiment, constructing a spatial ray equation based on the two-dimensional coordinates of the mandibular feature point and the internal parameters of the camera device that captures the facial image includes:

[0122] Input the two-dimensional coordinates of the mandibular feature points and normalize them using the internal parameters of the camera device;

[0123] Determine the ray direction vector;

[0124] The space ray equation is constructed based on the normalized two-dimensional coordinate points and the ray direction vector. The ray direction vector is

[0125]

[0126] Among them, K is the internal parameter of the camera device, x c ,y c is a two-dimensional coordinate point;

[0127] The space ray equation is

[0128]

[0129] Where t represents the distance the ray extends from the optical center along the direction of the ray direction vector.

[0130] By constructing the spatial ray equation, we prepare for the subsequent finding of the three-dimensional coordinate points of the mandibular feature points.

[0131] In one embodiment, finding the point closest to the ray represented by the spatial ray equation from the facial region point cloud subset as the three-dimensional coordinate point of the mandibular feature point includes:

[0132] The calculation formula is:

[0133]

[0134] Among them, o is the origin of the camera coordinate system, is the three-dimensional coordinate point of the mandibular feature point, and P is the nearest point.

[0135] Prepare input data for fitting the ground plane equation by finding the 3D coordinates of the mandibular feature points.

[0136] In one embodiment, obtaining the three-dimensional coordinates of at least three mandibular feature points and fitting the ground plane equation includes:

[0137] The least square method is used to fit the ground plane equation Ax+By+Cz+D=0;

[0138] Among them, A, B, C, D are the plane parameters to be determined;

[0139] Input the three-dimensional coordinates of the mandibular feature points into the ground plane equation to solve the optimal plane parameters.

[0140] When solving for the optimal plane parameters, the singular value decomposition (SVD) method and the normal equation formula solving method can be used to solve for the optimal plane parameters.

[0141] For example, the process of solving the normal equation formula is as follows:

[0142]

[0143] Solution:

[0144] In this case, C is the inverse of the coefficient of Z.

[0145] M is the design matrix, Z is the observation vector, M - is the transpose of M, (MT M)-1 is the matrix M - The inverse of M.

[0146] The SVD solution is to decompose the original matrix M and use its pseudo-inverse M + To directly calculate the solution. The two methods are mature algorithms and will not be described in detail here.

[0147] In one embodiment, a reference measurement value H is introduced. ref Make the fitted ground plane equation consistent with the actual ground, H ref The actual measured distance from the mandibular point to the ground. The plane parameters must meet the following conditions:

[0148]

[0149] Among them, X ref , Y ref , Z ref is the three-dimensional coordinate point of the measured mandible.

[0150] In order to accurately determine the ground position, it is necessary to input a reference measurement value obtained by conventional measurement means (such as a handheld laser rangefinder, etc.), that is, the actual measured distance H from the tester's jaw point to the ground in a level view state. ref , through H ref The introduction of makes the fitted ground plane equation consistent with the actual ground.

[0151] By introducing the fitted ground plane equation and actual measurement data, the plane parameters are optimized and adjusted so that the fitted ground plane is calibrated consistent with the actual ground height, and the plane parameters of the accurate ground plane equation are obtained.

[0152] In one embodiment, converting the three-dimensional coordinates of the mandibular feature points from the camera coordinate system to a world coordinate system with a ground plane equation as a reference, and calculating the distance from the mandibular feature points to the ground using the three-dimensional coordinates of the mandibular feature points in the world coordinate system includes:

[0153] The coordinate conversion formula for converting the three-dimensional coordinate point of the mandibular feature point from the camera coordinate system to the world coordinate system with the ground plane equation as the reference is:

[0154]

[0155] in, is the three-dimensional coordinate point in the world coordinate system, is the three-dimensional coordinate point in the camera coordinate system, R is the rotation matrix, and T is the translation vector. R and T are determined by the ground plane equation;

[0156] The calculation formula for calculating the distance d from the mandible to the ground using the three-dimensional coordinates of the mandibular feature points in the world coordinate system is:

[0157]

[0158] Among them, x w ,y w , z w is the three-dimensional coordinate point of the mandible in the world coordinate system.

[0159] The calculated distance d from the mandible to the ground can help the physician to efficiently adjust the height of the standing detector assembly. The physician can also accurately adjust the height of the standing detector assembly by automatically adjusting the height, thereby improving the shooting quality of DR images.

[0160] The technical solution proposed in this embodiment can realize the three-dimensional spatial positioning of the mandible in real time and without any sensation, without the patient's cooperation of additional actions, significantly improving the quality of DR chest X-ray images, reducing patient discomfort, and helping to improve the efficiency and accuracy of medical diagnosis.

[0161] The method for calculating the distance from the mandibular feature point to the ground provided in this embodiment is as follows in practical application:

[0162] 1. Early Deployment

[0163] 1. Install the following visual equipment: an RGB camera and a point cloud depth camera, at a certain angle to the standing detector assembly in the DR examination room and near the ceiling. These two devices are oriented toward the standing detector assembly where the person being examined will be imaged. They are used to capture real-time RGB images and depth point cloud images of the person being examined. The visual equipment's field of view must cover the person's face, frontally or sideways, within a certain range to facilitate effective positioning by the facial recognition algorithm.

[0164] 2. The relevant operator (non-specific) enters the visual field of the visual device and collects and reads the facial mandibular feature points in multiple non-collinear positions in sequence {P chin,i3D}, at least 3 points, collecting more points such as 5 to 7 will further improve the accuracy. At this time, the coordinate system of the 3D feature point set is the vision device coordinate system.

[0165] 3. Relevant personnel use other high-precision distance measuring equipment, such as a handheld laser rangefinder, to measure the height H from the lower jaw to the ground in a level view. ref .

[0166] 4. After obtaining an effective set of feature points for plane fitting and the height of the facial features relative to the ground, use the fitted ground plane equation to obtain the plane geometric parameters of the plane where the ground is located. Using this plane as a reference, complete the conversion from the visual device coordinate system (camera coordinate system) to the indoor world coordinate system.

[0167] 5. After completing the preliminary deployment and coordinate system conversion, the present invention enters the real-time use stage. When the patient stands in front of the standing detector assembly and prepares to take DR images, the system automatically starts the RGB camera and point cloud depth camera in real time to collect the patient's RGB image and depth point cloud data. Subsequently, the present invention system automatically performs the following real-time positioning process:

[0168] (1) Detect the patient’s facial area in real time in the RGB image and locate the two-dimensional coordinates of the mandibular feature points;

[0169] (2) Using the calibrated camera intrinsic parameter model, the spatial ray equation is calculated through the two-dimensional position of the mandible to determine the three-dimensional coordinates of the mandible in the camera coordinate system;

[0170] (3) converting the three-dimensional coordinate points of the mandible obtained above into the indoor world coordinate system determined in advance in real time;

[0171] (4) The actual vertical distance (height) from the patient's mandible to the ground is calculated in real time and displayed intuitively on the operation interface, so that the physician or the automated adjustment device can quickly and accurately adjust the height of the standing detector assembly or guide the patient to adjust the posture based on the real-time feedback information, ultimately obtaining the best DR image shooting quality, significantly improving shooting efficiency and patient comfort.

[0172] The above real-time process does not require any additional cooperation or action from the patient, and is completely non-sensory. It will not cause interference to the patient or increase additional discomfort, and has good clinical practicality and user experience advantages.

[0173] Reference Figure 4 The present invention also provides a hardware structure diagram of an electronic device, including:

[0174] at least one processor 301; and,

[0175] A memory 302 in communication with at least one of the processors 301; wherein,

[0176] The memory 302 stores instructions that can be executed by at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the above method for calculating the distance from the mandibular feature point to the ground.

[0177] Figure 4 A processor 301 is taken as an example.

[0178] The electronic device is preferably a controller of the vehicle. The electronic device may further include: an input device 303 and a display device 304 .

[0179] The processor 301 , the memory 302 , the input device 303 and the display device 304 may be connected via a bus or other means, with the bus connection being used as an example in the figure.

[0180] The memory 302 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the program instructions / modules corresponding to the method for calculating the distance from the mandibular feature point to the ground in the embodiment of the present application, for example, Figure 2 The processor 301 executes the non-volatile software programs, instructions and modules stored in the memory 302 to perform various functional applications and data processing, that is, to implement the method for calculating the distance from the mandibular feature point to the ground in the above embodiment.

[0181] Memory 302 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function; the data storage area may store data generated by the method for calculating the distance between the mandibular feature point and the ground, etc. Furthermore, memory 302 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, memory 302 may optionally include memory remote from processor 301. Such remote memory may be connected to a device executing the method for calculating the distance between the mandibular feature point and the ground via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0182] The input device 303 can receive user clicks and generate signal input related to user settings and function control of the method for calculating the distance from the mandibular feature point to the ground. The display device 304 can include a display device such as a display screen.

[0183] One or more modules are stored in the memory 302 and, when executed by one or more processors 301 , perform the method for calculating the distance from the mandibular feature point to the ground in any of the above method embodiments.

[0184] The present invention solves the technical problem in the prior art that chest X-rays require patients to cooperate with additional actions or equipment, which increases the patient's discomfort and the complexity of the doctor's operation. The method of the present invention enables patients to achieve real-time, non-sensing positioning without additional cooperation, improves DR image quality, shooting efficiency and patient comfort, reduces manual intervention, and the method provided by the present invention is simple to deploy and low-cost.

[0185] One embodiment of the present invention provides a storage medium storing computer instructions. When executed by a computer, the computer executes all steps of the method for calculating the distance from the mandibular feature point to the ground, as described above. This invention addresses the technical problem in the prior art of chest X-rays, which requires the patient to cooperate with additional actions or equipment, increasing patient discomfort and the complexity of the physician's operation. The method of the present invention enables real-time, non-invasive positioning without the patient's need for additional cooperation, improving DR image quality, shooting efficiency, and patient comfort, while reducing manual intervention. Furthermore, the method provided by the present invention is simple to deploy and low-cost.

[0186] In the context of the present disclosure, a storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. The storage medium may be a machine-readable signal medium or a machine-readable storage medium. Alternatively, the storage medium may be a non-transitory computer-readable storage medium, for example, a non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0187] One embodiment of the present invention provides a computer program product comprising a computer program / instructions that, when executed by a processor, implements the aforementioned method for calculating the distance from a mandibular feature point to the ground. This invention addresses the prior art technical problem that chest X-rays require additional patient actions or equipment, increasing patient discomfort and the complexity of the physician's operation. The method of the present invention enables real-time, seamless positioning without the need for additional patient cooperation, improving DR image quality, capture efficiency, and patient comfort while reducing manual intervention. Furthermore, the method provided by the present invention is simple to deploy and inexpensive.

[0188] The above description is only the principle and preferred embodiment of the present invention. It should be noted that for those skilled in the art, several other variations can be made based on the principle of the present invention, which should also be considered as the scope of protection of the present invention.

Claims

1. A method for calculating the distance from a mandibular feature point to the ground, characterized in that: include: Obtain a face image and extract a face bounding box, and extract the two-dimensional coordinate points of the mandibular feature points from the face bounding box; Obtain a face area point cloud subset based on the face point cloud image; Constructing a spatial ray equation based on the two-dimensional coordinates of the mandibular feature points and the internal parameters of the camera device that captures the facial image; Find the point closest to the ray represented by the spatial ray equation from the face area point cloud subset as the three-dimensional coordinate point of the mandibular feature point; Obtain three-dimensional coordinate points of at least three mandibular feature points and fit them to the ground plane equation; The three-dimensional coordinates of the mandibular feature points are converted from the camera coordinate system to the world coordinate system with the ground plane equation as a reference, and the three-dimensional coordinates of the mandibular feature points in the world coordinate system are used to calculate the distance from the mandibular to the ground.

2. The method for calculating the distance from the mandibular feature point to the ground according to claim 1, characterized in that: The step of obtaining a face area point cloud subset according to the face point cloud image comprises: A three-dimensional search space is constructed according to the center of the face bounding box, and the points that meet the screening conditions constitute the face area point cloud subset.

3. The method for calculating the distance from the mandibular feature point to the ground according to claim 1, characterized in that: The construction of the spatial ray equation based on the two-dimensional coordinate points of the mandibular feature points and the internal parameters of the camera device for capturing the facial image includes: Input the two-dimensional coordinates of the mandibular feature points and normalize them using the internal parameters of the camera device; Determine the ray direction vector; The space ray equation is constructed based on the normalized two-dimensional coordinate points and the ray direction vector. The ray direction vector is Among them, K is the internal parameter of the camera device, x c ,y c is a two-dimensional coordinate point; The space ray equation is Where t represents the distance the ray extends from the optical center along the direction of the ray direction vector.

4. The method for calculating the distance from the mandibular feature point to the ground according to claim 3, characterized in that: The method of finding the point closest to the ray represented by the spatial ray equation from the face region point cloud subset as the three-dimensional coordinate point of the mandibular feature point includes: The calculation formula is: Among them, o is the origin of the camera coordinate system, is the three-dimensional coordinate point of the mandibular feature point, and P is the nearest point.

5. The method for calculating the distance from the mandibular feature point to the ground according to claim 1, characterized in that: The step of obtaining the three-dimensional coordinates of at least three mandibular feature points and fitting the ground plane equation comprises: The least square method is used to fit the ground plane equation Ax+By+Cz+D=0; Among them, A, B, C, D are the plane parameters to be determined; Input the three-dimensional coordinates of the mandibular feature points into the ground plane equation to solve the optimal plane parameters.

6. The method for calculating the distance from the mandibular feature point to the ground according to claim 5, characterized in that: Introducing the reference measurement value H ref Make the fitted ground plane equation consistent with the actual ground, H ref The actual measured distance from the mandibular point to the ground. The plane parameters must meet the following conditions: Among them, X ref ,Y ref ,Z ref is the three-dimensional coordinate point of the measured mandible.

7. The method for calculating the distance from the mandibular feature point to the ground according to claim 6, characterized in that: The step of converting the three-dimensional coordinates of the mandibular feature points from the camera coordinate system to a world coordinate system with the ground plane equation as a reference, and calculating the distance from the mandibular feature points to the ground using the three-dimensional coordinates of the mandibular feature points in the world coordinate system comprises: The coordinate conversion formula for converting the three-dimensional coordinate point of the mandibular feature point from the camera coordinate system to the world coordinate system with the ground plane equation as the reference is: in, is the three-dimensional coordinate point in the world coordinate system, is the three-dimensional coordinate point in the camera coordinate system, R is the rotation matrix, and T is the translation vector. R and T are determined by the ground plane equation; The calculation formula for calculating the distance d from the mandible to the ground using the three-dimensional coordinates of the mandibular feature points in the world coordinate system is: Among them, x w ,y w ,z w is the three-dimensional coordinate point of the mandible in the world coordinate system.

8. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by at least one of the processors, and the instructions are executed by at least one of the processors to enable at least one of the processors to perform the method for calculating the distance from a mandibular feature point to the ground according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium stores computer instructions, and when a computer executes the computer instructions, it is used to perform all steps of the method for calculating the distance from a mandibular feature point to the ground according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for calculating the distance from a mandibular feature point to the ground as claimed in any one of claims 1 to 7 is implemented.

Images