Oral positioning method using 3D vision

Abstract

The application provides a 3D visual oral cavity positioning method, and the method steps of the application are as follows: first step: face point cloud collection, collecting point cloud data of a face front by using a 3D sensor; second step: point cloud noise removal and hole filling. The collected point cloud is uneven and has holes due to the interference of environmental light, the collected point cloud is subjected to median smoothing treatment first, the noise of the point cloud is removed, then the collected point cloud is subjected to mean smoothing treatment by using a fixed step step, the uniform human face smoothing point cloud is obtained, and the minimum value and the maximum value of the human face point cloud in the x direction and the y direction are calculated, the bottom position information of the oral cavity is calculated according to requirements and is sent to a robot to realize nucleic acid collection, the method has high calculation efficiency and strong universality, the positioning method can quickly and accurately position the oral cavity, has high positioning precision, and meets the requirements of a full-automatic nucleic acid detection workstation.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and more particularly to a 3D vision method for oral cavity localization. Background Technology

[0002] Oral cavity positioning is a crucial component of fully automated nucleic acid testing workstations. The stability of oral cavity position detection determines the overall stability of the workstation. Existing automated nucleic acid testing workstations generally require an oral cavity fixator to fix the position of the patient's mouth, which increases both cost and the risk of cross-infection. Therefore, there is an urgent need for a 3D vision-based oral cavity positioning method to reduce the risk of cross-infection. Summary of the Invention

[0003] To address the above technical problems, this invention provides a 3D vision-based method for oral cavity localization. The steps of this method are as follows:

[0004] Step 1: Facial point cloud acquisition, using a 3D sensor to acquire point cloud data of the front of the face.

[0005] Step 2: Noise removal and hole filling in the point cloud. Due to interference from ambient light, the acquired point cloud is uneven and contains gaps. First, median smoothing is performed on the acquired point cloud, and radius filtering is used to remove noise from the discrete point cloud. Then, mean smoothing is performed on the acquired point cloud using a fixed step size to obtain a uniform smoothed point cloud of the human face. The minimum and maximum values ​​of the human face point cloud in the x and y directions are calculated and denoted as (x...). min ,x max ), (y min ,y max The collected point cloud is divided into meshes using a fixed step size (step) to obtain (x max -x min )*(y max -y min There are ) / (step*step) regions, and the center of each region in the x and y directions is denoted as (x) / (step*step). c y c ), calculate the average height z of the point cloud in each region. c Each region retains a center point (x). c ,y c ,z c ), calculate the center point of each region sequentially, and form a new point cloud, denoted as the smoothed point cloud. The point cloud after radius filtering is as follows: Figure 2 As shown.

[0006] Step 3: Define the search rectangle and neighboring rectangles and calculate the z-axis mean. Define the search plane rectangle rect, with the position of rectangle rect as (x, y) (where the range of x is [x, y). min ,xmax The range of values ​​for y is [y min ,y max The rectangle rect has dimensions (l, w). Four adjacent search plane rectangles rect_up, rect_down, rect_left, and rect_right with the same dimensions (l, w) are defined. The positions of rect_up, rect_down, rect_left, and rect_right are (x, yw / 2), (x, y+w / 2), (xl / 2, y), and (x+l / 2, y). The average height of the x and y coordinates in the smoothed point cloud within the rectangles rect, rect_up, rect_down, rect_left, and rect_right is calculated and denoted as z. aver z aver_up z aver_down z aver_left z aver_right。

[0007] Step 4: Define the average deviation z in the z-direction. off If the average height of the point cloud within the middle rectangle is less than the average height of the point cloud within each of the four adjacent rectangles, calculate z. off , where z off =(z aver_up +z aver_down +z aver_left +z aver_right ) / 4-z aver .

[0008] Step 5: Obtain the point cloud of the oral cavity region. Repeat steps 3 and 4, setting the x-value range of rect in step 3 to [x...]. min ,x max The range of values ​​for y in rect is [y...]. min ,y max Calculate the maximum deviation in the z-direction, denoted as z. max_off Recorded in z max_off The position (x) of rectangle rect when the maximum value is obtained max_off ,y max_off ), and retain the point cloud whose x and y coordinates are within the current rectangle rect, and denot it as the oral cavity region point cloud.

[0009] Step 6: Obtain the point cloud of the floor of the oral cavity. Calculate the minimum value in the z-direction of the point cloud of the ROI region, denoted as z. min_mouth The point cloud of the oral cavity region obtained in step 5 is segmented in the z-axis, and the segmentation threshold is denoted as deep. Points with a z-axis depth within the range [z...] are retained. min_mouth ,zmin_mouth The point cloud within the range of +deep is denoted as the lower z-axis point cloud. The lower z-axis point cloud is clustered and segmented to obtain the center point and the number of points in each cluster. After deleting the point cloud of the cluster with the smaller number of points, the point cloud near the middle of the rectangle is denoted as the bottom of the oral cavity point cloud.

[0010] Step 7: Calculate the position of the oral cavity. Based on the floor of the oral cavity, calculate the minimum coordinates (x, y) of the oral cavity along the z-axis. rlt ,y rlt ,z rlt The robot can then perform nucleic acid sampling based on that location.

[0011] The beneficial effects of this invention are as follows: This method performs radius filtering and segmentation on the collected facial point cloud data to obtain point cloud data of the oral cavity and its surrounding area. Then, it segments the point cloud data of the oral cavity and its neighborhood to obtain the point cloud data of the deepest part of the oral cavity. According to the requirements, it calculates the position information of the bottom of the oral cavity and sends it to the robot to realize nucleic acid collection. This method has high computational efficiency and strong versatility. This positioning method can quickly and accurately locate the oral cavity with high positioning accuracy, which meets the needs of fully automated nucleic acid detection workstations. Attached Figure Description

[0012] Figure 1 Point cloud images of a human face acquired by a 3D sensor;

[0013] Figure 2 This is the point cloud image after denoising and smoothing according to the present invention;

[0014] Figure 3 This is a point cloud image of the oral cavity region of the present invention;

[0015] Figure 4 This is a point cloud image of the floor of the oral cavity according to the present invention.

[0016] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention:

[0017] Example 1, Figure 1 The point cloud images of human faces acquired by 3D sensors contain a lot of noise. The point cloud images of the mouth position will produce different types of holes due to different angles of head tilt. Headwear (glasses, etc.) will also affect the image quality and cause point cloud defects. To address the facial features of different groups of people and different shooting angles, fully automatic 3D vision is used for mouth positioning.

[0018] 1) Facial point cloud acquisition: In a specific area, a person faces a 3D sensor, tilts their head back, and opens their mouth. The 3D sensor acquires facial point cloud data. The angle and position of the point cloud acquisition are relatively free, without fixed limiting devices. The acquired point cloud data is as follows: Figure 1As shown.

[0019] 2) Point cloud noise removal and hole filling: Due to interference from ambient light, the collected point cloud is uneven and contains gaps. First, median smoothing is performed on the collected point cloud to remove boundary clutter and environmental noise, resulting in a denoised point cloud. The minimum and maximum values ​​of the denoised point cloud in the x and y directions are calculated and denoted as (x...). min ,x max ), (y min ,y max The collected point cloud is divided into meshes using a fixed step size (step) to obtain (x max -x min )*(y max -y min There are ) / (step*step) regions, and the center of each region in the x and y directions is denoted as (x) / (step*step). c y c ), calculate the average height z of the point cloud in each region. c Each region retains a center point (x). c ,y c ,z c ), calculate the center point of each region sequentially, and form a new point cloud, denoted as the smoothed point cloud. The point cloud after radius filtering is as follows: Figure 2 As shown.

[0020] 3) Define the search rectangle and neighboring rectangles and calculate the z-axis mean. Define the search plane rectangle rect, with the position of rectangle rect at (x, y) (the range of x is [x, y). min ,x max The range of values ​​for y is [y min ,y max The rectangle rect has length and width (l, w), where l is slightly smaller than the length of the oral cavity and w is slightly smaller than the width of the oral cavity. Four adjacent planar rectangles rect_up, rect_down, rect_left, and rect_right with the same length and width (l, w) are defined. The positions of rect_up are (x, yw / 2), rect_down are (x, y+w / 2), rect_left is (xl / 2, y), and rect_right is (x+l / 2, y). The average z-axis height of the x and y coordinates in the smoothed point cloud within the rectangles rect, rect_up, rect_down, rect_left, and rect_right is denoted as z. aver z aver_up z aver_down z aver_left zaver_right ;

[0021] 4) Define the average deviation z in the z-direction. off If the average height of the point cloud within the middle rectangle is less than the average height of the point cloud within each of the four adjacent rectangles, calculate z. off , where z off =(z aver_up +z aver_down +z aver_left +z aver_right ) / 4-z aver ;

[0022] 5) Obtain the point cloud of the oral cavity region. Repeat steps 3 and 4, setting the x-value range of rect in step 3 to [x...]. min ,x max The range of values ​​for y in rect is [y...]. min ,y max Calculate the maximum deviation in the z-direction, denoted as z. max_off Recorded in z max_off The position (x) of rectangle rect when the maximum value is obtained max_off ,y max_off The point cloud with smoothed x and y coordinates within the current rectangle rect is retained and denoted as the oral cavity region point cloud. The segmented point cloud is as follows: Figure 3 As shown.

[0023] 6) Obtain the point cloud of the floor of the oral cavity, calculate the minimum value in the z-direction of the point cloud of the oral cavity region, and denot it as z. min_mouth The point cloud of the oral cavity region obtained in step 5 is segmented in the z-axis, and the segmentation threshold is denoted as deep. Points with a z-axis depth within the range [z...] are retained. min_mouth ,z min_mouth The point cloud within the range of +deep is denoted as the lower z-axis point cloud. To prevent the lower z-value point cloud outside the oral cavity from interfering with the point cloud inside the oral cavity, the lower z-axis point cloud is clustered and segmented to obtain the center point and the number of points in each cluster. After deleting the point cloud in the cluster with the smaller number of points, the point cloud near the middle of the rectangle is obtained and denoted as the floor point cloud. The floor point cloud is shown in the figure. Figure 4 As shown.

[0024] 7) Calculate the position of the oral cavity. Based on the floor of the oral cavity, calculate the minimum coordinates (x, y) of the oral cavity along the z-axis. rlt ,y rlt ,z rlt The robot can then perform nucleic acid sampling based on that location.

[0025] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. The various components mentioned in this invention are common technologies in the existing field. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A 3D vision-based method for oral cavity localization, comprising the following steps: First step: facial point cloud acquisition, using a 3D sensor to acquire point cloud data of the front of the face; Step 2: Noise Removal and Hole Filling in Point Clouds; Due to interference from ambient light, the acquired point clouds are uneven and contain gaps. First, median smoothing is performed on the acquired point clouds, and radius filtering is used to remove noise from the discrete point clouds. Then, mean smoothing is performed on the acquired point clouds using a fixed step size to obtain a uniform smoothed point cloud of the human face. The minimum and maximum values ​​of the human face point cloud in the x and y directions are calculated and denoted as (x...). min , x max ), (y min , y max ); Step 3: Define the search rectangle and the nearest rectangle and calculate the z-axis mean; Define a search plane rectangle rect, with the position of rect denoted as (x, y) (where x takes values ​​in the range [x, y). min , x max The range of values ​​for y is [y min , y max The rectangle `rect` has dimensions (l, w). Four adjacent search plane rectangles `rect_up`, `rect_down`, `rect_left`, and `rect_right` with the same dimensions (l, w) are defined. The positions of rectangle `rect_up` are (x, y - w / 2), rectangle `rect_down` is (x, y + w / 2), rectangle `rect_left` is (x - l / 2, y), and rectangle `rect_right` is (x + l / 2, y). The average height of the x and y coordinates in the smoothed point cloud within the rectangles `rect`, `rect_up`, `rect_down`, `rect_left`, and `rect_right` is calculated and denoted as z. aver z aver_up z aver_down z aver_left z aver_right ; Step 4: Define the average deviation zoff in the z-direction. If the average height of the point cloud within the middle rectangle is less than the average height of the point cloud within each of the four adjacent rectangles, calculate zoff, where zoff = (z aver_up + z aver_down + z aver_left +z aver_right ) / 4 - z aver ; Step 5: Obtain the point cloud of the oral cavity region; repeat steps 3 and 4, setting the x-value range of rect in step 3 to [x...]. min , x max The range of values ​​for y in rect is [y...]. min , y max Calculate the maximum deviation in the z-direction, denoted as z. max_off Recorded in z max_off The position (x) of rectangle rect when the maximum value is obtained max_off ,y max_off ), and retain the point cloud whose x and y coordinates are within the current rectangle rect, and denot it as the oral cavity region point cloud; Step 6: Obtain the point cloud of the floor of the oral cavity; calculate the minimum value in the z-direction of the point cloud of the ROI region, denoted as z. min_mouth The point cloud of the oral cavity region obtained in step 5 is segmented in the z-axis, with the segmentation threshold denoted as deep. The segmentation depth is retained within the range [z...]. min_mouth , z min_mouth The point cloud within the range of +deep is denoted as the lower z-axis point cloud. The lower z-axis point cloud is clustered and segmented to obtain the center point and the number of points in each cluster. After deleting the point cloud of the cluster with the smaller number of points, the point cloud near the middle of the rectangle is denoted as the bottom of the oral cavity point cloud. Step 7: Calculate the position of the oral cavity. Based on the floor of the oral cavity, calculate the minimum coordinates (x, y) of the oral cavity along the z-axis. rlt ,y rlt ,z rlt The robot can then perform nucleic acid sampling based on that location.

2. The 3D vision-based oral cavity localization method according to claim 1, characterized in that... The second step is recorded as (x) min ,x max ), (y min , y max After that, the collected point cloud is divided into meshes using a fixed step size step, resulting in (x max - x min )×(y max - y min There are ) / (step × step) regions, and the center of each region in the x and y directions is denoted as (x) c y c ), calculate the average height z of the point cloud in each region. c Each region retains a center point (x). c ,y c ,z c The center point of each region is calculated sequentially, and a new point cloud is formed, which is denoted as a smooth point cloud.

Images