Device, system and method for precisely positioning human head

Abstract

The present disclosure provides devices, systems, and methods for precisely positioning a human head in space. According to the invention, accurate positioning of the head of the human body in the space is realized through drawing lines generated by computer facial recognition software and a mode of positioning the sensor to the head of the human body according to the permanent recognition mark of the sensor. For a patient, surgical operations, radiation imaging or imaging diagnosis procedures on the brain, intracranial or carotid-cranial structure are dangerous because the brain tissue of the human body is very sensitive, it is very difficult to identify and enter a specific position in the brain where an operation needs to be performed, and the brain tissue around the specific position where the operation needs to be performed may be damaged. This desire to accurately position and enter the brain leads to the development of stereotactic neurosurgical specialized procedures or "stereotactic procedures".

Description

priority This disclosure application claims priority to U.S. application serial number 17 / 707,153 filed on March 29, 2022, the contents of which are incorporated herein by reference. background

[0001] FIELD. The present disclosure relates generally to head and / or body positioning systems, and more particularly to devices, systems, and methods for accurately positioning a human head in space.

[0002] Description of the Related Art In many industrial applications, there is a need to constantly and repeatedly position and reposition the head, as well as position and reposition a head-mounted sensor device to the head.

[0003] Once such applications are used for medical purposes. Performing surgery, radiation imaging or diagnostic imaging procedures on the brain, intracranial or cervical structures is extremely dangerous for the patient because the human brain tissue is very sensitive, identifying and accessing the specific location in the brain where surgery is required is difficult, and there is the possibility of damaging brain tissue above or around the specific location where surgery is required. This desire to accurately locate and access the brain and other intracranial structures has led to the development of stereotactic neurosurgery, or "stereotactic surgery."

[0004] In stereotactic surgery, an external device that attaches to the patient's skull is often used during preoperative diagnostic procedures and during surgery. The device provides a grid or framework whose position is fixed relative to the patient's skull. The grid or framework can be used to establish a coordinate system to accurately locate the location of areas such as intracranial lesions in a repeatable manner.

[0005] However, it is difficult to control the position of the head in space, and it is also difficult to install and replace head-mounted sensor devices on the human head in a repeatable, accurate, predictable, user-friendly, cost-effective, time-saving, and easy manner.

[0006] For example, the cervicocranial spine is the region extending from the occipital bone, the atlas C1 (Atlas), and the axis C2 (Axis). The atlanto-occipital joint is a pure rotational joint, which makes it difficult to image traditionally, and therefore the most difficult joint to image without tools to control rotation during positioning. Any rotation of C1 / C2 directly results in a change in angle, which directly results in a diagnostically flawed x-ray image. Such images provide poor diagnostic quality for lateral mass (sliding of C1 relative to C2) measurements. The degree of symmetry of the position of C1 and C2, and the presence of lateral masses on both sides in the scoliotic open mouth view are indicators of alar / accessory ligament injury.

[0007] Unfortunately, imaging is severely underutilized in this area, as a large proportion of patients do not receive a definitive diagnosis despite seeking medical help. As a result, patients are often left with pain medications that are often hit-or-miss, primitive, and ineffective in treating the underlying condition.

[0008] Therefore, a technology is needed that can accurately position the head and reposition it to the same position at a later time. Summary of the invention

[0009] The present disclosure provides a device, system and method for accurately locating a human head in space. The present disclosure provides a method for accurately locating a human head in space by drawing lines generated by computer facial recognition software and locating a sensor to a human head according to a permanent identification mark carried by the sensor.

[0010] According to one aspect of the present disclosure, a method for positioning a human head includes imaging the face of a human head using at least one imaging device; generating at least one vertical facial line and at least two horizontal facial lines in the facial image, and generating at least one vertical facial line and at least two horizontal facial lines based on facial markers on the face; aligning the human head so that at least one vertical facial line is aligned with the generated vertical screen line, and each of the at least two horizontal facial lines is aligned with the generated horizontal screen line; storing the alignment line as a reference line; and repositioning the human head to a desired position, etc.

[0011] In one aspect, generating at least one vertical facial line and at least two horizontal facial lines includes using at least one facial recognition application to identify facial landmarks on a human head as key points, and connecting the key points to draw at least one vertical facial line and at least two horizontal facial lines.

[0012] On the other hand, at least one facial recognition application includes a convolutional neural network.

[0013] On the other hand, the generated vertical and horizontal screen lines indicate that the face is at a 90 degree angle to the imaging device.

[0014] In another aspect, among the aligned at least two horizontal face lines and the generated horizontal screen line, a lower line is stored as a default pitch line.

[0015] In another aspect, the default pitch line is split into two segments at at least one vertical facial line.

[0016] In another aspect, aligning the human head includes aligning two default pitch lines so that both are parallel to upper lines of the two horizontal facial lines.

[0017] On the other hand, the default pitch line is normalized to compensate for the differences in the facial structure of the human head.

[0018] In one aspect, each of the two default pitch lines is drawn by connecting an identified facial landmark at the tip of the nose of the face to at least one point on the perimeter of the face.

[0019] In another aspect, at least one vertical facial line is drawn by connecting the identified facial landmarks vertically distributed at the center of the nose of the face.

[0020] In another aspect, the yaw angle of the human head is determined based on at least one vertical face line and a generated vertical screen line.

[0021] On the other hand, at least two upper horizontal facial lines are drawn by connecting the identified facial landmarks at opposite ends of the brow line.

[0022] In one aspect, the roll angle of the human head is determined based on at least two horizontal facial lines and an upper line of the generated horizontal screen lines.

[0023] In another aspect, the method further includes storing coordinates associated with the alignment of the repositioned human head at a first point in time, and repositioning the human head according to the stored coordinates at a second point in time.

[0024] On the other hand, the method also includes placing a head-mounted sensor device on a human head, the head-mounted sensor device including a center mark and an edge mark for indicating the center position of the head-mounted sensor device; aligning the center mark of the head-mounted sensor device with at least one aligned vertical line; aligning the edge mark of the head-mounted sensor device with at least one aligned horizontal line; and storing the alignment status of the head-mounted sensor device and the human head, etc.

[0025] In another aspect, the method further includes storing coordinates associated with the alignment of the repositioned human head at the first point in time, and transmitting the stored coordinates to at least one sensor coupled to the at least one body part.

[0026] According to another aspect of the present disclosure, a human head positioning system includes at least one imaging device for imaging the face of a human head; at least one processing device that can generate at least one vertical facial line and at least two horizontal facial lines in facial imaging through a display device, and generate at least one vertical facial line and at least two horizontal facial lines based on facial markers on the face; at least one display device that can align the human head so that at least one vertical facial line is aligned with the generated vertical screen line, and each of at least two horizontal facial lines is aligned with the generated horizontal screen line; and at least one memory that can store the alignment line as a reference line, wherein, after the human head is repositioned to a desired position, at least one processing device can continuously update at least one vertical facial line and at least two horizontal facial lines based on the facial markers on the face.

[0027] In another aspect, at least one processing device can perform at least one facial recognition function for identifying facial landmarks on the face as key points and connecting the key points to draw at least one vertical facial line and at least two horizontal facial lines.

[0028] In one aspect, at least one facial recognition function includes a convolutional neural network.

[0029] On the other hand, the system also includes a head-mounted sensor device configured to be mounted on a human head, the head-mounted sensor device including a center mark and an edge mark for indicating the center position of the head-mounted sensor device; wherein the display device can align the center mark of the head-mounted sensor device with at least one aligned vertical line, and the edge mark of the head-mounted sensor device with at least one aligned horizontal line; and at least one processing device can store the alignment status of the head-mounted sensor device and the human head in at least one memory.

[0030] On the other hand, the devices, systems and methods of the present disclosure provide a user-friendly interface that allows for precise positioning of the head and future repositioning of the head to the same position. The devices, systems and methods of the present disclosure can control all axes of rotation. Among other tasks, the devices, systems and methods of the present disclosure also solve the following problems: improve the imaging diagnostic capabilities of the head, cervical and cervical structures; provide a method to reduce human exposure to ionizing radiation by reducing the amount of imaging required; enable medical diagnostic models to be used on a national / global electronic medical record network, greatly facilitating communication between registered stakeholders. Combining the devices, systems and methods of the present disclosure with a national / global electronic medical record network allows for larger-scale statistical analysis of data, thereby improving specific treatment plans and ultimately improving research on disease progression and prognosis.

[0031] According to another aspect of the present disclosure, a method is provided, including imaging the face of a human head using at least one imaging device; generating at least one vertical facial line and at least two horizontal facial lines in the facial image, generating at least one vertical facial line and at least two horizontal facial lines based on facial markers on the face; aligning the human head so that at least one vertical facial line is aligned with the generated vertical screen line, and each of the at least two horizontal facial lines is aligned with the generated horizontal screen line, so that the human head forms a zero-degree pitch angle with the at least one imaging device; storing the alignment line as a reference line; repositioning the human head to a desired position while maintaining the pitch angle at zero degrees; and providing an indication indicating that the human head is in a desired position and the pitch angle is zero. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] These and other objects, features and advantages of the present disclosure will become apparent upon reference to the following detailed description in conjunction with the accompanying drawings:

[0033] Figure 1 Shown is a facial line and a screen line located on a facial image according to one embodiment of the present disclosure;

[0034] Figure 2 Shown is the desired neutral orthogonal positioning of screen lines on a facial image according to one embodiment of the present disclosure;

[0035] Figure 3 Shown is an example of a head-mounted sensor device used in a positioning system according to an embodiment of the present disclosure;

[0036] Figure 4 Shown is an example of a head fixing device used in a positioning system according to an embodiment of the present disclosure;

[0037] Figure 5 Shown is the initial positioning of screen lines with normalized pitch angles on a head image according to one embodiment of the present disclosure;

[0038] Figure 6 Shown is the positioning of a default pitch line with a normalized pitch angle according to one embodiment of the present disclosure;

[0039] Figure 7 Shown is a repositioning of a head-mounted sensor device with a normalized pitch angle according to one embodiment of the present disclosure;

[0040] Figure 8 Shown is a correct leftward lateral movement of the head before the mouth opens according to one embodiment of the present disclosure;

[0041] Fig. 9Shown is a correct rightward lateral movement of the head before the mouth opens according to one embodiment of the present disclosure;

[0042] Fig.10 Shown is a pitch deviation between a generated line and a head position according to an embodiment of the present disclosure;

[0043] Fig.11 Shown is a block diagram of a positioning system according to an embodiment of the present disclosure;

[0044] Fig.12 Shown is a block diagram of an example computing device according to an embodiment of the present disclosure;

[0045] Fig.13 Shown is a facial key point mapping for calculating facial yaw, pitch and roll according to one embodiment of the present disclosure;

[0046] Fig.14 Shown is a reference system for calculating the facial pitch angle according to one embodiment of the present disclosure;

[0047] Fig.15 Shown is a method for generating vertical facial lines according to one embodiment of the present disclosure;

[0048] Fig.16 Shown is a method for upper horizontal facial line according to one embodiment of the present disclosure;

[0049] Fig.17 Shown is a method of moving an upper horizontal facial line using a user defined value according to one embodiment of the present disclosure;

[0050] Fig.18 Shown is a method for lower horizontal face line or pitch line according to one embodiment of the present disclosure;

[0051] Fig.19 The invention discloses a method for normalization processing according to an embodiment of the present invention. Fig.18 The method of the mid-lower horizontal facial line or pitch line;

[0052] Fig. 20 Shown is a flow chart for illustrating individual registration, authentication and positioning according to an embodiment of the present disclosure;

[0053] Fig.21 What is shown is the facial key points detected by the facial recognition function according to one embodiment of the present disclosure;

[0054] Fig. 22 Shown are roll, yaw, and pitch relative to an individual's head, according to one embodiment of the present disclosure;

[0055] Fig.23 It is shown that according to one embodiment of the present disclosure, the head rolls in a counterclockwise direction to calculate the resulting roll angle;

[0056] Fig.24 Shown is an embodiment of the present disclosure, in which the head is in a clockwise direction to calculate the resulting roll angle;

[0057] Fig.25 Shown is an embodiment of the present disclosure wherein the head is in a neutral position to calculate the resulting roll angle;

[0058] Fig.26 Shown is an embodiment of the present disclosure, with the head facing left to calculate the resulting yaw angle;

[0059] Fig. 27 What is shown is that according to one embodiment of the present disclosure, the head is turned right to calculate the resulting yaw angle;

[0060] Fig.28 Shown is an embodiment of the present disclosure, where the head looks straight ahead to calculate the resulting yaw angle;

[0061] Fig.29 Shown is an embodiment of the present disclosure in which the head moves upward to calculate the resulting yaw angle;

[0062] Fig.30 Shown is a head moving downward to calculate the resulting yaw angle according to one embodiment of the present disclosure;

[0063] Fig.31 Shown is an embodiment of the present disclosure wherein the head is in a neutral position to calculate the resulting pitch angle;

[0064] Fig.32 Shown is a screenshot of a graphical user interface according to an embodiment of the present disclosure;

[0065] Figure 33-40 Shown are various screenshots of a graphical user interface for directing an individual to a desired location according to one embodiment of the present disclosure;

[0066] Fig.41 Shown is a screenshot of a graphical user interface with an individual in a neutral position with their mouth open according to an embodiment of the present disclosure;

[0067] Fig.42 A screenshot of a graphical user interface showing an individual bending to the left with an open mouth according to an embodiment of the present disclosure; and

[0068] Fig.43Shown is a screenshot of a graphical user interface showing an individual bending to the right with their mouth open, according to an embodiment of the present disclosure. Detailed description

[0069] The embodiments of the present disclosure will be described below with reference to the accompanying drawings. Functions or structures that are already known to everyone are not described here to avoid obscuring the present disclosure by some unnecessary details. The word "example" used herein means "as an example, instance or illustration". Any configuration or design described as an "example" in this article shall not be understood as better or superior to other configurations or designs. "Coupled" here means directly or indirectly connected through one or more middleware. These middleware may include hardware-based and software-based components.

[0070] It should be further noted that, unless otherwise specified, all functions described herein may be implemented by hardware, software, or some combination of hardware and software. However, in one embodiment, unless otherwise specified, these functions are executed by at least one processor (e.g., a computer or electronic data processor, a digital signal processor, or an embedded microcontroller, etc.) according to code (e.g., computer program code, software, and / or integrated circuits, etc.).

[0071] It should be understood that the present disclosure can be implemented in many ways, including as a process, an apparatus, a system, a device, a method, or a computer-readable medium (for example, a computer-readable storage medium or a computer network in which the program is sent via an optical or electronic communication link, etc.).

[0072] Embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0073] The present disclosure relates to a system and method for determining the orientation of a human head using an imaging device (e.g., a camera, etc.). The system and method allow the positioning of the head to be controlled using lines on a computer-generated facial image (e.g., lines generated by facial recognition software) while the patient's head is moved. In addition, the present disclosure also relates to using computer-generated lines based on recognized facial landmarks. These facial landmarks are related to the facial lines of the human head and the lines recognized on a head-mounted sensor device, which can achieve accurate and controllable positioning of the head-mounted sensor device on the head.

[0074] The present disclosure provides a method for positioning a head and / or a head-mounted sensor device using computer facial recognition software. The computer facial recognition software can recognize special facial marks and generate lines visible to the naked eye on the facial image of a display device (e.g., a computer screen, etc.), thereby achieving accurate and controllable positioning and repeatable repositioning of the head relative to an imaging device (e.g., a camera or an object with a camera). In addition, the present disclosure can also enable accurate and controllable positioning and repeatable positioning of a head-mounted sensor device or a head-mounted object with permanent marks and a head relative to these facial lines.

[0075] To facilitate understanding of the present disclosure, the computer-generated lines used in the system and method of the present disclosure will first be described. In the system and method of the present disclosure, two types of computer-generated lines are used: 1.) screen lines and 2.) face lines. The description of screen lines and face lines is as follows: Figure 1 and Figure 2 shown.

[0076] Vertical screen line 1 is a vertical line that is used to display the center position of the imaging device projection to determine the position of the vertically drawn object on the computer screen at a 90-degree angle to the camera. Vertical face line 2 is located at the center of the face and is represented by a line drawn by certain marks on the face. Fig.13 and Fig.14 is described. An upper horizontal face line 3 is located on the forehead and is represented by a line drawn from certain markers of facial recognition. An upper horizontal screen line 4 is located on the forehead and is projected by the imaging device to show the position of the horizontally projected cameras at 90 degrees to each other. On the screen, a lower horizontal screen line 5 is located below the upper horizontal screen line 4. The lower horizontal screen line 5 is a line. This line will be at a 90 degree angle to the imaging device at the facial position and will show the position where the default pitch line can be marked after the baseline pitch orientation is met and saved. The lower horizontal face line 6 will form a separate horizontal line 9 after coinciding with the lower horizontal screen line 5. This line is also called the default pitch line and is located below the upper horizontal face line 4 on the facial image. It is represented by a line generated by the software based on the facial markers.

[0077] The single horizontal line 9 becomes the default pitch line 9, and its position is the same as the position of the lower horizontal face line 6 when the default pitch angle is registered. Figure 2 The pitch line shown in is a normalized line, which will be explained in detail below.

[0078] Once the face / object is at a 90 degree angle to the imaging device, the head is positioned at a default pitch angle of the ideal pitch position by electronic sensors or other means (e.g., pixel data, etc.), and the facial image marker is recorded according to the default pitch angle (normalized, see below). The default pitch angle extends from the point of the lower horizontal face line 6 and the lower horizontal screen line 5 and coincides with these two lines. The default pitch angle extends from the vertical face line 2 in the facial imaging and can be divided into two lines, such as Figure 5 When a positive or negative pitch angle occurs, the operator will see an angle that divides the default pitch angle line 9 into two lines 24, such as Figure 5 When the default pitch angle is zero, the relationship between it and the operator and the patient becomes a linear relationship, as shown in Figure 6 As shown on line 29.

[0079] According to an embodiment of the present disclosure, a positioning system 1100, such as Fig.11 The system 1100 includes at least one imaging device 1102, a head-mounted sensor device 1104, a head fixing device 1105, and a computing device 1106. It is understood that the imaging device 1102 can be a mobile device such as a camera, a video camera, a tablet computer, a mobile phone, etc. The detailed information of the computing device 1106 will be described below in conjunction with Fig.12 However, it is understood that the computing device 1106 includes input / output devices, such as a display device 1108 (e.g., a display, a touch screen, etc.), a keyboard 1110, a mouse 1112, and at least one speaker. It is also understood that the computing device 1106 can be a single device including an imaging device, a display device, and an input / output device.

[0080] Figure 3A head mounted sensor device 1104 is shown. An example of a head mounted sensor device is provided by Keeler USA, Inc. of Malvern, Pennsylvania. The head mounted sensor device 1104 includes a support portion 1140 placed on the top of the head to support other components of the head mounted sensor device 1104 placed on the head. In addition, the head mounted sensor device 1104 also includes at least one strap 1142 for securing the head mounted sensor device 1104 to the head. It is understood that any head mounted sensor device and any strap can be used with the system and method of the present disclosure as long as the head mounted sensor device includes at least two permanent marks. The head mounted sensor device 1104 includes a center mark 10 and an edge mark 11. The center mark 10 is a vertical line located on the head mounted sensor device 1104, representing the exact center position of the head mounted sensor device. The edge mark 11 is located at the front end of the strap 1142 of the head mounted sensor device, represented by a line on the lower edge of the strap 1142.

[0081] If necessary, an auxiliary fixing device (e.g., head fixing device 1105) may be used to ensure the stability of the head when it is positioned in front of the imaging device 1102. The head fixing device 1105 is used to position the head to a desired initial position. Figure 4 As shown, the head fixation device 1105 includes a horizontal frame 12 for initially positioning the head next to the imaging device 1102. The horizontal frame 12 can be moved up and down to accommodate different heights and vertical angles relative to the imaging device 1102. The position of the horizontal frame 12 can be recorded in each instance for future use as a reference when the head-mounted sensor device 1104 needs to be repositioned. In the methods and procedures of the present disclosure, the patient's lower jaw should be resting on the horizontal frame 12 with his chin. There is a chin rest 13 on the horizontal frame 12 for positioning the patient's chin during the procedure. The method of the present disclosure can use the head fixation device to navigate to the desired head position.

[0082] Optionally, the head fixture 1105 is an electric device. The device uses a plug 14 to connect to a socket and a control unit 15 to connect to motors 16 on both sides to move the horizontal frame 12 up and down when necessary through threaded rods 17 on both sides. There is one motor on each side. The head fixture 1105 uses a vertical directional rod 18 and a bearing 19 that can limit the up and down movement. The head fixture 1105 has four adjustable feet 20 that allow the head fixture 1105 to be fixed to a surface such as (but not limited to) a table. In some embodiments, the head fixture 1105 is connected to a computing device 1106, where the computing device 1106 controls the height of the upper horizontal frame 12 by sending signals to the control unit 15 and the motor 16.

[0083] For example, the systems and methods of the present disclosure can be used to position a facial image, or in this case a head image, without any angular distortion, such as in a specific task. The present disclosure solves the problem of having the same positioning in any XYZ coordinates. In this example, the basic goal of obtaining free angular image distortion is to position the object (e.g., face) orthogonally, at a 90 degree angle to the imaging device. The angular distortion value of this position is zero. Using this method, clean distortion-free default data for the initial positioning of any head position can be recorded and controlled for future accurate repositioning, for example, to control angular optical distortion when imaging, etc.

[0084] Initially, when an acceptable spatial relationship is formed between the head on the head fixture 1105 and the imaging device, the pitch angle value is registered as zero, and this pitch angle is recorded as the default position (i.e., 90 degrees to the imaging device), and the pitch degree is also acceptable. Figure 2 As shown, a zero degree pitch angle is registered when the head position is as follows: vertical face line 2 and vertical screen line 1 are moved to form a single vertical line 7; upper horizontal face line 3 and upper horizontal screen line 4 are moved to form an upper single horizontal line 8; and lower horizontal face line 6 and lower horizontal screen line 5 are coincident to form a single horizontal line 9. Once the operator of system 1100 is satisfied with the pitch angle of the head relative to imaging device 1102, the record is saved in the memory of computing device 1106 as baseline zero. After the zero degree pitch angle is saved, single horizontal line 9 becomes the default pitch line 9.

[0085] After achieving a 90 degree alignment of the face with the camera, the system may alert the operator through a signal or visual effect seen on the computer screen 1108 to let the operator know that the desired alignment has been achieved. For example, as the head moves within the imaging space, the computing device 1106 may emit a sound (played by a speaker) indicating that the generated face line and the screen line are either aligned or nearly aligned. In one embodiment, the pitch or tone of the sound emitted changes when the head is nearly aligned, and then a different, unique sound is emitted when the head is at an acceptable pitch angle (or zero pitch angle).

[0086] Operating Procedure

[0087] The patient sits in front of the imaging device 1102 and, if necessary, places his / her head on a head fixing device 1105 to keep the head stable. He / she will completely rest his / her chin on the horizontal frame 12 of the head fixing device 1105.

[0088] The computing device 1106 executing the facial recognition software then generates the face lines and screen lines. In one embodiment, the operator sees the face lines and screen lines in different colors for ease of identification. The vertical face lines are generated based on the facial markers. The vertical screen lines show the position of the vertical face lines at 90 degrees to the camera so that the patient's head is positioned at 90 degrees to the camera (e.g., a position that does not produce angular optical distortion). Four horizontal lines intersect their respective vertical lines at two different upper or lower levels: the upper horizontal face lines and the lower horizontal face lines are generated based on data obtained from the facial markers. The upper horizontal screen lines and the lower horizontal screen lines respectively project that the face with the respective horizontal face lines must be at a 90 degree position to the camera.

[0089] Now combine Figure 13-19 An example method for identifying facial landmarks and generating lines based on the identified points is described.

[0090] like Fig.13 As shown, the system calculates the yaw, pitch, and roll of the face from the camera's perspective by using simple geometry mapped to facial keypoints. Fig.14 As shown, a reference example is shown in the figure to facilitate understanding of the values ​​and lines explained in Table 1 below. It can be understood that the values ​​on the X-axis represent the image width and the values ​​on the Y-axis represent the image height. For example, point 1402 is the pixel at (0,0), that is, x=0 and y=0; point 1404 is the pixel at (18,6), that is, x=18 and y=6; point 1406 is the pixel at (6,13), that is, x=6 and y=13. In Table 1 below, it can be understood that the line described in column 2 is generated using point 1 described in column 3 and point 2 described in column 4, where Pn(x) refers to the x pixel value of facial point "n" and Pn(y) refers to the y pixel value of facial point "n". Table 1 -*" represents the multiplication sign -"-" represents a minus sign -Specify the "X, Y" order of coordinates

[0091] First, a convolutional neural network is used to detect facial key points in the image, e.g. Fig.13 1-68 shown in FIG. It is understood that the facial key points can be used to draw or generate a reference line on the image, for example, a virtual headband. Next, a line 1302 perpendicular to the face is drawn using the key points 28, 29, 30 and 31 on the head nose (i.e., the points 28, 29, 30 and 31 arranged vertically along the head nose). Fig.15 As shown, line 1302 extends vertically to the end of the image frame, wherein end points 1502 and 1504 are defined by Fig.15 It is understood that Fig.13 and 15 The line 1302 shown in FIG. Figure 1 Next, a line 1304 is drawn horizontally to the face between key points 18 and 27, i.e., points 18 and 27 are approximately located at the two ends of the facial brow line. Fig.16 As shown, line 1304 extends horizontally to the end of the image frame, where endpoints 1602 and 1604 are defined by Fig.16 It is understood that Fig.13 and 16 The line 1304 shown in FIG. Figure 1 The upper horizontal facial line shown in 3. Fig.17 As shown, line 1304 can be moved upward according to a user-defined value (e.g., an α value) so that line 1304 can be aligned with the head-mounted sensor device, as described in detail below.

[0092] Draw a line 1306 perpendicular to the camera and a line 1308 horizontal to the camera as reference coordinates, and this line always passes through the intersection of the upper horizontal face line 1304 and the upper vertical face line 1302. It can be understood that line 1306 is the vertical screen line 1, and line 1308 is the upper horizontal screen line 4. Figure 1 As shown in FIG. 13 . This intersection is approximately located at the center of the forehead. Through these four lines, the roll angle and yaw angle of the face in the image frame can be calculated. For example, the yaw angle can be determined by lines 1302 and 1306, and the roll angle can be determined by lines 1304 and 1308, as shown in FIG. Fig.13 shown.

[0093] Since the pitch angle is captured through the motion on the third axis (i.e., depth), and a simple two-dimensional (2D) camera cannot directly calculate the pitch angle, an adjustment method is used to calculate the pitch angle. First, a line 1802 is drawn between points 3, 34, and 15, and then a horizontal line 1804 is also drawn between points 3 and 15, such as Fig.18As shown. It will be appreciated that points 3 and 15 are located on the circumference of the face, while point 34 is approximately located at the tip of the nose. It will also be appreciated that points 1 to 17 define a portion of the circumference of the face, and points 32 to 36 define the lower portion of the nose. In some face shapes, when the pitch angle is zero, the two lines 1802 and 1804 should be parallel to each other. As the face moves up and down, the tip of the nose also moves, thereby calculating the pitch angle between the line drawn by points 3, 34 and 15 and the line drawn by points 3 and 15. However, due to facial structure, the initial angle at zero pitch angle may not be zero for most faces, and therefore, the initial angle needs to be normalized.

[0094] Therefore, after the patient's head is placed on the head fixing device with a pitch angle of zero and the input device such as the keyboard 1110 is activated, the system will calculate a normalized δ value of the pitch line (also called the baseline pitch line δ value, such as Fig.19 ) and calculates a rearranged point other than point 34 (e.g., slightly above point 34 in most cases). We call this shifted point 34'. Now, using only a 2D camera, the pitch angle of any face can be derived from the angles between 3-34'-15 and 3-15.

[0095] Since pitch is in the depth direction, it cannot be calculated directly using the camera. To solve this problem, the pitch needs to be adjusted or normalized. It can be seen that when the head moves in the pitch direction, the movement of point 34 is greater than that of points 3 and 15. Therefore, if the pitch changes, the angle between points 3, 34 and 15 will also change. This angle is not necessarily zero when the pitch is zero. This is why the operator needs to place the patient's head in a head fixation device to prevent the head from being unstable, moving or shaking when the operator presses button P on the keyboard to record it when he thinks or knows that the pitch is zero. After pressing button P, the Y coordinate of point 34 will be displaced by the formula Pitchnorm delta, which is the value P34(y)-(P3(y)+P15(y)) / 2 calculated for the frame when the operator recorded the pitch angle as zero. By subtracting this value from P34(y), the angle drawn as zero can be normalized. As shown in Fig.19 As shown, line 1804 is normalized by shifting upward by the baseline pitch delta value. The pitch angle can be calculated as the angle between line 1804 and 1802-1 or 1802-2.

[0096] It will be appreciated that in further steps, these lines will be used to assist in moving the head to any desired position, regardless of whether or not a head mounted sensor device is used. For example, these lines are used to control rotation in different directions, for example, vertical screen line 1 and vertical face line 2 are used to control yaw; upper horizontal screen line 4 and upper horizontal face line 3 are used to control roll; and the pitch line formed by lower horizontal screen line 5 and lower horizontal face line 6 is used to control pitch, where -Rotation around the Z axis (sagittal plane) is called roll. -Rotation around the X axis is called pitch. -Rotation around the vertical Y axis is called yaw. The operator can adjust the height of the upper horizontal screen line 5 by moving the line up or down (if necessary).

[0097] In an example, Figure 5 As shown, the system and method of the present disclosure can be used to align the human head with the imaging device at 90 degrees. Figure 5 As shown, because the head is not aligned according to the pre-recorded pitch, the default pitch line 24 is divided into two segments. The patient is then instructed to move the head until the pitch line 24 coincides with the zero pitch line 23. In addition, after the head is repositioned, the system and method can also be used to ensure that the head is in a zero pitch position, for example, Figure 8 As shown. Figure 8 In FIG. 3 , the pitch line 37 is parallel to the line 36 and perpendicular to the vertical line 35 .

[0098] Additionally, the coordinates of a specific location can be stored to allow the head to be repositioned at the same location at a later time. Figure 8 In a first time point (e.g., first visit), the coordinates of the position shown may be stored as yaw: 1, roll: 123.17, and pitch: 0. At a second time point (e.g., second visit), the head may be repositioned to the exact coordinates by retrieving the coordinates and displaying the retrieved coordinates on a display device. The operator may then instruct the patient to move the head until the head is aligned with the stored position. In one embodiment, the recorded lines may be displayed on a display device. Next, the system generates a facial line based on the identified facial points. As the facial line approaches the recorded line, the system may emit a sound, the pitch of which may change as the two lines get closer, and then an alert may be emitted once the two lines match.

[0099] In one embodiment, the system 1100 may include multiple imaging devices arranged in various ways. For example, the imaging devices may be arranged around the head to capture 360 ​​degrees of movement of the head relative to a starting or initial position. In this embodiment, a particular imaging device is activated as long as the facial image is within the frame of the imaging device. Once the patient turns his head, an adjacent imaging device detects the face within the range of the imaging device, thereby activating the imaging device. It is understood that multiple imaging devices may be arranged in any manner to capture the full range of movement of the patient, for example, multiple imaging devices are arranged in a spherical relationship. In another embodiment, once the head is positioned in a predetermined earlier position, another treatment device will immediately begin treatment. An example of a treatment device may be a robotic arm or a stereotactic device, both of which can emit focused radiation to treat intracranial lesions. Unless the positioning system has achieved the desired predetermined position, the treatment device will not begin to operate.

[0100] Now, a method for controlling the motion of a human head relative to a line is described. This method registers the head position and thereby positions the head-mounted sensor device with minimal distortion, avoiding any errors during repositioning. Figure 2 As shown, the operator overlaps the vertical screen line 1 and the vertical face line 2 to form a single line, i.e., the single vertical line 7; overlaps the upper horizontal screen line 4 and the upper horizontal face line 3 to form a single line, i.e., the upper single horizontal line 8; and overlaps the lower horizontal screen line 5 and the lower horizontal face line 6 to form a lower single horizontal line 9. Next, a head pitch angle position that meets the requirements is determined in a manner acceptable to the operator (with the help of electronic sensors on the head-mounted sensor device, facial clamps, pixel data, or other methods), and after finding this head pitch angle that meets the requirements, input is provided to the computing device 1106 to record the zero pitch angle and create a default pitch line 9. After the default pitch line 9 is recorded, it will overlap with the lower single horizontal line, and in repeated positions, it must also overlap in a purely linear manner. At this point, the operator can calculate the head pitch angle according to the head pitch angle. Figure 7 The rules described in ,position the head mounted sensor device.

[0101] Reinsertion of the Head-Mounted Sensor Unit and its Rules

[0102] When inserting or reinserting the head-mounted sensor device into a human head, the location of the upper individual horizontal line 8 on the forehead must already be known. This line is where the edge 11 of the head-mounted sensor device must coincide. The α value varies from individual to individual and is one of at least two necessary components for positioning and repositioning the head-mounted sensor device in order to determine the boundaries of the head-mounted sensor device. If the head-mounted sensor device 1104 needs to be positioned in a more convenient position, the α value can be moved up or down to facilitate operation. That is, the operator can enter the α value to move the upper individual horizontal line 8 on the display device up or down relative to the face, and then position the head-mounted sensor device on the head so that the edge 11 of the head-mounted sensor device coincides with the upper individual horizontal line 8. The center of the head-mounted sensor device 10 must coincide with the individual vertical line 7.

[0103] The head is positioned in the head fixture 1105 in front of the imaging device 1102. The lower jaw is completely against the horizontal frame 12 on the chin rest 13 of the head fixture. Figure 5 As shown, once the head is placed on the horizontal frame 12 of the head fixing device 1105, the facial image will appear on the display device. Figure 5 As shown, all of the following lines need to be realigned: upper horizontal face line 21, upper horizontal screen line 22, lower horizontal screen line 23, zero pitch angle / default pitch line 24, vertical screen line 25, and vertical face line 26.

[0104] Next, the patient is instructed to move his head so that the individual vertical lines 27 and the upper individual horizontal screen line 28 are formed on the computer screen, so that the lower horizontal face line 29 coincides with the lower horizontal screen line and forms a default pitch line with the previously recorded zero pitch angle. The facial markers help the face find the previously recorded default position according to the facial lines. This facial line must be coincident according to the rules to reposition the face so that the head returns to its original position, such as Figure 6 shown. When all screen lines are as described above, place the head mounted sensor device on the head, align the edge 11 of the head mounted sensor device with the upper single horizontal line 32, align the center 10 of the head mounted sensor device with the single vertical line 30, and the zero pitch angle line with the default pitch angle line 33, as shown in FIG. Figure 7 At this point, the head mounted sensor assembly 1105 may be locked in place.

[0105] The upper horizontal screen line and the face line 8 can be adjusted upward or downward by the α value or the control value. Therefore, any head-mounted sensor device or face system can be used synchronously with the system of the present invention, thereby making the method and system of the present invention more cost-effective and more versatile.

[0106] The method described herein is applicable whenever precise and controllable positioning of the head is required, with or without the use of positioning sensors. For example, during bloodless stereotactic brain surgery, or during imaging procedures that require a certain degree of precise positioning. This method is much more accurate than the mechanical clamp method, because traditional mechanical clamp systems cannot compensate for small movements of the head, so the method of the present disclosure is safer, more predictable and more controllable. The method can also be used to control object positioning for the purpose of using identified lines, or by using a head-mounted sensor device as described above to control angular image distortion of any type of head and neck image. The above-mentioned head-mounted sensor device has a three-axis electronic sensor that can be used to control roll, yaw and pitch.

[0107] Move your mouth to the sides correctly before opening it

[0108] In the neck, the open mouth view image in the scoliotic position can be used to evaluate damage to the cervicocranial supporting ligaments, for example, to diagnose cervicocranial instability. Since the atlantoaxial joint is a pure rotational joint, any rotation of the joint during the imaging process will cause a certain degree of image angle distortion. When performing open mouth imaging in the scoliotic position open mouth view using the method disclosed in the present invention, the default pitch line 37 must remain parallel to the upper horizontal facial line 36, and the angle between the vertical facial line 35, the default pitch line 37 and the upper horizontal facial line 36 must be strictly maintained at 90 degrees, such as Figure 8 As shown. The position of the open-mouth neck image in the scoliotic position is difficult to control, especially when taking accurate cervical and cranial images while controlling the rotation of the C1 / C2 atlantoaxial joint. If these rotations are introduced into the imaging because the positioning is not controlled, then a variety of different results may be seen on the images. In addition, without controlling the positioning, each image of the cervical and cranial region taken of the same patient will be different. In addition, it is also important to control the pitch angle because when taking the open-mouth view, the image of the palatine shelf and occipital bone may overlap with the image of the C1-C2 atlantoaxial. By using the method of the present disclosure, the patient can also be accurately repositioned to retake images of the cervical and cranial region to evaluate the progress of treatment or disease.

[0109] like Fig. 9 As shown, in order to correct the lateral movement to the right before opening the mouth, the default pitch line 41 must remain parallel to the upper horizontal facial line 39, and the angle between the vertical facial line 43, the default pitch line 41 and the upper horizontal facial line 39 must be strictly maintained at 90 degrees.

[0110] like Fig.10 As shown, we can see that relative to Fig. 9The pitch deviation is due to the fact that the pitch line 48 is not parallel to the upper horizontal face line 46. In addition, the angle between the vertical face line 49 and the default pitch line 48 is not 90 degrees, and the default pitch line 48 and the upper horizontal face line 46 are not parallel. To correct the deviation, the operator needs to instruct the patient to move his head until the default pitch line 48 is parallel to the upper horizontal face line 46 and the default pitch line 48 is perpendicular to the vertical face 49.

[0111] The method disclosed in the present invention can be used to achieve controllable left and right movement of the head without using any head-mounted sensor device or sensor. Figure 8-10 As shown, no head-mounted sensor device is shown in the figure. Figure 8-10 One significance is that the method disclosed herein allows diagnostic imaging of the atlantoaxial joint in an open mouth image of the cervicocranial scoliotic position without the use of any additional hardware to control positioning. The cervicocranial scoliotic position refers to the area extending from the occipital bone, C1 and C2, and is considered to be the most difficult area to image and analyze, which is why methods such as scoliotic position positioning and open mouth views for diagnosing injuries are significantly underutilized. Therefore, current radiological diagnostic technology is very limited. The method disclosed herein is a method for solving medical diagnostic imaging problems in anatomy.

[0112] Another significance of the above figure is that stereotactic surgery can be performed without the use of head-mounted sensor devices. Fig.13 and 14 As shown, accurate pixel data is sufficient to position the head in the desired position. Brain imaging is a complex process that requires a lot of electronic and electrical equipment. All electronic and electrical equipment generate magnetic fields that can distort the three-axis positioning sensors located close to these devices. Using pixel data to position the head based on facial landmarks is an important step to avoid the use of positioning electronics, which may fail in or around the magnetic fields or wires of MRI, CT, PET or X-ray equipment.

[0113] Diagnostic imaging of neck flexion and extension, and scoliosis can also be performed without the use of sensors, based on facial lines, or using a head-mounted sensor device positioned as described above.

[0114] Assisted sensor programming

[0115] After determining and recording the roll, yaw, and pitch line angle positions by default, such as Figure 6As shown, as long as the face is within the original frame of the computer, the data can be transferred to any other sensor as many times as needed. This position can be replicated and transferred to another sensor anywhere on the body and further used when other angle control positioning is required.

[0116] For example, an intraoral sensor prong made of a malleable dental impression material can be inserted into the patient's mouth, and the patient should close their mouth immediately after biting down on the sensor prong made of this material. As the vinyl polysiloxane (VPS) dental impression material is formed into a custom prong on the teeth, the material also molds around the teeth to form an interlocking pattern. A three-axis electronic sensor is fixed to the outside of the prong. The sensor is programmed when the default pitch line angle position is recorded. Alternatively, if you want to control the pure orthogonal positioning of the patient during spinal imaging, these coordinates can also be used or transmitted to the sensor on the spine via a BluetoothTM connection. The uses are endless.

[0117] In addition, if the positioning of the face and the anatomical morphology of other organs or other parts of the body are known, these other organs or parts can be positioned without auxiliary sensors. In this embodiment, the method establishes a coordinate relationship between the face and at least one other part of the body (e.g., an organ), and positions at least one other part of the body to a predetermined position by repositioning the head and / or face of the human body. For example, it is well known that the lateral orthogonal imaging of the spine is perpendicular to the front (frontal) orthogonal imaging of the face. If we program the orthogonal positioning of the face into a unit equipped with a camera or a unit representing a camera, then the positioning in the lateral orthogonal imaging of the spine will be the positioning of the face minus 90 degrees. We only need to rotate the patient 90 degrees toward the imaging source, so that the imaging source is located opposite the imaging device and 90 degrees from it, and let the camera perform orthogonal positioning imaging of the face. When programming the auxiliary device, we need to understand the relationship between the human face and anatomy. For example, the lateral orthogonal image of the spine is taken in a perpendicular manner to the face. With the aid of the auxiliary sensor, the disclosed method and system can image any object at any controlled position and later reposition that body part at the same position. For example, we know the exact position of the face towards the camera, and we program the position sensor on the wrist when saving the pitch angle data. This wrist sensor will save the XYZ coordinates related to the position of the face. As long as this wrist sensor can be accurately repositioned with or at the wrist, then we can position the wrist to the same XYZ coordinates. This is relativity.

[0118] The alpha value or control value is a unique control value for each individual face and has no effect on the distance between the face and the image capture device (e.g., camera). Therefore, when repeatedly positioning a head-mounted sensor device (e.g., head-mounted sensor device 1104), the alpha value allows the head-mounted sensor device to be positioned in exactly the same position as the first time without considering the distance between the face and the image capture device or camera.

[0119] Another use of the disclosed system and method is for targeting during an Air Force "dogfight," or for diagnostic measurements of neck mobility in video games, or in games that promote neck physical training and activity.

[0120] The devices, systems and methods disclosed herein can be used for the following use cases: positioning the human body to obtain correct posture by performing certain positioning next to a computer; physical medicine and rehabilitation testing to measure the range of motion of the neck; sports exercises and games; various imaging of the cervical skull, cervical spine and spine that requires controlled positioning; controlling the positioning of the head when using stereotactic brain surgery to diagnose and treat brain lesions; military aiming devices; positioning the patient before and during various imaging of the head, neck and spine when the control of image angle distortion becomes very important or when certain positioning protocols are used.

[0121] Fig.12 1 is a block diagram for illustrating the physical components of a computing device 1106, including a client computing device, a server, or any other computing device through which the present disclosure implements examples. The basic configuration of the computing device 1106 may include at least one processing unit 1204 and a system memory 1206. Depending on the configuration and type of the computing device, the system memory 1206 may include, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories. The system memory 1206 may include an operating system 1207 and one or more program modules 1208 suitable for running software programs / modules 1220 such as an IO manager 1224, other utilities 1226, and applications 1228. For example, the system memory 1206 may store instructions for execution. Other examples of the system memory 1206 include data that may be stored related to the application. For example, the operating system 1207 may be used to control the operation of the computing device 1106. In addition, the examples of the present disclosure may be used in conjunction with a graphics library, other operating systems, or any other application, and is not limited to any particular application or system. This basic configuration such as Fig.1212, which is shown by the components within the dashed line 1222. The computing device 1106 may have other features or functions. For example, the computing device 1106 may also include other data storage devices (removable memory and / or non-removable memory), such as magnetic disks, optical disks, tapes, etc. Such additional storage devices may include Fig.12 As shown, it is displayed by a removable memory 1209 and a non-removable memory 1210.

[0122] As described above, system memory 1206 may store a number of program modules and data files. When executed on processing unit 1204, program modules 1208 (e.g., input / output (I / O) manager 1224, other utilities 1226, and application programs 1228) may perform a number of processes, including but not limited to one or more of the operational stages described in the present disclosure. For example, one of application programs 1228 may execute Figure 1-10 The facial recognition software described in Figure 13-14 According to an example of the present disclosure, other program modules that may be used include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided application programs, photo editing applications, creative applications, etc. It is understood that multiple modules or applications 1228 may be executed simultaneously or approximately simultaneously and may share data.

[0123] In addition, examples of the present disclosure may be implemented in circuits consisting of discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or single chips containing electronic components or microprocessors. For example, examples of the present disclosure may be implemented via a system on a chip (SOC). Each or more components of the system on a chip (SOC) may be integrated onto a single integrated circuit, such as Fig.12 As shown. Such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") into a chip substrate as a single integrated circuit. When running through the SOC, the functions described herein may be run through application-specific logic integrated on a single integrated circuit (chip) together with other components of the computing device 1106. The examples of the present disclosure may also be implemented using other technologies capable of performing logical operations such as AND, OR, and NOT, including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, the examples of the present disclosure may also be implemented using a general-purpose computer or any other circuit or system.

[0124] The computing device 1106 may also include one or more input devices 1212, such as a keyboard, a mouse, a pen, a voice input device, a voice input / recognition device, a touch input device, etc. It may also include an output device 1214, such as a display, a speaker, a printer, etc. The above devices are only examples, and other devices may also be used. The computing device 1106 may include one or more communication connections 1216 that allow communication with other computing devices 918 (e.g., external servers) and / or other positioning system devices (e.g., imaging devices 1102, head-mounted sensor devices 1104, and head-fixed devices 1105, etc.). Examples of suitable communication connections 1216 include, but are not limited to: a network interface card; a radio frequency transmitter, receiver, and / or transceiver circuit; a universal serial bus (USB), parallel and / or serial port; and / or a wireless transceiver operating in accordance with (but not limited to) WIFI protocols, Bluetooth protocols, mesh protocols, etc.

[0125] The term "computer-readable medium" as used herein may include computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media that store information by any method or technology, such as computer-readable instructions, data structures, or program modules. System memory 1206, removable memory 1209, and non-removable memory 1210 are all examples of computer storage media (i.e., memory). Computer storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other storage technology, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and can be accessed by computing device 1202. Any such computer storage media may be part of computing device 1202. Computer storage media do not include carrier waves or other propagated or modulated data signals.

[0126] Communication media may be embodied by computer readable instructions, data structures, program modules or other data in a modulated data signal (e.g., a carrier wave or other transport mechanism), and includes any information transmission media. The term "modulated data signal" refers to a signal that has one or more characteristics set or changed so as to encode information in the signal. By way of example, communication media may include, but is not limited to, wired media (e.g., a wired network or direct-wired connection) and wireless media (e.g., acoustic waves, radio frequency (RF), infrared and other wireless media).

[0127] In further embodiments, the devices, systems, and methods of the present disclosure provide a user-friendly interface for precisely positioning a head and re-positioning it to the same position at a later time.

[0128] like Fig. 20As shown, a method for registering, authenticating, and locating a patient / individual is shown. In step 2002, an image of a patient is input into the system. For example, an imaging device 1102 can capture an image. In step 2004, facial recognition is performed in the input image. In one embodiment, a frontal view of a face is detected by using a pre-trained model called mediapipe in the Google API. The model detects 468 key points, such as Fig.21 As shown. As described below, these key points can be used to identify the patient and obtain the orientation of the patient's head. It is understood that the facial recognition function or algorithm can use key points to identify facial features such as eyes, brow lines, nose tips, corners of the mouth, etc.

[0129] In step 2006, the method may determine whether the recognized face is already in the database, for example, using an image matching function or algorithm. If the recognized face is not in the database, the patient is registered as a new patient in step 2008. In the registration step, data such as the patient's name, address, etc. may be stored in association with the image. If it is determined that the face already exists in the database, the patient data associated with the recognized face will be retrieved in step 2010. The retrieved data may include the patient's medical information, address, etc. The retrieved information may also include images of the patient previously stored in a particular location, which may need to be copied to update the patient's image. Whether registering a new patient or retrieving an existing patient, any recognition system can achieve this goal.

[0130] In step 2012, the roll, yaw and pitch of the patient's head (eg Fig. 22 ) will be determined by the detected key points. The patient will be instructed to move his head in different directions to achieve the desired position. In step 2014, the patient will be instructed to form an orthogonal view of his face with the imaging device 1102, that is, at a 90-degree angle to the imaging device. Maintaining this position, an MRI or CT scan can be performed after the patient closes his mouth. In addition, in this position, an X-ray can also be taken after the patient opens his mouth. As described below, the user can instruct the patient to reach the correct position based on the marks generated and displayed on the display device (e.g., the display device 1108 associated with the system 1100). Step 2016 will instruct the patient to achieve a left side bent view. Step 2018 will instruct the patient to achieve a right side bent view.

[0131] Understandably, Fig. 20 The method steps described in the embodiment can be performed by the processing unit 1204 of the computing device 1106. It can be further understood that the graphical user interface (GUI), the key point detection function or algorithm, the image matching function, etc. can be one of the application programs 1220 stored in the system memory 1206 and executed by the processing unit 1204.

[0132] Figure 23-25 This article describes how to calculate the patient's head roll. Roll refers to rotation in a clockwise or counterclockwise direction. If the face rolls counterclockwise, it means the face is rotating to the left, and the roll value is negative, as shown in 23. If the face rolls clockwise, it means the face is rotating to the right, and the roll value is positive, as shown in 24. If the face does not rotate at all, the roll value is zero, as shown in Fig.25 As shown. Fig.23 In the figure, line AB is a line parallel to the roll reference line PQ, which passes through the key points P and Q detected on the face. Line PQ is along the second principal component of the face, that is, if we regard the face as a space, it is equivalent to a horizontal line in this space. In one embodiment, point P and point Q are roughly located at the two ends of the eyebrow line of the face, as shown above. Fig.16 As described by points 18 and 27 in. Lines ST and UV are along the horizontal direction of the camera's viewing angle. To obtain the roll angle, the slopes of lines AB and ST need to be determined, where the slope of line AB is M and the slope of line ST is N. Since AB is parallel to PQ, the slopes of these two lines are equal. The slope of line PQ can be determined by the coordinates of points P and Q, which are obtained by the key point detection model. Therefore, the slope of line AB is: If the slope of line ST is N, the formula for calculating the roll angle a is as follows:

[0133] Figure 26-28 The calculation method of the patient's head yaw is introduced. The yaw is calculated as a pixel ratio. When the ratio value is 1, no rotation occurs, that is, like Fig.28 As shown, the individual is looking straight ahead at the imaging device 1102. When the individual looks to the right, the ratio value is greater than zero and less than one, that is, like Fig. 27 When the individual looks to the left, the ratio value will be greater than 1, that is, like Fig.26 As shown. Fig.26 In , line PQ is the line discussed above, i.e., the second principal component of facial key points. Line RN is perpendicular to line PQ, and line PQ always passes through a key point for nose N on the face.

[0134] Then, using point R we get the ratio x1:x2

[0135] Figure 29-31This article describes how to calculate the patient's head pitch. Pitch refers to the upward and downward movement of the head. Pitch is determined by an angle, which ranges from (-90,90). If the pitch value is positive, the head is moving upward, such as Fig.29 If the pitch value is negative, the head moves downward, as shown in Fig.30 If the pitch value is zero, that is, looking straight ahead, then the head movement will be neutral, as shown in Fig.31 shown.

[0136] When calculating pitch, the vector and Available through Fig.29 It is understood that points A and C are located on the facial circumference, and point B is approximately located on the nose tip, which is consistent with the above. Fig.18 The description of points 2, 15, and 34 is similar. The angle between the two vectors is calculated using the arctan function. This function gives the angle between the two vectors in any of the four quadrants, as shown in Table 2 below: Table 2 Quadrant Horizontal component (x) Vertical component (y) Quadrant 1 just just Second Quadrant burden just The third quadrant burden burden Quadrant 4 just burden

[0137] If vector A has two horizontal and vertical components, x and y, then the angle between the vector and the horizontal plane is Z, which is defined as: Z = arctan(x,y) AB and BC are two pitch lines. Vector and Calculated according to the following formula: α(degrees) = arctan((x a -x b ),(y a -y b ))-arctan((x c -x b ),(y c -y b ))

[0138] Figure 32-43 Shown are various screen shots of the graphical user interface (GUI) of the present disclosure. Fig.32As can be seen in FIG. 32 , the graphical user interface provides an image portion 3202 and optional items 3204 below the image portion 3202, such as “register patient”, “register doctor”, “open pitch”, “open mouth in neutral position”, “open mouth in left bent position” and “open mouth in right bent position”. After registering the patient, it is necessary to select a position from “open mouth in neutral position”, “open mouth in left bent position” and “open mouth in right bent position”, as shown in FIG.

[0139] exist Fig.33 , the user has selected the position "Neutral open mouth", but the patient / individual's face is rotated counter-clockwise, i.e. the roll value is negative, therefore the GUI should instruct the patient to rotate their face in the direction of the arrow to be in the neutral position.

[0140] exist Fig.34 , the patient has selected the position "neutral open mouth", but the patient / individual's face is rotated clockwise, i.e., the roll value is positive, therefore, the GUI should instruct the patient to rotate their face in the direction of the arrow to be in the neutral position.

[0141] exist Fig.35 In the example, the patient selected the position "neutral open mouth" but it is yawed to the left, therefore the GUI should instruct the patient to rotate their face in the direction of the arrow to be in the neutral position.

[0142] exist Fig.36 In the video, the patient selected the position "neutral open mouth" but it is yawed to the right, so the GUI should instruct the patient to rotate his face in the direction of the arrow to be in the neutral position.

[0143] exist Fig.37 In the image, the patient selected the position "Neutral open mouth", but the pitch value is positive, so the GUI should instruct the patient to rotate their face in the direction of the arrow to be in the neutral position.

[0144] exist Fig.38 In the image, the patient selected the position "Neutral open mouth" but the pitch value is negative, therefore the GUI should instruct the patient to rotate their face in the direction of the arrow to be in the neutral position.

[0145] exist Fig.39, the patient has selected the position "neutral open mouth". In the neutral position, the pitch value is zero, the yaw is neutral, and the roll value is zero. Here, the vertical face line 3902 and the horizontal face lines 3904, 3906 are aligned with the vertical screen line 3908 and the horizontal screen lines 3910, 3912 respectively (or at least within a predetermined tolerance range). However, the patient's mouth is not aligned with the target 3902, for example, when the alignment target is 3902, so a message will be displayed stating that the mouth is not aligned with the target. Therefore, the patient must now align his mouth with the target 3902 to meet the requirements of all values ​​of pitch, yaw, and roll. It will be understood that the center of the alignment target 3902 is the center of the mouth. As described below, the center of the mouth is determined as the midway point between the corners of the mouth.

[0146] exist Fig.40 , the patient selected the position "neutral open mouth". In the neutral position, the pitch value is zero, the yaw is neutral, and the roll value is zero. The patient is now aligned with the target 3902. In this position, the system is ready to take an x-ray. In addition, an MRI or CT scan can be performed after the patient closes his mouth. It is understood that once the patient is aligned with the target, an indication will be issued to confirm it, for example, various lines drawn on the screen may turn green, sounds may be triggered, etc. In addition, once the patient is aligned with the target, the system can automatically trigger the x-ray (or CT scan, MRI scan, etc.) without further input from the user.

[0147] exist Fig.41 In the test, the patient selected the position "Neutral position open mouth". In the neutral position, the pitch and yaw are neutral and the roll value is zero. In addition, the patient must open his mouth and look at the target box. In this case, the system is now ready to take an X-ray.

[0148] exist Fig.42 In the left bend position, the patient has selected the position "Open mouth in left bend position". In the left bend position, the pitch and yaw are neutral and the roll values ​​are negative. It can be understood that the lines on the screen are the desired positions for this selected mode (i.e., the generated face lines and screen lines), and the GUI will instruct the patient to move his head according to his positioning in the neutral position, as described above. The patient is now also aligned with the target point. In this case, the system is now ready to take an X-ray.

[0149] exist Fig.43In the right bend position, the patient has selected the position "Right bend position with open mouth". In the right bend position, the pitch and yaw are neutral and the roll values ​​are positive. It can be understood that the lines on the screen are the desired positions for this selected mode (i.e., the generated face lines and screen lines), and the GUI will instruct the patient to move his head according to his positioning in the neutral position, as described above. The patient is now also aligned with the target point. In this case, the system is now ready to take an X-ray.

[0150] exist Figure 42-43 In the figure, it can be understood that for the side-bending position with the mouth open, the center of the mouth is the rotation axis of the head (i.e., the rotation axis of the roll). The center of the mouth can be calculated as the point at the length 1Z of the left and right corners of the mouth. For example, Fig.31 As shown, the facial recognition function or algorithm may identify the corners of the mouth as points D and E. Half the distance between these two points is identified as the center of the mouth, which is point F. The individual's head may then be rotated to obtain a roll about the center of the mouth with a pitch value of 0 and a yaw value of neutral.

[0151] Advantages of the apparatus, system and method of the present disclosure include accurate positioning of the patient's head and neck for C1 / C2 diagnostic imaging. Due to the greatly improved positioning accuracy, the apparatus, system and method of the present disclosure can provide diagnostically acceptable optimal imaging studies. Therefore, the apparatus, system and method of the present disclosure can eliminate unnecessary X-ray exposure under ionizing radiation.

[0152] No other positioning system currently exists that can accurately position the head repeatedly in a space with high surrounding magnetic or electrostatic fields, and reposition it to the same position in the future. Mechanical head fixation devices do not provide the accuracy required to accurately position the head.

[0153] All of these benefits can lead to benefits such as improved diagnostic quality, provision of treatment proven to be medically necessary, improved prognosis, and safe treatment.

[0154] It is understood that the various features shown and described are interchangeable, that is, the features shown in one embodiment can be incorporated into another embodiment. It is also understood that the above methods, functions, algorithms, etc. can be implemented by any single device and / or a combination of multiple devices constituting a system, including but not limited to meters, IEDs, servers, storage devices, processors, memories, FPGAs, DSPs, etc.

[0155] Although the present disclosure discloses non-limiting embodiments, many variations may occur, and these variations are still within the concept and scope of the present disclosure. For those of ordinary skill in the art, they will understand these variations after reading the specification, drawings and claims. Therefore, the content of the present disclosure is not limited in any way except the spirit and scope of the attached claims.

[0156] Furthermore, although the above text has described numerous embodiments in detail, it should be understood that the legal scope of the present disclosure is defined by the words of the claims listed at the end of this patent. The detailed description should be understood as being only examples and not every possible embodiment, because it is neither possible nor practical to describe every possible embodiment. Many alternative embodiments can be implemented using current technology or technology developed after the filing date of this patent, and these embodiments still fall within the scope of the claims.

[0157] It should also be understood that unless a term is expressly defined in this patent using the phrase "As used in this disclosure, the term __________ is defined herein to mean..." or similar sentences, this patent is not intended to limit the meaning of the term, either expressly or impliedly, beyond its ordinary meaning, and the term should not be construed as limited in scope based on any statement in any part of this patent (except for the language used in the claims). Any term recited in the claims at the end of this patent is referred to in this patent in a manner consistent with a single meaning, which is only for the sake of clarity and to avoid confusion to the reader, and this patent is not intended to limit the term in the claim to that single meaning by implication or otherwise. Finally, unless a claim element is defined by reciting the word "means" and function without reciting any structure, it is not intended to interpret the scope of any claim element based on the application of 35 U.S.C. § 112, paragraph 6.

Claims

1. A method comprising: Imaging the face of a human head using at least one imaging device; generating at least one vertical face line and at least two horizontal face lines on the face image, and generating at least one vertical face line and at least two horizontal face lines based on facial landmarks on the face; Aligning the human head so that at least one vertical face line is aligned with the generated vertical screen line and each of at least two horizontal face lines is aligned with the generated horizontal screen line; storing the alignment line as a reference line; Repositioning the human head to a desired position; and Provides an indication of the desired position of the human head.

2. The method of claim 1 , wherein generating at least one vertical facial line and at least two horizontal facial lines comprises using at least one facial recognition application to identify facial landmarks on a human head as key points, and connecting the key points to draw at least one vertical facial line and at least two horizontal facial lines.

3. The method of claim 1, wherein the generated vertical screen lines and horizontal screen lines represent that the face is at a 90 degree angle to the imaging device.

4. The method of claim 3, wherein among the aligned at least two horizontal face lines and the generated horizontal screen line, a lower line is stored as a default pitch line. The method of claim 4 , wherein the default pitch line is divided into two segments at at least one vertical facial line.

6. The method according to claim 5, wherein aligning the human head comprises aligning two default pitch lines so that both are parallel to upper lines of the two horizontal facial lines.

7. The method of claim 4, wherein the default pitch line is normalized to compensate for differences in facial structures of human heads.

8. The method according to claim 6, wherein each of the two default pitch lines is drawn by connecting a recognized facial landmark at a nose tip of the face with at least one point on a facial circumference.

9. The method according to claim 2, wherein at least one vertical facial line is drawn by connecting the recognized facial landmarks vertically distributed at the center of the nose of the face.

10. The method of claim 9, wherein the yaw angle of the human head is determined based on at least one vertical face line and a generated vertical screen line.

11. The method of claim 2, wherein the upper lines of at least two horizontal facial lines are drawn by connecting the recognized facial landmarks at opposite ends of the facial brow line.

12. The method according to claim 11, wherein the roll angle of the human head is determined based on at least two horizontal face lines and an upper line of the generated horizontal screen lines.

13. The method of claim 1 further comprising storing coordinates associated with the alignment of the repositioned human head at a first point in time, and repositioning the human head at a second point in time based on the stored coordinates.

14. The method of claim 1, wherein the repositioning of the human head comprises rotating the human head around a rotation axis, wherein the rotation axis is about at least one of a center of a mouth and a tip of a nose of the human head.

15. The method according to claim 1, wherein aligning the human head so that the pitch angle of the human head relative to at least one imaging device is zero, further includes repositioning the human head to a desired position while ensuring that the pitch angle is zero, and providing an indication that the human head is in the desired position and the pitch angle is zero.

16. A system comprising: at least one imaging device for imaging the face of a human head; at least one processing device that generates at least one vertical face line and at least two horizontal face lines on the face image through the display device, and generates at least one vertical face line and at least two horizontal face lines based on facial landmarks on the face; A display device for aligning a human head so that at least one vertical facial line is aligned with a generated vertical screen line and so that each of at least two horizontal facial lines is aligned with a generated horizontal screen line; as well as at least one memory for storing the alignment line as a reference line, Wherein, when repositioning the human head to a desired position, at least one processing device continuously updates at least one vertical facial line and at least two horizontal facial lines based on facial markers on the face.

17. The system of claim 16, wherein at least one processing device is operable to perform at least one facial recognition function for identifying facial landmarks on the face as key points and connecting the key points to draw at least one vertical facial line and at least two horizontal facial lines.

18. The system of claim 16, wherein at least one processing device provides an indication that the person's head is in a desired position.

19. The system of claim 18, wherein the repositioning of the human head comprises rotating the human head around a rotation axis, wherein the rotation axis is about at least one of a center of a mouth and a tip of a nose of the human head.

20. The system of claim 18, wherein aligning the human head so that the pitch angle of the human head relative to at least one imaging device is zero further includes repositioning the human head to a desired position while ensuring that the pitch angle is zero, and providing an indication that the human head is in the desired position and the pitch angle is zero.

Images