Head posture detection methods, mobile CT equipment, electronic devices and storage media

Abstract

This invention discloses a head pose detection method, a mobile CT scanner, an electronic device, and a storage medium. The head pose detection method is applied to a mobile CT scanner, which includes a camera within its cavity. The method includes: acquiring a head image based on the camera and detecting the nose region within the head image to obtain a nose region image; performing ellipse recognition on the nose region image to obtain multiple elliptical regions; determining nose feature information based on the multiple elliptical regions; determining the degree of head offset based on the nose feature information; and detecting head pose based on the degree of head offset. This invention's head pose detection method performs ellipse detection on the nose region and extracts nose feature information, achieving head pose detection based on this nose feature information. This solves the problem of traditional head pose estimation methods heavily relying on complete facial key points, making the method simpler and more efficient.

Description

Technical Field

[0001] This application relates to the technical fields of medical devices and computer vision, and in particular to a head posture detection method, a mobile CT device, an electronic device, and a storage medium. Background Technology

[0002] Head pose estimation research is mainly applied in virtual reality, human-computer interaction, and intelligent monitoring. For traditional fixed CT equipment, subject positioning detection, such as for patients, involves detecting the posture of the torso and limbs, primarily relying on the detection of key skeletal points. However, mobile CT is often used for critically ill or unconscious patients, and the involuntary head movements during imaging can cause artifacts in the CT images. While minor head pose changes can be corrected through post-processing algorithms, larger changes can lead to severe artifacts in the original images received by the detectors, interfering with subsequent diagnoses. Summary of the Invention

[0003] Therefore, the purpose of this application is to propose a head pose detection method, a mobile CT device, an electronic device, and a storage medium, which performs ellipse detection on the nose region and extracts nose feature information, and realizes head pose detection based on the nose feature information. This solves the problem that traditional head pose estimation methods heavily rely on complete facial key points, and the method is simpler and more efficient.

[0004] This application provides a head posture detection method applied to a mobile CT scanner. The mobile CT scanner includes a camera inside its cavity. The method includes: acquiring a head image based on the camera and performing nose region detection on the head image to obtain a nose region image; performing ellipse recognition on the nose region image to obtain multiple elliptical regions; determining nose feature information based on the multiple elliptical regions; determining the degree of head offset based on the nose feature information; and detecting head posture based on the degree of head offset.

[0005] For example, the step of performing ellipse recognition on the nose region image to obtain multiple elliptical regions includes: performing binarization processing on the nose region image, and performing edge detection on the binarized nose region image to obtain edge contour information; and performing ellipse recognition on the edge contour information to obtain multiple elliptical regions.

[0006] For example, the nose feature information includes a nose feature line, and the step of determining the nose feature information based on the plurality of elliptical regions includes: determining the elliptical region with the largest area among the plurality of elliptical regions as the target elliptical region; and determining the minor axis of the target elliptical region as the nose feature line.

[0007] For example, the nose feature information includes the first coordinate information of the nose feature line in a first preset coordinate system, the head offset degree includes the head yaw angle, and determining the head offset degree based on the nose feature information includes: performing coordinate transformation on the first coordinate information to obtain the second coordinate information of the nose feature line in a second preset coordinate system for the head image; and calculating the head yaw angle based on the second coordinate information.

[0008] For example, the step of performing coordinate transformation on the first coordinate information to obtain the second coordinate information of the nose feature line in a second preset coordinate system for the head image includes: determining the nose region bounding box coordinate information in the first preset coordinate system and the nose region bounding box coordinate information in the second preset coordinate system, wherein the first preset coordinate system is a coordinate system established based on the nose region image, and the second preset coordinate system is a coordinate system established based on the head image; determining a coordinate transformation matrix based on the nose region bounding box coordinate information in the first preset coordinate system and the nose region bounding box coordinate information in the second preset coordinate system; and performing coordinate transformation on the first coordinate information based on the coordinate transformation matrix to obtain the second coordinate information of the nose feature line in the second preset coordinate system for the head image.

[0009] For example, the head image includes multiple head images, and the head posture detection based on the degree of head offset includes: detecting whether the multiple head images meet preset conditions; if they do, turning off the ray source and issuing a prompt message, wherein the prompt message represents the degree of head movement of the subject, and the preset conditions include at least one of the following: the difference between the head yaw angles corresponding to two adjacent head images is greater than a first preset threshold; the difference between the head yaw angle corresponding to the current head image and the head yaw angle corresponding to the initial head image is greater than a second preset threshold.

[0010] For example, the step of detecting the nose region in the head image to obtain a nose region image includes: detecting the nose region in the head image based on a trained object detection model to obtain a nose region detection result; and obtaining a nose region image based on the nose region detection result. The trained object detection model is obtained by: constructing a training dataset, which includes multiple head images and nose region annotation information corresponding to each head image; and training a preset object detection model based on the training dataset to obtain a trained model.

[0011] For example, the nose region detection result includes multiple third bounding boxes of the nose region and a confidence score corresponding to each third bounding box. Obtaining a nose region image based on the nose region detection result includes: determining the third bounding box corresponding to the maximum value among the confidence scores as the target bounding box, and filtering out third bounding boxes that are duplicates of the target bounding box; obtaining the nose region image based on the target bounding box, and displaying the target bounding box.

[0012] Another embodiment of this application provides a mobile CT device, the mobile CT device comprising: a mobile scanning cavity; a camera, the camera being placed at the top of the inner cavity of the scanning cavity and located obliquely above the head of the subject, the camera being used to capture head images; and a head posture detection device, the head posture detection device being used to implement the steps of the above method.

[0013] Another embodiment of this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any of the above embodiments.

[0014] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0015] Another embodiment of this application provides a computer program product, which includes instructions that, when executed by a processor of a computer device, enable the computer device to perform the steps of the method described in any of the above embodiments.

[0016] In the above embodiments, the head pose detection method is applied to a mobile CT scanner. The mobile CT scanner cavity includes a camera, and the method includes: acquiring a head image based on the camera, and detecting the nose region in the head image to obtain a nose region image; performing ellipse recognition on the nose region image to obtain multiple elliptical regions; determining nose feature information based on the multiple elliptical regions; determining the degree of head offset based on the nose feature information, and detecting head pose based on the degree of head offset. The head pose detection method of the present invention performs ellipse detection on the nose region and extracts nose feature information, and realizes head pose detection based on the nose feature information. This solves the problem that traditional head pose estimation methods heavily rely on complete facial key points, making the method simpler and more efficient. Attached Figure Description

[0017] Figure 1 A flowchart of the head pose detection method provided for the embodiments of this application;

[0018] Figure 2 A schematic diagram of a mobile CT device provided for an embodiment of this application;

[0019] Figure 3 A schematic diagram illustrating the positional relationship between the mobile CT camera and the patient, provided for an embodiment of this application;

[0020] Figure 4 A schematic diagram of a head image provided for an embodiment of this application;

[0021] Figure 5 A flowchart for nose region detection in a head image provided as an embodiment of this application;

[0022] Figure 6 A flowchart for obtaining a nose region image provided in an embodiment of this application;

[0023] Figure 7 A schematic diagram illustrating the nose region detection results provided in an embodiment of this application;

[0024] Figure 8 A flowchart for ellipse recognition of a nose region image provided for an embodiment of this application;

[0025] Figure 9 A flowchart for determining nose feature information provided in an embodiment of this application;

[0026] Figure 10 A schematic diagram of the elliptical region provided for an embodiment of this application;

[0027] Figure 11 A flowchart for determining the degree of head offset provided in this application embodiment;

[0028] Figure 12 A schematic diagram of three Euler angles provided for embodiments of this application;

[0029] Figure 13 A schematic diagram of the nose region image coordinate system provided for an embodiment of this application;

[0030] Figure 14 A flowchart of coordinate transformation provided for embodiments of this application;

[0031] Figure 15 A schematic diagram of the head image coordinate system provided for an embodiment of this application;

[0032] Figure 16 A flowchart of a head posture detection method based on mobile CT provided for an embodiment of this application.

[0033] Figure 17 A block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0034] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0035] Head pose estimation research is mainly applied in virtual reality, human-computer interaction, and intelligent monitoring. For traditional fixed CT equipment, subject positioning detection, such as for patients, involves detecting the posture of the torso and limbs, primarily relying on the detection of key skeletal points. However, mobile CT is often used for critically ill or unconscious patients, and the involuntary head movements during imaging can cause artifacts in the CT images. While minor head pose changes can be corrected through post-processing algorithms, larger changes can lead to severe artifacts in the original images received by the detectors, interfering with subsequent diagnoses.

[0036] Based on this, this application proposes a head posture detection method. This method is based on a mobile CT device. A camera is installed on the cavity of the CT device to acquire images of the patient's head. Ellipse recognition is performed on the patient's head images to obtain nose feature information. Based on the nose feature information, the degree of head displacement is determined, thereby realizing continuous monitoring of changes in the patient's head posture.

[0037] Figure 1 This is a flowchart of a head pose detection method according to an embodiment of this application.

[0038] As an implementation, such Figure 1 As shown, the head pose detection method includes:

[0039] S101: Acquire a head image based on a camera, and perform nose region detection on the head image to obtain a nose region image.

[0040] S102, Ellipse recognition is performed on the nose region image to obtain multiple elliptical regions.

[0041] S103, determines nose feature information based on multiple elliptical regions.

[0042] S104 determines the degree of head offset based on nose feature information and detects head posture based on the degree of head offset.

[0043] For example, the head pose detection method is applied to a mobile CT device, such as... Figure 2The diagram shows a mobile CT scanner, a portable computed tomography (CT) device. Unlike traditional fixed CT scanners, mobile CT scanners provide rapid and convenient bedside CT imaging services in various clinical environments, especially in hospitals and surrounding areas. They offer high mobility and convenience and can be mounted on dedicated vehicles, such as ambulances. Figure 2 As shown, mobile CT equipment uses devices such as headrest 01, shoulder rest 02 and examination bed 03 to restrict the patient's position during use. However, mobile CT is mainly used for critically ill or unconscious patients. During the imaging process, the patient's involuntary head movement can cause artifacts in the CT images.

[0044] For example, this application may install a camera inside the cavity of a mobile CT device, such as... Figure 3 As shown, due to the limitations of the mobile head CT scanner's size and aperture, the camera 301 can be placed inside the cavity, taking images at a position slightly higher than the front of the human body. The head support device 302 is used to support the patient's head, and the mobile CT device 303 has casters installed at its bottom, allowing it to be moved or fixed depending on the specific application scenario. The camera 301 can be a monocular camera or a depth camera.

[0045] For example, a mobile CT device acquires head images based on a camera. This can be done by capturing images with the camera or by acquiring a head image from a frame of a video stream recorded by the camera. Figure 4 The image shown is a schematic diagram of a head. The scanning process in a mobile CT scan is a dynamic process; due to the movement and camera angle, it is difficult for the camera to capture complete facial features, such as... Figure 4 The captured image only shows a small portion of the patient's facial features, making traditional head pose detection methods, which rely on complete facial landmarks, unsuitable for this scenario. This application performs nose region detection on the head image to obtain a nose region image, and then performs head pose detection based on this local nose region image, overcoming the limitations of traditional head pose detection methods that rely on complete facial landmarks. The head image is taken, for example, from a position slightly above or below the patient's head. For example, from a slightly above position... Figure 3 The camera 301 is positioned at a shooting angle relative to the patient, and the patient's nasal area is shown from this angle as follows: Figure 4 As shown. Of course, the patient's face can also be photographed from a position diagonally below the patient's head.

[0046] For example, after obtaining the nose region image, ellipse recognition is performed on the nose region image to obtain multiple elliptical regions. This application introduces a corresponding contour recognition algorithm in conjunction with the nose object features to determine the nose's feature information through multiple elliptical regions. The nose feature information can be, for example, a feature line at the bridge of the nose. The degree of head offset is determined based on the nose feature information, thereby realizing head pose detection.

[0047] The head posture detection method of this application obtains nose feature information by performing ellipse recognition on the patient's head image, and determines the degree of head offset based on the nose feature information, thereby realizing continuous monitoring of the patient's head posture changes and overcoming the limitations of traditional head posture detection methods that rely on complete facial key points.

[0048] As an example, such as Figure 5 As shown, nose region detection is performed on the head image to obtain a nose region image, including:

[0049] S501, based on the trained object detection model, performs nose region detection on the head image and obtains the nose region detection result.

[0050] S502, obtains a nose region image based on the nose region detection results.

[0051] The trained object detection model is obtained through the following method:

[0052] Construct a training dataset, which includes multiple head images and the corresponding nose region annotation information for each head image;

[0053] The pre-defined target detection model is trained based on the training dataset to obtain the trained model.

[0054] For example, nose region detection in head images can be performed using a trained object detection model. The object detection model can be a convolutional neural network (CNN). The specific training process is as follows: First, a training dataset is constructed, for example, collecting images from a moving CT camera containing nose images. The nose region for each head image is labeled, covering images at different distances, under different lighting conditions and backgrounds. The labeling information can be represented by rectangular bounding boxes indicating the nose region. Then, a suitable preset object detection model is selected, such as the YOLO model. This preset model is trained based on the training dataset to obtain a trained model. Images from the training dataset are input into the preset object detection model. The preset object detection model includes image preprocessing, such as resizing and normalization. The loss function of the preset object detection model can be cross-entropy loss and the IOU (Intersection over Union) function. The model undergoes continuous iterative training to finally obtain a trained object detection model.

[0055] For example, the acquired head image is input into a trained object detection model, the object detection model outputs the nose region detection result, and the required nose region image is obtained based on the nose region detection result.

[0056] As an example, such as Figure 6 As shown, the nose region detection results include multiple third bounding boxes of the nose region and the confidence score corresponding to each third bounding box. Based on the nose region detection results, a nose region image is obtained, including:

[0057] S601, determine the third bounding box corresponding to the maximum confidence value as the target bounding box, and filter out the third bounding boxes that are duplicates of the target bounding box.

[0058] S602, Obtain the nose region image based on the target bounding box and display the target bounding box.

[0059] For example, the object detection model outputs nose region detection results, which include multiple third bounding boxes of the nose region and the confidence score corresponding to each third bounding box. It can be understood that the third bounding box represents the location of the nose in the head image, and the confidence score represents the reliability of the detection result. The third bounding box corresponding to the maximum confidence score can be selected as the target bounding box, and the target bounding box is displayed in the head image, for example, as shown below. Figure 7The nose region detection results are shown. Duplicate third bounding boxes are filtered out, for example, using a non-maximum suppression algorithm. The nose region image is obtained based on the target bounding box. It can be understood that the image within the target bounding box is the nose region image, which can be obtained through operations such as cropping.

[0060] After obtaining the image of the nose region, ellipse recognition is performed on the nose region image to obtain multiple elliptical regions.

[0061] As an example, such as Figure 8 As shown, ellipse recognition is performed on the nose region image, resulting in multiple elliptical regions, including:

[0062] S801 performs binarization on the nose region image and performs edge detection on the binarized nose region image to obtain edge contour information.

[0063] S802 performs ellipse recognition on the edge contour information to obtain multiple elliptical regions.

[0064] For example, to perform elliptic recognition on a nose region image, the nose region image can first be binarized, for example, using the Otsu algorithm. Binarization represents each pixel of the nose region image with either 0 or 1. Edge detection is then performed on the binarized nose region image, for example, using the Canny edge detection algorithm, to extract edge contour information, where the extracted edge contour is the edge contour of the nose. Elliptic recognition is then performed on the edge contour information to obtain multiple elliptical regions. For example, a Hough transform is applied to the edge contour to identify the multiple elliptical regions contained within it.

[0065] Then, based on the multiple elliptical regions obtained from ellipse recognition, nose feature information is determined.

[0066] As an example, such as Figure 9 As shown, the nose feature information includes nose feature lines, and the nose feature information is determined based on multiple elliptical regions, including:

[0067] S901, determine the elliptical region with the largest area among multiple elliptical regions as the target elliptical region.

[0068] S902, the minor axis of the target elliptical region is determined to be the nose feature line.

[0069] For example, the nose feature information includes nose feature lines, which may be feature lines of the bridge of the nose. Based on multiple elliptical regions obtained through ellipse recognition, the area of ​​each elliptical region is calculated, and the elliptical region with the largest area is determined as the target elliptical region. The minor axis of the target elliptical region is then determined as the nose feature line. For example, as shown... Figure 10 The elliptical region shown includes a major axis and a minor axis; alternatively, the major axis can be taken as the nose feature line. The length of the feature line can be a preset length.

[0070] As an example, nose feature information includes nose feature lines. Determining nose feature information based on multiple elliptical regions involves: when multiple elliptical regions corresponding to the initial head image are obtained, receiving a selection instruction from the user and using the selected elliptical region as the initial target elliptical region; when multiple elliptical regions corresponding to any subsequent frame of the head image are obtained, selecting the elliptical region whose area and / or shape are closest to the initial target elliptical region as the target elliptical region for the corresponding head image. The shape can be represented by the minor axis and / or major axis of the ellipse. A weighted summation can be used to characterize the ellipse's features based on the area, minor axis, and major axis. Then, the elliptical features of the initial head image are compared with the elliptical features of the current frame of the head image, and the ellipse in the current frame that is closest to the elliptical features of the initial head image is determined as the target elliptical region. Using this method, an ellipse that better matches the actual area of ​​the subject's nose can be selected to describe its features, improving detection accuracy.

[0071] After obtaining the nose feature information, the degree of head offset is determined based on the nose feature information, and the head posture is detected based on the degree of head offset.

[0072] As an example, the nose feature information includes the first coordinate information of the nose feature line in a first preset coordinate system, and the head offset includes the head yaw angle, such as... Figure 11 As shown, based on nasal feature information, the degree of head offset is determined, including:

[0073] S1101, Perform coordinate transformation on the first coordinate information to obtain the second coordinate information of the nose feature line in the second preset coordinate system for the head image.

[0074] S1102, calculate the head yaw angle based on the second coordinate information.

[0075] For example, the orientation of a human head in three-dimensional space can be described by the three degrees of freedom: the pitch angle (vertical rotation), the yaw angle (horizontal rotation), and the rotation angle (left and right rotation). Figure 12As shown, in three-dimensional space, the rotation of the human head posture can be represented by three Euler angles. The rotation around the X-axis is called pitch, the rotation around the Y-axis is called yaw, and the rotation around the Z-axis is called roll.

[0076] For example, this application can represent the head posture estimation of a moving CT patient by calculating the head yaw angle, for the following reasons: Figure 3 As shown, during a mobile CT examination, when the patient's head is supported by a headrest, a coordinate system for the human head posture can be established on the rigid and completely fixed headrest. The central symmetry line of the headrest is defined as the Y-axis, the straight line along the direction from the ear to the left shoulder and perpendicular to the Y-axis is defined as the X-axis, and the straight line perpendicular to the plane containing the X and Y axes is called the Z-axis. Because patients are generally lying down during CT examinations, their heads are usually supported by the examination table and headrest, preventing them from making forward or backward pitch movements, thus restricting the pitch direction, i.e., the degree of freedom around the X-axis. Furthermore, because the headrest is generally a rigid, arc-shaped body, the sides of the patient's head are usually restricted by the headrest, which also restricts the roll direction, i.e., the degree of freedom around the Z-axis. Therefore, head posture detection during mobile CT examinations can be simplified to calculating the patient's head yaw angle. Of course, pitch and roll angles can also be calculated to achieve head posture detection based on multiple dimensions. Preferably, based on the physical characteristics of the mobile CT equipment, the method of head posture detection by calculating the patient's head yaw angle is simpler and more efficient.

[0077] For example, the mobile CT camera can be set at the top of the CT scanner cavity and at an angle above the head of the subject. The camera is located on one side of the CT scanning area and captures the patient's face from the direction of the top of the patient's head. The camera can be positioned in a vertical plane at a distance from the rotation axis of the CT scan, within a preset range that does not affect the accuracy of the nose area recognition.

[0078] The obtained two-dimensional image plane is perpendicular to the Y-axis of the head posture coordinate system. The projection of the patient's nasal feature line onto this two-dimensional image is perpendicular to the Y-axis of the head posture coordinate system, and therefore can be used to characterize the yaw angle.

[0079] For example, the head offset includes the head yaw angle, and the nose feature information includes the first coordinate information of the nose feature lines in a first preset coordinate system. The first preset coordinate system can be understood as a local coordinate system for the nose region image. For example, as Figure 13As shown, for the nose region image, a point in the third bounding box can be used as the origin to establish a two-dimensional coordinate system to obtain the first preset coordinate system, thereby obtaining the first coordinate information of the nose feature line. It should be noted that the first coordinate information of the nose feature line is not limited to the coordinate information of the two endpoints of the nose feature line, but can be the coordinate information of any point on the nose feature line.

[0080] For example, the first coordinate information is transformed to obtain the second coordinate information of the nose feature line in the head image. It can be understood that the coordinate transformation converts the coordinate information of the nose feature line from a first preset coordinate system, such as the local coordinate system for the nose region image, to a second preset coordinate system, such as the global coordinate system for the head image. The coordinate information of the nose feature line in the global coordinate system for the head image is the second coordinate information of the nose feature line. The head yaw angle is calculated based on the second coordinate information.

[0081] As an example, such as Figure 14 As shown, coordinate transformation is performed on the first coordinate information to obtain the second coordinate information of the nose feature line in a second preset coordinate system for the head image, including:

[0082] S1401, determine the bounding box coordinates of the nose region in the first preset coordinate system and the bounding box coordinates of the nose region in the second preset coordinate system, wherein the first preset coordinate system is established based on the nose region image and the second preset coordinate system is established based on the head image.

[0083] S1402, Based on the bounding box coordinates of the nose region in the first preset coordinate system and the bounding box coordinates of the nose region in the second preset coordinate system, determine the coordinate transformation matrix.

[0084] S1403, based on the coordinate transformation matrix, the first coordinate information is transformed to obtain the second coordinate information of the nose feature line in the second preset coordinate system for the head image.

[0085] For example, to perform coordinate transformation on the first coordinate information of the nose feature lines, it is necessary to obtain the coordinate transformation relationship between the local image coordinate system of the nose region and the overall image coordinate system of the head, such as a coordinate transformation matrix. A first preset coordinate system can be established on the nose region image, and the first preset coordinate system can be as follows: Figure 13 The coordinate system shown uses a point within the bounding box of the nose region as the origin, and the two boundaries bordering the origin as the X and Y axes. A second preset coordinate system is established on the head image. This second preset coordinate system can use the upper left corner of the overall head image as the origin to establish a two-dimensional coordinate system, for example... Figure 15As shown, the X-axis extends from left to right, and the Y-axis extends from top to bottom. The position of the nose region bounding box in the overall head image is known. Based on the principle of establishing coordinate systems, the bounding box coordinates of the nose region in the first preset coordinate system and the second preset coordinate system can be determined. It can be understood that the bounding box coordinates of the nose region in the first preset coordinate system are local bounding box coordinates for the nose region image, while the bounding box coordinates of the nose region in the second preset coordinate system are overall bounding box coordinates for the head image. Based on the bounding box coordinates of the nose region in the first and second preset coordinate systems, the coordinate transformation matrix between the two coordinate systems is determined.

[0086] For example, the bounding box coordinate information may include the coordinates of the four vertices of the rectangular bounding box. For instance, a point within the bounding box in the second preset coordinate system can be designated as follows: the top-left corner of the rectangular bounding box is (X0, y0), the top-right corner is (X0+w, y0), the bottom-left corner is (X0, y0+h), and the bottom-right corner is (X0+w, y0+h). Here, w is the length of the rectangular region on the X-axis, and h is the length of the rectangular region on the Y-axis. It should be noted that, as... Figure 15 The second preset coordinate system shown has the rectangular region tilted relative to it. In this case, w is the projected length relative to the X-axis, and h is the projected length relative to the Y-axis. Of course, the coordinate system can be established in other ways. For example, for ease of calculation, the X and Y axes of the coordinate system can be established in directions parallel to the rectangular bounding box. The purpose of establishing the coordinate system is to obtain the transformation relationship from the local nose region image to the original image. The form in which the coordinate system is established is not limited.

[0087] For example, based on the coordinate information of the four vertices in the bounding box, a coordinate transformation matrix is ​​obtained. This matrix represents the transformation relationship from a first preset coordinate system to a second preset coordinate system. The coordinate transformation matrix is ​​obtained using the following formula:

[0088]

[0089] Where A is the coordinate transformation matrix, (x′, y′) represents the bounding box coordinates in the first preset coordinate system, (X, Y) represents the bounding box coordinates in the second preset coordinate system, and (x0, y0) is the coordinate of the upper left corner of the bounding box in the second preset coordinate system. Therefore, the coordinate transformation matrix A can be solved using the known coordinates of the four points of the rectangle.

[0090] Based on the coordinate transformation matrix A, the first coordinate information is transformed to obtain the second coordinate information of the nose feature line in the head image (i.e., the coordinate information of the nose feature line in the second coordinate system). For example, the first coordinate information is the coordinates of any two points on the nose feature. Transforming these two coordinates using the coordinate transformation matrix A yields the second coordinate information of the nose feature line in the head image, which can be denoted as (X1, Y1) and (X2, Y2). The head yaw angle is then calculated based on the second coordinate information. The formula for calculating the head yaw angle is as follows:

[0091]

[0092] Where θ is the head yaw angle, and (X1,Y1) and (X2,Y2) are the second coordinate information of any two points on the nose feature line.

[0093] As an example, head images consist of multiple frames of head images, and head pose detection methods also include:

[0094] The system detects whether multiple frames of head images meet preset conditions. If they do, the X-ray source is turned off and a prompt message is issued. The prompt message indicates the degree of head movement of the subject. The preset conditions include at least one of the following:

[0095] The difference between the head yaw angles of two adjacent head images is greater than a first preset threshold.

[0096] The difference between the head yaw angle corresponding to the current frame's head image and the head yaw angle corresponding to the initial head image is greater than a second preset threshold. Specifically, the implementation methods are as follows:

[0097] In one implementation, the following conditions must be detected: whether the difference between the head yaw angles corresponding to two adjacent head images is greater than a first preset threshold, and whether the difference between the head yaw angle corresponding to the current head image and the head yaw angle corresponding to the initial head image is greater than a second preset threshold. If either condition is met, the ray source is turned off and a prompt message is issued; or, if both conditions are met, the ray source is turned off and a prompt message is issued.

[0098] In another implementation, detection is performed only on whether the difference between the head yaw angles of two adjacent head images is greater than a first preset threshold. If the condition is met, the ray source is turned off and a prompt message is issued.

[0099] In another implementation, detection is performed only on whether the difference between the head yaw angle corresponding to the current frame head image and the head yaw angle corresponding to the initial head image is greater than a second preset threshold. If the condition is met, the ray source is turned off and a prompt message is issued.

[0100] For example, the head posture detection method described above is performed in real time on head images. The acquired head video stream includes multiple frames of head images, with an interval of 0.2 seconds between each frame. The head yaw angle of each frame is calculated using the method described above. If the difference between the head yaw angles of two adjacent head images is greater than a first preset threshold, which can be 5°, then the difference indicates that the patient's head movement is significant. To ensure patient safety, the recording can be stopped immediately and a warning message can be given to avoid unnecessary radiation exposure to the patient. Simultaneously, it can promptly remind medical staff to take measures to prevent secondary injury during the examination.

[0101] For example, the second preset threshold mainly limits the cumulative movement range of the patient and is a larger threshold than the first preset threshold. If the difference between the head yaw angle corresponding to the current frame head image and the head yaw angle corresponding to the initial head image is greater than the second preset threshold, it indicates that the patient's cumulative movement range is large. To ensure patient safety and image quality, the imaging can be stopped in time and a prompt message can be given. Both the first and second preset thresholds can be determined based on whether the clarity of the CT images captured in the imaging experiment meets the actual requirements.

[0102] Figure 16 This is a flowchart of a head pose detection method based on mobile CT according to an embodiment of this application.

[0103] like Figure 16 As shown, S1: Based on a frame image acquired by a mobile CT camera, perform nose region detection on the image. S2: Perform ellipse detection on the segmented nose region image. S3: Extract the minor axis of the ellipse with the largest area as the feature line of the bridge of the nose. S4: Calculate the angle between the bridge of the nose feature line and the vertical direction in the image through coordinate transformation, thereby calculating the patient's head posture. S5: Compare the head posture angles in the preceding and following images. Compare the calculated head posture angles of two frames in the video stream at a certain time interval. Stop exposure when the difference exceeds a predetermined angle.

[0104] This application provides a solution for head pose detection based on mobile CT. Addressing the problem that traditional head pose estimation methods heavily rely on complete facial key points, this application uses a convolutional neural network to detect the nose region, performs ellipse detection on the nose region, and extracts the nasal bridge feature line. From this, the yaw angle of the patient's head within the mobile CT scan is calculated. By monitoring changes in the patient's head pose in real time, exposure can be stopped in time when the patient moves significantly, avoiding unnecessary radiation absorption and secondary damage during the examination.

[0105] This application also proposes a mobile CT device.

[0106] As an example, such as Figure 3 As shown, the mobile CT device includes: a mobile scanning cavity; a camera, which is placed at the top of the inner cavity of the scanning cavity and located diagonally above the head of the subject, and is used to capture head images; and a head posture detection device, which is used to implement the steps of the head posture detection method described above.

[0107] This application also proposes a computer-readable storage medium.

[0108] In this embodiment, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the head pose detection method described above.

[0109] Figure 17 A block diagram of an electronic device provided in an embodiment of this application.

[0110] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the head posture detection method described above.

[0111] like Figure 17 As shown, for ease of understanding, embodiments of this application illustrate a specific electronic device.

[0112] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0113] like Figure 17 As shown, the device includes a computing unit 1701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1702 or a computer program loaded from a storage unit 17017 into a random access memory (RAM) 1703. The RAM 1703 may also store various programs and data required for the operation of the electronic device. The computing unit 1701, ROM 1702, and RAM 1703 are interconnected via a bus 1704. An input / output (I / O) interface 1705 is also connected to the bus 1704.

[0114] Multiple components in the electronic device are connected to the I / O interface 1705. These components include: an input unit 1706, such as a keyboard or mouse; an output unit 1707, such as various types of displays or speakers; a storage unit 1708, such as a hard disk or optical disk; and a communication unit 1709, such as a network interface card (NIC), a modem, or a wireless transceiver. The communication unit 1709 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0115] The computing unit 1701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1701 performs the various methods described above, such as the head pose detection method. For example, in some embodiments, the head pose detection method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1708. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1702 and / or communication unit 1709. When the computer program is loaded into RAM 1703 and executed by the computing unit 1701, the head pose detection method described above can be executed. Alternatively, in other embodiments, the computing unit 1701 may be configured to perform the head pose detection method by any other suitable means (e.g., by means of firmware).

[0116] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this application, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0117] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0118] In the description of this application, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0119] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0120] Furthermore, the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this application can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this application, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly and specifically defined in the embodiments.

[0121] In this application, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication between two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific implementation.

[0122] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0123] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A head pose detection method, characterized in that, The method is applied to a mobile CT device, the mobile CT device including a camera, and the method includes: A head image is acquired based on the camera, and a nose region is detected in the head image to obtain a nose region image. Ellipse recognition is performed on the nose region image to obtain multiple elliptical regions; Nasal feature information is determined based on the multiple elliptical regions; Based on the nose feature information, the degree of head offset is determined, and the head posture is detected based on the degree of head offset.

2. The head posture detection method according to claim 1, characterized in that, The ellipse recognition process on the nose region image yields multiple elliptical regions, including: The nose region image is binarized, and edge detection is performed on the binarized nose region image to obtain edge contour information; Ellipse recognition is performed on the edge contour information to obtain multiple elliptical regions.

3. The head posture detection method according to claim 1, characterized in that, The nose feature information includes nose feature lines, and the determination of nose feature information based on the plurality of elliptical regions includes: The elliptical region with the largest area among the plurality of elliptical regions is identified as the target elliptical region. The minor axis of the target elliptical region is determined as the nose feature line.

4. The head posture detection method according to claim 3, characterized in that, The nose feature information includes the first coordinate information of the nose feature line in a first preset coordinate system, the head offset degree includes the head yaw angle, and the determination of the head offset degree based on the nose feature information includes: The first coordinate information is transformed to obtain the second coordinate information of the nose feature line in the second preset coordinate system for the head image; The head yaw angle is calculated based on the second coordinate information.

5. The head posture detection method according to claim 4, characterized in that, The step of performing coordinate transformation on the first coordinate information to obtain the second coordinate information of the nose feature line in the second preset coordinate system for the head image includes: Determine the bounding box coordinates of the nose region in the first preset coordinate system and the bounding box coordinates of the nose region in the second preset coordinate system, wherein the first preset coordinate system is a coordinate system established based on the nose region image and the second preset coordinate system is a coordinate system established based on the head image; Based on the bounding box coordinates of the nose region in the first preset coordinate system and the bounding box coordinates of the nose region in the second preset coordinate system, a coordinate transformation matrix is ​​determined. Based on the coordinate transformation matrix, the first coordinate information is transformed to obtain the second coordinate information of the nose feature line in the second preset coordinate system for the head image.

6. The head posture detection method according to claim 4, characterized in that, The head image includes multiple frames of head images, and the head pose detection based on the head offset degree includes: The system detects whether the multi-frame head images meet preset conditions. If they do, the X-ray source is turned off and a prompt message is issued. The prompt message indicates the degree of head movement of the subject. The preset conditions include at least one of the following: The difference between the head yaw angles of two adjacent head images is greater than a first preset threshold. The difference between the head yaw angle corresponding to the current frame head image and the head yaw angle corresponding to the initial head image is greater than the second preset threshold.

7. The head posture detection method according to claim 1, characterized in that, The step of detecting the nose region in the head image to obtain a nose region image includes: Based on the trained target detection model, the nose region is detected in the head image to obtain the nose region detection result. A nose region image is obtained based on the nose region detection results; The trained target detection model is obtained through the following method: Construct a training dataset, which includes multiple head images and the corresponding nose region annotation information for each head image; The preset target detection model is trained based on the training dataset to obtain the trained model.

8. The head posture detection method according to claim 7, characterized in that, The nose region detection result includes multiple third bounding boxes of the nose region and the confidence score corresponding to each third bounding box. Obtaining the nose region image based on the nose region detection result includes: The third bounding box corresponding to the maximum value in the confidence score is determined as the target bounding box, and third bounding boxes that are duplicates of the target bounding box are filtered out. The nose region image is obtained based on the target bounding box, and the target bounding box is displayed.

9. A mobile CT scanner, characterized in that, The mobile CT device includes: Moving the scanning cavity; A camera is placed at the top of the scanning cavity and diagonally above the head of the subject, and the camera is used to capture images of the head. A head posture detection device, wherein the head posture detection device is used to implement the steps of the method described in any one of claims 1-8.

10. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in any one of claims 1-8.

Images