Head posture detection method, mobile CT equipment, electronic equipment and storage medium

Abstract

The invention discloses a head posture detection method, mobile CT equipment, electronic equipment and a storage medium, the head posture detection method is applied to the mobile CT equipment, a cavity of the mobile CT equipment comprises a camera, and the method comprises the following steps: acquiring a head image based on the camera, and carrying out nose area detection on the head image to obtain a nose area image; performing ellipse identification on the nose area image to obtain a plurality of ellipse areas; determining nose feature information based on the plurality of elliptical regions; and determining a head offset degree based on the nose feature information, and detecting a head posture based on the head offset degree. According to the head posture detection method, ellipse detection is carried out on the nose area, the nose feature information is extracted, head posture detection is achieved based on the nose feature information, the problem that a traditional head posture estimation method seriously depends on complete face key points is solved, and the method is simpler and more efficient.

Description

Technical Field

[0001] The present application relates to technical fields such as medical equipment and computer vision, and in particular to a head posture detection method, a mobile CT device, an electronic device and a storage medium. Background Art

[0002] The research on head posture estimation is mainly used in virtual reality, human-computer interaction and intelligent monitoring. For traditional fixed CT equipment, the posture detection of the subject, such as the patient, is the posture detection of the human torso and limbs, which is mainly achieved by detecting the key points of the human skeleton. However, mobile CT is mostly used for patients with severe illness or unclear consciousness. During the shooting process, the patient's unconscious head shaking will cause artifacts in the CT image. For smaller changes in head posture, correction can be made through post-algorithm reconstruction and other methods, but for larger changes, it will cause serious artifacts to the original image received by the detector, which will interfere with subsequent diagnosis. Summary of the invention

[0003] To this end, the purpose of the implementation method of the present application is to propose a head posture detection method, a mobile CT device, an electronic device and a storage medium, to perform ellipse detection on the nose area and extract nose feature information, to realize head posture detection based on the nose feature information, thereby solving the problem that the traditional head posture estimation method is heavily dependent on complete facial key points, and the method is simpler and more efficient.

[0004] An embodiment of the present application provides a head posture detection method, which is applied to a mobile CT device, wherein the mobile CT device includes a camera in its cavity, and the method includes: acquiring a head image based on the camera, and performing nose area detection on the head image to obtain a nose area image; performing ellipse recognition on the nose area image to obtain multiple ellipse areas; determining nose feature information based on the multiple ellipse areas; determining a head deviation degree based on the nose feature information, and detecting the head posture based on the head deviation degree.

[0005] Exemplarily, performing ellipse recognition on the nose area image to obtain multiple ellipse areas includes: binarizing the nose area image and performing edge detection on the binarized nose area image to obtain edge contour information; performing ellipse recognition on the edge contour information to obtain multiple ellipse areas.

[0006] Exemplarily, the nose feature information includes a nose feature line, and determining the nose feature information based on the multiple elliptical areas includes: determining the elliptical area with the largest area among the multiple elliptical areas as the target elliptical area; and determining the short axis of the target elliptical area as the nose feature line.

[0007] Exemplarily, the nose feature information includes first coordinate information of a nose feature line in a first preset coordinate system, the head deviation degree includes a head yaw angle, and determining the head deviation degree based on the nose feature information includes: performing coordinate transformation on the first coordinate information to obtain second coordinate information of the nose feature line in a second preset coordinate system for the head image; and calculating the head yaw angle based on the second coordinate information.

[0008] Exemplarily, the coordinate transformation of the first coordinate information to obtain second coordinate information of the nose feature line in a second preset coordinate system for the head image includes: determining nose area bounding box coordinate information in the first preset coordinate system and nose area bounding box coordinate information in the second preset coordinate system, wherein the first preset coordinate system is a coordinate system established based on the nose area image, and the second preset coordinate system is a coordinate system established based on the head image; determining a coordinate transformation matrix based on the nose area bounding box coordinate information in the first preset coordinate system and the nose area bounding box coordinate information in the second preset coordinate system; and coordinate transformation of the first coordinate information based on the coordinate transformation matrix to obtain second coordinate information of the nose feature line in the second preset coordinate system for the head image.

[0009] Exemplarily, the head image includes multiple frames of head images, and the detecting of head posture based on the degree of head deviation includes: detecting whether the multiple frames of head images meet preset conditions, and if so, turning off the radiation source and issuing a prompt message, wherein the prompt message represents the degree of movement of the head of the person being photographed, and the preset condition includes at least one of the following: the difference between the head yaw angles corresponding to two adjacent frames of head images is greater than a first preset threshold; the difference between the head yaw angle corresponding to the current frame head image and the head yaw angle corresponding to the initial head image is greater than a second preset threshold.

[0010] Exemplarily, performing nose area detection on the head image to obtain a nose area image includes: performing nose area detection on the head image based on a trained target detection model to obtain a nose area detection result; and obtaining a nose area image based on the nose area detection result; wherein the trained target detection model is trained in the following manner: constructing a training data set, the training data set including multiple head images and nose area annotation information corresponding to each head image; and training a preset target detection model based on the training data set to obtain a trained model.

[0011] Exemplarily, the nose area detection result includes multiple third bounding boxes of the nose area and confidence levels corresponding to each third bounding box, and obtaining the nose area image based on the nose area detection result includes: determining the third bounding box corresponding to the maximum value in the confidence level as a target bounding box, and filtering third bounding boxes that are repeated with the target bounding box; obtaining the nose area image based on the target bounding box, and displaying the target bounding box.

[0012] Another embodiment of the present application provides a mobile CT device, which includes: a mobile scanning cavity; a camera, which is placed at the top of the inner cavity of the scanning cavity and is located diagonally above the head of the person being photographed, and the camera is used to capture a head image; and a head posture detection device, which is used to implement the steps of the above method.

[0013] Another embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of any of the above embodiments when executing the computer program.

[0014] Another embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method of any of the above embodiments are implemented.

[0015] Another embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device is enabled to perform the steps of the method of any of the above embodiments.

[0016] In the above implementation, the head posture detection method is applied to a mobile CT device, which includes a camera in the cavity, including: acquiring a head image based on the camera, and performing nose area detection on the head image to obtain a nose area image; performing ellipse recognition on the nose area image to obtain multiple ellipse areas; determining nose feature information based on multiple ellipse areas; determining the degree of head deviation based on the nose feature information, and detecting the head posture based on the degree of head deviation. The head posture detection method of the present invention performs ellipse detection on the nose area and extracts nose feature information, and realizes head posture detection based on the nose feature information, which solves the problem that the traditional head posture estimation method is heavily dependent on complete facial key points, and the method is simpler and more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a head posture detection method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of a mobile CT device provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of the positional relationship between the mobile CT camera and the patient provided in the embodiment of the present application;

[0020] Figure 4 A schematic diagram of a head image provided for an embodiment of the present application;

[0021] Figure 5 A flowchart of nose area detection on a head image provided by an embodiment of the present application;

[0022] Figure 6 A flowchart for obtaining a nose area image provided in an embodiment of the present application;

[0023] Figure 7 A schematic diagram of a nose area detection result provided in an embodiment of the present application;

[0024] Figure 8 A flow chart of performing ellipse recognition on a nose area image provided in an embodiment of the present application;

[0025] Fig. 9 A flowchart for determining nose feature information provided in an embodiment of the present application;

[0026] Fig.10 A schematic diagram of an elliptical area provided for an embodiment of the present application;

[0027] Fig.11 A flow chart for determining the degree of head deviation provided for an embodiment of the present application;

[0028] Fig.12 Schematic diagram of three Euler angles provided for implementation of the present application;

[0029] Fig.13 A schematic diagram of a nose region image coordinate system provided in an embodiment of the present application;

[0030] Fig.14 A flowchart of coordinate conversion provided for an embodiment of the present application;

[0031] Fig.15 A schematic diagram of a head image coordinate system provided in an embodiment of the present application;

[0032] Fig.16 A flowchart of a head posture detection method based on mobile CT provided in an embodiment of the present application.

[0033] Fig.17 A block diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0034] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0035] The research on head posture estimation is mainly used in virtual reality, human-computer interaction and intelligent monitoring. For traditional fixed CT equipment, the posture detection of the subject, such as the patient, is the posture detection of the human torso and limbs, which is mainly achieved by detecting the key points of the human skeleton. However, mobile CT is mostly used for patients with severe illness or unclear consciousness. During the shooting process, the patient's unconscious head shaking will cause artifacts in the CT image. For smaller changes in head posture, correction can be made through post-algorithm reconstruction and other methods, but for larger changes, it will cause serious artifacts to the original image received by the detector, which will interfere with subsequent diagnosis.

[0036] Based on this, the present application proposes a head posture detection method, which is based on a mobile CT device. A camera is installed on the inner cavity of the CT device to obtain the patient's head image, and ellipse recognition is performed on the patient's head image to obtain nose feature information. Based on the nose feature information, the degree of head deviation is determined to achieve continuous monitoring of the patient's head posture changes.

[0037] Figure 1 This is a flowchart of a head posture detection method according to an embodiment of the present application.

[0038] As an implementation, Figure 1 As shown, the head posture detection method includes:

[0039] S101, obtaining a head image based on a camera, and performing nose region detection on the head image to obtain a nose region image.

[0040] S102, performing ellipse recognition on the nose region image to obtain a plurality of ellipse regions.

[0041] S103, determining nose feature information based on the multiple elliptical regions.

[0042] S104, determining the degree of head deviation based on the nose feature information, and detecting the head posture based on the degree of head deviation.

[0043] Exemplarily, the head posture detection method is applied to a mobile CT device, such as Figure 2The schematic diagram of the mobile CT device shown in the figure is a mobile computed tomography device. Different from the traditional fixed CT scanner, the mobile CT can provide fast and convenient bedside CT imaging services in different clinical environments, especially in various scenarios inside and outside the hospital. It has high mobility and convenience, and can also be equipped on special vehicles, for example, in ambulances. Figure 2 As shown, when the mobile CT device is in use, devices such as a head rest 01, a shoulder rest 02 and an examination bed 03 are used to restrict the patient's body position. However, the mobile CT is mostly used for patients with serious illnesses or unclear consciousness. During the shooting process, the patient's unconscious head shaking will cause artifacts in the CT image.

[0044] For example, the present application can install a camera in the inner cavity of a mobile CT device, such as Figure 3 As shown, due to the volume and aperture size of the mobile head CT itself, the camera 301 can be placed inside the cavity and take pictures at a position slightly higher than the front of the human body. The head carrying device 302 is used to carry the patient's head. The bottom of the mobile CT device 303 is equipped with a pulley, which can be moved or fixed according to the specific application scenario. The camera 301 can be a monocular camera or a depth camera.

[0045] Exemplarily, the mobile CT device obtains the head image based on the camera. The head image can be obtained by shooting with the camera, or the head image can be obtained from a certain frame in the video stream recorded by the camera, for example. Figure 4 The scanning process of mobile CT is a dynamic process. Due to the movement and camera angle, it is difficult for the camera to capture complete facial features, such as Figure 4 The picture shown only contains a small part of the facial features of the patient's head. Thus, the traditional head posture detection that relies on complete facial key points is not applicable to this scenario. The present application performs nose area detection on the head image to obtain a nose area image, and performs head posture detection on the patient based on the nose local area image, thus overcoming the limitation of the traditional head posture detection that relies on complete facial key points. The head image is, for example, taken from a position diagonally above or diagonally below the patient's head. For shooting angles such as diagonally above, Figure 3 The camera 301 is at a shooting angle relative to the patient. The patient's nose area presented at this shooting angle is as follows: Figure 4 Of course, the patient's face can also be photographed from a position diagonally below the patient's head.

[0046] For example, after obtaining the nose region image, ellipse recognition is performed on the nose region image to obtain multiple ellipse regions. The present application introduces a corresponding contour recognition algorithm in combination with the nose object features, and determines the characteristic information of the nose through multiple ellipse regions. The nose characteristic information may be, for example, a characteristic line at the bridge of the nose. The degree of head deviation is determined based on the nose characteristic information, thereby realizing head posture detection.

[0047] The head posture detection method of the present application performs ellipse recognition on the patient's head image to obtain nose feature information, determines the degree of head deviation based on the nose feature information, and realizes continuous monitoring of the patient's head posture changes, thereby overcoming the limitations of traditional head posture detection that relies on complete facial key points.

[0048] As an example, Figure 5 As shown, the nose area is detected on the head image to obtain a nose area image, including:

[0049] S501, performing nose region detection on the head image based on the trained target detection model to obtain a nose region detection result.

[0050] S502: Obtain a nose region image based on the nose region detection result.

[0051] Among them, the trained target detection model is trained in the following way:

[0052] Construct a training data set, which includes multiple head images and nose area annotation information corresponding to each head image;

[0053] The preset target detection model is trained based on the training data set to obtain a trained model.

[0054] Exemplarily, nose region detection is performed on a head image, and nose region detection can be performed on the head image using a trained target detection model. The target detection model can use a convolutional neural network, and a trained target detection model is obtained by training the convolutional neural network. Specific training process: First, a training data set is constructed. For example, mobile CT camera photos containing nose images are collected, and the nose region corresponding to each head image is annotated, which can cover images at different distances, illuminations, and backgrounds. The annotation information can mark the nose region with a rectangular frame. Then, a suitable preset target detection model is selected. The preset target detection model can be a YOLO model. The preset target detection model is trained based on the training data set to obtain a trained model. The images in the training data set are input into the preset target detection model. The preset target detection model includes preprocessing the image, such as resizing, normalization, etc. The loss function of the preset target detection model can use a cross entropy loss and an IOU (Intersection over Union) function. The model is continuously iterated and trained to finally obtain a trained target detection model.

[0055] Exemplarily, the acquired head image is input into a trained target detection model, the target detection model outputs a nose area detection result, and the desired nose area image is obtained according to the nose area detection result.

[0056] As an example, Figure 6 As shown, the nose region detection result includes multiple third bounding boxes of the nose region and the confidence level corresponding to each third bounding box. The nose region image is obtained based on the nose region detection result, including:

[0057] S601: Determine a third bounding box corresponding to the maximum value in the confidence as a target bounding box, and filter third bounding boxes that overlap with the target bounding box.

[0058] S602, obtaining a nose region image based on the target bounding box, and displaying the target bounding box.

[0059] Exemplarily, the target detection model outputs a nose region detection result, which includes a plurality of third bounding boxes of the nose region and a confidence level corresponding to each third bounding box. It can be understood that the third bounding box indicates the position of the nose in the head image, and the confidence level indicates the credibility of the detection result. The third bounding box corresponding to the maximum value in the confidence level can be selected as the target bounding box, and the target bounding box is displayed in the head image, for example, as shown in FIG. Figure 7The nose region detection result is shown. The third bounding box that is repeated in the target bounding box is filtered, for example, the repeated third bounding box is filtered using a non-maximum suppression algorithm. The nose region image is obtained according to the target bounding box. It can be understood that the image in the target bounding box is the nose region image, and the nose region image can be obtained by operations such as cutting.

[0060] After the nose region image is obtained, ellipse recognition is performed on the nose region image to obtain a plurality of ellipse regions.

[0061] As an example, Figure 8 As shown, ellipse recognition is performed on the nose area image to obtain multiple ellipse areas, including:

[0062] S801, binarizing the nose region image, and performing edge detection on the binarized nose region image to obtain edge contour information.

[0063] S802, performing ellipse recognition on edge contour information to obtain multiple ellipse areas.

[0064] Exemplarily, to perform ellipse recognition on the nose region image, the nose region image may be firstly binarized, for example, the OTSU algorithm may be used to binarize the nose region image, and the binarization process may represent each pixel of the nose region image with 0 or 1. The binarized nose region image is subjected to edge detection, for example, the Canny edge detection algorithm is used to process the image, and edge contour information is extracted, wherein the extracted edge contour is the edge contour of the nose. Ellipse recognition is performed on the edge contour information to obtain multiple ellipse regions, and ellipse recognition is performed on the edge contour information, for example, the Hough transform is performed on the edge contour to identify multiple ellipse regions contained in the edge contour.

[0065] Then, the nose feature information is determined based on the multiple ellipse areas obtained by ellipse recognition.

[0066] As an example, Fig. 9 As shown, the nose feature information includes a nose feature line, and the nose feature information is determined based on a plurality of elliptical regions, including:

[0067] S901: Determine the elliptical region with the largest area among multiple elliptical regions as the target elliptical region.

[0068] S902, determining the short axis of the target elliptical area as the nose feature line.

[0069] Exemplarily, the nose feature information includes a nose feature line, which may be a feature line of the bridge of the nose. Based on the multiple elliptical regions obtained by ellipse recognition, the area of ​​each elliptical region is calculated, the elliptical region with the largest area is determined as the target elliptical region, and the short axis of the target elliptical region is determined as the nose feature line. For example, Fig.10 The ellipse region shown includes a major axis and a minor axis, and of course the major axis can be taken as the nose feature line. The length of the feature line can be a preset length.

[0070] As an example, the nose feature information includes a nose feature line, and the nose feature information is determined based on multiple elliptical areas, including: when multiple elliptical areas corresponding to the initial head image are obtained, a selection instruction input by the user is received, and the selected elliptical area is used as the initial target elliptical area; when multiple elliptical areas corresponding to any subsequent frame head image are obtained, the elliptical area whose area and / or shape is closest to the initial target elliptical area is selected as the target elliptical area of ​​the corresponding head image. The shape can be the short axis and / or long axis of the ellipse, and the area, short axis, and long axis can be characterized by a weighted summation method. Then, the elliptical features of the initial head image and the elliptical features of the current frame head image are compared, and the elliptical features of the current frame head image that are closest to the elliptical features of the initial head image are determined as the target elliptical area. In the above manner, an ellipse that is more consistent with the actual area of ​​the subject's nose can be selected to describe the nose feature, thereby improving the accuracy of detection.

[0071] After obtaining the nose feature information, the head deviation degree is determined based on the nose feature information, and the head posture is detected based on the head deviation degree.

[0072] As an example, the nose feature information includes the first coordinate information of the nose feature line in the first preset coordinate system, and the head deviation degree includes the head yaw angle, such as Fig.11 As shown, based on the nose feature information, the degree of head deviation is determined, including:

[0073] S1101, performing coordinate transformation on the first coordinate information to obtain second coordinate information of the nose feature line in a second preset coordinate system for the head image.

[0074] S1102: Calculate the head yaw angle based on the second coordinate information.

[0075] For example, the orientation of a human head in three-dimensional space can be described by the degrees of freedom in three directions: the pitch angle of vertical rotation, the yaw angle of horizontal rotation, and the rotation angle of left and right rotation. Fig.12As shown in the figure, in three-dimensional space, the rotation of the human head posture can be represented by three Euler angles. The rotation around the X-axis is called pitch, the rotation around the Y-axis is called yaw, and the rotation around the Z-axis is called roll.

[0076] For example, the present application may express the head posture estimation of the mobile CT patient by calculating the yaw angle of the head, for the following reasons: Figure 3 As shown, during the mobile CT examination, when the patient's head is supported by the headrest, a coordinate system of the human head posture can be established on the rigid and completely fixed headrest. The central symmetry line of the headrest is defined as the Y axis, the straight line along the direction from the human ear to the left shoulder and perpendicular to the Y axis is defined as the X axis, and the straight line perpendicular to the plane where the X axis and the Y axis are located is called the Z axis. Because the patient is generally in a lying state during the CT examination, the head is usually supported by the examination bed and the headrest, so that the head cannot continue to make forward or backward pitching movements, which limits the pitch direction, that is, the degree of freedom around the X axis. And because the headrest is generally an arc-shaped rigid body, there are generally headrest restrictions on both sides of the patient's head, which will also limit the flip direction, that is, the degree of freedom around the Z axis. Therefore, the head posture detection for mobile CT examination can be simplified to the calculation of the yaw angle of the patient's head. Of course, the pitch angle and the flip angle can also be calculated, and the head posture detection is realized according to the angles of multiple dimensions. Preferably, according to the physical characteristics of the mobile CT device, the method of head posture detection by calculating the yaw angle of the patient's head in this application is simpler and more efficient.

[0077] Exemplarily, the mobile CT camera can be set at the top of the inner cavity of the CT machine and located diagonally above the head of the person being photographed. The camera is located on one side of the CT scanning area and photographs the patient's face from the top of the patient's head. The camera can be set with reference to the vertical plane position of the CT scan rotation center axis, and can be set within a preset range that does not affect the recognition accuracy of the nose area.

[0078] The obtained two-dimensional image plane is perpendicular to the Y axis of the head posture coordinate. The projection of the patient's nose feature line on the two-dimensional image is perpendicular to the Y axis of the head posture coordinate, and can therefore be used to characterize the yaw angle.

[0079] Exemplarily, the head deviation degree includes the head yaw angle, and the nose feature information includes the first coordinate information of the nose feature line in the first preset coordinate system, and the first preset coordinate system can be understood as a local coordinate system for the nose area image. Fig.13As shown, for the nose area image, a point in the third boundary box can be used as the origin to establish a two-dimensional coordinate system to obtain a first preset coordinate system, thereby obtaining the first coordinate information of the nose feature line. It should be noted that the first coordinate information of the nose feature line is not limited to the coordinate information of the two endpoints of the nose feature line, and can be the coordinate information of any point on the nose feature line.

[0080] Exemplarily, the first coordinate information is subjected to coordinate transformation to obtain second coordinate information of the nose feature line for the head image. It can be understood that the coordinate transformation is to transform the coordinate information of the nose feature line from a first preset coordinate system, such as a local coordinate system for the nose region image, to a second preset coordinate system, such as a global coordinate system for the head image. The coordinate information of the nose feature line in the global coordinate system for the head image is the second coordinate information of the nose feature line. The head yaw angle is calculated based on the second coordinate information.

[0081] As an example, Fig.14 As shown, the first coordinate information is transformed to obtain the second coordinate information of the nose feature line in the second preset coordinate system for the head image, including:

[0082] S1401, determining coordinate information of a nose region boundary box in a first preset coordinate system and coordinate information of a nose region boundary box in a second preset coordinate system under the first preset coordinate system, wherein the first preset coordinate system is established based on a nose region image, and the second preset coordinate system is established based on a head image.

[0083] S1402: Determine a coordinate conversion matrix based on the coordinate information of the nose region bounding box in the first preset coordinate system and the coordinate information of the nose region bounding box in the second preset coordinate system.

[0084] S1403: Perform coordinate transformation on the first coordinate information based on the coordinate transformation matrix to obtain second coordinate information of the nose feature line in a second preset coordinate system for the head image.

[0085] For example, in order to realize the coordinate transformation of the first coordinate information of the nose feature line, it is necessary to obtain the coordinate transformation relationship between the local image coordinate system of the nose area and the whole head image coordinate system, for example, a coordinate transformation matrix. A first preset coordinate system can be established on the nose area image, and the first preset coordinate system can be as follows: Fig.13 The coordinate system shown in FIG. 1 takes a certain point in the nose region boundary box as the origin, and the two boundaries bordering the origin as the X and Y axes. A second preset coordinate system is established on the head image. The second preset coordinate system can use the upper left corner of the head image as the coordinate origin to establish a two-dimensional coordinate system, such as Fig.15As shown, the X axis extends from left to right, and the Y axis extends from top to bottom. The position of the nose area bounding box in the overall head image is known. According to the principle of establishing a coordinate system, the coordinate information of the nose area bounding box in the first preset coordinate system and the coordinate information of the nose area bounding box in the second preset coordinate system can be determined. It can be understood that the coordinate information of the nose area bounding box in the first preset coordinate system is the local bounding box coordinate information for the nose area image, and the coordinate information of the nose area bounding box in the second preset coordinate system is the overall bounding box coordinate information for the head image. According to the coordinate information of the nose area bounding box in the first preset coordinate system and the coordinate information of the nose area bounding box in the second preset coordinate system, the coordinate conversion matrix between the two coordinate systems is determined.

[0086] Exemplarily, the bounding box coordinate information may include the coordinate information of the four vertices in the rectangular bounding box. For example, the coordinates of a point in the bounding box under the second preset coordinate system, for example, the coordinates of the upper left corner of the rectangular bounding box are (X0, y0), the upper right corner is (X0+w, y0), the lower left corner is (X0, y0+h), and the lower right corner is (X0+w, y0+h). Among them, w is the length of the rectangular area on the X axis, and h is the length of the rectangular area on the Y axis. It should be noted that, if Fig.15 In the second preset coordinate system shown, the rectangular area is skewed relative to the coordinate system. In this case, w is the projection length relative to the X axis, and h is the projection length relative to the Y axis. Of course, the coordinate system can also be established in other ways. For example, in order to facilitate calculation, the X axis and Y axis of the coordinate system can also be established in a direction parallel to the rectangular bounding box. The coordinate system is established to obtain the transformation relationship from the local nose area image to the original image. The form of establishing the coordinate system is not limited.

[0087] Exemplarily, according to the coordinate information of the four vertices in the bounding box, a coordinate transformation matrix is ​​obtained, and the coordinate transformation matrix represents the transformation relationship from the first preset coordinate system to the second preset coordinate system. The coordinate transformation matrix is ​​obtained by the following formula:

[0088]

[0089] Where A is the coordinate conversion matrix, (x', y') represents the coordinate information of the bounding box in the first preset coordinate system, (X, Y) represents the coordinate information of the bounding box in the second preset coordinate system, and (x0, y0) is the coordinate of the upper left corner of the bounding box in the second preset coordinate system. Therefore, the coordinate conversion matrix A can be solved by the coordinates of the four known points of the rectangle.

[0090] Based on the coordinate transformation matrix A, the first coordinate information is transformed to obtain the second coordinate information of the nose feature line for the head image (that is, the coordinate information of the nose feature line in the second coordinate system). For example, the first coordinate information is the coordinates of any two points on the nose feature. The coordinates of these two points are transformed according to the coordinate transformation matrix A to obtain the second coordinate information of the nose feature line for the head image. The second coordinate information can be recorded as (X1, Y1) and (X2, Y2). The head yaw angle is calculated based on the second coordinate information. The calculation formula of the head yaw angle is shown as follows:

[0091]

[0092] Among them, θ is the head yaw angle, (X1, Y1) and (X2, Y2) are the second coordinate information of any two points on the nose feature line.

[0093] As an example, the head image includes multiple frames of head images, and the head posture detection method further includes:

[0094] Detect whether the multiple frames of head images meet the preset conditions. If they meet the conditions, turn off the radiation source and issue a prompt message, wherein the prompt message represents the degree of movement of the head of the person being photographed. The preset conditions include at least one of the following:

[0095] The difference between the head yaw angles corresponding to two adjacent frames of head images is greater than a first preset threshold;

[0096] The difference between the head yaw angle corresponding to the current frame head image and the head yaw angle corresponding to the initial head image is greater than a second preset threshold. Specific implementations include the following:

[0097] In one embodiment, both conditions need to be detected, namely, whether the difference between the head yaw angles corresponding to two adjacent frames of head images is greater than a first preset threshold, and whether the difference between the head yaw angles corresponding to the current frame head image and the head yaw angles corresponding to the initial head image is greater than a second preset threshold. If either of the conditions is met, the radiation source is turned off and a prompt message is issued, or if both conditions are met, the radiation source is turned off and a prompt message is issued.

[0098] In another implementation, only the condition that whether the difference between the head yaw angles corresponding to two adjacent frames of head images is greater than a first preset threshold is detected. If so, the ray source is turned off and a prompt message is issued.

[0099] In another embodiment, only the condition that the difference between the head yaw angle corresponding to the current frame head image and the head yaw angle corresponding to the initial head image is greater than a second preset threshold is detected. If so, the radiation source is turned off and a prompt message is issued.

[0100] Exemplarily, the head posture detection method is performed in real time on the head image. The collected head video stream includes multiple frames of head images. The interval time in the video stream can be 0.2s. The head yaw angle of each frame is calculated according to the above method. The difference between the head yaw angles corresponding to two adjacent frames of head images is greater than the first preset threshold value, which can be 5°. If the difference between the head yaw angles corresponding to two adjacent frames of head images is greater than the first preset threshold value, it means that the patient's head movement amplitude is large. In order to ensure the safety of the patient, the shooting can be stopped in time and a prompt message can be given to prevent the patient from being exposed to unnecessary radiation. At the same time, it can promptly remind medical staff to take measures to avoid secondary injuries during the examination.

[0101] Exemplarily, the second preset threshold mainly limits the patient's cumulative movement amplitude, which is a threshold greater than the first preset threshold. If the difference between the head yaw angle corresponding to the current frame head image and the head yaw angle corresponding to the initial head image is greater than the second preset threshold, it means that the patient's cumulative movement amplitude is large. In order to ensure the safety of the patient and the image quality, the shooting can also be stopped in time and a prompt message can be given. Both the first preset threshold and the second preset threshold can be determined according to whether the clarity of the CT image captured in the shooting experiment meets the actual needs.

[0102] Fig.16 This is a flowchart of a head posture detection method based on mobile CT according to an embodiment of the present application.

[0103] like Fig.16 As shown, S1: Based on a frame image acquired by a mobile CT camera, the image is subjected to nose area detection. S2: Ellipse detection is performed on the segmented nose area image. S3: The short axis of the ellipse with the largest area is extracted as the characteristic line of the bridge of the nose. S4: The angle between the characteristic line of the bridge of the nose and the vertical direction in the image is calculated by coordinate transformation, thereby calculating the patient's head posture. S5: Compare the head posture angles in the previous and next images, and compare the head posture angle calculation results of two frames of images separated by a certain time in the video stream, and stop the exposure when the difference exceeds the predetermined angle.

[0104] The present application provides a solution for head posture detection based on mobile CT. In view of the problem that traditional head posture estimation methods heavily rely on complete facial key points, a convolutional neural network is used to detect the nose area, and ellipse detection is performed on the nose area to extract the characteristic line of the nose bridge, thereby calculating the yaw angle of the patient's head in the mobile CT. By real-time monitoring of changes in the patient's head posture, exposure can be stopped in time when the patient has a large abnormal movement, thereby avoiding the patient from absorbing unnecessary radiation and secondary damage during the examination process.

[0105] The application also proposes a mobile CT device.

[0106] As an example, Figure 3 As shown, the mobile CT device includes: a mobile scanning cavity; a camera, which is placed at the top of the inner cavity of the scanning cavity and is located diagonally above the head of the person being photographed, and the camera is used to capture the head image; a head posture detection device, which is used to implement the steps of the above-mentioned head posture detection method.

[0107] The application also proposes a computer-readable storage medium.

[0108] In this embodiment, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned head posture detection method are implemented.

[0109] Fig.17 A block diagram of an electronic device provided for an embodiment of the present application.

[0110] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned head posture detection method when executing the computer program.

[0111] like Fig.17 As shown, for ease of understanding, the embodiment of the present application shows a specific electronic device.

[0112] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0113] like Fig.17 As shown, the device includes a computing unit 1701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1702 or a computer program loaded from a storage unit 17017 into a random access memory (RAM) 1703. In RAM 1703, various programs and data required for the operation of the electronic device can also be stored. The computing unit 1701, ROM 1702, and RAM 1703 are connected to each other via a bus 1704. An input / output (I / O) interface 1705 is also connected to the bus 1704.

[0114] Multiple components in the electronic device are connected to the I / O interface 1705, including: an input unit 1706, such as a keyboard, a mouse, etc.; an output unit 1707, such as various types of displays, speakers, etc.; a storage unit 1708, such as a disk, an optical disk, etc.; and a communication unit 1709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1709 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0115] The computing unit 1701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1701 performs the various methods described above, such as the head posture detection method. For example, in some embodiments, the head posture detection method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1708. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1702 and / or a communication unit 1709. When the computer program is loaded into RAM 1703 and executed by the computing unit 1701, the head posture detection method described above may be executed. Alternatively, in other embodiments, the computing unit 1701 may be configured to perform the head posture detection method in any other appropriate manner (e.g., by means of firmware).

[0116] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this application, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0117] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0118] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0119] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0120] In addition, the terms "first", "second", etc. used in the embodiments of the present application are only used for descriptive purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in the present embodiment. Therefore, the features defined by the terms "first", "second", etc. in the embodiments of the present application can explicitly or implicitly indicate that at least one of the features is included in the embodiment. In the description of the present application, the word "multiple" means at least two or two or more, such as two, three, four, etc., unless otherwise clearly and specifically defined in the embodiments.

[0121] In this application, unless otherwise clearly specified or limited in the embodiments, the terms "installed", "connected", "connected" and "fixed" etc. appearing in the embodiments should be understood in a broad sense. For example, the connection can be a fixed connection, a detachable connection, or an integrated connection. It can be understood that it can also be a mechanical connection, an electrical connection, etc.; of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements, or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific implementation situation.

[0122] In the present application, unless otherwise clearly specified and limited, a first feature being “above” or “below” a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being “above”, “above”, and “above” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being “below”, “below”, and “below” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0123] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A head posture detection method, characterized in that: The method is applied to a mobile CT device, the mobile CT device includes a camera, and the method includes: Acquire a head image based on the camera, and perform nose region detection on the head image to obtain a nose region image; Performing ellipse recognition on the nose region image to obtain a plurality of ellipse regions; Determining nose feature information based on the multiple elliptical areas; Based on the nose feature information, a head deviation degree is determined, and a head posture is detected based on the head deviation degree.

2. The head posture detection method according to claim 1, characterized in that: The step of performing ellipse recognition on the nose region image to obtain a plurality of ellipse regions includes: Binarizing the nose region image, and performing edge detection on the binarized nose region image to obtain edge contour information; Ellipse recognition is performed on the edge contour information to obtain a plurality of ellipse areas.

3. The head posture detection method according to claim 1, characterized in that: The nose feature information includes a nose feature line, and determining the nose feature information based on the multiple elliptical areas includes: Determine the elliptical region with the largest area among the multiple elliptical regions as the target elliptical region; The short axis of the target elliptical area is determined as the nose feature line.

4. The head posture detection method according to claim 3, characterized in that: The nose feature information includes first coordinate information of a nose feature line in a first preset coordinate system, the head deviation degree includes a head yaw angle, and determining the head deviation degree based on the nose feature information includes: Performing coordinate transformation on the first coordinate information to obtain second coordinate information of a nose feature line in a second preset coordinate system for the head image; The head yaw angle is calculated based on the second coordinate information.

5. The head posture detection method according to claim 4, characterized in that: The performing coordinate transformation on the first coordinate information to obtain second coordinate information of the nose feature line in a second preset coordinate system for the head image includes: Determine the coordinate information of the nose region bounding box in the first preset coordinate system and the coordinate information of the nose region bounding box in the second preset coordinate system, wherein the first preset coordinate system is a coordinate system established based on the nose region image, and the second preset coordinate system is a coordinate system established based on the head image; determining a coordinate transformation matrix based on the coordinate information of the nose region bounding box in the first preset coordinate system and the coordinate information of the nose region bounding box in the second preset coordinate system; The first coordinate information is subjected to coordinate transformation based on the coordinate transformation matrix to obtain second coordinate information of the nose feature line in a second preset coordinate system for the head image.

6. The head posture detection method according to claim 4, characterized in that: The head image includes multiple frames of head images, and the detecting the head posture based on the head deviation degree includes: Detect whether the multiple frames of head images meet a preset condition. If so, turn off the radiation source and issue a prompt message, wherein the prompt message represents the degree of head movement of the person being photographed. The preset condition includes at least one of the following: The difference between the head yaw angles corresponding to two adjacent frames of head images is greater than a first preset threshold; A difference between the head yaw angle corresponding to the current frame head image and the head yaw angle corresponding to the initial head image is greater than a second preset threshold.

7. The head posture detection method according to claim 1, characterized in that: The performing nose region detection on the head image to obtain a nose region image includes: Perform nose region detection on the head image based on the trained target detection model to obtain a nose region detection result; Obtaining a nose region image based on the nose region detection result; The trained target detection model is trained in the following way: Constructing a training data set, wherein the training data set includes a plurality of head images and nose region annotation information corresponding to each head image; The preset target detection model is trained based on the training data set to obtain a trained model.

8. The head posture detection method according to claim 7, characterized in that: The nose region detection result includes a plurality of third bounding boxes of the nose region and a confidence level corresponding to each third bounding box, and obtaining a nose region image based on the nose region detection result includes: Determine a third bounding box corresponding to the maximum value in the confidence as a target bounding box, and filter third bounding boxes that are repeated with the target bounding box; The nose region image is obtained based on the target bounding box, and the target bounding box is displayed.

9. A mobile CT device, characterized in that: The mobile CT device comprises: Moving scanning cavity; A camera, the camera is placed at the top of the inner cavity of the scanning cavity and is located obliquely above the head of the person being photographed, and the camera is used to capture a head image; A head posture detection device, wherein the head posture detection device is used to implement the steps of the method described in any one of claims 1-8.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the method described in any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images