A method and apparatus for determining a scoliosis angle

Abstract

The application relates to a method and a device for determining a scoliosis angle, wherein the method comprises: acquiring a three-dimensional ultrasound image of a user to be measured, the three-dimensional ultrasound image comprising at least an image of a spine; determining a projection image of the spine on a coronal plane according to the three-dimensional ultrasound image; determining a fitting curve of the spine according to the projection image; and determining at least one scoliosis angle of the spine according to the fitting curve. The embodiments of the application at least solve the problem that a scoliosis angle cannot be determined in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ultrasonic imaging, and in particular to a method and device for determining a scoliosis angle. BACKGROUND

[0002] Scoliosis is a popular orthopedic disease at present. Early detection and timely treatment of scoliosis is the key to maintaining bone health. Ultrasonic examination has the advantages of no radiation and reasonable price, and is often used to obtain ultrasonic images of the human spine. In the related art, after obtaining the ultrasonic images of a patient, a doctor often only obtains the degree of scoliosis by visual observation. However, in actual scenarios, it can be further necessary to obtain accurate bending parameters of the patient's spine to make orthotic devices for users to use.

[0003] At present, there is no way to obtain accurate bending parameters of the spine from ultrasonic images in the related art. SUMMARY

[0004] Embodiments of the present application provide a method and device for determining a scoliosis angle to at least solve the problem that the scoliosis angle cannot be determined in the related art.

[0005] In a first aspect, the embodiments of the present application provide a method for determining a scoliosis angle, comprising:

[0006] obtaining a three-dimensional ultrasonic image of a user to be measured, the three-dimensional ultrasonic image comprising at least an image of a spine;

[0007] determining a projection image of the spine on a coronal plane according to the three-dimensional ultrasonic image;

[0008] determining a fitting curve of the spine according to the projection image;

[0009] determining at least one scoliosis angle of the spine according to the fitting curve.

[0010] Optionally, in an embodiment of the present application, the determining the fitting curve of the spine according to the projection image comprises:

[0011] inputting the projection image into a lamina pair detection model, and outputting a detection box of a lamina pair in the projection image through the lamina pair detection model, wherein the lamina pair detection model is trained using a plurality of coronal plane ultrasonic sample images of the spine, and the coronal plane ultrasonic sample images of the spine are labeled with positions of the lamina pair;

[0012] fitting the fitting curve of the spine according to the center points of the detection boxes.

[0013] Optionally, in an embodiment of the present application, the lamina pair detection model comprises a region generation network and a target detection network, an output end of the region generation network is connected with an input end of the target detection network, wherein,

[0014] the region generation network is configured to determine at least one candidate detection box of each lamina pair from the projection image;

[0015] the target detection network is configured to determine a detection box of the lamina pair from the at least one candidate detection box of each lamina pair.

[0016] Optionally, in an embodiment of the present application, the determining the at least one lateral curvature angle of the spine according to the fitting curve comprises:

[0017] determining at least one extreme point of the fitting curve;

[0018] respectively acquiring adjacent inflection points on both sides of the at least one extreme point of the fitting curve;

[0019] taking an included angle between tangent lines at the adjacent inflection points on both sides of each extreme point as a lateral curvature angle at the extreme point.

[0020] Optionally, in an embodiment of the present application, the acquiring the three-dimensional ultrasound image of the user to be measured, the three-dimensional ultrasound image comprising at least an image of a spine, comprises:

[0021] acquiring a plurality of two-dimensional ultrasound images of the user to be measured and position information of a probe of a two-dimensional ultrasound instrument by using the probe;

[0022] respectively determining a transformation relationship between the plurality of two-dimensional ultrasound images and a preset three-dimensional space according to the position information of the probe;

[0023] converting the plurality of two-dimensional ultrasound images into the preset three-dimensional space according to the transformation relationship to acquire the three-dimensional ultrasound image of the user to be measured.

[0024] Optionally, in an embodiment of the present application, the method further comprises:

[0025] displaying the projection image of the spine on the coronal plane, the detection box of the lamina pair and the at least one lateral curvature angle.

[0026] In a second aspect, the embodiments of the present application further provide a device for determining a lateral curvature angle of a spine, comprising:

[0027] an ultrasound image acquisition module configured to acquire a three-dimensional ultrasound image of a user to be measured, the three-dimensional ultrasound image comprising at least an image of a spine;

[0028] a coronal plane image determination module configured to determine a projection image of the spine on a coronal plane according to the three-dimensional ultrasound image;

[0029] a fitting curve determination module configured to determine a fitting curve of the spine according to the projection image;

[0030] a scoliosis angle determination module configured to determine at least one scoliosis angle of the spine according to the fitting curve.

[0031] In a third aspect, an apparatus for determining a scoliosis angle is provided, and the apparatus includes an ultrasound device, a processor, and a display, wherein

[0032] the ultrasound device is configured to scan an ultrasound image of a user to be measured;

[0033] the processor is configured to reconstruct a three-dimensional ultrasound image of the user to be measured according to the ultrasound image, and implement the method for determining a scoliosis angle when executed;

[0034] the display is configured to display the projection image and / or the at least one scoliosis angle.

[0035] In a fourth aspect, a non-transitory computer-readable storage medium having computer program instructions stored therein is provided, and the computer program instructions, when executed by a processor, implement the method.

[0036] In a fifth aspect, a computer program product is provided, including computer readable code or a non-transitory computer readable storage medium carrying computer readable code, and when the computer readable code is run in a processor of an electronic device, the processor in the electronic device implements the method.

[0037] The embodiments of the present application provide a method and an apparatus for determining a scoliosis angle. Firstly, a scoliosis angle of a spine is determined according to a three-dimensional ultrasound image of a user to be measured. The three-dimensional ultrasound image is directly or indirectly obtained by an ultrasound device, which is non-radiative to human body and low in cost, and meets the requirement of a user suffering from scoliosis to observe frequently. Secondly, a projection image of the spine on a coronal plane in the three-dimensional ultrasound image can comprehensively and accurately reflect a state of the spine, and a fitting curve of the spine can be obtained, so that a more accurate scoliosis angle can be obtained.

[0038] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0040] Figure 1 is a method flowchart of a method for determining a scoliosis angle provided by an embodiment of the application;

[0041] Figure 2 is a schematic diagram of determining a scoliosis angle provided by an embodiment of the application;

[0042] Figure 3 is a schematic diagram of a scene provided by an embodiment of the application;

[0043] Figure 4 is a software use flowchart provided by an embodiment of the application;

[0044] Figure 5 is a user interface 500 provided by an embodiment of the application;

[0045] Figure 6 is a method flowchart of a method for determining a scoliosis angle provided by an embodiment of the application;

[0046] Figure 7 is a module structure schematic diagram of a scoliosis angle determination device 700 provided by an embodiment of the application;

[0047] Figure 8 is a module structure schematic diagram of a processing device 1200 provided by an embodiment of the application;

[0048] Figure 9 is a conceptual partial view of a computer program product 1300 provided by an embodiment of the application. DETAILED DESCRIPTION

[0049] In order to make the objects, technical solutions and advantages of the application clearer, the application is described and explained below in connection with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and should not be used to limit the application. Based on the embodiments provided by the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the application. In addition, it should be understood that although the effort made in this development process can be complex and lengthy, some design, manufacture or production changes made by those of ordinary skill in the art related to the content disclosed in the application on the basis of the technical content disclosed in the application are only routine technical means and should not be understood as insufficient disclosure of the content disclosed in the application.

[0050] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0051] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application means greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in this application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0052] The method for determining the scoliosis angle described in this application is described in detail below with reference to the accompanying drawings. Figure 1 1 is a flow chart of an embodiment of the method for determining scoliosis angle provided by the present application. Although the present application provides the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiments of the present application. In the actual process of determining the scoliosis angle or when the method is executed, the method can be executed in the order of the methods shown in the embodiments or drawings or in parallel (for example, in an environment of parallel processors or multi-threaded processing).

[0053] Specifically, an embodiment of the method for determining the scoliosis angle provided by the present application is as follows: Figure 1 As shown, the method may include:

[0054] S101: Obtain a three-dimensional ultrasound image of a user to be measured, the three-dimensional ultrasound image at least including an image of a spine.

[0055] In the embodiments of the present application, ultrasound imaging refers to scanning a human body by using an ultrasound probe to emit an ultrasound beam, and obtaining an image by receiving and processing signals reflected by the human body. Compared with a two-dimensional ultrasound image, a three-dimensional ultrasound image has a larger contrast between the structure of an object and liquid gray scale, and can clearly show the three-dimensional shape, surface features, spatial position relationship, and the like of the scanned object. The various embodiments of the present application need to determine the lateral bending angle of the spine of a human body, but since the spine has a curved property, the lateral bending angle of the spine cannot be determined from a planar image. Based on this, in the embodiments of the present application, a three-dimensional ultrasound image of the spine can be obtained first, and then the lateral bending angle of the spine is determined according to the three-dimensional ultrasound image.

[0056] In an embodiment of the present application, the three-dimensional ultrasound image can be obtained by reconstructing a two-dimensional ultrasound image. Specifically, the obtaining of the three-dimensional ultrasound image of the user to be measured, the three-dimensional ultrasound image at least including an image of a spine, comprises:

[0057] S201: Obtain a plurality of two-dimensional ultrasound images of a user to be measured and position information of a probe of a two-dimensional ultrasound instrument by using the probe;

[0058] S203: Determine a transformation relationship between each of the plurality of two-dimensional ultrasound images and a preset three-dimensional space according to the position information of the probe;

[0059] S205: Convert the plurality of two-dimensional ultrasound images into the preset three-dimensional space according to the transformation relationship, and obtain a three-dimensional ultrasound image of the user to be measured.

[0060] In the embodiment of the present application, according to different working principles, the two-dimensional ultrasound instrument may include various types of ultrasound instruments such as type A, type M, type B, and type D. In terms of instrument volume, the two-dimensional ultrasound instrument may include a traditional large-scale ultrasound instrument, and may also include a handheld two-dimensional ultrasound instrument. The handheld two-dimensional ultrasound instrument has the advantages of fast imaging, small size, light weight, and easy portability. The present application does not limit the type of two-dimensional ultrasound instrument. In the embodiment of the present application, not only can the probe of the two-dimensional ultrasound instrument be used to obtain multiple two-dimensional ultrasound images of the user being tested, but the position information of the probe can also be obtained. In a possible embodiment, the position information of the probe can be obtained using a positioning module. The positioning module can be integrated into the interior of the two-dimensional ultrasound instrument, or it can be set outside the two-dimensional ultrasound instrument and can track any position of the probe. In one example, the positioning module may include a position sensor with six degrees of freedom. Then, the obtained probe position may include, for example, Cartesian coordinates (x, y, z) and Euler angles (A, E, R).

[0061] Based on the position information of the probe, the transformation relationship between the multiple two-dimensional ultrasound images and the preset three-dimensional space can be determined respectively. Specifically, in the process of reconstructing the three-dimensional ultrasound image, a two-dimensional ultrasound image coordinate system, a source coordinate system of the positioning module, and an observer coordinate system are involved, wherein the three-dimensional ultrasound image is based on the observer coordinate system. Based on this, in an embodiment of the present application, a transformation matrix M1 can be determined based on at least three calibration points in the source coordinate system. Using the transformation matrix M1, the two-dimensional ultrasound image can be transformed from the two-dimensional ultrasound image coordinate system to the source coordinate system. Then, based on the two-dimensional ultrasound image transformed to the source coordinate system and the parameters of the observer coordinate system, a transformation matrix M2 can be determined. Using the transformation matrix M2, the two-dimensional ultrasound image can be transformed from the source coordinate system to the preset three-dimensional space of the observer coordinate system. The preset three-dimensional space may include a mapping space for placing the multiple two-dimensional ultrasound images. Using the above method, each pixel in the multiple two-dimensional ultrasound images can be transformed into the observer coordinate system to obtain a three-dimensional ultrasound image of the measured user.

[0062] Through the above embodiment, the position information of the probe of the two-dimensional ultrasound instrument during movement can be recorded and used to reconstruct the three-dimensional ultrasound image. This not only allows the operator to adjust the scanning angle and scanning speed at will, but also allows the operator to use the above reconstruction method to obtain a three-dimensional ultrasound image of the spine of the user being measured in real time.

[0063] Of course, in other embodiments, the three-dimensional ultrasound image can also be directly acquired using a stereo ultrasound device, or reconstructed using other three-dimensional ultrasound image reconstruction methods, and this application does not impose any limitations thereto.

[0064] S103: Determine a projection image of the spine on the coronal plane according to the three-dimensional ultrasound image.

[0065] In the human body structure, when observing the human spine from the side, the spine has a curvature. Therefore, it is impossible to obtain a complete spinal image observed from the rear view angle of the human body through a coronal plane image. The coronal plane image may refer to a cross-sectional image that longitudinally cuts the human body into two parts, front and back, along the left and right directions. Based on this, in an embodiment of the present application, the projection image of the spine on the coronal plane can be determined based on the three-dimensional ultrasound image. Specifically, the three-dimensional image of the spine can be extracted from the three-dimensional ultrasound image, and the three-dimensional image of the spine can be projected onto the coronal plane to obtain the projection image of the spine on the coronal plane. The projection image can fully display the skeletal structure of the spine of the user being tested from the rear view angle.

[0066] S105: Determine a fitting curve of the spine according to the projection image.

[0067] Since the spine has a linear structure, the spine may correspond to a fitting curve in the projection image, and then, at least one scoliosis angle of the spine may be determined by the fitting curve. Therefore, it is first necessary to obtain an expression of the fitting curve, which may include, for example, a curve equation. In order to obtain an accurate expression of the fitting curve, in an embodiment of the present application, the position of the lamina pair may be used to determine the expression of the fitting curve. In the structure of the spine, when viewed from the rear of the human body, the lamina pair is located at the center of the spine. Therefore, the position of the lamina pair is determined, and based on the position of the lamina pair, the fitting curve corresponding to the spine can be accurately determined. In one embodiment of the present application, the position of the lamina pair may be detected by machine learning. Specifically, determining the fitting curve of the spine based on the projection image may include:

[0068] S301: Inputting the projection image into a lamina pair detection model, and outputting a detection frame of the lamina pair in the projection image through the lamina pair detection model, wherein the lamina pair detection model is trained using a plurality of spinal coronal ultrasound sample images, wherein the spinal coronal ultrasound sample images are annotated with the positions of the lamina pairs;

[0069] S303: According to the center points of the detection frames, a fitting curve of the spine is obtained by fitting.

[0070] In the embodiments of the present application, the lamina pair detection model can be trained by machine learning. Specifically, the lamina pair detection model is trained by a plurality of spine coronal plane projection sample images, and the lamina pair positions are labeled in the spine coronal plane projection sample images. The spine coronal plane projection sample images can be training samples obtained before training the lamina pair detection model. The training samples can include the projection images of the spine in the coronal plane obtained in the same manner as in the above embodiments. Of course, the training samples can also be obtained from a medical image database, and the present application does not limit the manner of obtaining the training samples. The spine coronal plane ultrasound sample images can be labeled with the positions of the lamina pairs, for example, the images of each lamina pair can be framed in the sample images by using a bounding box.

[0071] In an embodiment of the present application, in the process of constructing the lamina pair detection model, first, an initial model of the lamina pair detection model can be constructed, and the training parameters are set in the initial model. Then, the plurality of spine coronal plane ultrasound sample images can be input into the constructed lamina pair detection model respectively to generate a prediction result. Finally, based on the difference between the prediction result and the labeled position of the lamina pair, the training parameters are iteratively adjusted until the difference meets the preset requirement. In this way, the lamina pair detection model can be generated. It should be noted that the machine learning method of the lamina pair detection model can include any neural network-based object detection algorithm, such as Yolo (You Only Look Once: Unified, Real-Time Object Detection), DEtection Transformer (DETR), Region-Convolutional Neural Network (R-CNN), Fast Region-Convolutional Neural Network (Fast R-CNN), AlexNet, LeNet, ResNet, ResNet1001 (pre-activation), etc., which are not limited in the present application.

[0072] In the embodiments of the present application, the position of the lamina pair in the spine is detected by using the machine learning-based method, which can avoid the difference between different users to be detected, and can more robustly monitor the scoliosis angle of the same user at different times.

[0073] After the pairs of laminae are detected in the projection image, a fitting curve of the spine can be fitted according to the center points of the detection boxes. Since the pairs of laminae are located at the center of the spine, the center points of the pairs of laminae are also the center points of the spine in the coronal plane. Therefore, the fitting curve of the spine can be accurately fitted according to the center points of the detection boxes.

[0074] Further, in order to improve the detection efficiency of the pair of laminae detection model, a region generation module (RPN) for generating candidate detection boxes can be arranged in the pair of laminae detection model. Based on this, in an embodiment of the present application, the pair of laminae detection model includes a region generation network and a target detection network, an output end of the region generation network is connected to an input end of the target detection network, wherein,

[0075] S401: The region generation network is used to obtain at least one candidate detection box of each pair of laminae from the projection image.

[0076] S403: The target detection network is used to determine the detection box of the pair of laminae from the at least one candidate detection box.

[0077] In an embodiment of the present application, the region generation network has the advantages of short time consumption and easy coupling with the target detection module, and can quickly obtain at least one candidate detection box of each pair of laminae in the projection image. Based on the at least one candidate detection box, the target detection network can more quickly determine the detection box of each pair of laminae, and the efficiency and accuracy can be improved.

[0078] In an embodiment of the present application, after the key points for fitting the curve, such as the center points of the detection boxes, are determined, the curve fitting can be completed according to the key points. In an embodiment, the fitting curve can include a quintic power curve. It is found through multiple experiments that the quintic power curve can more accurately express the spine curve. Of course, in other embodiments, the present application does not limit the highest power of the fitting curve of the spine, for example, the highest power of the fitting curve can be determined according to the actual spine characteristics of the user to be detected.

[0079] It should be noted that in the process of determining the fitting curve of the spine, a short curve phenomenon may occur. A short curve refers to a situation where the number of lamina pairs between two inflection points is less than a preset threshold value, for example, the number of lamina pairs contained in the short curve is less than or equal to five lamina pairs. The short curve phenomenon may be caused by image loss, reconstruction error, etc. In this case, some key points can be added between the short curves, for example, some points can be added by interpolation, and then the fitting curve of the spine can be re-determined. In addition, in the case where the short curve appears at both ends of the spine, some key points can be added to the extension line of the spine, and the highest power of the fitting curve can be increased before re-fitting, for example, the fitting curve can be adjusted from the fifth power to the sixth power to conform to the actual characteristics of the spine.

[0080] S107: Determine at least one scoliosis angle of the spine according to the fitting curve.

[0081] The healthy spinal state is basically a straight line when viewed from the rear, but in the case of scoliosis, the state of the spine may include one or more scoliosis radians. Therefore, the angles of these scoliosis radians are obtained according to the fitting curve of the spine, and at least one scoliosis angle of the spine can be determined. In one embodiment of the present application, the at least one scoliosis angle of the spine can be determined based on the extreme points and inflection points of the fitting curve. Specifically, the determination of the at least one scoliosis angle of the spine according to the fitting curve may include:

[0082] S501: Determine at least one extreme point of the fitting curve;

[0083] S503: Obtain adjacent inflection points of the fitting curve on both sides of the at least one extreme point respectively;

[0084] S505: Taking the angle between the tangent lines at the adjacent inflection points on both sides of each extreme point as the scoliosis angle at the extreme point.

[0085] like Figure 2 As shown, for a section of the fitting curve, the extreme point is determined to be A, and the adjacent inflection points on both sides of point A are points B and C. The tangent lines of the fitting curve at points B and C are determined, and then the angle α between the two tangent lines is equal to the scoliosis angle β of the spine at point A. It should be noted that when calculating the scoliosis angle at the end of the spine, the inflection point needs to be specially processed. For example, the average value point (the median value between the endpoint and the adjacent inflection point) calculated by the Lagrange mean method can be used instead of the inflection point to accurately calculate the scoliosis angle.

[0086] Of course, in other embodiments, the scoliosis angle can also be determined in other ways. After the fitting curve is determined, the inflection points on the fitting curve can be determined, and then the tangent lines of each inflection point on the fitting curve can be determined. The included angle of adjacent tangent lines is taken as the scoliosis angle. The application does not limit the way of determining the scoliosis angle.

[0087] The embodiment of the application provides a method for determining a scoliosis angle. First, the scoliosis angle is determined according to a three-dimensional ultrasound image of a user to be measured. The three-dimensional ultrasound image is directly or indirectly obtained by using an ultrasound instrument. The method is radiation-free to human body and has low cost, which meets the requirement of users suffering from scoliosis to observe frequently. Second, in the three-dimensional ultrasound image, the image of the scoliosis projected on the coronal plane can comprehensively and accurately reflect the state of the scoliosis, and a relatively accurate scoliosis curve can be fitted, so that a more accurate scoliosis angle can be obtained.

[0088] It should be noted that, as shown in Figure 3 The final result or indirect result of the method for determining the scoliosis angle according to any of the above embodiments can be displayed through a user interface of a client 301, and further, the start and end of the action can be triggered through a control in the user interface of the client 301. In the case that the client 301 and the ultrasound instrument 303 have data processing capability, the client 301 can be coupled with the ultrasound instrument 303, and the reconstruction of the three-dimensional ultrasound image, the determination of the scoliosis angle and the like can be completed by one party having data processing capability. The specific coupling manner can include wired connection or wireless connection. The wired connection can include connecting the ultrasound instrument 303 and the client 301 by using a coaxial cable, an optical fiber, a twisted pair and the like. The wireless connection can include 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultrawideband) connection and other now known or future developed wireless connection manners. Of course, in the case that the three-dimensional ultrasound image can be obtained only after the processor 305 performs three-dimensional reconstruction on the two-dimensional ultrasound image obtained by the ultrasound instrument 303, the client 301 can be coupled with the ultrasound instrument 303 by using a wired connection or a wireless connection. Figure 3As shown, the client 301 can also be coupled to the processor 305. Optionally, in one embodiment, the processor 305 and the client 301 can be an electronic device integrated into one with data transmission capability, data processing capability and data display function, and the electronic device can include, for example, a mobile smart phone, a computer (including a laptop computer, a desktop computer), a tablet electronic device, a personal digital assistant (PDA) or an intelligent wearable device, etc., which is not limited in this application. In this way, the client 301 and the processor 305 can be matched with different ultrasound machines, and the function improvement can be achieved at a lower cost. Optionally, in one embodiment, the client 301, the ultrasound machine 303 and the processor 305 can be an electronic device integrated into one with ultrasound scanning capability, data transmission capability, data processing capability and data display function.

[0089] The above control and display methods are described below in conjunction with the usage of the software. The software may include multiple controls, and the functions of the multiple controls are described below using Table 1.

[0090] Table 1 Controls and function descriptions

[0091]

[0092] The following combination Figure 4 The software usage process shown in the figure explains how to use the above controls.

[0093] like Figure 4 As shown, after starting the software, the staff can use the control 401 to establish a connection with the ultrasound machine 303. After the establishment is successful, the pre-scan link can be started through the control 403 to determine the scanning range for the subsequent formal scan. Of course, if the connection is unsuccessful, the staff can be prompted that the connection failed. Of course, the pre-scan interface can also be displayed. In the pre-scan interface, the staff can set the scanning voxel size (Voxel Size), specifically including the width (Width), height (Height) and depth (Depth) of the voxel. In addition, the range that can be expanded (Enlarge the volume) based on the scanning range determined by the pre-scan can be set to increase a certain degree of inclusiveness and prevent the scanning range from being expanded during the scanning process. For example, if the expandable range is set to 1mm, then the scanning range determined by the pre-scan can be expanded by 1mm in each direction up, down, left, right, front and back.

[0094] like Figure 4As shown, after the pre-scan, the staff can control the ultrasound machine 303 to enter the formal scanning phase through the control 402, or can pause the scanning of the ultrasound machine 303 through the control 402. During the scanning process, the staff can use the control 408 to reconstruct the three-dimensional ultrasound image in real time. Figure 5 The 3D ultrasound images reconstructed at four moments during the scanning process are displayed, as shown in user interface 500. The staff reconstructed the 3D ultrasound images of the user being tested in real time at 13 seconds, 27 seconds, 40 seconds, and 54 seconds from the start of the scan. By displaying the 3D ultrasound images of the user being tested in real time, the staff can understand the user's spinal health status in real time.

[0095] The software's user interface can include multiple data display areas. The first data display area can be used to display a cross-sectional image of the 3D ultrasound image in real time, the second data display area can be used to display a coronal image of the 3D ultrasound image in real time, and the third data display area can be used to display a sagittal image of the 3D ultrasound image in real time. This allows for a comprehensive, multi-angle display of the 3D ultrasound image. Furthermore, a segmentation control can be used to remove the muscle layer from the 3D ultrasound image, displaying only the skeletal portion, meeting the needs of users observing the spinal column in this embodiment of the application.

[0096] like Figure 4 As shown, after the scan is completed, the scan results can be saved through control 409 to help establish the profile of the user being tested. After triggering control 409, the user interface of the software can jump to the data saving interface. In the data saving interface, you can set the data saving path, identification information of the user being tested, data name and other information. In addition, in the data saving interface, you can also select the type of data to be saved, wherein the data type can include at least one of the real-time reconstructed three-dimensional ultrasound image, two-dimensional cross-sectional image (the position of the cross-section can be customized by the staff), the position information of the three-dimensional ultrasound image (such as the spatial coordinates of each voxel in the three-dimensional ultrasound image), and the reconstructed coronal image.

[0097] like Figure 4As shown, after saving the data, two tasks can be performed, namely automatic detection and measurement and precise reconstruction. Among them, automatic detection and measurement is the process of determining the scoliosis angle mainly described in each embodiment of this application. Based on this, the software also provides a scoliosis angle display interface. In the scoliosis angle display interface, some basic data need to be imported before obtaining the scoliosis angle. The basic data may include a lamina pair detection model for detecting the position of the lamina pair, a projection image of the spine on the coronal plane, and program instructions for automatic detection and measurement. The scoliosis angle display interface may include three data display areas. Among them, the first data display area is used to display the original projection image of the spine on the coronal plane. The second data display area is used to display the detection frame 201 of the lamina pair detected from the projection image, and can also display the position of the inflection point on the fitting curve obtained by fitting the center point of the lamina pair detection frame 201. The third data display area is used to display the scoliosis angle obtained based on the inflection point.

[0098] In an embodiment of the present application, precise reconstruction means that after the scan is completed, three-dimensional reconstruction can be performed again according to the required algorithm. After the software jumps to the precise reconstruction user interface, the staff can define the voxel size of the reconstructed three-dimensional image and select the three-dimensional reconstruction algorithm, such as the voxel nearest neighbor method (Voxel Nearest Neighbor, VNN), multiplane image 2 (Multiplane Image2, MPI2), multiplane image 4 (Multiplane Image4, MPI4), pixel nearest neighbor method (Pixel Nearest Neighbor, PNN) and other algorithms to determine a more accurate three-dimensional ultrasound image.

[0099] The following combination Figure 6 The following describes a specific application scenario of the method for determining the scoliosis angle of the present application. Figure 6 As shown, after obtaining the projection image, it is possible to first determine whether there is a key part (such as the spinal endpoint) missing in the projection image. The projection image without the key part cannot be put into the program for operation, and the three-dimensional ultrasound image needs to be reconstructed.

[0100] If the projection image is complete, the lamina pair detection model can be used to detect the lamina pair in the projection image to determine the detection frame of the lamina pair. After the detection frame of the lamina pair is detected, it can be checked whether the detected lamina pair is correct. If there is any lamina pair that is not detected or the detection result is biased, the detection frame of the lamina pair can be manually adjusted. Of course, based on the detection frame of the lamina pair, it is also necessary to determine whether there is a short curve in the spine. If there is no short curve, a quintic function can be used to fit the center point of the detection frame to obtain the fitting curve of the spine. If there is a short curve, there are mainly two cases here. The first case is that if there is a short curve (≤5 pairs of lamina), a key point can be added between the lamina pairs in the short curve. The second case is that if the short curve is at both ends of the entire spine, 1-2 key points need to be added on the extension line of the spine, and a sextic fitting function is used for curve fitting to obtain the fitting curve of the spine.

[0101] Finally, the inflection points in the fitting curve are determined, and the tangent lines at all the inflection points are determined. The bending angle of the spine is determined by calculating the included angle of the tangent lines of each pair of adjacent inflection points. It should be noted that special treatment is needed for the inflection points when calculating the bending angle of the end of the spine. For example, the average point (the middle value of the end point and the adjacent inflection point) calculated by the Lagrange mean value method can be used instead of the inflection point to accurately calculate the bending angle of the spine.

[0102] The method for determining the scoliosis angle provided in the present application is described in detail above. The device 700 for determining the scoliosis angle provided in the present application will be described below in combination with the accompanying drawings. Figure 7 , the device 700 for determining the scoliosis angle provided in the present application comprises:

[0103] The ultrasound image acquisition module 701 is configured to acquire a three-dimensional ultrasound image of a user to be measured, and the three-dimensional ultrasound image at least includes an image of a spine.

[0104] The coronal plane image determination module 703 is configured to determine a projection image of the spine on a coronal plane according to the three-dimensional ultrasound image.

[0105] The fitting curve determination module 705 is configured to determine a fitting curve of the spine according to the projection image.

[0106] The scoliosis angle determination module 707 is configured to determine at least one scoliosis angle of the spine according to the fitting curve.

[0107] Optionally, in an embodiment of the present application, the fitting curve determination module 705 is specifically configured to:

[0108] input the projection image into a lamina pair detection model, and output a detection box of a lamina pair in the projection image through the lamina pair detection model, wherein the lamina pair detection model is trained by using a plurality of coronal plane ultrasound sample images of a spine, and the coronal plane ultrasound sample images are labeled with positions of lamina pairs;

[0109] fit a fitting curve of the spine according to center points of the detection boxes.

[0110] Optionally, in an embodiment of the present application, the lamina pair detection model comprises a region generation network and a target detection network, an output end of the region generation network is connected with an input end of the target detection network, wherein,

[0111] the region generation network is configured to determine at least one candidate detection box of each lamina pair from the projection image;

[0112] the target detection network is configured to determine the detection box of the lamina pair from the at least one candidate detection box of each lamina pair.

[0113] Optionally, in an embodiment of the present application, the lateral bending angle determination module 707 is specifically configured to:

[0114] determine at least one extreme point of the fitting curve;

[0115] respectively acquire adjacent inflection points on both sides of the at least one extreme point of the fitting curve;

[0116] take an included angle between tangent lines at the adjacent inflection points on both sides of each extreme point as a lateral bending angle at the extreme point.

[0117] Optionally, in an embodiment of the present application, the ultrasound image acquisition module 701 is specifically configured to:

[0118] acquire a plurality of two-dimensional ultrasound images of a user to be measured and position information of a probe by using the probe of a two-dimensional ultrasound instrument;

[0119] respectively determine transformation relationships between the plurality of two-dimensional ultrasound images and a preset three-dimensional space according to the position information of the probe;

[0120] convert the plurality of two-dimensional ultrasound images into the preset three-dimensional space according to the transformation relationships, and acquire a three-dimensional ultrasound image of the user to be measured.

[0121] Optionally, in an embodiment of the present application, the method further comprises:

[0122] a display configured to display the projection image of the spine on the coronal plane, the detection box of the lamina pair, and the at least one lateral bending angle.

[0123] According to the embodiment of the present application, the scoliosis angle determination device 700 may correspond to executing the method described in the embodiment of the present application, and the above-mentioned and other operations and / or functions of each module in the scoliosis angle determination device 700 are respectively for implementing the corresponding processes of the methods provided in the above-mentioned embodiments. For the sake of brevity, they will not be repeated here.

[0124] It should also be noted that the embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed across multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0125] An embodiment of the present application further provides a device for determining the scoliosis angle, the device comprising an ultrasound device, a processor, and a display, wherein:

[0126] The ultrasonic device is used to scan an ultrasonic image of the user being tested;

[0127] The processor is configured to implement the method for determining the scoliosis angle described in any one of the above embodiments when executing;

[0128] The display is used to display the projection image and / or the at least one scoliosis angle.

[0129] The embodiments of the present application further provide a processing device. The processing device may be a physical device or a cluster of physical devices, or a virtualized cloud device, such as at least one cloud computing device in a cloud computing cluster. For ease of understanding, the present application illustrates the structure of the processing device using the processing device as an independent physical device.

[0130] like Figure 8As shown, the processing device 1200 includes a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described apparatus when executing the instructions. The processing device 1200 includes a memory 1201, a processor 1203, a bus 1205 and a communication interface 1207. The memory 1201, the processor 1203 and the communication interface 1207 communicate through the bus 1201. The bus 1205 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The communication interface 1207 is configured to communicate with the outside.

[0131] The processor 1203 can be a central processing unit (CPU). The memory 1201 can include a volatile memory, such as a random access memory (RAM). The memory 1201 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, an HDD or an SSD.

[0132] The memory 1201 stores executable code, and the processor 1203 executes the executable code to implement the above-described method of constructing a test scenario.

[0133] Embodiments of the present application provide a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions being executed by a processor to implement the above-described method.

[0134] Embodiments of the present application provide a computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described method.

[0135] In some embodiments, the disclosed method can be implemented as computer program instructions encoded in a machine-readable storage medium in a machine-readable format or in other non-transitory media or articles of manufacture. Figure 9Schematically illustrates a conceptual partial view of an example computer program product, arranged in accordance with at least some embodiments presented herein, comprising a computer program for executing a computer process on a computing device. In one embodiment, the example computer program product 1300 is provided using a signal-bearing medium 1301. The signal-bearing medium 1301 may include one or more program instructions 1303, which, when executed by one or more processors, may provide the above-described instructions for executing a computer process. Figure 1 The functions or parts of functions described. In addition, Figure 9 Program instructions 1303 in also describe example instructions.

[0136] In some examples, the signal-bearing medium 1301 may include a computer-readable medium 1305, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc. In some embodiments, the signal-bearing medium 1301 may include a computer-recordable medium 1307, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal-bearing medium 1301 may include a communication medium 1305, such as, but not limited to, a digital and / or analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communication link, a wireless communication link, etc.). Thus, for example, the signal-bearing medium 1301 may be communicated by a wireless form of communication medium 1309 (e.g., a wireless communication medium that complies with the IEEE 802.11 standard or other transmission protocol). The one or more program instructions 1303 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, such as for Figure 2 The computing device of the computing device described can be configured to, in response to being communicated to the program instruction 1303 of computing device by one or more in computer-readable medium 1305, computer recordable medium 1307 and / or communication medium 1309, provide various operations, functions or actions.Should be understood that the arrangement described here is only for the purpose of example.Thus, it will be understood by those skilled in the art that other arrangements and other elements (for example, machines, interfaces, functions, sequences, and functional groups etc.) can be used instead, and some elements can be omitted together according to the desired result.In addition, many of the described elements can be implemented as discrete or distributed components or in any appropriate combination and position to combine the functional entities implemented by other components.

[0137] The computer program product of the present application can be a computer program product comprising a computer readable storage medium having stored, thereon, computer readable program code configured such that upon execution by one or more computer processors, the computer readable program code enables performing the methods as described herein. The computer readable storage medium can be a tangible computer readable storage medium. The tangible computer readable storage medium can be a storage medium such as, e.g., memory.

[0138] It is also noted that each of the blocks of the flowchart and / or the combination of blocks in the flowchart can be implemented by hardware, e.g., circuitry or an ASIC (Application Specific Integrated Circuit), or can be implemented by a combination of hardware and software, e.g., firmware or the like.

[0139] Although the present application has been described in connection with various embodiments thereof, it will be understood that other modifications will be apparent to those of ordinary skill in the art, and this application is intended to cover any adaptations or variations of the preferred embodiments discussed herein. Single processor or other units can implement several of the functions recited in the claims. Certain measures recited in mutually different dependent claims are not meant to be mutually exclusive, but can be implemented in combination.

[0140] The foregoing description of the exemplary embodiments of this application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the application be limited not with this detailed description, but rather by the claims appended hereto. The terms used in this document, including the specification, examples, and claims, are not limited to the specific meanings provided here. Rather, these terms are used only in their broadest, ordinary sense and are intended to include all compatible meanings.

Claims

1. A method for determining a scoliosis angle, characterized in that: include: Acquiring a three-dimensional ultrasound image of the user being measured, wherein the three-dimensional ultrasound image at least includes an image of the spine; determining a projection image of the spine on a coronal plane according to the three-dimensional ultrasound image; Determining a fitting curve of the spine based on the projection image; in the process of determining the fitting curve of the spine, if short curves appear at both ends of the spine, adding key points on an extension line of the spine and increasing the highest power of the fitting curve before refitting, wherein the short curve refers to a fitting curve in which the number of lamina pairs between two inflection points is less than a preset number threshold; Determining at least one scoliosis angle of the spine according to the fitting curve; when calculating the scoliosis angle of the spine end, using the median of the endpoint calculated by the Lagrange mean method and the inflection point adjacent to the spine end to replace the inflection point of the spine end to calculate the scoliosis angle; Wherein, determining the fitting curve of the spine based on the projection image includes: inputting the projection image into a lamina pair detection model, and outputting a detection frame of the lamina pairs in the projection image through the lamina pair detection model, wherein the lamina pair detection model is trained using a plurality of spinal coronal ultrasound sample images, and the spinal coronal ultrasound sample images are marked with the positions of the lamina pairs; and fitting the fitting curve of the spine based on the center points of each of the detection frames.

2. The method according to claim 1, characterized in that The lamina pair detection model includes a region generation network and a target detection network, wherein the output end of the region generation network is connected to the input end of the target detection network, wherein: The region generation network is used to determine at least one candidate detection frame of each lamina pair from the projection image; The object detection network is used to determine the detection frame of the lamina pair from the at least one candidate detection frame of each lamina pair.

3. The method according to claim 1, characterized in that Determining at least one scoliosis angle of the spine according to the fitting curve comprises: determining at least one extreme point of the fitting curve; Respectively obtaining adjacent inflection points of the fitting curve on both sides of the at least one extreme point; The angle between the tangent lines at adjacent inflection points on both sides of each extreme point is taken as the scoliosis angle at the extreme point.

4. The method according to claim 1, wherein The step of obtaining a three-dimensional ultrasound image of the user being measured, wherein the three-dimensional ultrasound image at least includes an image of the spine, comprises: Acquiring multiple two-dimensional ultrasound images of the user being measured and position information of the probe using a probe of a two-dimensional ultrasound apparatus; determining, according to the position information of the probe, transformation relationships between the plurality of two-dimensional ultrasound images and a preset three-dimensional space; According to the transformation relationship, the multiple two-dimensional ultrasound images are transformed into the preset three-dimensional space to obtain a three-dimensional ultrasound image of the measured user.

5. The method according to claim 1, wherein Also includes: The projection image of the spine on the coronal plane, the detection frame of the vertebral plate pair and the at least one scoliosis angle are displayed.

6. A device for determining the angle of scoliosis, characterized in that: include: An ultrasonic image acquisition module, configured to acquire a three-dimensional ultrasonic image of the user being tested, wherein the three-dimensional ultrasonic image at least includes an image of the spine; a coronal plane image determination module, configured to determine a projection image of the spine on the coronal plane based on the three-dimensional ultrasound image; a fitting curve determination module, configured to determine a fitting curve of the spine based on the projection image; in the process of determining the fitting curve of the spine, if short curves appear at both ends of the spine, then adding key points to the extension line of the spine and refitting after increasing the highest power of the fitting curve, wherein the short curve refers to a fitting curve in which the number of lamina pairs between two inflection points is less than a preset number threshold; a scoliosis angle determination module, configured to determine at least one scoliosis angle of the spine based on the fitting curve; when calculating the scoliosis angle of the spinal end, using the median of the endpoint calculated by the Lagrange mean method and the inflection point adjacent to the spinal end to replace the inflection point of the spinal end to calculate the scoliosis angle; Wherein, determining the fitting curve of the spine based on the projection image includes: inputting the projection image into a lamina pair detection model, and outputting a detection frame of the lamina pairs in the projection image through the lamina pair detection model, wherein the lamina pair detection model is trained using a plurality of spinal coronal ultrasound sample images, and the spinal coronal ultrasound sample images are marked with the positions of the lamina pairs; and fitting the fitting curve of the spine based on the center points of each of the detection frames.

7. A device for determining the angle of scoliosis, characterized in that It includes an ultrasound machine, a processor, and a display, wherein: The ultrasonic device is used to scan an ultrasonic image of the user being tested; The processor is configured to reconstruct a three-dimensional ultrasound image of the user being measured based on the ultrasound image, and implement the method according to any one of claims 1 to 5 when executed; The display is used to display the projection image and / or the at least one scoliosis angle.

8. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

9. A computer program product, characterized in that The method comprises a computer-readable code or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the method according to any one of claims 1 to 5.

Images