Path planning method and device for automatic liver scanning, equipment and medium

Abstract

The invention discloses a path planning method and device for automatic liver scanning, equipment and a medium, and the method comprises the steps: obtaining human body point cloud data, reconstructing a human body surface based on the human body point cloud data, determining the position information of a liver probe based on the image data of a rib position, and on the basis of the position information and the human body surface, estimating a probe posture so as to form a planned path for automatic liver scanning. According to the method, the scanning path is more reasonable through position planning, and both liver parenchyma and intrahepatic vascular diseases can be detected. Then, through pose planning, when the probe is attached to the skin, a smaller angle can be formed, and the ultrasonic image quality is improved.

Description

Technical Field

[0001] The present application relates to the field of biomedical technology, and in particular to a path planning method, device, equipment and medium for automatic liver scanning. Background Art

[0002] Medical ultrasound examination has gradually become the preferred technology for liver disease screening and detection due to its non-invasive, real-time and efficient characteristics. Currently, clinical ultrasound examinations are performed by doctors holding probes. Long-term hand burden and bad posture can cause carpal tunnel syndrome, tendonitis, shoulder and neck discomfort and other diseases. As an automated tool, the robotic arm can greatly reduce the burden of doctors' scanning, and also help to improve the standardization process of liver scanning and reduce the differences in operations between different doctors.

[0003] The robotic arm can operate according to the preset scanning path and parameters. However, the existing path planning for the liver area lacks good planning for the posture angle, which makes it difficult for the probe to stick to the skin during actual operation, resulting in a significant decrease in the quality of the obtained ultrasound image.

[0004] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0005] The technical problem to be solved by the present application is to provide a path planning method, device, equipment and medium for automatic liver scanning in view of the deficiencies in the prior art.

[0006] In order to solve the above technical problems, the first aspect of the present application provides a path planning method for automatic liver scanning, wherein the path planning method for automatic liver scanning specifically includes:

[0007] Obtaining human body point cloud data and human body image of the object to be scanned;

[0008] Determine a human body surface based on the human body point cloud data, and obtain a normal vector of each surface point in the human body surface;

[0009] Determine a liver region of the object to be scanned based on the human body image, and construct an initial scanning path for the object to be scanned based on the liver region;

[0010] Based on the normal vector of each surface point in the human body surface, the probe posture corresponding to each path point in the initial scanning path is determined, and the probe posture is added to the path point to obtain the target scanning path of the object to be scanned.

[0011] The path planning method for automatic liver scanning, wherein the determining the liver region of the object to be scanned based on the human body image specifically comprises:

[0012] Inputting the human body image into a trained human body segmentation model, and outputting the rib position of the object to be scanned through the human body segmentation model;

[0013] According to the correspondence between the rib positions and the liver, the liver region of the object to be scanned is determined based on the rib positions of the object to be scanned.

[0014] The path planning method for automatic liver scanning, wherein the step of constructing an initial scanning path for the object to be scanned based on the liver region specifically comprises:

[0015] Taking a vertex of a region in the liver region as a starting point, constructing a transverse scanning path in a round-trip scanning manner;

[0016] Taking a vertex of a region in the liver region as a starting point, constructing a longitudinal scanning path in a round-trip scanning manner;

[0017] Taking the xiphoid process as the starting point, a lateral scanning path is constructed along the lower edge of the lateral ribs;

[0018] The path set consisting of the transverse scanning path, the longitudinal scanning path and the side scanning path is used as the initial scanning path of the object to be scanned.

[0019] In the path planning method for automatic liver scanning, the step of constructing an initial scanning path for the object to be scanned based on the liver region further comprises:

[0020] Acquiring the transverse width and the longitudinal width of the liver region;

[0021] Based on the transverse width and the longitudinal width, a width of an ultrasound probe used for performing ultrasound examination on the object to be scanned is determined.

[0022] The path planning method for automatic liver scanning, wherein the first distance between two adjacent rows of the transverse scanning path in the transverse scanning path is equal to the width of the ultrasound probe; the second distance between two adjacent rows of the longitudinal scanning path in the longitudinal scanning path is equal to the width of the ultrasound probe.

[0023] The path planning method for automatic liver scanning, wherein the step of determining the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface specifically includes:

[0024] For each path point in the initial scanning path, determining the probe coordinate system corresponding to the path point based on the normal vector of the corresponding surface point and the moving direction of the path point;

[0025] A rotation matrix between the probe coordinate system and a global coordinate system of the liver region is determined, and a probe posture corresponding to the path point is determined based on the rotation matrix.

[0026] The path planning method for automatic liver scanning, wherein the step of determining the probe coordinate system corresponding to the path point based on the normal vector of the corresponding surface point and the moving direction of the path point specifically includes:

[0027] Obtain the surface point corresponding to the path point, and use the normal vector direction of the surface point as the Z-axis direction of the path point;

[0028] Reading the moving direction of the path point, and using the moving direction as the X-axis direction of the path point;

[0029] The Y-axis direction of the path point is determined based on the X-axis direction and the Z-axis direction to obtain a probe coordinate system of the ultrasound probe at the path point.

[0030] The second aspect of the present application provides a path planning device for automatic liver scanning, wherein the path planning device for automatic liver scanning specifically comprises:

[0031] An acquisition module is used to acquire human body point cloud data and human body image of the object to be scanned;

[0032] A determination module, used to determine a human body surface based on the human body point cloud data, and obtain a normal vector of each surface point in the human body surface;

[0033] A construction module, used for determining a liver region of the object to be scanned based on the human body image, and constructing an initial scanning path for the object to be scanned based on the liver region;

[0034] An adding module is used to determine the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and add the probe posture to the path point to obtain the target scanning path of the object to be scanned.

[0035] A third aspect of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in any of the above-described path planning methods for automatic liver scanning.

[0036] A fourth aspect of the present application provides a terminal device, comprising: a processor and a memory;

[0037] The memory stores a computer-readable program executable by the processor;

[0038] When the processor executes the computer-readable program, the steps in any of the above-mentioned path planning methods for automatic liver scanning are implemented.

[0039] Beneficial effects: Compared with the prior art, the present application provides a path planning method, device, equipment and medium for automatic liver scanning, the method comprising obtaining human body point cloud data and human body image of the object to be scanned; determining the human body surface based on the human body point cloud data, and obtaining the normal vector of each surface point in the human body surface; determining the liver area of ​​the object to be scanned based on the human body image, and constructing an initial scanning path for the object to be scanned based on the liver area; determining the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and adding the probe posture to the path point to obtain the target scanning path of the object to be scanned. The present application firstly makes the scanning path more reasonable through position planning, which can not only detect the liver parenchyma but also the intrahepatic blood vessels, and then makes the probe fit the skin at a smaller angle through posture planning, thereby improving the quality of ultrasonic images. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 This is a flow chart of a path planning method for automatic liver scanning provided in an embodiment of the present application.

[0042] Figure 2 Schematic diagram of the transverse scanning path and the longitudinal scanning path.

[0043] Figure 3 A schematic diagram of the side scan inspection path.

[0044] Figure 4 Schematic diagram of the probe coordinate system.

[0045] Figure 5 This is a principle block diagram of a path planning device for automatic liver scanning provided in an embodiment of the present application.

[0046] Figure 6 A functional block diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The embodiments of the present application provide a path planning method, device, equipment and medium for automatic liver scanning. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0049] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0050] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0051] After research, it was found that medical ultrasound examination has gradually become the preferred technology for liver disease screening and detection due to its non-invasive, real-time and efficient characteristics. At present, clinical ultrasound examinations are all performed by doctors holding probes. Long-term hand burden and bad posture can cause carpal tunnel syndrome, tendonitis, shoulder and neck discomfort and other diseases. As an automated tool, the robotic arm can greatly reduce the burden of doctors' scanning, and at the same time, it can help improve the standardization process of liver scanning and reduce the differences in operations between different doctors.

[0052] The robotic arm can operate according to the preset scanning path and parameters. However, the existing path planning for the liver area lacks good planning for the posture angle, which makes it difficult for the probe to stick to the skin during actual operation, resulting in a significant decrease in the quality of the obtained ultrasound image.

[0053] In order to solve the above problems, in an embodiment of the present application, the human body point cloud data and human body image of the object to be scanned are obtained; the human body surface is determined based on the human body point cloud data, and the normal vector of each surface point in the human body surface is obtained; the liver area of ​​the object to be scanned is determined based on the human body image, and an initial scanning path is constructed for the object to be scanned based on the liver area; based on the normal vector of each surface point in the human body surface, the probe posture corresponding to each path point in the initial scanning path is determined, and the probe posture is added to the path point to obtain the target scanning path of the object to be scanned. The present application first makes the scanning path more reasonable through position planning, which can not only detect liver parenchyma but also intrahepatic blood vessels, and then through posture planning, the probe can have a smaller angle when it fits the skin, thereby improving the quality of ultrasonic images.

[0054] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0055] This embodiment provides a path planning method for automatic liver scanning. Figure 1 As shown, the method includes:

[0056] S10, obtaining human body point cloud data and human body image of the object to be scanned.

[0057] Specifically, the human body point cloud data includes point cloud data of the human body surface corresponding to the liver area, and the human body image includes the liver area. The human body point cloud data is obtained by scanning the object to be scanned, for example, scanning the object to be scanned by a kinect camera to obtain the human body point cloud data of the object to be scanned. The human body image is the RGB-D image data of the object to be scanned, for example, the image of the human body is obtained by taking an image of the object to be scanned by a depth camera. In addition, the human body point cloud data and the human body image can be acquired in real time before the liver of the object to be scanned is automatically scanned, or can be pre-collected and stored locally, or can be sent by an external device, etc.

[0058] S20. Determine a human body surface based on the human body point cloud data, and obtain a normal vector of each surface point in the human body surface.

[0059] Specifically, the human body surface is obtained by reconstructing the human body through human body point cloud data, wherein the human body surface is a 3D model of the human body. That is to say, after obtaining the human body point cloud data, a three-dimensional reconstruction is performed based on the human body point cloud data to obtain a 3D model of the human body. After obtaining the human body surface, the normal vector of each target surface point on the human body surface is obtained, wherein each target surface point corresponds to a point cloud data, and the point cloud data corresponding to all target surface points constitute the human body point cloud data of the object to be scanned. In other words, the human body surface is processed to obtain the normal vector of the surface position of each point cloud data in the human body point cloud data of the object to be scanned. For example, the human body surface is processed by the PCL library to obtain the normal vector of the surface position of each point cloud data in the human body point cloud data of the object to be scanned.

[0060] S30, determining a liver region of the object to be scanned based on the human body image, and constructing an initial scanning path for the object to be scanned based on the liver region.

[0061] Specifically, the liver area is the image area of ​​the liver in the human body image, which can be obtained by performing image segmentation on the human body image, or by first identifying the rib position in the human body image, and then determining the liver area of ​​the object to be scanned based on the positional relationship between the ribs and the liver.

[0062] Exemplarily, determining the liver region of the object to be scanned based on the human body image specifically includes:

[0063] Inputting the human body image into a trained human body segmentation model, and outputting the rib position of the object to be scanned through the human body segmentation model;

[0064] According to the correspondence between the rib positions and the liver, the liver region of the object to be scanned is determined based on the rib positions of the object to be scanned.

[0065] Specifically, the human body segmentation model is a trained deep learning model (e.g., CNN, DNN, etc.) for identifying the rib positions in the human body image. That is, the input item of the human body segmentation model is the human body image, and the output item is the rib position. The rib position includes the position information of each human body rib included in the human body image in the human body image, and each human body rib can be marked in the human body image.

[0066] The corresponding relationship between the rib position and the liver is that the upper boundary is located in the gap between the 5th and 6th ribs on the right, the lower boundary of the liver is located at the 12th rib at the level of the right mid-axillary line, the right lower boundary liver width extends from the rib edge to the junction of the 7th and 8th ribs on the right, and extends obliquely from the junction to the upper left to the 6th rib on the left, the left boundary is close to the liver at the edge of the left rib, and the right boundary is below the nipple. Therefore, after obtaining the rib position, the liver area of ​​the object to be scanned can be outlined based on the corresponding relationship between the rib position and the liver, where the liver area is distributed in a right-angle trapezoid.

[0067] Further, after the liver region is acquired, the scanning range to be covered by the ultrasound scan can be determined based on the liver region, that is, the scanning range to be covered by the ultrasound scan covers the liver region. To this end, when constructing an initial scanning path for the object to be scanned based on the liver region, the scanning range of the liver region can be firstly based on the scanning range, and then the initial scanning path can be constructed for the object to be scanned based on the scanning range, wherein the scanning range can be the liver region, or the smallest right-angle trapezoidal region including the liver region, etc. Here, the liver region is taken as an example of the scanning range for explanation.

[0068] Exemplarily, constructing an initial scanning path for the object to be scanned based on the liver region specifically includes:

[0069] Taking a vertex of a region in the liver region as a starting point, constructing a transverse scanning path in a round-trip scanning manner;

[0070] Taking a vertex of a region in the liver region as a starting point, constructing a longitudinal scanning path in a round-trip scanning manner;

[0071] Taking the xiphoid process as the starting point, a lateral scanning path is constructed along the lower edge of the lateral ribs;

[0072] The path set consisting of the transverse scanning path, the longitudinal scanning path and the side scanning path is used as the initial scanning path of the object to be scanned.

[0073] Specifically, each path point in the transverse scanning path, the longitudinal scanning path, and the lateral scanning path includes position information, wherein the position information includes the x-direction coordinate, the y-direction coordinate, and the z-direction coordinate. That is, the position information of each path point can be expressed as (x, y, z). The transverse scanning path and the longitudinal scanning path are used to observe liver lesions, and the lateral scanning path is used to evaluate the condition of liver blood vessels. The starting position of the lateral scanning path is the xiphoid process position, and the lateral scanning path starts from the xiphoid process position and extends along the lower edge of the right rib to the lowest end of the right rib.

[0074] Furthermore, the starting point of the transverse scanning path is a regional vertex of the liver region, and the starting point of the longitudinal scanning path is also a regional vertex of the liver region, and the starting point of the transverse scanning path and the starting point of the longitudinal scanning path may be different, for example, the starting point of the transverse scanning path is the upper left vertex of the liver region, and the starting point of the longitudinal scanning path is the upper right vertex of the liver region. Of course, the starting point of the transverse scanning path and the starting point of the longitudinal scanning path can be the same, for example, the starting point of the transverse scanning path and the starting point of the longitudinal scanning path are both the upper left vertex of the right-angled trapezoid region.

[0075] For example: Select the upper right vertex of the liver area as the starting point of the transverse scanning path, and then construct the scanning path from top to bottom in a round-trip scanning manner to obtain the following Figure 2 Then, the upper left vertex of the liver region is selected as the starting point of the longitudinal scanning path, and then the scanning path is constructed from left to right in a round-trip scanning manner to obtain the following: Figure 2 The longitudinal scanning path shown in . Select the xiphoid process as the starting position, and extend from the xiphoid process along the lower edge of the right rib to the bottom of the right rib to obtain the following Figure 3 Side scan path shown.

[0076] The embodiment of the present application uses a round-trip scanning method to construct a transverse scanning path and a longitudinal scanning path, which can not only minimize the movement distance of the robot arm during actual operation, but also perform a comprehensive scan of the liver area, thereby improving the accuracy of liver detection. At the same time, the embodiment of the present application also constructs a side scanning path, through which the intrahepatic blood vessels can be scanned, so that the intrahepatic blood vessels and liver contours can be detected through the initial scanning path provided by the embodiment of the present application, further improving the comprehensiveness of liver scanning.

[0077] In one implementation, in order to minimize the moving distance of the robot arm during actual operation, when constructing the initial scanning path, the width of the ultrasound probe can also be determined according to the liver area. Based on this, constructing the initial scanning path for the object to be scanned based on the liver area also includes:

[0078] Acquiring the transverse width and the longitudinal width of the liver region;

[0079] Based on the transverse width and the longitudinal width, a width of an ultrasound probe used for performing ultrasound examination on the object to be scanned is determined.

[0080] Specifically, the transverse width is the maximum distance of the liver region in the transverse direction, and the longitudinal width is the maximum distance of the liver region in the longitudinal direction. That is, obtaining the transverse width and longitudinal width of the liver region refers to obtaining the maximum distance of the liver region in the transverse direction and the maximum distance of the liver region in the longitudinal direction.

[0081] After obtaining the transverse width and the longitudinal width, the greatest common divisor of the transverse width and the longitudinal width can be obtained, and then the ultrasonic probe width can be determined based on the greatest common divisor. For example, the greatest common divisor can be used as the ultrasonic probe width, or 1 / n of the greatest common divisor can be used as the ultrasonic probe width, where n is a positive integer greater than 1. Of course, in practical applications, when determining the ultrasonic probe width based on the greatest common divisor, the ultrasonic probe width can be determined based on the greatest common divisor first, and then it can be detected whether there is an ultrasonic probe with the ultrasonic probe width. If there is no ultrasonic probe with the ultrasonic probe width, then 1 / 2, 1 / 3, ..., 1 / n can be used, where n is a positive integer, until there is an ultrasonic probe with the ultrasonic probe width, or there is an ultrasonic probe whose difference with the ultrasonic probe width meets the preset requirements. In addition, when the greatest common divisor is obtained and the ultrasonic probe widths of all ultrasonic probes are greater than the greatest common divisor, the greatest common divisor can be expanded by n times, where n is a positive integer, and then the ultrasonic probe width can be determined based on 1 / n of the greatest common divisor. For example, Figure 2 In the liver region shown, the ultrasound probe width can be 5.5 cm.

[0082] Further, after determining the width of the ultrasound probe, the first distance between two adjacent rows of transverse scanning paths and the second distance between two adjacent rows of longitudinal scanning paths can be determined according to the width of the ultrasound probe, so that both the transverse scanning path and the longitudinal scanning path can cover the liver area and will not repeat too many repeated areas. In a specific implementation, the first distance between two adjacent rows of transverse scanning paths in the transverse scanning path is equal to the ultrasound probe width; the second distance between two adjacent rows of longitudinal scanning paths in the longitudinal scanning path is equal to the ultrasound probe width. Of course, in actual applications, the first distance between two adjacent rows of transverse scanning paths in the transverse scanning path and the second distance between two adjacent rows of longitudinal scanning paths in the longitudinal scanning path are both less than the width of the ultrasound probe, and the difference between the ultrasound probe width and the first distance and the difference between the ultrasound probe width and the second distance both meet preset conditions (for example, less than a preset difference threshold, etc.).

[0083] S40. Based on the normal vector of each surface point in the human body surface, determine the probe posture corresponding to each path point in the initial scanning path, and add the probe posture to the path point to obtain the target scanning path of the object to be scanned.

[0084] Specifically, the probe posture includes the rotation angle of the x-axis, the rotation angle of the y-axis and the rotation angle of the z-axis, wherein the rotation angle of the x-axis can be expressed as the Euler angle r x To describe, the rotation angle of the y-axis can be expressed as the Euler angle r y To describe, the rotation angle of the z-axis can be expressed as the Euler angle r zTo describe this, the probe posture can be expressed as (r x , r y , r z ), then when the probe posture is added to the path point, the path point can be expressed as (x, y, z, r x , r y , r z ). The embodiment of the present application adds the probe posture in the path point, so that the probe can be kept in contact with the skin at each path point, thereby improving the quality of the ultrasound image.

[0085] In one implementation, determining the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface specifically includes:

[0086] For each path point in the initial scanning path, determining the probe coordinate system corresponding to the path point based on the normal vector of the corresponding surface point and the moving direction of the path point;

[0087] A rotation matrix between the probe coordinate system and a global coordinate system of the liver region is determined, and a probe posture corresponding to the path point is determined based on the rotation matrix.

[0088] Specifically, the probe coordinate system is the coordinate system of the ultrasound probe when scanning at the path point, that is, the probe coordinate system is the local coordinate system of the path point. The global coordinate system is the global coordinate system of the liver region, and the rotation matrix is ​​the transformation relationship between the local coordinate system of the path point and the global coordinate system of the liver region.

[0089] To this end, we can first determine the rotation matrix based on the angle between the probe coordinate system and the global coordinate system. The rotation order is ZYX. The rotation matrix can be expressed as:

[0090]

[0091] Where R represents the rotation matrix, R z (γ) represents the rotation γ around the Z axis, γ represents the rotation angle, R y (β) represents the rotation β around the Y axis, β represents the nutation angle, R x (α) represents a rotation of α around the X-axis, and α represents the precession angle.

[0092] Secondly, the rotation matrix can also be obtained from the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of the probe coordinate system. Correspondingly, the rotation matrix can be expressed as:

[0093] R = [X l Y l Z l ]

[0094] Among them, Xl Represents the vector coordinate of the X axis of the local coordinate system in the global coordinate system, Y l Represents the vector coordinate of the Y axis of the local coordinate system in the global coordinate system, Z l The vector coordinate representing the Z axis of the local coordinate system in the global coordinate system.

[0095] Finally, the Euler angles γ, β, and α can be extracted based on the two expressions of the rotation matrix, where the formulas for γ, β, and α are:

[0096] γ=atan2(R 21 ,R 11 )

[0097] β=-asin(R 31 )

[0098] α=atan2(R 32 ,R 33 )

[0099] Among them, atan2(·1,·2) is a function in C++. The method of atan2(·1,·2) is to use atan2(·2 / ·1) when the absolute value of ·1 is larger than the absolute value of ·2, and to use atan2(·1 / ·2) otherwise. In this way, the global coordinate system can be obtained by rotating r in the rotation order of ZYX. x , r y , r z angle, the ultrasound probe can be placed closer to the skin, and combined with position planning, the robot arm position (x, y, z, r) can be determined. x , r y , r z ).

[0100] In one implementation, determining the probe coordinate system corresponding to the path point based on the normal vector of the corresponding surface point and the moving direction of the path point specifically includes:

[0101] Obtain the surface point corresponding to the path point, and use the normal vector direction of the surface point as the Z-axis direction of the path point;

[0102] Reading the moving direction of the path point, and using the moving direction as the X-axis direction of the path point;

[0103] The Y-axis direction of the path point is determined based on the X-axis direction and the Z-axis direction to obtain a probe coordinate system of the ultrasound probe at the path point.

[0104] Specifically, after obtaining the normal vector of the surface point, the direction of the central axis during the probe scanning can be specified. The probe coordinate system is as follows: Figure 4As shown, the center point of the probe scan is taken as the origin, and the normal vector direction of the surface point is taken as the Z-axis direction of the path point to ensure that the probe scans the tissue skin surface at a vertical angle, reducing the angle that may appear during scanning and ensuring image quality. In addition, during the scanning process, the wide surface of the probe should remain perpendicular to the moving position. To this end, the moving direction is taken as the X-axis direction of the path point, and then the remaining Y-axis direction can be obtained by cross product to obtain the probe coordinate system of the path point.

[0105] In summary, this embodiment provides a path planning method for automatic liver scanning, the method comprising obtaining human body point cloud data and human body image of the object to be scanned; determining the human body surface based on the human body point cloud data, and obtaining the normal vector of each surface point in the human body surface; determining the liver area of ​​the object to be scanned based on the human body image, and constructing an initial scanning path for the object to be scanned based on the liver area; determining the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and adding the probe posture to the path point to obtain the target scanning path of the object to be scanned. The present application firstly makes the scanning path more reasonable through position planning, which can not only detect the liver parenchyma but also the intrahepatic blood vessels, and then uses posture planning to make the probe fit the skin at a smaller angle to improve the quality of ultrasonic images.

[0106] Based on the above-mentioned path planning method for automatic liver scanning, this embodiment provides a path planning device for automatic liver scanning, such as Figure 5 As shown, the path planning device for automatic liver scanning specifically includes:

[0107] An acquisition module 100 is used to acquire human body point cloud data and human body image of the object to be scanned;

[0108] A determination module 200 is used to determine a human body surface based on the human body point cloud data, and obtain a normal vector of each surface point in the human body surface;

[0109] A construction module 300 is used to determine the liver region of the object to be scanned based on the human body image, and to construct an initial scanning path for the object to be scanned based on the liver region;

[0110] The adding module 400 is used to determine the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and add the probe posture to the path point to obtain the target scanning path of the object to be scanned.

[0111] Based on the above-mentioned path planning method for automatic liver scanning, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the path planning method for automatic liver scanning as described in the above-mentioned embodiment.

[0112] Based on the above-mentioned path planning method for automatic liver scanning, the present application also provides a terminal device, such as Figure 6 As shown, it includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0113] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0114] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0115] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0116] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A path planning method for automatic liver scanning, characterized in that: The path planning method for automatic liver scanning specifically includes: Obtaining human body point cloud data and human body image of the object to be scanned; Determine a human body surface based on the human body point cloud data, and obtain a normal vector of each surface point in the human body surface; Determine a liver region of the object to be scanned based on the human body image, and construct an initial scanning path for the object to be scanned based on the liver region; Based on the normal vector of each surface point in the human body surface, the probe posture corresponding to each path point in the initial scanning path is determined, and the probe posture is added to the path point to obtain the target scanning path of the object to be scanned.

2. The path planning method for automatic liver scanning according to claim 1, characterized in that: Determining the liver region of the object to be scanned based on the human body image specifically includes: Inputting the human body image into a trained human body segmentation model, and outputting the rib position of the object to be scanned through the human body segmentation model; According to the correspondence between the rib positions and the liver, the liver region of the object to be scanned is determined based on the rib positions of the object to be scanned.

3. The path planning method for automatic liver scanning according to claim 1, characterized in that: The constructing of an initial scanning path for the object to be scanned based on the liver region specifically includes: Taking a vertex of a region in the liver region as a starting point, constructing a transverse scanning path in a round-trip scanning manner; Taking a vertex of a region in the liver region as a starting point, constructing a longitudinal scanning path in a round-trip scanning manner; Taking the xiphoid process as the starting point, a lateral scanning path is constructed along the lower edge of the lateral ribs; The path set consisting of the transverse scanning path, the longitudinal scanning path and the side scanning path is used as the initial scanning path of the object to be scanned.

4. The path planning method for automatic liver scanning according to claim 3, characterized in that: The constructing an initial scanning path for the object to be scanned based on the liver region further comprises: Acquiring the transverse width and the longitudinal width of the liver region; Based on the transverse width and the longitudinal width, a width of an ultrasound probe used for performing ultrasound examination on the object to be scanned is determined.

5. The path planning method for automatic liver scanning according to claim 4, characterized in that: A first distance between two adjacent rows of transverse scanning paths in the transverse scanning path is equal to the width of the ultrasonic probe; a second distance between two adjacent rows of longitudinal scanning paths in the longitudinal scanning path is equal to the width of the ultrasonic probe.

6. The path planning method for automatic liver scanning according to claim 1, characterized in that: The step of determining the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface specifically includes: For each path point in the initial scanning path, determining the probe coordinate system corresponding to the path point based on the normal vector of the corresponding surface point and the moving direction of the path point; A rotation matrix between the probe coordinate system and a global coordinate system of the liver region is determined, and a probe posture corresponding to the path point is determined based on the rotation matrix.

7. The path planning method for automatic liver scanning according to claim 6, characterized in that: Determining the probe coordinate system corresponding to the path point based on the normal vector of the corresponding surface point and the moving direction of the path point specifically includes: Obtain the surface point corresponding to the path point, and use the normal vector direction of the surface point as the Z-axis direction of the path point; Reading the moving direction of the path point, and using the moving direction as the X-axis direction of the path point; The Y-axis direction of the path point is determined based on the X-axis direction and the Z-axis direction to obtain a probe coordinate system of the ultrasound probe at the path point.

8. A path planning device for automatic liver scanning, characterized in that: The path planning device for automatic liver scanning specifically includes: An acquisition module is used to acquire human body point cloud data and human body image of the object to be scanned; A determination module, used to determine a human body surface based on the human body point cloud data, and obtain a normal vector of each surface point in the human body surface; A construction module, used for determining a liver region of the object to be scanned based on the human body image, and constructing an initial scanning path for the object to be scanned based on the liver region; An adding module is used to determine the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and add the probe posture to the path point to obtain the target scanning path of the object to be scanned.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the path planning method for automatic liver scanning as described in any one of claims 1-7.

10. A terminal device, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the path planning method for automatic liver scanning as described in any one of claims 1-7 are implemented.

Images