A method, apparatus, equipment, and medium for automated liver scanning path planning

Abstract

This application discloses a method, apparatus, device, and medium for automatic liver scanning path planning. The method includes acquiring human body point cloud data, reconstructing the human body surface based on the point cloud data, determining the position information of the liver probe based on image data of the rib positions, and estimating the probe posture based on the position information and the human body surface to form a planned path for automatic liver scanning. This application first uses position planning to make the scanning path more reasonable, enabling the detection of not only liver parenchyma but also intrahepatic vascular diseases. Then, it uses pose planning to allow for a smaller angle when the probe adheres to the skin, improving ultrasound image quality.

Description

Technical Field

[0001] This application relates to the field of biomedical technology, and in particular to a path planning method, apparatus, equipment and medium for automated liver scanning. Background Technology

[0002] Medical ultrasound, with its non-invasive, real-time, and efficient characteristics, is gradually becoming the preferred technology for liver disease screening and detection. Currently, clinical ultrasound examinations are performed by doctors holding the probe, and the long-term strain on the hands and poor posture can lead to carpal tunnel syndrome, tendinitis, and neck and shoulder discomfort. Robotic arms, as an automated tool, can greatly reduce the workload of doctors during scans and also help standardize liver scans, reducing operational differences between doctors.

[0003] The robotic arm can operate according to the preset scanning path and parameters. However, the existing path planning for the liver area lacks good planning for the pose angle, which makes it difficult for the probe to fit tightly against the skin during actual operation, resulting in a significant decrease in the quality of the obtained ultrasound images.

[0004] Therefore, the existing technology still needs to be improved and enhanced. Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a path planning method, device, equipment and medium for automatic liver scanning, which addresses the shortcomings of the prior art.

[0006] To address the aforementioned technical problems, the first aspect of this application provides a path planning method for automatic liver scanning, wherein the automatic liver scanning path planning method specifically includes:

[0007] Acquire human point cloud data and human images of the object to be scanned;

[0008] The human body surface is determined based on the human body point cloud data, and the normal vector of each surface point in the human body surface is obtained;

[0009] The liver region of the object to be scanned is determined based on the human image, and an initial scanning path is constructed for the object to be scanned based on the liver region;

[0010] Based on the normal vector of each surface point in the human body surface, the probe posture corresponding to each path point in the initial scanning path is determined, and the probe posture is added to the path point to obtain the target scanning path of the object to be scanned.

[0011] The path planning method for automatic liver scanning, wherein determining the liver region of the object to be scanned based on the human image specifically includes:

[0012] The human image is input into a trained human segmentation model, and the human segmentation model outputs the rib location of the object to be scanned.

[0013] Based on the correspondence between the rib positions and the liver, the liver region of the subject to be scanned is determined according to the rib positions of the subject to be scanned.

[0014] The path planning method for automatic liver scanning, wherein constructing an initial scanning path for the object to be scanned based on the liver region specifically includes:

[0015] Starting from a vertex in the liver region, a transverse scanning path is constructed using a round-trip scanning method;

[0016] Starting from a vertex in the liver region, a longitudinal scanning path is constructed using a round-trip scanning method;

[0017] Starting from the xiphoid process, construct a lateral scanning path along the lower edge of the lateral ribs;

[0018] The path set consisting of the transverse scanning path, longitudinal scanning path, and side scanning path is used as the initial scanning path for the object to be scanned.

[0019] The path planning method for automatic liver scanning, wherein constructing an initial scanning path for the object to be scanned based on the liver region further includes:

[0020] Obtain the horizontal and vertical widths of the liver region;

[0021] Based on the horizontal and vertical widths, the width of the ultrasound probe used for performing ultrasound examination on the object to be scanned is determined.

[0022] The method for automatic liver scanning path planning is described above, wherein the first distance between two adjacent rows of transverse scanning paths is equal to the width of the ultrasound probe; and the second distance between two adjacent rows of longitudinal scanning paths is equal to the width of the ultrasound probe.

[0023] The path planning method for automatic liver scanning, wherein determining the probe pose corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface specifically includes:

[0024] For each path point in the initial scanning path, the probe coordinate system corresponding to the path point is determined based on the normal vector of the corresponding surface point and the movement direction of the path point.

[0025] Determine the rotation matrix between the probe coordinate system and the global coordinate system of the liver region, and determine the probe pose corresponding to the path point based on the rotation matrix.

[0026] The path planning method for automated liver scanning, wherein determining the probe coordinate system corresponding to the path point based on the normal vector of the corresponding surface point and the movement direction of the path point specifically includes:

[0027] Obtain the surface point corresponding to the path point, and take the normal vector direction of the surface point as the Z-axis direction of the path point;

[0028] Read the movement direction of the path point and use the movement direction as the X-axis direction of the path point;

[0029] The Y-axis direction of the path point is determined based on the X-axis and Z-axis directions to obtain the probe coordinate system of the ultrasonic probe at the path point.

[0030] A second aspect of this application provides a path planning device for automatic liver scanning, wherein the automatic liver scanning path planning device specifically includes:

[0031] The acquisition module is used to acquire human point cloud data and human images of the object to be scanned.

[0032] The determination module is used to determine the human body surface based on the human body point cloud data, and to obtain the normal vector of each surface point in the human body surface;

[0033] A construction module is used to determine the liver region of the object to be scanned based on the human image, and to construct an initial scanning path for the object to be scanned based on the liver region;

[0034] An addition module is used to determine the probe pose corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and add the probe pose to the path point to obtain the target scanning path of the object to be scanned.

[0035] A third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the path planning method for automatic liver scanning as described above.

[0036] A fourth aspect of this application provides a terminal device, which includes: a processor and a memory;

[0037] The memory stores a computer-readable program that can be executed by the processor;

[0038] When the processor executes the computer-readable program, it implements the steps in the path planning method for automatic liver scanning as described above.

[0039] Beneficial Effects: Compared with existing technologies, this application provides a path planning method, apparatus, device, and medium for automatic liver scanning. The method includes acquiring human point cloud data and human images of the object to be scanned; determining the human body surface based on the human point cloud data and acquiring the normal vector of each surface point in the human body surface; determining the liver region of the object to be scanned based on the human body image and constructing an initial scanning path for the object to be scanned based on the liver region; determining the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and adding the probe posture to the path point to obtain the target scanning path for the object to be scanned. This application first uses position planning to make the scanning path more reasonable, which can detect not only liver parenchyma but also intrahepatic blood vessels. Then, it uses pose planning to make the probe fit with the skin at a smaller angle, improving the quality of ultrasound images. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart of a path planning method for automatic liver scanning provided in an embodiment of this application.

[0042] Figure 2 This is a schematic diagram of the transverse and longitudinal scanning paths.

[0043] Figure 3 This is a schematic diagram of the side scan path.

[0044] Figure 4 This is a schematic diagram of the probe coordinate system.

[0045] Figure 5 A schematic diagram of the path planning device for automatic liver scanning provided in this application embodiment.

[0046] Figure 6 A schematic block diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0047] This application provides a method, apparatus, device, and medium for automatic liver scanning path planning. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0048] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0049] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0050] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0051] Research has shown that medical ultrasound, with its non-invasive, real-time, and efficient characteristics, is gradually becoming the preferred technology for liver disease screening and detection. Currently, clinical ultrasound examinations are performed by doctors holding the probe, and the long-term strain on the hands and poor posture can lead to carpal tunnel syndrome, tendinitis, and neck and shoulder discomfort. Robotic arms, as an automated tool, can greatly reduce the workload of doctors during scans and also help standardize liver scans, reducing operational differences between doctors.

[0052] The robotic arm can operate according to the preset scanning path and parameters. However, the existing path planning for the liver area lacks good planning for the pose angle, which makes it difficult for the probe to fit tightly against the skin during actual operation, resulting in a significant decrease in the quality of the obtained ultrasound images.

[0053] To address the aforementioned issues, this application embodiment acquires human point cloud data and a human image of the object to be scanned; determines a human surface based on the human point cloud data and obtains the normal vector of each surface point in the human surface; determines the liver region of the object to be scanned based on the human image, and constructs an initial scanning path for the object based on the liver region; determines the probe pose corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human surface, and adds the probe pose to the path point to obtain the target scanning path for the object to be scanned. This application first uses position planning to make the scanning path more reasonable, enabling detection not only of liver parenchyma but also of intrahepatic blood vessels. Then, it uses pose planning to allow for a smaller angle when the probe is in contact with the skin, improving the quality of the ultrasound image.

[0054] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0055] This embodiment provides a path planning method for automated liver scanning, such as... Figure 1 As shown, the method includes:

[0056] S10. Obtain the human point cloud data and human image of the object to be scanned.

[0057] Specifically, the human point cloud data includes point cloud data of the human body surface corresponding to the liver region, and the human body image includes the liver region. The human point cloud data is obtained by scanning the object to be scanned, for example, by scanning the object with a Kinect camera. The human body image is the RGB-D image data of the object to be scanned, for example, by capturing an image of the object with a depth camera. Furthermore, the human point cloud data and human body image can be acquired in real-time before the automatic liver scan, or they can be pre-collected and stored locally, or sent from an external device, etc.

[0058] S20. Determine the human body surface based on the human body point cloud data, and obtain the normal vector of each surface point in the human body surface.

[0059] Specifically, the human body surface is obtained through human body point cloud data reconstruction, whereby the human body surface is a 3D human body model. That is, after acquiring the human body point cloud data, a 3D reconstruction is performed based on the human body point cloud data to obtain the 3D human body model. After acquiring the human body surface, the normal vector of each target surface point on the human body surface is obtained, where each target surface point corresponds to a point cloud data, and the point cloud data corresponding to all target surface points constitute the human body point cloud data of the object to be scanned. In other words, the human body surface is processed to obtain the normal vector of the surface position of each point cloud data in the human body point cloud data of the object to be scanned. For example, the human body surface is processed using the PCL library to obtain the normal vector of the surface position of each point cloud data in the human body point cloud data of the object to be scanned.

[0060] S30. Determine the liver region of the object to be scanned based on the human body image, and construct an initial scanning path for the object to be scanned based on the liver region.

[0061] Specifically, the liver region is the image region of the liver in a human body image. It can be obtained by image segmentation of the human body image, or by first identifying the position of the ribs in the human body image and then determining the liver region of the object to be scanned based on the positional relationship between the ribs and the liver.

[0062] For example, determining the liver region of the object to be scanned based on the human image specifically includes:

[0063] The human image is input into a trained human segmentation model, and the human segmentation model outputs the rib location of the object to be scanned.

[0064] Based on the correspondence between the rib positions and the liver, the liver region of the subject to be scanned is determined according to the rib positions of the subject to be scanned.

[0065] Specifically, the human segmentation model is a trained deep learning model (e.g., CNN, DNN, etc.) used to identify the rib locations in a human image. In other words, the input to the human segmentation model is a human image, and the output is the rib locations. The rib locations include the positional information of each rib in the human image, and each rib can be marked on the human image.

[0066] The correspondence between the rib locations and the liver is as follows: the upper boundary is located between the 5th and 6th ribs on the right side; the lower boundary of the liver is located at the 12th rib at the level of the right midaxillary line; the lower right boundary of the liver extends from the edge of the ribs to the junction of the 7th and 8th ribs on the right side, and then extends obliquely to the upper left from the junction to the 6th rib on the left side. The left boundary is close to the edge of the liver along the left rib, and the right boundary extends below the nipple. Therefore, after obtaining the rib locations, the liver region of the object to be scanned can be delineated according to the correspondence between the rib locations and the liver, where the liver region is distributed in a right-angled trapezoidal shape.

[0067] Furthermore, after acquiring the liver region, the scanning area to be covered by the ultrasound scan can be determined based on this region; that is, the scanning area to be covered by the ultrasound scan must encompass the liver region. Therefore, when constructing the initial scanning path for the object to be scanned based on the liver region, the scanning area can be determined first, and then the initial scanning path can be constructed based on this scanning area. The scanning area can be the entire liver region, or it can be a minimum right-angled trapezoidal region containing the liver region, etc. Here, we will use the liver region as an example for explanation.

[0068] For example, constructing an initial scanning path for the object to be scanned based on the liver region specifically includes:

[0069] Starting from a vertex in the liver region, a transverse scanning path is constructed using a round-trip scanning method;

[0070] Starting from a vertex in the liver region, a longitudinal scanning path is constructed using a round-trip scanning method;

[0071] Starting from the xiphoid process, construct a lateral scanning path along the lower edge of the lateral ribs;

[0072] The path set consisting of the transverse scanning path, longitudinal scanning path, and side scanning path is used as the initial scanning path for the object to be scanned.

[0073] Specifically, each path point in the transverse, longitudinal, and lateral scanning paths includes location information, which includes x-coordinates, y-coordinates, and z-coordinates. That is, the location information of each path point can be represented as (x, y, z). The transverse and longitudinal scanning paths are used to observe liver lesions, while the lateral scanning path is used to assess the condition of liver vessels. The lateral scanning path starts at the xiphoid process and extends along the lower edge of the right rib to the lowest point of the right rib.

[0074] Furthermore, the starting point of the transverse scanning path is a vertex of the liver region, and the starting point of the longitudinal scanning path is also a vertex of the liver region. The starting points of the transverse and longitudinal scanning paths can be different; for example, the starting point of the transverse scanning path can be the upper left vertex of the liver region, while the starting point of the longitudinal scanning path can be the upper right vertex of the liver region. Alternatively, the starting points of the transverse and longitudinal scanning paths can be the same; for example, both can be the upper left vertex of a right-angled trapezoidal region.

[0075] For example: Select the upper right vertex of the liver region as the starting point of the transverse scanning path, and then construct the scanning path from top to bottom using a round-trip scanning method to obtain the following... Figure 2 The transverse scanning path is shown in the image. Then, the upper left vertex of the liver region is selected as the starting point of the longitudinal scanning path, and the scanning path is constructed from left to right using a reciprocating scanning method to obtain the desired result. Figure 2 The longitudinal scanning path is shown in the image. The xiphoid process is selected as the starting point, and the scan extends along the lower edge of the right rib to the bottom of the right rib, as shown in the image. Figure 3 The side scan path is shown.

[0076] This application embodiment constructs transverse and longitudinal scanning paths using a reciprocating scanning method. This minimizes the robotic arm's movement distance during actual operation while enabling a comprehensive scan of the liver region, thus improving the accuracy of liver detection. Simultaneously, this application embodiment also constructs a lateral scanning path, which allows for the scanning of intrahepatic vessels. Therefore, the initial scanning path provided by this application embodiment can detect intrahepatic vessels and the liver contour, further enhancing the comprehensiveness of liver scanning.

[0077] In one implementation, to minimize the actual movement distance of the robotic arm during operation, the width of the ultrasound probe can be determined based on the liver region when constructing the initial scanning path. Therefore, constructing the initial scanning path for the object to be scanned based on the liver region further includes:

[0078] Obtain the horizontal and vertical widths of the liver region;

[0079] Based on the horizontal and vertical widths, the width of the ultrasound probe used for performing ultrasound examination on the object to be scanned is determined.

[0080] Specifically, the horizontal width is the maximum distance of the liver region in the horizontal direction, and the vertical width is the maximum distance of the liver region in the vertical direction. In other words, obtaining the horizontal and vertical widths of the liver region means obtaining the maximum distance of the liver region in the horizontal direction and the maximum distance in the vertical direction.

[0081] After obtaining the horizontal and vertical widths, their greatest common divisor (GCD) can be obtained. The width of the ultrasound probe can then be determined based on this GCD. For example, the GCD can be used as the ultrasound probe width, or 1 / n of the GCD can be used as the ultrasound probe width, where n is a positive integer greater than 1. In practical applications, when determining the ultrasound probe width based on the GCD, one can first determine the ultrasound probe width using the GCD, then check if an ultrasound probe with that width exists. If not, the width can be increased by 1 / 2, 1 / 3, ..., 1 / n, where n is a positive integer, until an ultrasound probe with that width exists, or if the difference between the width and the GCD meets a preset requirement. Furthermore, if, after obtaining the GCD, the width of all ultrasound probes is greater than the GCD, the GCD can be multiplied by n, where n is a positive integer, and then the ultrasound probe width can be determined by using 1 / n of the GCD. For example... Figure 2 The ultrasound probe width can be 5.5cm for the liver region shown.

[0082] Furthermore, after determining the width of the ultrasound probe, a first distance between two adjacent transverse scanning paths and a second distance between two adjacent longitudinal scanning paths can be determined based on the ultrasound probe width. This ensures that both the transverse and longitudinal scanning paths can cover the liver region without excessive overlap. In one specific implementation, the first distance between two adjacent transverse scanning paths is equal to the ultrasound probe width; the second distance between two adjacent longitudinal scanning paths is equal to the ultrasound probe width. However, in practical applications, both the first distance between two adjacent transverse scanning paths and the second distance between two adjacent longitudinal scanning paths are less than the ultrasound probe width, and the differences between the ultrasound probe width and the first distance, and the differences between the ultrasound probe width and the second distance, satisfy preset conditions (e.g., less than a preset difference threshold).

[0083] S40. Based on the normal vector of each surface point in the human body surface, determine the probe posture corresponding to each path point in the initial scanning path, and add the probe posture to the path point to obtain the target scanning path of the object to be scanned.

[0084] Specifically, the probe orientation includes rotation angles along the x-axis, y-axis, and z-axis. The x-axis rotation angle can be expressed as Euler angles θ. x To describe the rotation angle of the y-axis, we can use Euler angles r. y To describe the rotation angle along the z-axis, we can use Euler angles r. zTo describe this, the probe orientation can be expressed as (r x r y r z If the probe orientation is added to the path point, then the path point can be represented as (x, y, z, r). x r y r z This application embodiment improves ultrasound image quality by adding probe posture within the path points, thus maintaining probe contact with the skin at each path point.

[0085] In one implementation, determining the probe pose corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface specifically includes:

[0086] For each path point in the initial scanning path, the probe coordinate system corresponding to the path point is determined based on the normal vector of the corresponding surface point and the movement direction of the path point.

[0087] Determine the rotation matrix between the probe coordinate system and the global coordinate system of the liver region, and determine the probe pose corresponding to the path point based on the rotation matrix.

[0088] Specifically, the probe coordinate system is the coordinate system of the ultrasound probe when scanning at the path point; that is, the probe coordinate system is the local coordinate system of the path point. The global coordinate system is the global coordinate system of the liver region, and the rotation matrix represents the transformation relationship between the local coordinate system of the path point and the global coordinate system of the liver region.

[0089] Therefore, the rotation matrix can be determined first based on the angle between the probe coordinate system and the global coordinate system, with the rotation order being ZYX. The rotation matrix can be expressed as:

[0090]

[0091] Where R represents the rotation matrix, R z (γ) represents rotation about the Z-axis by γ, where γ represents the rotation angle, and R y (β) represents rotation about the Y-axis by β, β represents the nutation angle, and R x (α) represents rotation α around the X-axis, where α represents the precession angle.

[0092] Secondly, the rotation matrix can also be obtained from the X-axis, Y-axis, and Z-axis coordinates of the probe coordinate system. Accordingly, the rotation matrix can be expressed as:

[0093] R = [X] l Y l Z l ]

[0094] Among them, Xl The X-axis represents the vector coordinate of the local coordinate system in the global coordinate system, and the Y-axis represents the vector coordinate of the local coordinate system. l The Z represents the vector coordinates of the Y-axis in the local coordinate system within the global coordinate system. l This represents the vector coordinates of the Z-axis in the local coordinate system within the global coordinate system.

[0095] Finally, Euler angles γ, β, and α can be extracted from the two expressions of the rotation matrix, where the formulas for γ, β, and α are as follows:

[0096] γ=atan2(R 21 ,R 11 )

[0097] β=-asin(R 31 )

[0098] α=atan2(R 32 ,R 33 )

[0099] Here, `atan2(·1,·2)` is a C++ function. `atan2(·1,·2)` works by using `atan2(·2 / ·1)` when the absolute value of `·1` is greater than the absolute value of `·2`, and vice versa. This allows obtaining the coordinates in the global coordinate system according to the ZYX rotation order, by rotating `r` respectively. x r y r z By adjusting the angle, the ultrasound probe can be positioned closer to the skin. Combined with position planning, the final pose of the robotic arm (x, y, z, r) can be determined. x r y r z ).

[0100] In one implementation, determining the probe coordinate system corresponding to the path point based on the normal vector of the corresponding surface point and the movement direction of the path point specifically includes:

[0101] Obtain the surface point corresponding to the path point, and take the normal vector direction of the surface point as the Z-axis direction of the path point;

[0102] Read the movement direction of the path point and use the movement direction as the X-axis direction of the path point;

[0103] The Y-axis direction of the path point is determined based on the X-axis and Z-axis directions to obtain the probe coordinate system of the ultrasonic probe at the path point.

[0104] Specifically, after obtaining the normal vectors of the surface points, the orientation of the central axis during probe scanning can be defined, and the probe coordinate system is as follows: Figure 4As shown, with the probe scanning center point as the origin, the normal vector direction of the curved surface point is taken as the Z-axis direction of the path point. This ensures that the probe scans the tissue and skin surface at a perpendicular angle, reducing possible angles during scanning and guaranteeing image quality. Furthermore, during the scanning process, the probe's wide face should remain perpendicular to the moving position. Therefore, the moving direction is taken as the X-axis direction of the path point, and the remaining Y-axis direction is obtained through a cross product, thus deriving the probe coordinate system of the path point.

[0105] In summary, this embodiment provides a path planning method for automatic liver scanning. The method includes acquiring human point cloud data and human images of the object to be scanned; determining the human body surface based on the human point cloud data and acquiring the normal vector of each surface point in the human body surface; determining the liver region of the object to be scanned based on the human body image and constructing an initial scanning path for the object to be scanned based on the liver region; determining the probe pose corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and adding the probe pose to the path point to obtain the target scanning path for the object to be scanned. This application first uses position planning to make the scanning path more reasonable, which can not only detect liver parenchyma but also detect intrahepatic blood vessels. Then, it uses pose planning to make the probe fit with the skin at a smaller angle, thereby improving the quality of ultrasound images.

[0106] Based on the above-mentioned path planning method for automatic liver scanning, this embodiment provides a path planning device for automatic liver scanning, such as... Figure 5 As shown, the automatic liver scanning path planning device specifically includes:

[0107] The acquisition module 100 is used to acquire human point cloud data and human images of the object to be scanned.

[0108] The determination module 200 is used to determine the human body surface based on the human body point cloud data, and to obtain the normal vector of each surface point in the human body surface;

[0109] The construction module 300 is used to determine the liver region of the object to be scanned based on the human body image, and to construct an initial scanning path for the object to be scanned based on the liver region;

[0110] The addition module 400 is used to determine the probe posture corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and add the probe posture to the path point to obtain the target scanning path of the object to be scanned.

[0111] Based on the above-described automatic liver scanning path planning method, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the automatic liver scanning path planning method described in the above embodiment.

[0112] Based on the above-mentioned path planning method for automated liver scanning, this application also provides a terminal device, such as... Figure 6 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0113] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0114] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0115] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0116] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A path planning method for automatic liver scanning, characterized in that, The aforementioned automatic liver scanning path planning method specifically includes: Acquire human point cloud data and human images of the object to be scanned; The human body surface is determined based on the human body point cloud data, and the normal vector of each surface point in the human body surface is obtained; The liver region of the object to be scanned is determined based on the human image, and an initial scanning path is constructed for the object to be scanned based on the liver region; Based on the normal vector of each surface point in the human body surface, the probe pose corresponding to each path point in the initial scanning path is determined, and the probe pose is added to the path point to obtain the target scanning path of the object to be scanned. Specifically, constructing an initial scanning path for the object to be scanned based on the liver region includes: Starting from a vertex in the liver region, a transverse scanning path is constructed using a round-trip scanning method; Starting from a vertex in the liver region, a longitudinal scanning path is constructed using a round-trip scanning method; Starting from the xiphoid process, construct a lateral scanning path along the lower edge of the lateral ribs; The path set consisting of the transverse scanning path, longitudinal scanning path, and side scanning path is used as the initial scanning path for the object to be scanned. The step of determining the probe pose corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface specifically includes: For each path point in the initial scanning path, obtain the surface point corresponding to the path point, and take the normal vector direction of the surface point as the Z-axis direction of the path point. Read the movement direction of the path point and take the movement direction as the X-axis direction of the path point. Determine the Y-axis direction of the path point based on the X-axis direction and the Z-axis direction to obtain the probe coordinate system of the ultrasound probe at the path point. Determine the rotation matrix between the probe coordinate system and the global coordinate system of the liver region, and determine the probe pose corresponding to the path point based on the rotation matrix.

2. The path planning method for automatic liver scanning according to claim 1, characterized in that, The step of determining the liver region of the subject to be scanned based on the human image specifically includes: The human image is input into a trained human segmentation model, and the human segmentation model outputs the rib location of the object to be scanned. Based on the correspondence between the rib positions and the liver, the liver region of the subject to be scanned is determined according to the rib positions of the subject to be scanned.

3. The path planning method for automatic liver scanning according to claim 1, characterized in that, The step of constructing an initial scanning path for the object to be scanned based on the liver region also includes: Obtain the horizontal and vertical widths of the liver region; Based on the horizontal and vertical widths, the width of the ultrasound probe used for performing ultrasound examination on the object to be scanned is determined.

4. The path planning method for automatic liver scanning according to claim 3, characterized in that, The first distance between two adjacent rows of transverse scanning paths in the transverse scanning path is equal to the width of the ultrasound probe; the second distance between two adjacent rows of longitudinal scanning paths in the longitudinal scanning path is equal to the width of the ultrasound probe.

5. A path planning device for automatic liver scanning, characterized in that, The aforementioned automatic liver scanning path planning device specifically includes: The acquisition module is used to acquire human point cloud data and human images of the object to be scanned. The determination module is used to determine the human body surface based on the human body point cloud data, and to obtain the normal vector of each surface point in the human body surface; A construction module is used to determine the liver region of the object to be scanned based on the human image, and to construct an initial scanning path for the object to be scanned based on the liver region; An addition module is used to determine the probe pose corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface, and add the probe pose to the path point to obtain the target scanning path of the object to be scanned. Specifically, constructing an initial scanning path for the object to be scanned based on the liver region includes: Starting from a vertex in the liver region, a transverse scanning path is constructed using a round-trip scanning method; Starting from a vertex in the liver region, a longitudinal scanning path is constructed using a round-trip scanning method; Starting from the xiphoid process, construct a lateral scanning path along the lower edge of the lateral ribs; The path set consisting of the transverse scanning path, longitudinal scanning path, and side scanning path is used as the initial scanning path for the object to be scanned. The step of determining the probe pose corresponding to each path point in the initial scanning path based on the normal vector of each surface point in the human body surface specifically includes: For each path point in the initial scanning path, obtain the surface point corresponding to the path point, and take the normal vector direction of the surface point as the Z-axis direction of the path point. Read the movement direction of the path point and take the movement direction as the X-axis direction of the path point. Determine the Y-axis direction of the path point based on the X-axis direction and the Z-axis direction to obtain the probe coordinate system of the ultrasound probe at the path point. Determine the rotation matrix between the probe coordinate system and the global coordinate system of the liver region, and determine the probe pose corresponding to the path point based on the rotation matrix.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the path planning method for automatic liver scanning as described in any one of claims 1-4.

7. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps of the path planning method for automatic liver scanning as described in any one of claims 1-4.

Images