Scanning trajectory generation method and device, surgical robot, and electronic device

Abstract

The application discloses a scanning track generation method and device, a surgical robot and electronic equipment. The method comprises the following steps: acquiring a first three-dimensional CT image, and the first three-dimensional CT image comprises a target object. A skin region and an organ region are determined from the first three-dimensional CT image, and the organ region corresponds to a target organ of the target object. The coordinates of the pixels in the organ region in a pixel coordinate system of the first three-dimensional CT image are determined, and a first coordinate set is obtained. A first vector is determined based on the first coordinate set. A target plane is obtained based on the first vector and the organ region. A target region corresponding to the organ region is determined from the skin region based on the coordinates of the organ region in the pixel coordinate system. A scanning track is generated based on the intersection line of the target plane and the target region, and the scanning track is a track for an ultrasonic scanning device to scan the target organ of the target object. The scanning track generated based on the scanning track generation method provided in the application can improve the quality of the scanning track.

Description

Technical Field

[0001] This application relates to the field of medical imaging technology, and in particular to a scanning trajectory generation method and apparatus, a surgical robot, and an electronic device. Background Technology

[0002] By using an ultrasound scanning device to scan an object, ultrasound images of the organs inside the object can be obtained, and information about the organs inside the object can be obtained based on the ultrasound images.

[0003] Current technology relies on human experience to determine the scanning trajectory of ultrasound scanning equipment, but this can easily lead to low quality of the scanning trajectory. Summary of the Invention

[0004] This application provides a scanning trajectory generation method and apparatus, a surgical robot, and an electronic device to improve the quality of scanning trajectories in ultrasound scanning equipment.

[0005] Firstly, a method for generating a scanning trajectory is provided, the method comprising: Acquire a first three-dimensional computed tomography (CT) image, wherein the first three-dimensional CT image includes the target object; Skin regions and organ regions are determined from the first three-dimensional CT image, wherein the organ regions correspond to the target organs of the target object; Determine the coordinates of the pixels in the organ region in the pixel coordinate system of the first three-dimensional CT image to obtain a first coordinate set; Based on the first set of coordinates, a first vector is determined; wherein, the coordinates of pixels in the organ region are projected onto the first vector, and the resulting projection component has the greatest degree of dispersion. Based on the first vector and the organ region, a target plane is obtained, wherein the normal vector of the target plane is the first vector, and the target plane passes through a reference point within the organ region; Based on the coordinates of the organ region in the pixel coordinate system, a target region corresponding to the organ region is determined from the skin region; A scanning trajectory is generated based on the intersection of the target plane and the target region. The scanning trajectory is the trajectory of the ultrasound scanning device scanning the target organ of the target object.

[0006] In this aspect, after acquiring a first 3D CT image, the generating device determines skin and organ regions from the first 3D CT image. Then, it determines the coordinates of pixels in the organ region in the pixel coordinate system of the first 3D CT image, obtaining a first coordinate set. Based on the first coordinate set, it determines a first vector, wherein the direction indicated by the first vector is highly probable to be the direction of the long axis of the target organ. Based on the first vector and the organ region, it obtains a target plane. Based on the coordinates of the organ region in the pixel coordinate system, it determines the target region corresponding to the organ region from the skin region. Based on the intersection of the target plane and the target region, it generates a scanning trajectory. When the ultrasound scanning device scans the target organ of the target object based on the scanning trajectory, it can obtain more information about the target organ, thereby improving the quality of the scanning trajectory. Furthermore, since no human intervention is required, labor costs can be reduced.

[0007] In conjunction with any embodiment of this application, before determining the target region corresponding to the organ region from the skin region based on the coordinates of the organ region in the pixel coordinate system, the method further includes: Determine the circumscribed cuboid of the organ region; The step of determining the target region corresponding to the organ region from the skin region based on the coordinates of the organ region in the pixel coordinate system includes: Based on the coordinates of the organ region in the pixel coordinate system, the coordinates of the points in the circumscribed cuboid in the pixel coordinate system are determined to obtain the second coordinate set. Determine the projection coordinates of the coordinates in the second coordinate set onto the coronal plane of the target object to obtain the third coordinate set; Based on the coordinates in the third coordinate set, a coordinate range is obtained. The maximum value of the coordinate range is determined based on the maximum value of the coordinates in the third coordinate set, and the minimum value of the coordinate range is determined based on the minimum value of the coordinates in the third coordinate set.

[0008] Based on the coordinate range, the target region is determined from the skin region, the projection coordinates of the target coordinates on the coronal plane are within the coordinate range, and the target coordinates are the coordinates of the pixels in the target region under the pixel coordinates.

[0009] In conjunction with any embodiment of this application, the method further includes: The first three-dimensional CT image is input into the first model to obtain a predicted trajectory, which is the trajectory of the ultrasound scanning device scanning the target object. The loss of the first model is obtained based on the difference between the predicted trajectory and the scanning trajectory; Based on the loss of the first model, the parameters of the first model are updated to obtain a trained first model. The trained first model is used to generate the trajectory of the target organ of the object scanned by the ultrasound scanning device based on the three-dimensional CT image of the object.

[0010] In any embodiment of this application, generating a scanning trajectory based on the intersection of the target plane and the target region includes: Determine the coordinates of the pixels in the pixel coordinate system along the intersection line of the target plane and the target region to obtain a fourth coordinate set; Based on the coordinates in the fourth coordinate set, the equation of the intersection line is obtained; Based on the equation of the intersection line, the equation of the scanning trajectory is generated; The loss of the first model is obtained based on the difference between the predicted trajectory and the scanning trajectory, including: Based on the equation of the scanning trajectory, the curvature information of the scanning trajectory is obtained; Based on the curvature information of the scan trajectory and the curvature information of the predicted trajectory, a curvature difference is obtained, which is the difference between the curvature of the scan trajectory and the curvature of the predicted trajectory. Based on the curvature difference, the loss of the first model is obtained, and the curvature difference is positively correlated with the loss of the first model.

[0011] In conjunction with any embodiment of this application, before determining the first vector based on the first coordinate set, the method further includes: The organ region is input into the second model so that the second model determines the direction of the long axis of the target organ, thereby obtaining a second vector; The organ region is divided into n sub-regions along the second vector, where n is an integer greater than 1; The step of determining the first vector based on the first coordinate set includes: Based on the first coordinate set and the n sub-regions, n fifth coordinate sets are obtained, where each fifth coordinate set is a set of coordinates of pixels in the sub-regions. Principal component analysis (PCA) is performed on each of the n fifth coordinate sets to obtain n third vectors; wherein each third vector corresponds one-to-one with a fifth coordinate set; the projection of the coordinates in the fifth coordinate set onto the third vectors yields the most discrete projection components; The first vector is obtained based on the sum of the n third vectors.

[0012] In any embodiment of this application, obtaining the first vector based on the sum of the n third vectors includes: The sum of the n third vectors is determined to obtain the fourth vector; Determine the angles between the n third vectors and the fourth vector to obtain the angles between the n vectors; If there is an abnormal angle among the n vector angles, the vector among the n third vectors that corresponds to the abnormal angle is determined to be an abnormal vector, and the abnormal angle is an angle that is greater than or equal to a second threshold. The first vector is obtained by summing the vectors other than the abnormal vector among the n third vectors.

[0013] In conjunction with any embodiment of this application, before obtaining the target plane based on the first vector, the method further includes: Based on the first set of coordinates, the center coordinates of the organ region are obtained; The step of obtaining the target plane based on the first vector includes: Based on the first vector and the center coordinates, the target plane is obtained, and the target plane passes through the point corresponding to the center coordinates.

[0014] Secondly, a scanning trajectory generation device is provided, the scanning trajectory generation device comprising: An acquisition unit is used to acquire a first three-dimensional CT image, wherein the first three-dimensional CT image includes a target object; A processing unit is configured to determine skin regions and organ regions from the first three-dimensional CT image, wherein the organ regions correspond to the target organs of the target object; The processing unit is further configured to determine the coordinates of the pixels in the organ region in the pixel coordinate system of the first three-dimensional CT image, and obtain a first coordinate set; Based on the first set of coordinates, a first vector is determined; wherein, the coordinates of pixels in the organ region are projected onto the first vector, and the resulting projection component has the greatest degree of dispersion. Based on the first vector and the organ region, a target plane is obtained, wherein the normal vector of the target plane is the first vector, and the target plane passes through a reference point within the organ region; Based on the coordinates of the organ region in the pixel coordinate system, a target region corresponding to the organ region is determined from the skin region; A scanning trajectory is generated based on the intersection of the target plane and the target region. The scanning trajectory is the trajectory of the ultrasound scanning device scanning the target organ of the target object.

[0015] In conjunction with any embodiment of this application, the processing unit is further configured to determine the circumscribed cuboid of the organ region; The processing unit is further configured to determine the coordinates of points in the circumscribed cuboid in the pixel coordinate system based on the coordinates of the organ region in the pixel coordinate system, thereby obtaining a second set of coordinates; The processing unit is further configured to determine the projection coordinates of the coordinates in the second coordinate set onto the coronal plane of the target object, thereby obtaining a third coordinate set; The processing unit is further configured to obtain a coordinate range based on the coordinates in the third coordinate set, wherein the maximum value of the coordinate range is determined based on the maximum value of the coordinates in the third coordinate set, and the minimum value of the coordinate range is determined based on the minimum value of the coordinates in the third coordinate set.

[0016] The processing unit is further configured to determine the target region from the skin region based on the coordinate range, wherein the projection coordinates of the target coordinates on the coronal plane are within the coordinate range, and the target coordinates are the coordinates of the pixels in the target region under the pixel coordinates.

[0017] In any embodiment of this application, the processing unit is further configured to input the first three-dimensional CT image into the first model to obtain a predicted trajectory, wherein the predicted trajectory is the trajectory of the ultrasound scanning device scanning the target object; The processing unit is further configured to obtain the loss of the first model based on the difference between the predicted trajectory and the scanning trajectory; The processing unit is further configured to update the parameters of the first model based on the loss of the first model to obtain a trained first model, wherein the trained first model is used to generate the trajectory of the target organ of the object scanned by the ultrasound scanning device based on the three-dimensional CT image of the object.

[0018] In conjunction with any embodiment of this application, the processing unit is further configured to determine the coordinates of pixels in the pixel coordinate system at the intersection of the target plane and the target region, thereby obtaining a fourth coordinate set; The processing unit is further configured to obtain the equation of the intersection line based on the coordinates in the fourth coordinate set; The processing unit is further configured to generate the equation of the scanning trajectory based on the equation of the intersection line; The processing unit is further configured to obtain the curvature information of the scanning trajectory based on the equation of the scanning trajectory; The processing unit is further configured to obtain a curvature difference based on the curvature information of the scanning trajectory and the curvature information of the predicted trajectory, wherein the curvature difference is the difference between the curvature of the scanning trajectory and the curvature of the predicted trajectory. The processing unit is further configured to obtain the loss of the first model based on the curvature difference, wherein the curvature difference is positively correlated with the loss of the first model.

[0019] In any embodiment of this application, the processing unit is further configured to input the organ region into a second model, so that the second model determines the direction of the long axis of the target organ and obtains a second vector; The processing unit is further configured to divide the organ region into n sub-regions along the second vector, where n is an integer greater than 1; The processing unit is further configured to obtain n fifth coordinate sets based on the first coordinate set and the n sub-regions, wherein the fifth coordinate sets are the sets of coordinates of pixels in the sub-regions; The processing unit is further configured to perform PCA on the n fifth coordinate sets respectively to obtain n third vectors; wherein the third vectors correspond one-to-one with the fifth coordinate sets; and project the coordinates in the fifth coordinate sets onto the third vectors to obtain the projection components with the greatest dispersion. The processing unit is further configured to obtain the first vector based on the sum of the n third vectors.

[0020] In any embodiment of this application, the processing unit is further configured to determine the sum of the n third vectors to obtain a fourth vector; The processing unit is further configured to determine the angles between the n third vectors and the fourth vector, thereby obtaining the angles between the n vectors; The processing unit is further configured to, when there is an abnormal angle among the n vector angles, determine that the vector among the n third vectors corresponding to the abnormal angle is an abnormal vector, wherein the abnormal angle is an angle greater than or equal to a second threshold. The processing unit is further configured to obtain the first vector based on the sum of the n third vectors excluding the abnormal vector.

[0021] In any embodiment of this application, the processing unit is further configured to obtain the center coordinates of the organ region based on the first coordinate set; The processing unit is further configured to obtain the target plane based on the first vector and the center coordinates, wherein the target plane passes through the point corresponding to the center coordinates.

[0022] Thirdly, a surgical robot is provided, including the scan trajectory generation device as described in the second aspect. In this third aspect, the surgical robot can execute a scan trajectory generation method via the scan trajectory generation device to improve the quality of the scan trajectory.

[0023] Fourthly, an electronic device is provided, comprising: a processor and a memory, the memory for storing computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs a method as described in the first aspect above and any possible implementation thereof.

[0024] Fifthly, another electronic device is provided, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs a method as described in the first aspect above and any possible implementation thereof.

[0025] In a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored therein, the computer program including program instructions that, when executed by a processor, cause the processor to perform a method as described in the first aspect above and any possible implementation thereof.

[0026] In a seventh aspect, a computer program product is provided, the computer program product comprising a computer program or instructions, wherein, when the computer program or instructions are executed on a computer, the computer performs the method described in the first aspect and any possible implementation thereof.

[0027] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments of this application will be described below.

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0030] Figure 1 A flowchart illustrating a scanning trajectory generation method provided in an embodiment of this application; Figure 2 A schematic diagram of a scanning trajectory provided for an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a scanning trajectory generation device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0032] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0033] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one" means one or more, "more" means two or more, and "at least two" means two or three or more.

[0034] The execution subject of this application embodiment is a scan trajectory generation device (hereinafter referred to as the generation device), wherein the generation device can be any electronic device capable of executing the technical solution disclosed in the method embodiment of this application. Optionally, the generation device can be one of the following: a computer or a server.

[0035] It should be understood that the method embodiments of this application can also be implemented by a processor executing computer program code. The embodiments of this application are described below with reference to the accompanying drawings. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating a scanning trajectory generation method provided in an embodiment of this application.

[0036] 101. The generating device acquires a first three-dimensional CT image, the first three-dimensional CT image including the target object.

[0037] In this embodiment of the application, the target object is a human being. The first three-dimensional CT image is a three-dimensional image obtained by performing a CT scan on the target object. The first three-dimensional CT image includes the skin region of the target object and the target organ within the target object's body. For example, the target organ is one of the following: kidney or lung.

[0038] 102. The generating device determines the skin region and organ region from the first three-dimensional CT image.

[0039] The skin region is the image region in the first 3D CT image that corresponds to the skin of the target object. In one optional embodiment, the generating device first determines the contour of the target object from the object region in the first 3D CT image, wherein the object region corresponds to the target object. Pixels whose distance to the contour of the target object is less than a third threshold are determined from the image region of the target object to obtain skin pixels. The skin region is obtained based on the skin pixels. This ensures that there are no holes in the skin region, making the skin region a continuous region. Optionally, the generating device processes the object region based on an erosion algorithm to obtain the skin region. In another optional embodiment, the generating device performs semantic segmentation on the first 3D CT image to obtain pixels whose semantic meaning is skin. The skin region is obtained based on the pixels whose semantic meaning is skin.

[0040] The organ region is the image region in the first 3D CT image corresponding to the target organ. In an optional embodiment, the generation device uses a segmentation model to segment the first 3D CT image to obtain organ pixels, where organ pixels are pixels that semantically represent an organ. The segmentation model is used to perform semantic segmentation on the 3D CT image.

[0041] 103. The generating device determines the coordinates of pixels in the organ region in the pixel coordinate system of the first three-dimensional CT image, and obtains the first coordinate set.

[0042] The first coordinate set is the set of coordinates of pixels in the first three-dimensional CT image, that is, the first coordinate set includes the three-dimensional coordinates of all pixels in the first three-dimensional CT image in the pixel coordinate system.

[0043] 104. The generating device determines the first vector based on the first set of coordinates.

[0044] In this embodiment, projecting the coordinates of pixels in the organ region onto the first vector results in the largest dispersion of the projected components. In other words, the dispersion of the projected components obtained by projecting the coordinates of pixels in the organ region onto the first vector is greater than the dispersion of the projected components obtained by projecting the coordinates of pixels in the organ region onto directions other than the first vector. Optionally, the dispersion of the projected components is determined based on the variance of the projected components.

[0045] For ease of description, if the projection component of the coordinates of a pixel in an image region has the greatest dispersion in a certain direction, then that direction is called the principal direction. For example, the direction indicated by the first vector mentioned above is the principal direction of the organ region.

[0046] In some schemes, the generating device obtains the principal direction by performing PCA on the first coordinate set. A first vector is then obtained based on the principal direction. Optionally, a covariance matrix is ​​obtained based on the coordinates in the first coordinate set. A first eigenvector is obtained based on the covariance matrix, where the eigenvalue corresponding to the first eigenvector is the maximum value among the eigenvalues ​​corresponding to the eigenvectors of the covariance matrix. This first eigenvector is then used as the first vector.

[0047] Projecting the coordinates of pixels in the organ region onto the first vector results in the largest dispersion of the projected components, indicating that the organ region has the largest size in the direction indicated by the first vector. In other words, the direction indicated by the first vector is more likely to be the direction of the organ's major axis. For example, if the target organ is the kidney, then the first vector is more likely to be the direction of the kidney's major axis.

[0048] Optionally, the generating device first converts the coordinates in the first coordinate set to coordinates in the world coordinate system based on the transformation relationship to obtain the transformed coordinate set. Then, it determines the first vector based on the transformed coordinate set.

[0049] 105. The generating device obtains the target plane based on the first vector and the organ region.

[0050] The normal vector of the target plane is the first vector. That is, the target plane is a plane perpendicular to the first vector. The target plane passes through a reference point within the organ region, meaning the target plane intersects the organ region. In an optional embodiment, the generating device determines a reference point from the organ region. Based on the first vector and the reference point, the target plane is obtained, wherein the target plane is not only perpendicular to the first vector but also passes through the reference point. In some embodiments, the reference point is any point within the organ region. In other embodiments, the reference point is one of the following: the center of the organ region, the centroid of the organ region, or the pixel with the largest gray value in the organ region. Optionally, before performing the step "obtain the target plane based on the first vector", the generating device further performs the following steps: obtaining the center coordinates of the organ region based on a first set of coordinates. Then, based on the first vector and the center coordinates, the target plane is obtained, wherein the target plane is not only perpendicular to the first vector but also passes through the point corresponding to the center coordinates, where the point corresponding to the center coordinates is the center of the organ region.

[0051] 106. The generating device determines the target region corresponding to the organ region from the skin region based on the coordinates of the organ region in the pixel coordinate system.

[0052] The target region is a region within the skin area. In an optional embodiment, the generating device obtains the target region based on the projection of the organ region onto the coronal plane of the target object. The coronal plane of the target object is the plane that divides the target object into anterior and posterior portions.

[0053] Since the target plane intersects with the organ region, and the target region is the area in the skin region that corresponds to the organ region, the target plane intersects with the target region, meaning there is an intersection line between the target plane and the target region.

[0054] 107. The generating device generates a scanning trajectory based on the intersection of the target plane and the target area.

[0055] In this embodiment, the scanning trajectory is the trajectory of the target organ scanned by the ultrasound scanning device. The ultrasound scanning device is used to acquire ultrasound images; exemplarily, the ultrasound scanning device is an ultrasound probe. In some embodiments, the ultrasound probe scans the target organ along the scanning trajectory to obtain an ultrasound image of the target organ.

[0056] Considering that when an ultrasound scanning device scans along the short axis of the target organ, the acquired ultrasound image can include information about the long axis of the target organ, thus allowing the ultrasound image to carry more information about the target organ, the direction of the scanning trajectory generated by the generating device should be as consistent as possible with the direction of the short axis of the target organ.

[0057] On the one hand, because the minor axis of the target organ is perpendicular to its major axis, and the direction indicated by the first vector is more likely to be the direction of the kidney's major axis, the extension direction of the scanning trajectory should be as perpendicular as possible to the first vector. Furthermore, because the target plane is perpendicular to the first vector, the extension direction of the scanning trajectory should be within the target plane; that is, the scanning trajectory is a trajectory within the target plane. On the other hand, the ultrasound scanning device scans the target object by moving across the skin surface, therefore the scanning trajectory should be a trajectory within the skin area. Moreover, scanning the target organ within the target area allows the ultrasound scanning device to obtain more information about the target organ, thereby improving the quality of the obtained ultrasound image. Therefore, based on these three aspects, the generating device generates a scanning trajectory based on the intersection of the target plane and the target area. In this way, when the ultrasound scanning device obtains an ultrasound image based on the scanning trajectory, it can enrich the information of the target organ in the ultrasound image. For example, Figure 2 This is a schematic diagram of a scanning trajectory provided in an embodiment of this application. Figure 2 As shown, the scanning trajectory is the trajectory within the target area, and the extension direction of the scanning trajectory is perpendicular to the first vector. It should be understood that because the skin region of the target object is a curved surface rather than a plane, the target area is a curved surface rather than a plane, and correspondingly, the scanning trajectory within the target area is a curve rather than a straight line. Figure 2It uses a two-dimensional plane to briefly represent the target area and a straight line to briefly represent the scanning trajectory.

[0058] exist Figure 1 In the method shown, after acquiring a first 3D CT image, the generating device determines the skin region and organ region from the first 3D CT image. Then, it determines the coordinates of pixels in the organ region in the pixel coordinate system of the first 3D CT image, obtaining a first coordinate set. Based on the first coordinate set, a first vector is determined, wherein the direction indicated by the first vector is more likely to be the direction of the major axis of the target organ. Based on the first vector and the organ region, a target plane is obtained. Based on the coordinates of the organ region in the pixel coordinate system, a target region corresponding to the organ region is determined from the skin region. Finally, based on the intersection of the target plane and the target region, a scanning trajectory is generated. When the ultrasound scanning device scans the target organ of the target object based on the scanning trajectory, it can obtain more information about the target organ, thereby improving the quality of the scanning trajectory.

[0059] As an optional implementation, before executing the step "determining the target region corresponding to the organ region from the skin region based on the coordinates of the organ region in the pixel coordinate system," the generation device further performs the following steps: determining the circumscribed cuboid of the organ region. After determining the circumscribed cuboid, during the execution of the step "determining the target region corresponding to the organ region from the skin region based on the coordinates of the organ region in the pixel coordinate system," the following steps are performed: determining the coordinates of points in the circumscribed cuboid in the pixel coordinate system based on the coordinates of the organ region in the pixel coordinate system, obtaining a second coordinate set. Determining the projection coordinates of the coordinates in the second coordinate set onto the coronal plane of the target object, obtaining a third coordinate set. Based on the coordinates in the third coordinate set, a coordinate range is obtained, where the maximum value of the coordinate range is determined based on the maximum value of the coordinates in the third coordinate set, and the minimum value of the coordinate range is determined based on the minimum value of the coordinates in the third coordinate set. Based on the coordinate range, the target region is determined from the skin region, where the projection coordinates of the target coordinates onto the coronal plane are within the coordinate range, and the target coordinates are the coordinates of pixels in the target region in the pixel coordinate system.

[0060] In some schemes, the generating apparatus determines the outer bounding box of the organ region as the outer cuboid of the organ region. Exemplarily, the outer bounding box is one of the following: an axis-aligned bounding box. Aligned bounding box (AABB) and rotated bounding box (OBB).

[0061] In this embodiment, the coronal plane is parallel to the plane formed by the horizontal axis and the vertical axis of the pixel coordinate system of the first 3D CT image. The coordinates in the third coordinate set include horizontal and vertical coordinates, where the horizontal coordinate is the coordinate corresponding to the horizontal axis and the vertical coordinate is the coordinate corresponding to the vertical axis. The coordinate range includes the range of the horizontal coordinate and the range of the vertical coordinate. For example, the range of the horizontal coordinate is -10 to 25, and the range of the vertical coordinate is -33 to 45. The minimum value of the horizontal coordinate range is determined based on the minimum value of the horizontal coordinate in the third coordinate set, and the maximum value of the horizontal coordinate range is determined based on the maximum value of the horizontal coordinate in the third coordinate set. Similarly, the minimum value of the vertical coordinate range is determined based on the minimum value of the vertical coordinate in the third coordinate set, and the maximum value of the vertical coordinate range is determined based on the maximum value of the vertical coordinate in the third coordinate set.

[0062] Optionally, the minimum value of the x-coordinate range is the difference between the minimum x-coordinate in the third coordinate set and a preset value, where the preset value is a positive number. The maximum value of the x-coordinate range is the sum of the maximum value of the x-coordinate in the third coordinate set and the preset value. Optionally, the minimum value of the y-coordinate range is the difference between the minimum y-coordinate in the third coordinate set and a preset value. The maximum value of the y-coordinate range is the sum of the maximum value of the y-coordinate in the third coordinate set and the preset value. This increases the probability that the target region includes the complete outline of the target organ.

[0063] Optionally, the target organ is a kidney. Based on the coordinates in the third coordinate set, it can be determined whether the target organ is the left or right kidney. For example, the direction of the horizontal axis of the pixel coordinate system is from the left to the right of the target object. Therefore, the horizontal coordinate of the left kidney is smaller than that of the right kidney, and in this case, the target organ can be determined as the left or right kidney based on the magnitude of the horizontal coordinates in the third coordinate set.

[0064] Optionally, based on a coordinate range, two candidate regions can be determined from the skin region. The coordinates of pixels within a candidate region in the pixel coordinate system are called candidate coordinates, and the projected coordinates of these candidate coordinates in the coronal plane are within the coordinate range. Based on the vertical coordinates of the two candidate regions in the pixel coordinate system of the first 3D CT image, it can be determined whether the candidate region is a region within the abdominal skin region or a region within the back skin region of the target object. For example, the vertical axis of the pixel coordinate system of the first 3D CT image points from the abdomen of the target object to the back of the target object. Therefore, the region with the larger vertical coordinate among the two candidate regions is the region within the back skin region of the target object, and the region with the smaller vertical coordinate among the two candidate regions is the region within the abdominal skin region of the target object.

[0065] In some cases, the target region can be selected from two candidate regions based on actual needs. For example, if the target is lying face down on a bed, the ultrasound scanner needs to scan the target's back, and the generating device selects a candidate region within the skin area of ​​the target's back as the target region. If the target is lying face up on a bed, the ultrasound scanner needs to scan the target's abdomen, and the generating device selects a candidate region within the skin area of ​​the target's abdomen as the target region.

[0066] In this implementation, the coordinate range is first determined based on the circumscribed cuboid of the organ region. Then, based on the coordinate range, the target region is determined from the skin region, which can improve the accuracy of the target region.

[0067] As an optional implementation, the generation device further performs the following steps: inputting a first three-dimensional CT image into a first model to obtain a predicted trajectory, the predicted trajectory being the trajectory of the ultrasound scanning device scanning the target object. Based on the difference between the predicted trajectory and the scanning trajectory, a loss of the first model is obtained. Based on the loss of the first model, the parameters of the first model are updated to obtain a trained first model. The trained first model is used to generate the trajectory of the target organ of the ultrasound scanning device scanning the object based on the three-dimensional CT image of the object.

[0068] In this embodiment, the generating device uses the first 3D CT image and the scan trajectory as training data to train the first model, so that the first model learns how to generate the trajectory of the target organ of the object scanned by the ultrasound scanning device based on the object's 3D CT image. The scan trajectory serves as supervisory information; that is, the generating device uses the scan trajectory to supervise the predicted trajectory generated by the first model during training.

[0069] In one alternative implementation, when it is necessary to acquire ultrasound images of the target organ of an object, a second three-dimensional CT image of the object to be acquired can be acquired first. Then, the second three-dimensional CT image is input into a trained first model to obtain the target trajectory of the ultrasound scanning device scanning the target organ of the object. The ultrasound scanning device is then controlled based on the target trajectory to perform a scan, obtaining the target ultrasound image of the object. This allows the target ultrasound image to carry more information about the target organ.

[0070] Optionally, after acquiring the target ultrasound image, the target organ in the second 3D CT image and the target organ in the target ultrasound image are registered to obtain registration parameters. Based on the registration parameters, the second 3D CT image is processed to align the target organ in the second 3D CT image with the target organ in the target ultrasound image, resulting in a third 3D CT image. The coordinates of the lesion in the target organ in the pixel coordinate system of the target ultrasound image are determined based on the third 3D CT image. Based on the target ultrasound image, the coordinates of the starting point in the pixel coordinate system of the target ultrasound image are determined, where the starting point is a point in the skin region of the object to be acquired. Based on the coordinates of the lesion in the pixel coordinate system of the target ultrasound image and the coordinates of the starting point in the pixel coordinate system of the target ultrasound image, a target path is obtained, where the target path is the path from the skin region of the object to be acquired to the lesion. In this embodiment, because planning the target path requires using both the ultrasound image and the 3D CT image of the object to be acquired, and the target trajectory of the ultrasound scanning device is generated based on a trained first model, planning the target path based on this embodiment can reduce the labor cost of acquiring ultrasound images and improve planning efficiency.

[0071] As an optional implementation, the generation device performs the following steps during the step "generating a scanning trajectory based on the intersection of the target plane and the target region": determining the coordinates of pixels in the pixel coordinate system along the intersection of the target plane and the target region to obtain a fourth coordinate set; obtaining the equation of the intersection line based on the coordinates in the fourth coordinate set; generating the equation of the scanning trajectory based on the equation of the intersection line. After generating the equation of the scanning trajectory, the generation device performs the following steps during the step "obtaining the loss of the first model based on the difference between the predicted trajectory and the scanning trajectory": obtaining the curvature information of the scanning trajectory based on the equation of the scanning trajectory; obtaining the curvature difference based on the curvature information of the scanning trajectory and the curvature information of the predicted trajectory, where the curvature difference is the difference between the curvature of the scanning trajectory and the curvature of the predicted trajectory; and obtaining the loss of the first model based on the curvature difference, where the curvature difference is positively correlated with the loss of the first model.

[0072] In this implementation, when training the first model, the generation device needs to calculate the loss of the first model based on the difference between the curvature information of the predicted trajectory generated by the first model and the curvature information of the scan trajectory. The curvature information of the scan trajectory needs to be calculated based on the equation of the scan trajectory. Therefore, the generation device first obtains the equation of the intersection line based on the coordinates in the fourth coordinate set. Then, based on the equation of the intersection line, it generates the equation of the scan trajectory. Thus, when training the first model, the curvature information of the scan trajectory can be obtained based on the equation of the scan trajectory, and the loss of the first model can be calculated based on this curvature information. Moreover, this implementation calculates the equation of the scan trajectory before training, eliminating the need to calculate it during the training process, thereby improving training efficiency.

[0073] In some schemes, since the pixel coordinate system of the first 3D CT image is a 3D coordinate system, and the scanning trajectory is a trajectory in the pixel coordinate system of the first 3D CT image, the scanning trajectory is a curve in 3D space. Therefore, the equation of the scanning trajectory can be a curve polynomial. Optionally, the curve polynomial includes the following two equations: y = a0 + a1x + a2x² + a3x³, z = b0 + b1x + b2x² + b3x³. Where a0, a1, a2, a3, b0, b1, b2, and b3 are constants. x represents the abscissa of the scanning trajectory, y represents the ordinate of the scanning trajectory, and z represents the ordinate of the scanning trajectory. Based on the coordinates in the fourth coordinate set, the values ​​of the constants in the above two equations can be determined, and thus the curve polynomial can be determined. Optionally, since the scanning trajectory passes through pixels in the intersection line, and the coordinates in the fourth coordinate set are the coordinates of pixels in the intersection line, the generating device can fit the coordinates in the fourth coordinate set to determine the values ​​of the constants. Optionally, the generating device uses the equation of the intersection line as the equation of the scanning trajectory, that is, the scanning trajectory is the same as the intersection line.

[0074] The curvature information of the trajectory includes the curvature of the trajectory. Optionally, the curvature information of the trajectory includes the curvature at any point in the trajectory. In some schemes, the generating device can obtain the curvature difference based on the curvature of the scanned trajectory and the curvature of the predicted trajectory. Optionally, the generating device determines a first curvature of the inflection points in the scanned trajectory based on the scanned trajectory. It determines a second curvature of the inflection points in the predicted trajectory based on the curvature of the predicted trajectory. The curvature difference is obtained based on the difference between the first curvature and the second curvature. This reduces the amount of data processing required to determine the curvature difference and increases the speed of determining the curvature difference.

[0075] As an optional implementation, before executing the step "determine the first vector based on the first coordinate set", the generation device further performs the following steps: inputting the organ region into the second model so that the second model determines the direction of the major axis of the target organ, obtaining a second vector. Dividing the organ region into n sub-regions along the second vector, where n is an integer greater than 1. After obtaining the n sub-regions, the generation device performs the following steps during the execution of the step "determine the first vector based on the first coordinate set": obtaining n fifth coordinate sets based on the first coordinate set and the n sub-regions, where the fifth coordinate sets are the sets of coordinates of pixels in the sub-regions. Performing PCA on each of the n fifth coordinate sets to obtain n third vectors. The third vectors correspond one-to-one with the fifth coordinate sets; projecting the coordinates in the fifth coordinate sets onto the third vectors results in the largest dispersion of the projected components. The first vector is obtained based on the sum of the n third vectors.

[0076] Considering that different regions within an organ region have different principal directions, directly determining the principal direction of the entire organ region can easily overlook the principal directions of local regions within the organ region, leading to low accuracy in determining the principal direction of the organ region. Therefore, in this embodiment, the generating device first determines the principal direction of local regions within the organ region (i.e., the direction indicated by the third vector). Then, based on the principal directions of the local regions, it obtains the principal direction of the organ region (i.e., the direction indicated by the first vector). This improves the accuracy of the principal direction of the organ region, thus improving the accuracy of the first vector.

[0077] In some cases, malformed areas exist within organ regions, such as when a disease within the organ causes distortion in a specific area. If the principal direction of the entire organ region is determined directly, the malformed area will directly influence this overall principal direction, leading to low accuracy in determining the organ region's principal direction. However, if the principal direction of the malformed area is determined separately, and then the principal direction of the entire organ region is derived from it along with the principal directions of other local regions, the influence of the malformed area's principal direction on the overall organ region's principal direction can be reduced by processing it (e.g., reducing its weight or filtering it out), thereby improving the accuracy of the organ region's principal direction.

[0078] Since the method of segmenting the organ region directly determines the segmentation result, and the segmentation result affects the determination of the principal orientation of the organ region, in this embodiment, the generating device first processes the first 3D CT image using a second model to roughly estimate the orientation of the major axis of the target organ, where the orientation of the major axis of the target organ is represented by a second vector. Then, the organ region is divided into n sub-regions along the second vector. Next, based on the coordinates of the pixels in each sub-region, the principal orientation of each sub-region is obtained, where the principal orientation of each sub-region is represented by a third vector. Finally, by summing all the vectors used to represent the principal orientations of the sub-regions, a first vector representing the principal orientation of the organ region is obtained. This improves the accuracy of the first vector.

[0079] As an optional implementation, the first vector is obtained based on the sum of n third vectors, including the following steps: First, the sum of the n third vectors is determined to obtain a fourth vector. Second, the angles between the n third vectors and the fourth vector are determined to obtain n vector angles. If there are abnormal angles among the n vector angles, the vector corresponding to the abnormal angle among the n third vectors is identified as the abnormal vector, and the abnormal angle is an angle greater than or equal to a second threshold. Third, the first vector is obtained based on the sum of the n third vectors excluding the abnormal vector.

[0080] As mentioned earlier, after obtaining the main direction of a local region, its influence on the overall main direction of the organ region can be reduced by processing it (e.g., reducing weight or filtering). Therefore, in this implementation, the generating device first determines the sum of n third vectors to obtain a fourth vector. Then, it determines the angle between each third vector and the fourth vector. If the angle between the third vector and the fourth vector is large, it indicates a higher probability that the third vector is an abnormal vector. For example, the direction of the third vector corresponding to the deformed region differs significantly from the direction of the third vector corresponding to other local regions, thus leading to a larger angle between the third vector and the fourth vector corresponding to the deformed region.

[0081] The generating device uses a second threshold to determine whether the angle between the third and fourth vectors is large or small. Therefore, when there is an abnormal angle among the n vector angles, the generating device identifies the vector corresponding to the abnormal angle among the n third vectors as the abnormal vector. Then, based on the sum of the n third vectors excluding the abnormal vector, the first vector is obtained, which can improve the accuracy of the first vector.

[0082] Optionally, the vectors among the n third vectors, excluding the outlier vectors, include at least two fifth vectors; that is, the fifth vectors are the vectors among the n third vectors excluding the outlier vectors, and the number of fifth vectors is greater than 1. If there are still outlier vectors among the at least two fifth vectors, the generating device removes the outlier vectors from the at least two fifth vectors until there are no outlier vectors in the removed vectors, and then obtains the first vector based on the sum of the removed vectors.

[0083] Optionally, the vectors among the n third vectors, excluding the outlier vectors, include at least two fifth vectors. Obtaining the first vector based on the sum of the n third vectors (excluding the outlier vectors) involves the following steps: Based on the at least two fifth vectors, obtain at least two unit vectors, where each unit vector corresponds one-to-one with a fifth vector, and the direction of the unit vector is the same as the direction of the fifth vector. Summate the at least two unit vectors to obtain the first vector. Since the magnitude of a unit vector is 1, the magnitude of the sum of different unit vectors can reflect the difference in direction between the different unit vectors. The larger the magnitude of the sum of different unit vectors, the smaller the difference in direction between the different unit vectors. For example, if the direction of the first unit vector is the same as the direction of the second unit vector, then the magnitude of the vector obtained by summing the first and second unit vectors is the sum of the magnitudes of the first and second unit vectors. Therefore, the direction of the first vector can indicate the main direction of the organ region, and the magnitude of the first vector can indicate the confidence level of the main direction of the organ. That is, the magnitude of the first vector can indicate the confidence level of the first vector, wherein the magnitude of the first vector is positively correlated with the confidence level of the first vector.

[0084] In some implementation scenarios, the target organ is the kidney. After acquiring the first 3D CT image of the target object, trajectory-related information can be obtained based on the first 3D CT image. This trajectory-related information includes the following: whether the kidney is the left or right kidney, the equation of the scan trajectory, a first vector, a preset value, and the center coordinates of the kidney. Then, detailed trajectory-related information can be output so that the user can understand the trajectory-related information.

[0085] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0086] The methods of the embodiments of this application have been described in detail above, and the apparatus of the embodiments of this application is provided below.

[0087] Please see Figure 3 , Figure 3 This is a schematic diagram of a scanning trajectory generation device provided in an embodiment of this application. The scanning trajectory generation device 1 includes: an acquisition unit 11 and a processing unit 12, wherein: Acquisition unit 11 is used to acquire a first three-dimensional CT image, wherein the first three-dimensional CT image includes a target object; Processing unit 12 is used to determine skin regions and organ regions from the first three-dimensional CT image, wherein the organ regions correspond to the target organs of the target object; The processing unit 12 is further configured to determine the coordinates of the pixels in the organ region in the pixel coordinate system of the first three-dimensional CT image, and obtain a first coordinate set; Based on the first set of coordinates, a first vector is determined; wherein, the coordinates of pixels in the organ region are projected onto the first vector, and the resulting projection component has the greatest degree of dispersion. Based on the first vector and the organ region, a target plane is obtained, wherein the normal vector of the target plane is the first vector, and the target plane passes through a reference point within the organ region; Based on the coordinates of the organ region in the pixel coordinate system, a target region corresponding to the organ region is determined from the skin region; A scanning trajectory is generated based on the intersection of the target plane and the target region. The scanning trajectory is the trajectory of the ultrasound scanning device scanning the target organ of the target object.

[0088] In any embodiment of this application, the processing unit 12 is further configured to determine the circumscribed cuboid of the organ region; The processing unit 12 is further configured to determine the coordinates of the points in the circumscribed cuboid in the pixel coordinate system based on the coordinates of the organ region in the pixel coordinate system, thereby obtaining a second set of coordinates. The processing unit 12 is further configured to determine the projection coordinates of the coordinates in the second coordinate set onto the coronal plane of the target object, thereby obtaining a third coordinate set; The processing unit 12 is further configured to obtain a coordinate range based on the coordinates in the third coordinate set, wherein the maximum value of the coordinate range is determined based on the maximum value of the coordinates in the third coordinate set, and the minimum value of the coordinate range is determined based on the minimum value of the coordinates in the third coordinate set.

[0089] The processing unit 12 is further configured to determine the target region from the skin region based on the coordinate range, wherein the projection coordinates of the target coordinates on the coronal plane are within the coordinate range, and the target coordinates are the coordinates of the pixels in the target region under the pixel coordinates.

[0090] In any embodiment of this application, the processing unit 12 is further configured to input the first three-dimensional CT image into the first model to obtain a predicted trajectory, wherein the predicted trajectory is the trajectory of the ultrasound scanning device scanning the target object; The processing unit 12 is further configured to obtain the loss of the first model based on the difference between the predicted trajectory and the scanning trajectory; The processing unit 12 is further configured to update the parameters of the first model based on the loss of the first model to obtain a trained first model, wherein the trained first model is used to generate the trajectory of the target organ of the object scanned by the ultrasound scanning device based on the three-dimensional CT image of the object.

[0091] In conjunction with any embodiment of this application, the processing unit 12 is further configured to determine the coordinates of pixels in the pixel coordinate system at the intersection of the target plane and the target region, thereby obtaining a fourth coordinate set; The processing unit 12 is further configured to obtain the equation of the intersection line based on the coordinates in the fourth coordinate set; The processing unit 12 is also used to generate the equation of the scanning trajectory based on the equation of the intersection line; The processing unit 12 is also used to obtain the curvature information of the scanning trajectory based on the equation of the scanning trajectory; The processing unit 12 is further configured to obtain a curvature difference based on the curvature information of the scanning trajectory and the curvature information of the predicted trajectory, wherein the curvature difference is the difference between the curvature of the scanning trajectory and the curvature of the predicted trajectory. The processing unit 12 is further configured to obtain the loss of the first model based on the curvature difference, wherein the curvature difference is positively correlated with the loss of the first model.

[0092] In any embodiment of this application, the processing unit 12 is further configured to input the organ region into a second model, so that the second model determines the direction of the long axis of the target organ and obtains a second vector; The processing unit 12 is further configured to divide the organ region into n sub-regions along the second vector, where n is an integer greater than 1; The processing unit 12 is further configured to obtain n fifth coordinate sets based on the first coordinate set and the n sub-regions, wherein the fifth coordinate sets are the sets of coordinates of pixels in the sub-regions; The processing unit 12 is further configured to perform PCA on the n fifth coordinate sets respectively to obtain n third vectors; wherein the third vectors correspond one-to-one with the fifth coordinate sets; and project the coordinates in the fifth coordinate sets onto the third vectors to obtain the projection components with the greatest dispersion. The processing unit 12 is further configured to obtain the first vector based on the sum of the n third vectors.

[0093] In any embodiment of this application, the processing unit 12 is further configured to determine the sum of the n third vectors to obtain a fourth vector; The processing unit 12 is also used to determine the angle between the n third vectors and the fourth vector to obtain the angle between the n vectors; The processing unit 12 is further configured to, when there is an abnormal angle among the n vector angles, determine that the vector among the n third vectors corresponding to the abnormal angle is an abnormal vector, wherein the abnormal angle is an angle greater than or equal to a second threshold. The processing unit 12 is further configured to obtain the first vector based on the sum of the vectors other than the abnormal vector among the n third vectors.

[0094] In any embodiment of this application, the processing unit 12 is further configured to obtain the center coordinates of the organ region based on the first coordinate set; The processing unit 12 is further configured to obtain the target plane based on the first vector and the center coordinates, wherein the target plane passes through the point corresponding to the center coordinates.

[0095] In this embodiment, after acquiring a first 3D CT image, the generating device determines a skin region and an organ region from the first 3D CT image. Then, it determines the coordinates of pixels in the organ region in the pixel coordinate system of the first 3D CT image, obtaining a first coordinate set. Based on the first coordinate set, it determines a first vector, wherein the direction indicated by the first vector is more likely to be the direction of the major axis of the target organ. Based on the first vector and the organ region, it obtains a target plane. Based on the coordinates of the organ region in the pixel coordinate system, it determines the target region corresponding to the organ region from the skin region. Finally, based on the intersection of the target plane and the target region, it generates a scanning trajectory. When the ultrasound scanning device scans the target organ of the target object based on the scanning trajectory, it can obtain more information about the target organ, thereby improving the quality of the scanning trajectory.

[0096] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0097] Figure 4This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, memory 22, input device 23, and output device 24 are coupled together via connectors, which include various interfaces, transmission lines, or buses, etc., and are not limited in this embodiment. It should be understood that in the various embodiments of this application, coupling refers to mutual connection in a specific way, including direct connection or indirect connection through other devices, such as through various interfaces, transmission lines, buses, etc.

[0098] Processor 21 can be one or more graphics processing units (GPUs). If processor 21 is a GPU, the GPU can be a single-core GPU or a multi-core GPU. Optionally, processor 21 can be a processor group composed of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Optionally, the processor can also be other types of processors, etc., which are not limited in this embodiment.

[0099] The memory 22 can be used to store computer program instructions, as well as various types of computer program code, including program code for executing the scheme of this application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0100] Input device 23 is used to input data and / or signals, and output device 24 is used to output data and / or signals. Input device 23 and output device 24 can be independent devices or an integrated device.

[0101] It is understood that in this embodiment of the application, the memory 22 can be used not only to store related instructions, but also to store related data. This embodiment of the application does not limit the specific data stored in the memory.

[0102] Understandable Figure 4This is merely a simplified design of an electronic device. In practical applications, the electronic device may also include other necessary components, including, but not limited to, any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of this application are within the protection scope of this application.

[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0104] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will also readily understand that the various embodiments of this application have different focuses, and for the sake of convenience and brevity, the same or similar parts may not be repeated in different embodiments. Therefore, parts not described or not described in detail in one embodiment can be referred to the descriptions in other embodiments.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0106] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0108] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0109] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or RAM, magnetic disks, or optical disks.

Claims

1. A method for generating a scanning trajectory, characterized in that, The method includes: Acquire a first 3D CT image, the first 3D CT image including the target object; Skin regions and organ regions are determined from the first three-dimensional CT image, wherein the organ regions correspond to the target organs of the target object; Determine the coordinates of the pixels in the organ region in the pixel coordinate system of the first three-dimensional CT image to obtain a first coordinate set; Based on the first set of coordinates, a first vector is determined; wherein, the coordinates of pixels in the organ region are projected onto the first vector, and the resulting projection component has the greatest degree of dispersion. Based on the first vector and the organ region, a target plane is obtained, wherein the normal vector of the target plane is the first vector, and the target plane passes through a reference point within the organ region; Based on the coordinates of the organ region in the pixel coordinate system, a target region corresponding to the organ region is determined from the skin region; A scanning trajectory is generated based on the intersection of the target plane and the target region. The scanning trajectory is the trajectory of the ultrasound scanning device scanning the target organ of the target object.

2. The method according to claim 1, characterized in that, Before determining the target region corresponding to the organ region from the skin region based on the coordinates of the organ region in the pixel coordinate system, the method further includes: Determine the circumscribed cuboid of the organ region; The step of determining the target region corresponding to the organ region from the skin region based on the coordinates of the organ region in the pixel coordinate system includes: Based on the coordinates of the organ region in the pixel coordinate system, the coordinates of the points in the circumscribed cuboid in the pixel coordinate system are determined to obtain the second coordinate set. Determine the projection coordinates of the coordinates in the second coordinate set onto the coronal plane of the target object to obtain the third coordinate set; Based on the coordinates in the third coordinate set, a coordinate range is obtained. The maximum value of the coordinate range is determined based on the maximum value of the coordinates in the third coordinate set, and the minimum value of the coordinate range is determined based on the minimum value of the coordinates in the third coordinate set. Based on the coordinate range, the target region is determined from the skin region, the projection coordinates of the target coordinates on the coronal plane are within the coordinate range, and the target coordinates are the coordinates of the pixels in the target region under the pixel coordinates.

3. The method according to claim 2, characterized in that, The method further includes: The first three-dimensional CT image is input into the first model to obtain a predicted trajectory, which is the trajectory of the ultrasound scanning device scanning the target object. The loss of the first model is obtained based on the difference between the predicted trajectory and the scanning trajectory; Based on the loss of the first model, the parameters of the first model are updated to obtain a trained first model. The trained first model is used to generate the trajectory of the target organ of the object scanned by the ultrasound scanning device based on the three-dimensional CT image of the object.

4. The method according to claim 3, characterized in that, The step of generating a scanning trajectory based on the intersection of the target plane and the target region includes: Determine the coordinates of the pixels in the pixel coordinate system along the intersection line of the target plane and the target region to obtain a fourth coordinate set; Based on the coordinates in the fourth coordinate set, the equation of the intersection line is obtained; Based on the equation of the intersection line, the equation of the scanning trajectory is generated; The loss of the first model is obtained based on the difference between the predicted trajectory and the scanning trajectory, including: Based on the equation of the scanning trajectory, the curvature information of the scanning trajectory is obtained; Based on the curvature information of the scan trajectory and the curvature information of the predicted trajectory, a curvature difference is obtained, which is the difference between the curvature of the scan trajectory and the curvature of the predicted trajectory. Based on the curvature difference, the loss of the first model is obtained, and the curvature difference is positively correlated with the loss of the first model.

5. The method according to claim 1 or 2, characterized in that, Before determining the first vector based on the first set of coordinates, the method further includes: The organ region is input into the second model so that the second model determines the direction of the long axis of the target organ, thereby obtaining a second vector; The organ region is divided into n sub-regions along the second vector, where n is an integer greater than 1; The step of determining the first vector based on the first coordinate set includes: Based on the first coordinate set and the n sub-regions, n fifth coordinate sets are obtained, where each fifth coordinate set is a set of coordinates of pixels in the sub-regions. Principal component analysis is performed on each of the n fifth coordinate sets to obtain n third vectors; wherein each third vector corresponds one-to-one with a fifth coordinate set; the projection of the coordinates in the fifth coordinate set onto the third vectors yields the most discrete projection components; The first vector is obtained based on the sum of the n third vectors.

6. The method according to claim 5, characterized in that, The process of obtaining the first vector based on the sum of the n third vectors includes: The sum of the n third vectors is determined to obtain the fourth vector; Determine the angles between the n third vectors and the fourth vector to obtain the angles between the n vectors; If there is an abnormal angle among the n vector angles, the vector among the n third vectors that corresponds to the abnormal angle is determined to be an abnormal vector, and the abnormal angle is an angle that is greater than or equal to a second threshold. The first vector is obtained by summing the vectors other than the abnormal vector among the n third vectors.

7. The method according to claim 1 or 2, characterized in that, Before obtaining the target plane based on the first vector, the method further includes: Based on the first set of coordinates, the center coordinates of the organ region are obtained; The step of obtaining the target plane based on the first vector includes: Based on the first vector and the center coordinates, the target plane is obtained, and the target plane passes through the point corresponding to the center coordinates.

8. A scanning trajectory generation device, characterized in that, The scanning trajectory generation device includes: An acquisition unit is used to acquire a first three-dimensional CT image, wherein the first three-dimensional CT image includes a target object; A processing unit is configured to determine skin regions and organ regions from the first three-dimensional CT image, wherein the organ regions correspond to the target organs of the target object; The processing unit is further configured to determine the coordinates of the pixels in the organ region in the pixel coordinate system of the first three-dimensional CT image, and obtain a first coordinate set; Based on the first set of coordinates, a first vector is determined; wherein, the coordinates of pixels in the organ region are projected onto the first vector, and the resulting projection component has the greatest degree of dispersion. Based on the first vector and the organ region, a target plane is obtained, wherein the normal vector of the target plane is the first vector, and the target plane passes through a reference point within the organ region; Based on the coordinates of the organ region in the pixel coordinate system, a target region corresponding to the organ region is determined from the skin region; A scanning trajectory is generated based on the intersection of the target plane and the target region. The scanning trajectory is the trajectory of the ultrasound scanning device scanning the target organ of the target object.

9. A surgical robot, characterized in that, Includes the scanning trajectory generation device as described in claim 8.

10. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1 to 7.

Images