Contour segmentation method and related device based on ultrasound images

Abstract

This application provides a contour segmentation method and related apparatus based on ultrasound images. The method includes: acquiring multiple ultrasound images obtained by scanning a target organ using an ultrasound device; determining a first region contour from the multiple ultrasound images, wherein the distance between pixels on the first region contour and the ultrasound device is less than or equal to a first threshold; and using the first region contour as the contour of the target organ. This application can improve the accuracy of the target organ contour reconstructed from ultrasound images, thereby improving registration accuracy during registration based on the target organ contour.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a contour segmentation method and related apparatus based on ultrasound images. Background Technology

[0002] With the continuous development of computer science and technology, medical technology has also ushered in significant breakthroughs. For example, ultrasound images of organs can be acquired using ultrasound equipment, and CT images of organs can be acquired using computed tomography (CT) equipment. Then, the ultrasound contour of the organ can be segmented from the ultrasound image, and the CT contour of the organ can be segmented from the CT image. The ultrasound contour and CT contour of the organ are then registered so that surgeons and other operators can obtain information about the internal organs during surgery, thereby reducing surgical errors.

[0003] However, the high number of false positives in ultrasound images leads to low accuracy in the segmented ultrasound contours. Summary of the Invention

[0004] This application provides a contour segmentation method and related apparatus based on ultrasound images. This application can improve the accuracy of the target organ contour reconstructed based on ultrasound images, thereby improving the registration accuracy when performing registration based on the target organ contour.

[0005] In a first aspect, embodiments of this application provide a contour segmentation method based on ultrasound images, comprising:

[0006] Multiple ultrasound images are acquired, which are obtained by scanning the target organ using an ultrasound device.

[0007] A first region contour is determined from the above multiple ultrasound images, wherein the distance between the pixels on the first region contour and the ultrasound device is less than or equal to a first threshold.

[0008] The outline of the first region is used as the outline of the target organ.

[0009] In conjunction with the first aspect, in one possible implementation, determining the contour of the first region from the aforementioned plurality of ultrasound images includes:

[0010] Each of the above ultrasound images is segmented to obtain multiple target images, each of which includes a set of pixels of the target organ on a cross section.

[0011] A first set of pixels is determined from the target image above, wherein the pixels in the first set of pixels are located on the outline of the target organ above;

[0012] The first region contour is obtained based on the set of first pixels in each of the aforementioned target images.

[0013] In conjunction with the first aspect, in one possible implementation, obtaining the first region contour based on the first set of sub-pixels in each of the aforementioned target images includes:

[0014] A transformation matrix is ​​obtained, which is used to locate the ultrasound image within the tracking range of the optical tracking device;

[0015] The spatial location information of each of the ultrasound images is determined based on the transformation matrix described above.

[0016] Based on the aforementioned spatial location information, the set of first pixels in each of the aforementioned target images is reconstructed into the outline of the aforementioned first region.

[0017] In conjunction with the first aspect, in one possible implementation, determining the first set of pixels from the target image includes:

[0018] Determine the midpoint between the maximum and minimum coordinate values ​​of a pixel in the target image along a first direction; the first direction is the direction in which the target organ points towards the ultrasound device.

[0019] The pixels whose coordinates in the first direction are greater than or equal to the intermediate number are defined as the first set of pixels.

[0020] In conjunction with the first aspect, in one possible implementation, using the outline of the first region as the outline of the target organ includes:

[0021] Based on the distance between the points on the first region contour and the ultrasound device, a second region contour is selected from the first region contour.

[0022] The outline of the second region is used as the outline of the target organ.

[0023] In conjunction with the first aspect, in one possible implementation, the process of filtering out a second region contour from the first region contour based on the distance between points on the first region contour and the ultrasound device includes:

[0024] The target point is determined from the outline of the first region, where the target point is any point on the outline of the first region.

[0025] Candidate points are determined on the outline of the first region based on the target point. The absolute value of the difference between the coordinate values ​​of the candidate point and the target point in the second direction is less than or equal to a second threshold. The second direction is perpendicular to a third direction, which is the direction in which the target organ points to the ultrasound device.

[0026] The point closest to the ultrasound device among the target point and candidate points is taken as the point on the contour of the second region.

[0027] Secondly, embodiments of this application provide a contour segmentation device based on ultrasound images, comprising:

[0028] The acquisition unit is used to acquire multiple ultrasound images, which are obtained by scanning the target organ using an ultrasound device.

[0029] A determining unit is configured to determine a first region contour from the plurality of ultrasound images, wherein the distance between a pixel on the first region contour and the ultrasound device is less than or equal to a first threshold.

[0030] The determining unit is further configured to use the contour of the first region as the contour of the target organ.

[0031] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to invoke the program instructions to execute the method of the first aspect or any possible implementation of the first aspect.

[0032] Fourthly, embodiments of this application provide a chip including logic circuitry and an interface, wherein the logic circuitry and the interface are coupled; the interface is used to input and / or output code instructions, and the logic circuitry is used to execute the code instructions to cause the method in the first aspect or any possible implementation thereof to be executed.

[0033] Fifthly, embodiments of this application disclose a computer program product, which includes program instructions that, when executed by a processor, cause the method in the first aspect or any possible implementation thereof to be executed.

[0034] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed on a processor, causes the method in the first aspect or any possible implementation thereof to be performed. Exemplarily, the computer program product may be a software installation package.

[0035] In this embodiment, after acquiring multiple ultrasound images of a target organ using an ultrasound device, a first region contour is determined from these ultrasound images. The distance between pixels on the first region contour and the ultrasound device is less than or equal to a first threshold. Because lesions (e.g., stones) in the target organ inevitably cause acoustic interference during ultrasound scanning, the accuracy of the ultrasound image in the vicinity of the lesion (especially in areas far from the ultrasound device) is low, resulting in a large deviation in the accuracy of the segmented ultrasound contour. This application uses the first region contour as the contour of the target organ, which can effectively remove low-precision contour data from the aforementioned multiple ultrasound images, improve the accuracy of the target organ contour reconstructed from the ultrasound images, and thus improve registration accuracy when registering based on the target organ contour. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments or background art of this application, the accompanying drawings used in the embodiments or background art of this application will be briefly introduced below.

[0037] Figure 1 This is a schematic diagram of a scene for acquiring ultrasound images provided in an embodiment of this application;

[0038] Figure 2 This is a schematic flowchart of a contour segmentation method based on ultrasound images provided in an embodiment of this application;

[0039] Figure 3 This is a schematic diagram of the outline of a first region provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of a target image provided in an embodiment of this application;

[0041] Figure 5 This is a schematic diagram of a first pixel set provided in an embodiment of this application;

[0042] Figure 6 This is a schematic diagram of another set of first pixels provided in an embodiment of this application;

[0043] Figure 7 This is a schematic diagram of a scenario where an optical tracking device is used to determine the spatial location information of an ultrasound image, as provided in an embodiment of this application.

[0044] Figure 8 This is a schematic diagram of the outline of a first region with hierarchical division provided in an embodiment of this application;

[0045] Figure 9 This is a schematic diagram of a second region outline provided in an embodiment of this application;

[0046] Figure 10This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0047] Figure 11 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0048] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0049] It should be noted that the terms "first," "second," "third," and "fourth," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. It should also be understood that the numbering before the steps in the embodiments of this application is for ease of understanding and description of the scheme, and should not be construed as a limitation on the order in which the steps are performed.

[0050] With the continuous development of computer technology, medical technology has also ushered in significant breakthroughs. Image 3D reconstruction technology has wide applications in fields such as computer vision and medical surgical navigation. For example, ultrasound images of a target organ can be acquired to reconstruct its contour, facilitating lesion analysis and surgical planning by doctors and other operators.

[0051] In this embodiment, an ultrasound image can be understood as an image acquired using an ultrasound device. It is understood that a wave that can cause a sound sensation in the auditory organs can be called a sound wave, while a sound wave that cannot be perceived by human sensory organs can be called an ultrasound wave. In this embodiment, the aforementioned ultrasound device can be understood as a device that uses an ultrasound beam to scan the object to be scanned, then receives the reflected signal of the ultrasound beam, and processes the reflected signal to obtain an image of the internal organs of the object to be scanned.

[0052] For ease of understanding, please refer to the example provided. Figure 1 , Figure 1 This is a schematic diagram of a scenario for acquiring ultrasound images provided in an embodiment of this application. For example... Figure 1 As shown, part 101 can be understood as the object to be scanned. In this embodiment of the application, the ultrasonic device may include an ultrasonic probe, such as... Figure 1The ultrasound probe 102 in the image may include an ultrasonic wave transmitting unit and an ultrasonic wave receiving unit, and can be used to transmit ultrasonic waves and receive ultrasonic waves reflected from the object to be scanned. The ultrasonic waves emitted by the probe are attenuated by the tissue of the object to be scanned and then reflected back to the probe. It is understood that the aforementioned ultrasound probe may also be referred to as a probe, ultrasonic probe, etc.

[0053] In this embodiment of the application, the object to be scanned can be understood as any object whose ultrasound image is acquired. For example, the above-mentioned object can be a person, such as a patient or volunteer, or an animal. This application does not limit this.

[0054] For example, the ultrasound device may also include a data processing unit, such as Figure 1 The data processing unit 104 is connected to the ultrasound probe 102 and processes the ultrasonic waves reflected from the object to be scanned received by the ultrasound probe 102 to obtain an ultrasound image. Optionally, the ultrasound device may also include a display unit, such as... Figure 1 The display unit 103 can be used to display the ultrasound images obtained by the data processing unit 104.

[0055] It is understandable that ultrasound images acquired through ultrasound equipment are two-dimensional images corresponding to a single cross-section. Since the target organ or other tissue is a three-dimensional spatial structure, the image acquirer using an ultrasound probe can rotate or move the probe to acquire multiple relatively complete ultrasound images, thereby obtaining ultrasound image information of the entire target organ. Specifically, as shown... Figure 1 As shown in section 105.

[0056] Ultrasound imaging offers advantages such as real-time performance and low examination cost. However, compared to computed tomography (CT) images, ultrasound images have lower resolution, providing less information to the surgeon during operation. Therefore, a CT scan can be performed preoperatively to acquire CT images. Then, the CT contour of the target organ is segmented from the CT image, and the ultrasound contour is segmented from the ultrasound image. The CT and ultrasound contours are then registered to obtain the registration result. Because CT images have high resolution and retain more detail, the surgeon can obtain more information from the registration result, thus reducing the difficulty of the surgery.

[0057] During ultrasound scanning, lesions in the target organ (such as stones) inevitably cause acoustic interference, resulting in low accuracy of ultrasound images in the vicinity of the lesion (especially in areas far from the ultrasound equipment). This leads to a large deviation in the accuracy of the segmented ultrasound contour, which in turn affects the subsequent registration accuracy and results in a large surgical puncture error.

[0058] To address the aforementioned problems, this application provides a contour segmentation method and related apparatus based on ultrasound images. This method improves the accuracy of reconstructing the ultrasound contour of a target organ from ultrasound images, thereby enhancing registration accuracy during ultrasound contour-based registration. The method provided in this application can be executed by an electronic device. This electronic device can be any device capable of executing the technical solutions disclosed in the method embodiments of this application. The electronic device can be a tablet computer, PDA, laptop computer, mobile Internet device (MID), wearable device, or scanning device, etc., and can also be a terminal device, server, or a server cluster composed of multiple servers, etc., without limitation. Optionally, the method embodiments of this application can also be implemented by a processor executing computer program code.

[0059] To facilitate understanding of the methods provided in the embodiments of this application, please refer to the following examples. Figure 2 , Figure 2 This is a schematic flowchart illustrating a contour segmentation method based on ultrasound images provided in an embodiment of this application. The method includes:

[0060] 201: Acquire multiple ultrasound images, which are obtained by scanning the target organ using an ultrasound device.

[0061] In this application embodiment, the target organ can be any organ capable of acquiring ultrasound images. For example, the target organ can be the kidney (including the left kidney and / or the right kidney), gallbladder, spleen, etc., and this application does not limit it.

[0062] It is understood that the target organ is a three-dimensional spatial structure, and multiple ultrasound images can be obtained by scanning various regions of the target organ using ultrasound equipment (such as rotating or moving the ultrasound probe). These multiple ultrasound images can be understood as images acquired after scanning the entire target organ, with each ultrasound image including a cross-section of the target organ. Integrating the information from these multiple ultrasound images yields a complete picture of the target organ.

[0063] In one possible implementation, the electronic device can establish a communication connection with the ultrasound device to acquire multiple ultrasound images. In another possible implementation, the electronic device may include a storage unit from which it can acquire multiple ultrasound images.

[0064] 202: Determine a first region contour from the plurality of ultrasound images, wherein the distance between the pixels on the first region contour and the ultrasound device is less than or equal to a first threshold.

[0065] It is understandable that the target organ is a three-dimensional structure; therefore, the complete outline of the target organ can be segmented from the aforementioned multiple ultrasound images. For ease of description, the complete outline of the target organ can be referred to as the overall outline, where the segmented overall outline can be understood as a three-dimensional outline.

[0066] In this embodiment, the distance between the pixels on the first region contour and the ultrasound device is less than or equal to a first threshold, which is used to express that the first region contour is a portion of the overall contour that is closer to the ultrasound device. This embodiment does not limit the specific value of the first threshold; it can be set according to actual conditions or experience. It can be understood that since the overall contour is a three-dimensional contour, the first region contour can also be understood as a three-dimensional contour.

[0067] It is understandable that when acquiring multiple ultrasound images using ultrasound equipment, the relative position between the ultrasound equipment and the target organ remains constant. Therefore, for the acquired ultrasound images or the segmented overall contour, the orientation of the ultrasound equipment can be reflected by the magnitude (or direction) of its coordinate values. Thus, when determining the contour of the first region, it can be based on the relative positional relationship between the ultrasound equipment and the overall contour or the ultrasound images.

[0068] For example, in the three-dimensional coordinate system to which the overall contour belongs, the x-direction points towards the ultrasound device. Therefore, pixels with larger x-coordinate values ​​on the overall contour are closer to the ultrasound device. Thus, the set of pixels on the overall contour with x-coordinate values ​​greater than a certain threshold can be understood as the aforementioned first region contour.

[0069] For easier understanding, please refer to Figure 3 , Figure 3 This is a schematic diagram of the outline of a first region provided in an embodiment of this application.

[0070] For example, such as Figure 3 The contour 300 shown in (a) can be understood as the overall contour (three-dimensional contour) of the target organ. Assume that in the coordinate system containing the overall contour, the ultrasound device is located in the x-direction of the overall contour (e.g., Figure 3 (As indicated by the arrow in the image), meaning that pixels with larger x-coordinate values ​​on the overall contour are closer to the ultrasound device. Therefore, the set of pixels on the overall contour with x-coordinate values ​​greater than a certain threshold can be understood as the first region contour mentioned above.

[0071] For example Figure 3 As shown in (b), the x-coordinate value is greater than x. a The outline 301 can be understood as the outline of the first region mentioned above. For example... Figure 3 As shown in (c), the x-coordinate value is greater than x. bThe outline 302 can also be understood as the outline of the first region mentioned above.

[0072] In one possible implementation, the electronic device can first segment the overall contour of the target organ from the ultrasound image, and then segment a first region contour from the overall contour. In another possible implementation, the electronic device can determine the required set of pixels from the pixels of the target organ in a cross-section of the ultrasound image, and then obtain the first region contour based on the set of pixels determined in each ultrasound image. Therefore, in some embodiments, step 202 above, determining the first region contour from the multiple ultrasound images, includes:

[0073] 2021: Each ultrasound image is segmented to obtain multiple target images, each containing a set of pixels on a cross section of the target organ.

[0074] In this step, the electronic device can segment each ultrasound image based on an artificial intelligence segmentation algorithm to obtain multiple target images. Each target image includes a set of pixels representing a cross-section of the target organ.

[0075] For easier understanding, please refer to Figure 4 , Figure 4 This is a schematic diagram of a target image provided in an embodiment of this application. For example... Figure 4 As shown in the shaded area, each target image comprises a set of pixels on a cross-section of the target organ.

[0076] 2022: Determine a first set of pixels from the target image, wherein the pixels in the first set of pixels are located on the contour of the target organ.

[0077] 2023: The first region contour is obtained based on the first set of sub-pixels in each target image.

[0078] In this embodiment, the electronic device first determines the first set of pixels from each two-dimensional ultrasound image, and then obtains the first region contour based on the first set of sub-pixels in each target image. Compared with directly segmenting the three-dimensional overall contour and then segmenting the first region contour from the three-dimensional overall contour, the above method is simpler to implement and the obtained first region contour is more accurate.

[0079] In this embodiment, the electronic device determines a first set of pixels from the target image, and then obtains the first region contour based on the first set of pixels determined in each ultrasound image. It is understood that since the first set of pixels ultimately yields the first region contour, the distance between the pixels in the first set and the ultrasound device is also less than or equal to the first threshold.

[0080] Understandably, in practical implementations, the orientation of the ultrasound device can be reflected by the magnitude (or direction) of coordinate values ​​in the target image. Therefore, the electronic device can determine the aforementioned first set of pixels based on the coordinate values ​​of the set of pixels in the target image. For example, in the two-dimensional coordinate system of the target image, the y-direction points towards the ultrasound device; thus, pixels with larger y-coordinate values ​​are closer to the ultrasound device. Therefore, the set of pixels in the target image whose y-coordinate values ​​are greater than a certain threshold and located on the contour of the target organ can be understood as the aforementioned first set of pixels.

[0081] In this embodiment, the pixels in the first pixel set are located on the outline of the target organ. For easier understanding, please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of a first pixel set provided in an embodiment of this application.

[0082] like Figure 5 As shown, assuming the y-direction points towards the ultrasound device in the two-dimensional coordinate system of the target image, then the pixels located on the contour of the target organ and whose y-coordinate is greater than a certain threshold can be understood as the aforementioned first set of pixels. For example, pixels located on the contour of the target organ and whose y-coordinate value is greater than a certain threshold... a The set of pixels 501 can be understood as the first set of pixels.

[0083] It is understood that when determining the first set of pixels from the target image, the electronic device may extract the outline of the upper half of the target organ as the first set of pixels based on the cross-section of the target organ in the target image. In some embodiments, step 2022 above, determining the first set of pixels from the target image, includes:

[0084] 20221: Determine the median between the maximum and minimum coordinate values ​​of a pixel in the target image in the first direction; the first direction is the direction in which the target organ points towards the ultrasound device.

[0085] 20222: The first set of pixels is defined as the pixels whose coordinates are greater than or equal to the median in the first direction.

[0086] In this embodiment, the first direction is the direction from the target organ to the ultrasound device, used to express the relative positional relationship between pixels in the target image and the ultrasound device. In specific implementations, the relative positional relationship between the ultrasound device and the target image is determined, and the aforementioned first direction can be reflected by the magnitude (or direction) of coordinate values ​​in the target image. For example, the aforementioned first direction can be the x-direction of the coordinate system in the target image, or it can be the y-direction, etc.

[0087] For easier understanding, please refer to Figure 6 , Figure 6 This is a schematic diagram of another set of first pixels provided in an embodiment of this application.

[0088] like Figure 6 As shown, assuming that in the two-dimensional coordinate system of the target image, the y-direction is the first direction mentioned above, that is, the direction in which the target organ points towards the ultrasound device. The electronic device can determine the maximum coordinate value y of the pixel in the target image in the first direction. max and the minimum value of coordinate y min Based on the maximum coordinate y max and the minimum value of coordinate y min Get the median y mid Among them, the middle number y mid It can be the maximum value of coordinate y max and the minimum value of coordinate y min Half of the sum. Then, the y-coordinate in the target image is greater than or equal to the median y. mid The set of pixels 601 located on the outline of the target organ can be understood as the first set of pixels mentioned above.

[0089] In this embodiment, it can be understood that the half of the contour of the target organ that is closer to the ultrasound device is taken as the final contour of the target organ. This contour selection method can retain sufficient contour information and remove low-precision contour data as much as possible, so as to ensure the registration accuracy of subsequent registration based on the first region contour.

[0090] Optionally, the electronic device can obtain a first region contour from a first set of pixels determined in each target image using an optical tracking device. In some embodiments, step 2023 above, obtaining the first region contour based on the first sub-pixel set in each target image, includes:

[0091] 20231: Obtain the transformation matrix, which is used to locate the ultrasound image within the tracking range of the optical tracking device.

[0092] 20232: Determine the spatial location information of each ultrasound image based on the transformation matrix.

[0093] 20233: Based on spatial location information, the set of first pixels in each target image is reconstructed into the contour of the first region.

[0094] In this embodiment, the optical tracking device can be understood as a device for locating objects within a tracking range. Exemplarily, the optical tracking device can use markers within the tracking range to determine a transformation matrix, and then determine the positions of other objects within the tracking range based on the transformation matrix. For easier understanding, please refer to [link to relevant documentation]. Figure 7 , Figure 7This is a schematic diagram of a scenario where an optical tracking device is used to determine the spatial location information of an ultrasound image, as provided in an embodiment of this application.

[0095] For example, taking the acquisition of ultrasound images of the kidney as an example, such as Figure 7 As shown, part 701 can be understood as the object to be scanned. The ultrasound device may include an ultrasound probe 702, which scans the kidney of the object to be scanned. The ultrasound probe 702 is connected to a support 703, which includes multiple markers. For example, the number of markers on the support 703 is greater than or equal to three. Figure 7 The example shown uses four markers, but five or six markers can also be set according to the actual situation. This application does not limit this.

[0096] First, it's understandable that without optical tracking for positioning, the ultrasound images acquired by the ultrasound equipment are merely two-dimensional images, lacking three-dimensional spatial information. After activating the optical tracking device, the three-dimensional spatial position information of the two-dimensional ultrasound image can be obtained through the coordinates of markers on the ultrasound probe.

[0097] It is understood that the optical tracking device 704 can determine the three-dimensional spatial coordinates of each marker on the support 703 through the reflective coating on the surface of the marker. Then, the optical tracking device 704 can obtain a transformation matrix from the coordinates of the marker, where the transformation matrix can be understood as the transformation matrix between the reference coordinates of the reference object (e.g., the coordinates of the marker mentioned above) acquired by the optical tracking device and the coordinates of the origin. Since it is a three-dimensional spatial coordinate system, the above transformation matrix can include rotation components and translation components. For example, the above transformation matrix can be represented by equation (1):

[0098]

[0099] Among them, H us This represents the transformation matrix. Subsequently, other objects with a defined positional relationship to the marker can have their spatial position information determined using the transformation matrix. It is understandable that, since the structure of the support 703 is fixed, the spatial position information within the scanning range below the ultrasound probe 702, i.e., the spatial position information of the acquired ultrasound image, can be determined through the structural relationship between the marker and the support 703 and the transformation matrix.

[0100] It is understandable that, based on determining the spatial location information of each ultrasound image, the spatial location information of each target image can also be determined. Based on the determined spatial location information, the electronic device can reconstruct a first region contour from the first set of pixels in each target image, wherein the aforementioned first region contour is a three-dimensional contour.

[0101] For example, an electronic device can acquire data of the first set of pixels in each target image. Then, multiply the matrix by the data to obtain the outline of the first region, i.e.

[0102] 203: Use the outline of the first region as the outline of the target organ.

[0103] In this embodiment, after acquiring multiple ultrasound images of a target organ using an ultrasound device, a first region contour is determined from these ultrasound images. The distance between pixels on the first region contour and the ultrasound device is less than or equal to a first threshold. Because lesions (e.g., stones) in the target organ inevitably cause acoustic interference during ultrasound scanning, the accuracy of the ultrasound image in the vicinity of the lesion (especially in areas far from the ultrasound device) is low, resulting in a large deviation in the accuracy of the segmented ultrasound contour. This application uses the first region contour as the contour of the target organ, which can effectively remove low-precision contour data from the aforementioned multiple ultrasound images, improve the accuracy of the target organ contour reconstructed from the ultrasound images, and thus improve registration accuracy when registering based on the target organ contour.

[0104] In this embodiment, steps 201-203 can be understood as the first reconstruction of the target organ contour, i.e., obtaining the first region contour based on multiple ultrasound images. However, in reality, due to the influence of segmentation algorithms or reconstruction methods, the first region contour reconstructed by the electronic device may have hierarchical divisions. For ease of understanding, please refer to... Figure 8 , Figure 8 This is a schematic diagram of the outline of a first region with hierarchical division provided in an embodiment of this application. For example... Figure 8 As shown, the first region contour reconstructed by the electronic device includes layers 801 and 802.

[0105] Understandable, Figure 8 This is merely an example; in reality, the first region contour may have two layers in some areas, while having one or three layers in other areas, etc. To further improve the accuracy of the target organ contour, a second reconstruction can optionally be performed based on the obtained first region contour. Therefore, in some embodiments, step 203, using the first region contour as the contour of the target organ, includes:

[0106] 2031: Based on the distance between points on the first region contour and the ultrasound device, select the second region contour from the first region contour.

[0107] 2032: Use the contour of the second region as the contour of the target organ.

[0108] It is understandable that the first region contour is a three-dimensional contour determined from the ultrasound image, and the distance between the points on the first region contour and the ultrasound device can be determined based on the coordinates of the points on the first region contour.

[0109] It is understood that the relative positional relationship between the ultrasound device and the first region contour is fixed. Therefore, the distance between points on the first region contour and the ultrasound device can be reflected by coordinate values. For example, the electronic device can select the set of points closest to the ultrasound device in the direction from the target organ to the ultrasound device as the second region contour.

[0110] In some embodiments, step 2031, selecting a second region contour from the first region contour based on the distance between points on the first region contour and the ultrasound device, includes:

[0111] 20311: Determine the target point from the outline of the first region. The target point is any point on the outline of the first region.

[0112] 20312: Based on the target point, candidate points are determined from the contour of the first region. The absolute value of the difference between the coordinate values ​​of the candidate point and the target point in the second direction is less than or equal to a second threshold. The second direction is perpendicular to the third direction, which is the direction in which the target organ points to the ultrasound device.

[0113] 20313: Select the point closest to the ultrasound device among the target point and candidate points as the point on the second region contour.

[0114] It is understood that the first region contour is a three-dimensional structure, including multiple points. In this embodiment, each point on the first region contour is selected as a target point, and then points on the second region contour are determined based on the target points. Therefore, the above embodiment can be understood as a loop, and the loop ends after all points on the first region contour have been determined as target points.

[0115] In this embodiment, the second direction is perpendicular to the third direction, where the third direction is the direction from which the target organ points to the ultrasound device, used to express the relative positional relationship between points in the first region contour and the ultrasound device. In specific implementations, the relative positional relationship between the ultrasound device and points in the first region contour is determined, and the aforementioned first direction can be reflected by the magnitude and direction of the coordinate values ​​in the coordinate system containing the first region contour. For example, the aforementioned third direction can be the x-direction in the coordinate system containing the first region contour, then the aforementioned second direction can be either the y-direction or the z-direction. Optionally, the aforementioned third direction can be understood as the first direction.

[0116] After determining the target point, the electronic device identifies candidate points on the first region contour. Understandably, the number of candidate points identified can be at least one or zero. If at least one candidate point is identified, the electronic device uses one target point and the point among the at least one candidate point that is closest to the ultrasound device as a point on the second region contour; if no candidate points are identified (i.e., zero candidate points), the electronic device uses the target point as a point on the second region contour.

[0117] In this embodiment, the specific value of the second threshold is not limited and can be determined empirically, such as 0.1, 0.5, etc. For ease of understanding, exemplarily, assume the coordinates of the target point are (1,1,1), the second threshold is 0.5, and the third direction is the x-direction of the coordinate system where the first region contour is located. Then, points whose distance from the target point's coordinates in the y-direction is less than or equal to 0.5, and whose distance from the target point's coordinates in the z-direction is less than or equal to 0.5, can be selected as candidate points. For example, points (0.3,1.1,1.3), (1.6,0.9,0.8), and (2,1,1.4) will all be selected as candidate points. In the above example, point (2,1,1.4) is closest to the ultrasound device; therefore, point (2,1,1.4) is selected as a point on the second region contour.

[0118] Using the above method, the outline of the layered first region can be reconstructed into the outline of the single-layered second region, for example... Figure 9 As shown. From Figure 8 The first region contour is obtained Figure 9 The second region contour can be understood as a second reconstruction. Because the second region contour has higher precision, the registration accuracy is also higher and the surgical error is smaller in the subsequent registration process.

[0119] The methods of the embodiments of this application have been described in detail above. The apparatus of the embodiments of this application is provided below.

[0120] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 is used to execute the contour segmentation method based on ultrasound images provided in the above embodiment. It is understood that any device capable of implementing the method provided in this application falls within the protection scope of this application. Exemplarily, the electronic device 100 may be a laptop computer or a desktop computer, etc., and this embodiment of the application does not limit its scope.

[0121] like Figure 10 As shown, the electronic device 100 includes an acquisition unit 1001 and a determination unit 1002, and optionally, may also include a segmentation unit 1003 and a reconstruction unit 1004. The descriptions of each unit are as follows:

[0122] The acquisition unit 1001 is used to acquire multiple ultrasound images, which are obtained by scanning the target organ using an ultrasound device.

[0123] The determining unit 1002 is used to determine a first region contour from the plurality of ultrasound images, wherein the distance between the pixels on the first region contour and the ultrasound device is less than or equal to a first threshold.

[0124] The determining unit 1002 is also used to use the outline of the first region as the outline of the target organ.

[0125] Optionally, it also includes a segmentation unit 1003 for segmenting each of the above ultrasound images to obtain multiple target images, wherein the target images include a set of pixels of the target organ on a cross section.

[0126] The determining unit 1002 is further configured to determine a first set of pixels from the target image, wherein the pixels in the first set of pixels are located on the outline of the target organ;

[0127] The reconstruction unit 1004 is used to obtain the outline of the first region based on the first set of pixels in each of the above target images.

[0128] Optionally, the acquisition unit 1001 is further configured to acquire a transformation matrix, which is used to locate the ultrasound image within the tracking range of the optical tracking device;

[0129] The determining unit 1002 is also used to determine the spatial location information of each of the above-mentioned ultrasound images based on the above-mentioned transformation matrix;

[0130] The reconstruction unit 1004 is specifically used to reconstruct the first pixel set in each of the above target images into the outline of the first region based on the above spatial location information.

[0131] Optionally, the determining unit 1002 is further configured to determine the median number between the maximum and minimum coordinate values ​​of a pixel in the target image in a first direction; the first direction is the direction in which the target organ points toward the ultrasound device.

[0132] The determining unit 1002 is further configured to include the pixel points whose coordinate values ​​in the first direction are greater than or equal to the intermediate number as the first pixel point set.

[0133] Optionally, the determining unit 1002 is further configured to filter out a second region contour from the first region contour based on the distance between points on the first region contour and the ultrasound device.

[0134] The determining unit 1002 is also used to use the outline of the second region as the outline of the target organ.

[0135] Optionally, the determining unit 1002 is further configured to determine a target point from the outline of the first region, wherein the target point is any point on the outline of the first region;

[0136] The determining unit 1002 is further configured to determine candidate points on the contour of the first region based on the target point, wherein the absolute value of the difference between the coordinate values ​​of the candidate points and the target points in the second direction is less than or equal to a second threshold, the second direction is perpendicular to a third direction, and the third direction is the direction in which the target organ points to the ultrasound device.

[0137] The determining unit 1002 is further configured to use the point closest to the ultrasonic device among the target point and the candidate points as a point on the contour of the second region.

[0138] Please see Figure 11 , Figure 11 This is a schematic diagram of another electronic device provided in an embodiment of this application. The electronic device 110 can be used to implement the above-described contour segmentation method based on ultrasound images.

[0139] like Figure 11 As shown, the electronic device 110 includes a memory 1101 and a processor 1102. Optionally, the electronic device 110 may also include a communication interface 1103 and a bus 1104; wherein the memory 1101, the processor 1102 and the communication interface 1103 are connected to each other through the bus 1104.

[0140] The memory 1101 is used to provide storage space, in which data such as the operating system and computer programs can be stored. The memory 1101 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0141] Processor 1102 is a module that performs arithmetic and logical operations, and can be one or a combination of processing modules such as a central processing unit (CPU), a graphics processing unit (GPU), or a microprocessor unit (MPU). Additionally, memory 1101 stores computer programs, and processor 1102 can call the computer programs stored in memory 1101 to execute corresponding methods.

[0142] In the embodiments of this application, such as Figure 11 When the electronic device 110 shown executes the above-described contour segmentation method based on ultrasound images, the processor 1102 can control the data communication function of the communication interface 1103.

[0143] For example, in some embodiments, processor 1102 is configured to acquire multiple ultrasound images;

[0144] Processor 1102 is used to determine the contour of a first region from multiple ultrasound images;

[0145] Processor 1102 is configured to use the contour of the first region as the contour of the target organ.

[0146] In other embodiments, the processor 1102 may be used to implement the functions of the acquisition unit 1001, the determination unit 1002, the segmentation unit 1003, and the reconstruction unit 1004 in the electronic device 100. Optionally, the image acquired by the acquisition unit 1001 in the electronic device 100 may also be acquired through the communication interface 1103.

[0147] This application also provides a computer-readable storage medium storing a computer program that, when run on a processor, can implement the methods described in this application.

[0148] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the above claims.

Claims

1. A contour segmentation method based on ultrasound images, characterized in that, The method includes: Multiple ultrasound images are acquired, which are obtained by scanning the target organ using an ultrasound device; A first region contour is determined from the multiple ultrasound images, wherein the distance between the pixels on the first region contour and the ultrasound device is less than or equal to a first threshold; the first region contour is a portion of the overall contour that is closer to the ultrasound device. The first region contour is used as the contour of the target organ, and the contour of the target organ is used for registration with the CT contour. The CT contour is obtained by segmenting from the CT image, and the CT image is obtained by performing a CT scan on the target organ. Determining the contour of the first region from the plurality of ultrasound images includes: Each ultrasound image is segmented to obtain multiple target images, each target image including a set of pixels of the target organ on a cross section; A first set of pixels is determined from the target image, wherein the pixels in the first set of pixels are located on the outline of the target organ; The first region contour is obtained based on the first set of pixels in each target image; The step of obtaining the first region contour based on the first sub-pixel set in each target image includes: A transformation matrix is ​​obtained, which is used to locate the ultrasound image within the tracking range of the optical tracking device; the optical tracking device is used to obtain the three-dimensional spatial position information of the ultrasound image through the coordinates of the markers on the ultrasound probe of the ultrasound device, and the transformation matrix is ​​the transformation matrix between the coordinates of the markers acquired by the optical tracking device and the coordinates of the origin; The spatial location information of each ultrasound image is determined based on the transformation matrix. Based on the spatial location information, the first set of pixels in each target image is reconstructed into the outline of the first region; Determining the first set of pixels from the target image includes: Determine the midpoint between the maximum and minimum coordinate values ​​of a pixel in the target image along a first direction; the first direction is the direction in which the target organ points towards the ultrasound device. The first set of pixels is defined as the pixels whose coordinates in the first direction are greater than or equal to the intermediate number.

2. The method according to claim 1, characterized in that, The step of using the contour of the first region as the contour of the target organ includes: Based on the distance between points on the first region contour and the ultrasound device, a second region contour is selected from the first region contour. The contour of the second region is used as the contour of the target organ.

3. The method according to claim 2, characterized in that, The step of filtering out a second region contour from the first region contour based on the distance between points on the first region contour and the ultrasound device includes: A target point is determined from the contour of the first region, wherein the target point is any point on the contour of the first region. Candidate points are determined on the contour of the first region based on the target point. The absolute value of the difference between the coordinate values ​​of the candidate point and the target point in the second direction is less than or equal to a second threshold. The second direction is perpendicular to a third direction, which is the direction in which the target organ points to the ultrasound device. The point closest to the ultrasound device among the target point and the candidate points is taken as a point on the contour of the second region.

4. A contour segmentation device based on ultrasound images, characterized in that, The device includes: The acquisition unit is used to acquire multiple ultrasound images, which are obtained by scanning the target organ using an ultrasound device. A determining unit is configured to determine a first region contour from the plurality of ultrasound images; the distance between the first region contour and the ultrasound device is less than the distance between other region contours and the ultrasound device, wherein the other region contours are contours on the plurality of ultrasound images other than the first region contour; the first region contour is a portion of the overall contour that is closer to the ultrasound device. The determining unit is further configured to use the first region contour as the contour of the target organ, the contour of the target organ is used to register with the CT contour, the CT contour is obtained by segmenting from the CT image, and the CT image is obtained by performing a CT scan on the target organ. A segmentation unit is used to segment each of the ultrasound images to obtain multiple target images, wherein the target images include a set of pixels of the target organ on a cross section; The determining unit is further configured to determine a first set of pixels from the target image, wherein the pixels in the first set of pixels are located on the outline of the target organ; A reconstruction unit is used to obtain the contour of the first region based on the first set of pixels in each of the target images; The acquisition unit is further configured to acquire a transformation matrix, which is used to locate the ultrasound image within the tracking range of the optical tracking device; the optical tracking device is configured to obtain the three-dimensional spatial position information of the ultrasound image through the coordinates of the markers on the ultrasound probe of the ultrasound device, and the transformation matrix is ​​the transformation matrix between the coordinates of the markers acquired by the optical tracking device and the coordinates of the origin; The determining unit is further configured to determine the spatial location information of each ultrasound image based on the transformation matrix; The reconstruction unit is specifically used to reconstruct the first region contour from the first set of pixels in each target image based on the spatial location information. The determining unit is further configured to determine the median number between the maximum and minimum coordinate values ​​of a pixel in the target image in a first direction; the first direction is the direction in which the target organ points towards the ultrasound device; The determining unit is further configured to include pixels whose coordinate values ​​in the first direction are greater than or equal to the intermediate number as the first set of pixels.

5. An electronic device, characterized in that, The device includes a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions such that the method as described in any one of claims 1-3 is executed.

6. A chip, characterized in that, Includes logic circuits and interfaces, wherein the logic circuits and interfaces are coupled; The interface is used to input and / or output code instructions, and the logic circuit is used to execute the code instructions to cause the method of any one of claims 1-3 to be performed.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the method as described in any one of claims 1-3 to be performed.

Images