Rib-based respiration monitoring method, surgical robot and related products

Abstract

The application discloses a rib-based respiration monitoring method, a surgical robot and related products. The method comprises the following steps: acquiring at least one first two-dimensional ultrasound image, and the image in the at least one first two-dimensional ultrasound image comprises a first object. A region corresponding to a rib of the first object is determined from the at least one first two-dimensional ultrasound image, and at least one first rib region is obtained. At least one second rib region corresponding to the first rib of the first object is obtained based on the at least one first rib region. The respiration rule of the first object is obtained based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time at which the at least one first two-dimensional ultrasound image is acquired. The respiration rule of the first object obtained based on the method provided in the application can improve the accuracy of the respiration rule, and in turn, the accuracy of monitoring the respiration of the first object can be improved.

Description

Technical Field

[0001] This application relates to the field of medical imaging technology, and in particular to a rib-based respiratory monitoring method, a surgical robot, and related products. Background Technology

[0002] After acquiring medical images of a subject using medical imaging equipment, information about the subject can be obtained based on these images. These medical images include ultrasound images and computed tomography (CT) images. The information obtained includes the location information of points within the subject's skin and tissues. However, because the subject's breathing causes movement within the skin and tissues, the location information of points within these areas changes with the breathing pattern. Therefore, understanding the subject's breathing patterns is of great importance.

[0003] Current technologies use the following method to obtain the subject's breathing patterns: optical markers are attached to the skin area of ​​the subject's body, then the position information of the optical markers at different times is determined by an optical tracking device, and finally the subject's breathing patterns are obtained based on the position information of the optical markers at different times.

[0004] However, the accuracy of breathing patterns obtained based on this technology is low. Summary of the Invention

[0005] This application provides a rib-based respiratory monitoring method, a surgical robot, and related products. The related products include a rib-based respiratory monitoring device, electronic equipment, and a computer-readable storage medium to improve the accuracy of respiratory patterns, thereby improving the accuracy of monitoring the respiration of a first subject.

[0006] Firstly, a rib-based respiratory monitoring method is provided, the method comprising:

[0007] Acquire at least one first two-dimensional ultrasound image, wherein the image in the at least one first two-dimensional ultrasound image includes a first object;

[0008] Determine the region corresponding to the rib of the first object from the at least one first two-dimensional ultrasound image to obtain at least one first rib region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region in the at least one first rib region;

[0009] Based on the at least one first rib region, at least one second rib region is obtained, the at least one second rib region corresponds to the first rib of the first object, and the region in the at least one second rib region corresponds one-to-one with the image in the at least one first two-dimensional ultrasound image;

[0010] Based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time, the breathing pattern of the first object is obtained, wherein the time in the at least one acquisition time is the time when the image in the at least one first two-dimensional ultrasound image is acquired, and the time in the at least one acquisition time corresponds one-to-one with the image in the at least one first two-dimensional ultrasound image.

[0011] In conjunction with any embodiment of this application, determining the region corresponding to the rib of the first object from the at least one first two-dimensional ultrasound image to obtain at least one first rib region includes:

[0012] The region corresponding to the pleura of the first object is determined from the at least one first two-dimensional ultrasound image to obtain at least one first pleural region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region among the at least one first pleural region;

[0013] From the at least one first two-dimensional ultrasound image, a region with a gray value greater than or equal to a gray value threshold is determined to obtain at least one rib candidate region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region among the at least one rib candidate region;

[0014] Based on the at least one first pleural region, the at least one first rib region is determined from the at least one rib candidate region.

[0015] In any embodiment of this application, the at least one first two-dimensional ultrasound image is obtained by scanning the first object using an ultrasound probe; the pixels in the ultrasound image obtained by scanning the object using the ultrasound probe are ultrasound pixels, and the coordinates of the ultrasound pixels are negatively correlated with the distance from the scanned object corresponding to the ultrasound pixel to the ultrasound probe; when obtaining the at least one first two-dimensional ultrasound image using an ultrasound probe, the distance from the ribs of the first object to the ultrasound probe is smaller than the distance from the pleura of the first object to the ultrasound probe;

[0016] Determining the at least one first rib region from the at least one rib candidate region based on the at least one first pleural region includes:

[0017] At least one pleural location is obtained based on the coordinates of the at least one first pleural region in the at least one first two-dimensional ultrasound image;

[0018] Based on the coordinates of the at least one rib candidate region in the at least one first two-dimensional ultrasound image, at least one rib candidate position is obtained;

[0019] The difference between each of the at least one candidate rib location and each of the at least one pleural location is determined to obtain at least one first difference value;

[0020] At least one second difference is determined from the at least one first difference, each of the at least one second difference being greater than or equal to a first threshold and less than or equal to a second threshold, wherein the first threshold is a positive number;

[0021] Based on the at least one second difference, at least one first rib position is determined from the at least one rib candidate position, wherein the at least one second difference is obtained based on the at least one first rib position;

[0022] The at least one first rib region is obtained based on the region corresponding to the position of the at least one first rib in at least one rib candidate region.

[0023] In any embodiment of this application, the at least one first two-dimensional ultrasound image is obtained by scanning while the pose of the ultrasound probe remains unchanged;

[0024] The process of obtaining at least one second rib region based on the at least one first rib region includes:

[0025] Clustering is performed on at least one first rib position in the at least one first rib region to obtain at least one rib position class, wherein the difference between any two first rib positions in any one rib position class is less than or equal to a third threshold.

[0026] The at least one second rib region is determined from the at least one first rib region based on the first rib position in any one of the at least one rib position classes.

[0027] In conjunction with any embodiment of this application, obtaining the respiratory pattern of the first object based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time includes:

[0028] Determine at least one second two-dimensional ultrasound image from the at least one first two-dimensional ultrasound image that includes the at least one second rib region;

[0029] Determine the location of the region corresponding to the clavicle of the first object in the at least one second two-dimensional ultrasound image to obtain at least one clavicle location;

[0030] Determine the position of the at least one second rib region in the at least one second two-dimensional ultrasound image to obtain the position of at least one second rib.

[0031] Determine the difference between the position of the at least one clavicle and the position of the at least one second rib to obtain at least one third difference;

[0032] Based on the at least one third difference and the at least one acquisition time, the breathing pattern of the first object is obtained.

[0033] Optionally, obtaining the breathing pattern of the first object based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time includes: obtaining the breathing pattern of the first object based on the position of the center of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time.

[0034] In conjunction with any embodiment of this application, the method further includes:

[0035] Obtain a fourth difference value, which is the difference between the position of the first object's clavicle at the planning time and the position of the first rib at the planning time. The planning time is the time when the target path is planned. The starting point of the target path is a point in the skin region of the first object, and the ending point of the target path is a point in the tissue of the first object.

[0036] Based on the fourth difference and the breathing pattern of the first object, a target time is obtained. The difference between the breathing pattern and the target time is the target difference. The absolute value of the difference between the target difference and the fourth difference is less than or equal to a third threshold. The target time is the time when the object moves from its skin area to its tissue based on the target path.

[0037] Secondly, a rib-based respiratory monitoring device is provided, the rib-based respiratory monitoring device comprising:

[0038] An acquisition unit is configured to acquire at least one first two-dimensional ultrasound image, wherein the image in the at least one first two-dimensional ultrasound image includes a first object;

[0039] A processing unit is configured to determine, from the at least one first two-dimensional ultrasound image, a region corresponding to the rib of the first object, and to obtain at least one first rib region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region among the at least one first rib region.

[0040] The processing unit is further configured to obtain at least one second rib region based on the at least one first rib region, wherein the at least one second rib region corresponds to the first rib of the first object, and the region in the at least one second rib region corresponds one-to-one with the image in the at least one first two-dimensional ultrasound image;

[0041] The processing unit is further configured to obtain the breathing pattern of the first object based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time, wherein the time in the at least one acquisition time is the time when the image in the at least one first two-dimensional ultrasound image is acquired, and the time in the at least one acquisition time corresponds one-to-one with the image in the at least one first two-dimensional ultrasound image.

[0042] In conjunction with any embodiment of this application, the processing unit is further configured to:

[0043] The region corresponding to the pleura of the first object is determined from the at least one first two-dimensional ultrasound image to obtain at least one first pleural region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region among the at least one first pleural region;

[0044] From the at least one first two-dimensional ultrasound image, a region with a gray value greater than or equal to a gray value threshold is determined to obtain at least one rib candidate region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region among the at least one rib candidate region;

[0045] Based on the at least one first pleural region, the at least one first rib region is determined from the at least one rib candidate region.

[0046] In any embodiment of this application, the at least one first two-dimensional ultrasound image is obtained by scanning the first object using an ultrasound probe; the pixels in the ultrasound image obtained by scanning the object using the ultrasound probe are ultrasound pixels, and the coordinates of the ultrasound pixels are negatively correlated with the distance from the scanned object corresponding to the ultrasound pixel to the ultrasound probe; when obtaining the at least one first two-dimensional ultrasound image using an ultrasound probe, the distance from the ribs of the first object to the ultrasound probe is smaller than the distance from the pleura of the first object to the ultrasound probe;

[0047] The processing unit is further configured to:

[0048] At least one pleural location is obtained based on the coordinates of the at least one first pleural region in the at least one first two-dimensional ultrasound image;

[0049] Based on the coordinates of the at least one rib candidate region in the at least one first two-dimensional ultrasound image, at least one rib candidate position is obtained;

[0050] The difference between each of the at least one candidate rib location and each of the at least one pleural location is determined to obtain at least one first difference value;

[0051] At least one second difference is determined from the at least one first difference, each of the at least one second difference being greater than or equal to a first threshold and less than or equal to a second threshold, wherein the first threshold is a positive number;

[0052] Based on the at least one second difference, at least one first rib position is determined from the at least one rib candidate position, wherein the at least one second difference is obtained based on the at least one first rib position;

[0053] The at least one first rib region is obtained based on the region corresponding to the position of the at least one first rib in at least one rib candidate region.

[0054] In any embodiment of this application, the at least one first two-dimensional ultrasound image is obtained by scanning while the pose of the ultrasound probe remains unchanged;

[0055] The processing unit is further configured to:

[0056] Clustering is performed on at least one first rib position in the at least one first rib region to obtain at least one rib position class, wherein the difference between any two first rib positions in any one rib position class is less than or equal to a third threshold.

[0057] The at least one second rib region is determined from the at least one first rib region based on the first rib position in any one of the at least one rib position classes.

[0058] In conjunction with any embodiment of this application, the processing unit is further configured to:

[0059] Determine at least one second two-dimensional ultrasound image from the at least one first two-dimensional ultrasound image that includes the at least one second rib region;

[0060] Determine the location of the region corresponding to the clavicle of the first object in the at least one second two-dimensional ultrasound image to obtain at least one clavicle location;

[0061] Determine the position of the at least one second rib region in the at least one second two-dimensional ultrasound image to obtain the position of at least one second rib.

[0062] Determine the difference between the position of the at least one clavicle and the position of the at least one second rib to obtain at least one third difference;

[0063] Based on the at least one third difference and the at least one acquisition time, the breathing pattern of the first object is obtained.

[0064] Optionally, obtaining the breathing pattern of the first object based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time includes: obtaining the breathing pattern of the first object based on the position of the center of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time.

[0065] In conjunction with any embodiment of this application, the acquisition unit is further configured to acquire a fourth difference, the fourth difference being the difference between the position of the first object's clavicle at the planning time and the position of the first rib at the planning time, the planning time being the time when the target path is planned, the starting point of the target path being a point in the skin region of the first object, and the ending point of the target path being a point in the tissue of the first object;

[0066] The processing unit is further configured to obtain a target time based on the fourth difference and the breathing pattern of the first object, wherein the difference between the breathing pattern and the target time is the target difference, the absolute value of the difference between the target difference and the fourth difference is less than or equal to a third threshold, and the target time is the time when the object moves from the skin area of ​​the first object to the tissue of the first object based on the target path.

[0067] Thirdly, a surgical robot is provided, including a rib-based respiratory monitoring device as described in the second aspect. In this third aspect, the surgical robot can perform a rib-based respiratory monitoring method using the rib-based respiratory monitoring device, thereby improving the accuracy of respiratory patterns.

[0068] Fourthly, an electronic device is provided, comprising: a processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs as described in the first aspect and any of its embodiments.

[0069] Fifthly, another electronic device is provided, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs as described in the first aspect and any of its embodiments.

[0070] In a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored therein, the computer program including program instructions that, when executed by a processor, cause the processor to perform the first aspect and any of its embodiments described above.

[0071] In a seventh aspect, a computer program product is provided, the computer program product comprising a computer program or instructions that, when the computer program or instructions are executed on a computer, cause the computer to perform the first aspect and any of its embodiments described above.

[0072] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application.

[0073] In this embodiment, at least one first two-dimensional ultrasound image includes a first object. After acquiring at least one first two-dimensional ultrasound image, the rib-based respiratory monitoring device determines the region corresponding to the ribs of the first object from the at least one first two-dimensional ultrasound image, obtaining at least one first rib region. Then, based on the at least one first rib region, at least one second rib region is obtained, wherein the at least one second rib region corresponds to the first ribs of the first object, and the regions within the at least one second rib region correspond one-to-one with the images in the at least one first two-dimensional ultrasound image. Based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time, the respiratory pattern of the first object is obtained. This allows the respiratory pattern of the object to be obtained without using an optical tracking device to determine the position information of optical markers, thereby improving the accuracy of the respiratory pattern and reducing damage to the object's skin area. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0075] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0076] Figure 1A schematic flowchart of a rib-based respiratory monitoring device method provided in this application embodiment;

[0077] Figure 2a A schematic diagram of a first two-dimensional CT image provided for an embodiment of this application;

[0078] Figure 2b A schematic diagram of another first two-dimensional CT image provided in an embodiment of this application;

[0079] Figure 2c A schematic diagram of yet another first two-dimensional CT image provided in an embodiment of this application;

[0080] Figure 3 A schematic diagram of a three-dimensional CT image provided in an embodiment of this application;

[0081] Figure 4 A schematic diagram of a target path provided for an embodiment of this application;

[0082] Figure 5 A schematic diagram illustrating another target path provided in an embodiment of this application;

[0083] Figure 6 A schematic diagram illustrating yet another target path provided in an embodiment of this application;

[0084] Figure 7 A schematic diagram illustrating the movement of an ultrasound probe along the clavicle, provided as an embodiment of this application;

[0085] Figure 8 A schematic diagram of a first two-dimensional ultrasound image provided in an embodiment of this application;

[0086] Figure 9 A schematic diagram of another first two-dimensional ultrasound image provided in an embodiment of this application;

[0087] Figure 10 This application provides a schematic diagram illustrating the relationship between a fourth difference and respiratory patterns in an embodiment of the present application.

[0088] Figure 11 A schematic diagram of a rib-based respiratory monitoring device provided in an embodiment of this application;

[0089] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0090] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0091] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0092] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one" means one or more, "more" means two or more, and "at least two" means two or three or more.

[0093] As described in the background art, some current technologies use the following method to obtain the breathing pattern of an object: attaching optical markers to the skin area of ​​the object's body surface, then determining the position information of the optical markers at different times through an optical tracking device, and finally obtaining the breathing pattern of the object based on the position information of the optical markers at different times.

[0094] However, this technology has the following problems: 1. When the optical marker is obstructed, the optical tracking device cannot determine the position information of the optical marker, resulting in low accuracy of the subject's breathing pattern. 2. The optical marker is attached to the skin area of ​​the subject's body, which can easily cause damage to the skin area.

[0095] Based on this, embodiments of this application provide a rib-based respiratory monitoring method, a surgical robot, and related products to solve the above problems. Since the ribs of the subject move with breathing, the rib-based respiratory monitoring method can obtain the subject's breathing pattern based on the positional information of the ribs at different times. This allows the subject's breathing pattern to be obtained without using an optical tracking device to determine the positional information of optical markers, thereby improving the accuracy of the breathing pattern and reducing damage to the subject's skin area.

[0096] The executing entity of this application embodiment is a rib-based respiratory monitoring device (hereinafter referred to as the monitoring device). The monitoring device can be any electronic device capable of executing the technical solutions disclosed in the method embodiments of this application. Optionally, the monitoring device can be one of the following: a computer or a server.

[0097] It should be understood that the method embodiments of this application can also be implemented by a processor executing computer program code. The embodiments of this application are described below with reference to the accompanying drawings. Please refer to... Figure 1 , Figure 1 This is a schematic flowchart of a rib-based respiratory monitoring device method provided in an embodiment of this application.

[0098] 101. Acquire at least one first two-dimensional ultrasound image, wherein the image in the at least one first two-dimensional ultrasound image includes the first object.

[0099] In this embodiment, each of at least one first two-dimensional ultrasound image includes a first object. Optionally, the first object is a person. In some embodiments, at least one first two-dimensional ultrasound image may be obtained by scanning the first object using an ultrasound probe.

[0100] Optionally, at least one first two-dimensional ultrasound image is obtained by scanning the first object while keeping the position of the ultrasound probe unchanged. Wherein, if the number of two-dimensional ultrasound images in at least one first two-dimensional ultrasound image is greater than 1, different first two-dimensional ultrasound images are acquired at different times.

[0101] Optionally, each of at least one of the first two-dimensional ultrasound images includes the ribs of the first object.

[0102] Optionally, each of at least one of the first two-dimensional ultrasound images includes the lungs of the first subject.

[0103] In one implementation of acquiring at least one first two-dimensional ultrasound image, the monitoring device receives at least one first two-dimensional ultrasound image input by a user through an input component. The input component includes a keyboard, mouse, touchscreen, touchpad, and audio input device.

[0104] In another implementation of acquiring at least one first two-dimensional ultrasound image, the monitoring device receives at least one first two-dimensional ultrasound image sent by the terminal. Optionally, the terminal can be any of the following: a mobile phone, a computer, a tablet computer, or a server.

[0105] 102. Determine the region corresponding to the rib of the first object from at least one first two-dimensional ultrasound image to obtain at least one first rib region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region in the at least one first rib region.

[0106] In this embodiment, the first rib region is the region in the first two-dimensional ultrasound image corresponding to the rib of the first object. The number of first rib regions in a single first two-dimensional ultrasound image is non-negative; that is, a single first two-dimensional ultrasound image may have no first rib regions, or it may have one or more first rib regions.

[0107] In one possible implementation, the monitoring device determines a region corresponding to the pleura of the first object from at least one first two-dimensional ultrasound image, obtaining at least one first pleural region, wherein one image from the at least one first two-dimensional ultrasound image corresponds to at least one region among the at least one first pleural region. Regions with grayscale values ​​greater than or equal to a grayscale threshold are determined from the at least one first two-dimensional ultrasound image, obtaining at least one rib candidate region, wherein one image from the at least one first two-dimensional ultrasound image corresponds to at least one region among the at least one rib candidate region. Based on the at least one first pleural region, at least one first rib region is determined from the at least one rib candidate region.

[0108] The first pleural region is the region in the first two-dimensional ultrasound image that corresponds to the pleura of the first object. The number of first pleural regions in a single first two-dimensional ultrasound image is non-negative; that is, a single first two-dimensional ultrasound image may have no first pleural region, or it may have one or more first pleural regions.

[0109] Rib candidate regions are areas in a first-dimensional ultrasound image whose grayscale values ​​are greater than or equal to a grayscale threshold. In some schemes, the grayscale value of any pixel in a rib candidate region is greater than or equal to the grayscale threshold. In other schemes, the average grayscale value of the pixels in the rib candidate region is greater than or equal to the grayscale threshold. The number of rib candidate regions in a first-dimensional ultrasound image is non-negative; that is, a first-dimensional ultrasound image may have no rib candidate regions or may have one or more rib candidate regions.

[0110] On the one hand, due to the high acoustic impedance of ribs, when ribs are present within the scanning area of ​​the ultrasound probe, artifact regions corresponding to the ribs exist in the ultrasound image obtained by the ultrasound probe. These artifact regions are areas with higher grayscale values. On the other hand, if we define the direction from the back of the object to the chest as the lateral direction, the lateral distance between the object's ribs and the surface of the object's chest as the rib lateral distance, and the lateral distance between the object's pleura and the surface of the object's chest as the pleural lateral distance, then the rib lateral distance is smaller than the pleural lateral distance. The first two-dimensional ultrasound image is obtained by emitting ultrasound waves in the lateral direction or the opposite direction of the lateral direction using the ultrasound probe. Therefore, the coordinates of pixels in the first two-dimensional ultrasound image, the scanning object corresponding to the pixel, and the lateral distance of the ultrasound probe are related. Optionally, when the first two-dimensional ultrasound image is obtained by emitting ultrasound waves in the lateral direction using the ultrasound probe, the closer the scanning object is to the ultrasound probe in the lateral direction, the larger the coordinates of the region corresponding to the scanning object in the first two-dimensional ultrasound image.

[0111] Based on the above two aspects, it can be seen that the region corresponding to the rib can be determined from the first two-dimensional ultrasound image based on the grayscale value and the pleural region in the first two-dimensional ultrasound image. Therefore, in this implementation, the monitoring device determines the region corresponding to the pleura of the first object from at least one first two-dimensional ultrasound image to obtain at least one first pleural region, and determines the region with a grayscale value greater than or equal to a grayscale threshold from at least one first two-dimensional ultrasound image to obtain at least one rib candidate region, wherein a grayscale value greater than or equal to the grayscale threshold indicates a large grayscale value. Then, based on at least one first pleural region, at least one first rib region can be determined from at least one rib candidate region.

[0112] In some embodiments, at least one first two-dimensional ultrasound image is obtained by scanning a first object using an ultrasound probe. When obtaining at least one first two-dimensional ultrasound image using an ultrasound probe, the distance from the ribs of the first object to the ultrasound probe is smaller than the distance from the pleura of the first object to the ultrasound probe. If the pixels in the ultrasound image obtained by scanning the object using an ultrasound probe are considered ultrasound pixels, the coordinates of the ultrasound pixels are negatively correlated with the distance from the scanned object to the ultrasound probe corresponding to the ultrasound pixel. Therefore, in the first two-dimensional ultrasound image, the coordinates of the region corresponding to the ribs of the first object are larger than the coordinates of the region corresponding to the pleura of the first object. Optionally, the coordinates mentioned in the embodiments of this application are the ordinates of the pixel coordinate system of the first two-dimensional ultrasound image.

[0113] At this time, the monitoring device can achieve "determining at least one first rib region from at least one rib candidate region based on at least one first pleural region" by performing the following steps: At least one pleural location is obtained based on the coordinates of at least one first pleural region in at least one first two-dimensional ultrasound image. Optionally, the position in the at least one pleural location corresponds one-to-one with the region in the at least one first pleural region, and the magnitude of the position in the at least one pleural location is positively correlated with the coordinates of the region in the at least one pleural region in the first two-dimensional ultrasound image. In some implementations, the at least one pleural location is the coordinates of at least one first pleural region in at least one first two-dimensional ultrasound image. At least one rib candidate location is obtained based on the coordinates of at least one rib candidate region in at least one first two-dimensional ultrasound image. Optionally, the position in the at least one rib candidate location corresponds one-to-one with the region in the at least one rib candidate region, and the magnitude of the position in the at least one rib candidate location is positively correlated with the coordinates of the region in the at least one rib candidate region in the first two-dimensional ultrasound image. In some implementations, the at least one rib candidate location is the coordinates of at least one rib candidate region in at least one first two-dimensional ultrasound image.

[0114] The difference between each position in at least one candidate rib location and each position in at least one pleural location is determined to obtain at least one first difference value. At least one second difference value is determined from the at least one first difference value, where each of the at least one second difference value is greater than or equal to a first threshold and less than or equal to a second threshold, wherein the first threshold is a positive number. At least one first rib location is determined from the at least one candidate rib location based on the at least one second difference value, wherein the at least one second difference value is based on the at least one first rib location, specifically, the difference among the at least one second difference value is the difference between the position in the at least one first rib location and the position in the at least one pleural location. At least one first rib region is obtained based on the region in the at least one candidate rib region corresponding to the at least one first rib location.

[0115] On the one hand, as mentioned earlier, in the first two-dimensional ultrasound image, the coordinates of the region corresponding to the ribs of the first subject are larger than the coordinates of the region corresponding to the pleura of the first subject. Therefore, the coordinates of the region corresponding to the ribs should be larger than the coordinates of the first pleural region. On the other hand, within the first subject's body, the distance between the ribs and the pleura is relatively short. Therefore, based on the above two aspects, a second difference is determined from at least one first difference that satisfies the following condition: greater than or equal to a first threshold and less than or equal to a second threshold. Thus, obtaining at least one first rib region based on at least one second difference can improve the accuracy of at least one first rib region.

[0116] In some embodiments, obtaining at least one pleural location based on the coordinates of at least one first pleural region in at least one first two-dimensional ultrasound image includes: determining the coordinates of at least one first pleural region in at least one first two-dimensional ultrasound image to obtain at least one pleural coordinate; and determining the relative position between each coordinate in the at least one pleural coordinate and the coordinates of the clavicle of the first object in the first two-dimensional ultrasound image to obtain at least one pleural location. Optionally, the location in the at least one pleural location is the distance between the coordinates in the at least one pleural coordinate and the coordinates of the clavicle of the first object in the first two-dimensional ultrasound image. Obtaining at least one rib candidate location based on the coordinates of at least one rib candidate region in at least one first two-dimensional ultrasound image includes: determining the coordinates of at least one rib candidate region in at least one first two-dimensional ultrasound image to obtain at least one rib candidate coordinate; and determining the relative position between each coordinate in the at least one rib candidate coordinate and the coordinates of the clavicle of the first object in the first two-dimensional ultrasound image to obtain at least one rib candidate location. Optionally, the location in the at least one rib candidate location is the distance between the coordinates in the at least one rib candidate coordinate and the coordinates of the clavicle of the first object in the first two-dimensional ultrasound image.

[0117] Considering that if there are at least two first two-dimensional ultrasound images, and the pose of the first object changes during the acquisition of at least two first two-dimensional ultrasound images by the ultrasound probe, the coordinates of the rib candidate region in different first two-dimensional ultrasound images will differ significantly. In this case, the change in the coordinates of the rib candidate region in at least two first two-dimensional ultrasound images includes changes caused by the first object's breathing and changes caused by changes in the first object's pose. Similarly, the change in the coordinates of the pleural region in at least two first two-dimensional ultrasound images includes changes caused by the first object's breathing and changes caused by changes in the first object's pose. Therefore, if the coordinates of at least one rib candidate region in at least one first two-dimensional ultrasound image are taken as at least one rib candidate position, and the coordinates of at least one first pleural region in at least one first two-dimensional ultrasound image are taken as at least one pleural position, and at least one first rib region is determined based on at least one rib candidate position and at least one pleural position, the change in the position of at least one first rib region in at least one first two-dimensional ultrasound image will include changes caused by the first object's breathing and changes caused by changes in the first object's pose. This results in the following: when at least one second rib region is obtained based on at least one first rib region, the change in the position of the at least one second rib region in at least one first two-dimensional ultrasound image includes changes caused by the breathing of the first subject and changes caused by the pose of the first subject. Therefore, obtaining the breathing pattern of the first subject based on the position of the at least one second rib region in at least one first two-dimensional ultrasound image and at least one acquisition time leads to a low error in the breathing pattern.

[0118] Because the clavicle of the first subject does not move with the first subject's breathing, and the change in the position of the first subject's clavicle caused by changes in the first subject's posture is the same as the change in the position of the first subject's ribs caused by changes in the first subject's posture, this scheme allows for more accurate reflection of changes in the first subject's breathing over time in at least one pleural location and at least one candidate rib location. Therefore, the accuracy of the breathing pattern can be improved when the breathing pattern is obtained based on at least one pleural location and at least one candidate rib location.

[0119] It should be understood that in the aforementioned current technology, if the position of the first object changes during the process of the optical tracking device determining the position information of the optical marker, the position of the optical marker attached to the surface of the first object will also change. In this case, the change in the position information of the optical marker includes changes caused by the first object's breathing and changes caused by changes in the first object's pose. Therefore, obtaining the object's breathing pattern based on the position information of the optical marker at different times will also lead to a large error in the breathing pattern. In other words, the breathing pattern of the first object obtained based on this method is more accurate than the breathing pattern obtained based on current technology.

[0120] In one possible implementation, at least one first two-dimensional ultrasound image is obtained by scanning at least one second rib region while the pose of the ultrasound probe remains unchanged. In this case, with the pose of the first object unchanged, the coordinate differences of the ribs of the first object in different first two-dimensional ultrasound images are small. Therefore, the monitoring device can achieve "obtaining at least one second rib region based on at least one first rib region" by performing the following steps: clustering at least one first rib position within at least one first rib region to obtain at least one rib position class, wherein the difference between any two first rib positions in any one of the at least one rib position classes is less than or equal to a third threshold. Based on the first rib positions in any one of the at least one rib position classes, at least one second rib region is determined from the at least one first rib region.

[0121] Optionally, the monitoring device determines the number of first rib locations in each class of at least one rib location class, obtaining at least one number of coordinates. The class corresponding to the maximum value among the at least one number of coordinates in the at least one rib location class is determined as the target rib location class. Based on the first rib locations in the target rib class, at least one second rib region is determined from the at least one first rib region, wherein the coordinates of the at least one second rib region in at least one first two-dimensional ultrasound image are the first rib locations in the target rib location class.

[0122] 103. Based on at least one first rib region, at least one second rib region is obtained, wherein at least one second rib region corresponds to the first rib of the first object, and the region in at least one second rib region corresponds one-to-one with an image in at least one first two-dimensional ultrasound image.

[0123] In this embodiment of the application, the first rib is any one of the ribs in the first object, and at least one second rib region corresponds to the first rib of the first object, that is, at least one second rib region corresponds to the same rib of the first object.

[0124] In one possible implementation, at least one first two-dimensional ultrasound image is obtained by scanning with the ultrasound probe in a constant pose. Because the range of motion of the ribs due to respiration is small, the difference in position of the same rib in different two-dimensional ultrasound images is small. Therefore, at least one second rib region can be obtained based on the position of at least one first rib region in at least one first two-dimensional ultrasound image. Specifically, if the position of a region in at least one second rib region in the first two-dimensional ultrasound image is referred to as the second rib position, then the difference between any two different second rib positions is less than or equal to a fourth threshold.

[0125] 104. Based on the position of at least one second rib region in at least one first two-dimensional ultrasound image and at least one acquisition time, the respiratory pattern of the first object is obtained, wherein the time in the at least one acquisition time is the time when the image in the at least one first two-dimensional ultrasound image is acquired, and the time in the at least one acquisition time corresponds one-to-one with the image in the at least one first two-dimensional ultrasound image.

[0126] In one possible implementation, the location of at least one region within a second rib area in a first two-dimensional ultrasound image is referred to as a second rib location. Therefore, the location of at least one second rib region in at least one first two-dimensional ultrasound image is defined as at least one second rib location. Based on the location within the at least one second rib location and the corresponding acquisition time, acquisition data is obtained, thus yielding at least one acquisition data point. By fitting the at least one acquisition data point, the respiratory pattern of the first subject can be obtained. Optionally, the respiratory pattern of the first subject can be used to determine the respiratory cycle of the first subject.

[0127] exist Figure 1 In the rib-based respiratory monitoring method shown, at least one first two-dimensional ultrasound image includes a first subject. After acquiring at least one first two-dimensional ultrasound image, the monitoring device determines the region corresponding to the ribs of the first subject from the at least one first two-dimensional ultrasound image, obtaining at least one first rib region. Then, based on the at least one first rib region, at least one second rib region is obtained, wherein the at least one second rib region corresponds to the first ribs of the first subject, and the regions within the at least one second rib region correspond one-to-one with the images in the at least one first two-dimensional ultrasound image. Based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time, the respiratory pattern of the first subject is obtained. This method can obtain the respiratory pattern of the subject without using an optical tracking device to determine the position information of optical markers, thereby improving the accuracy of the respiratory pattern and reducing damage to the subject's skin area.

[0128] In one optional implementation, the monitoring device achieves "obtaining the respiratory pattern of a first subject based on the position of at least one second rib region in at least one first two-dimensional ultrasound image and at least one acquisition time" by performing the following steps: Determining at least one second two-dimensional ultrasound image including at least one second rib region from at least one first two-dimensional ultrasound image; determining the position of the region corresponding to the clavicle of the first subject in at least one second two-dimensional ultrasound image to obtain at least one clavicle position; determining the position of at least one second rib region in at least one second two-dimensional ultrasound image to obtain at least one second rib position; determining the difference between the at least one clavicle position and the at least one second rib position to obtain at least one third difference; and obtaining the respiratory pattern of the first subject based on at least one third difference and at least one acquisition time. This allows the third difference to more accurately reflect the change in the position of at least one second rib region in at least one second two-dimensional ultrasound image caused by the breathing of the first subject, thereby improving the accuracy of the respiratory pattern.

[0129] In an optional implementation, the monitoring device further performs the following steps: acquiring a fourth difference value, wherein the fourth difference value is the difference between the position of the clavicle of the first object and the position of the first rib at the planning time, the planning time is the time when the target path is planned, the starting point of the target path is a point in the skin region of the first object, and the ending point of the target path is a point in the tissue of the first object. Based on the fourth difference value and the breathing pattern of the first object, a target time is obtained, wherein the difference in the breathing pattern corresponding to the target time is the target difference value, the absolute value of the difference between the target difference value and the fourth difference value is less than or equal to a third threshold value, and the target time is the time when the object moves from the skin region of the first object to the tissue of the first object based on the target path.

[0130] Because the difference between the positions of the first object's clavicle and first rib changes with the object's breathing, and the fourth difference is the difference between the positions of the first object's clavicle and first rib at the planning time, the time when the breathing state is the same as the target time can be determined based on the fourth difference; this is the target time. Thus, moving from the first object's skin area to the first object's tissue based on the target path at the target time improves the accuracy of reaching the endpoint of the target path. Optionally, the target time is later than the planning time. The first object's tissue is the first object's lungs.

[0131] In some schemes, the target path is obtained based on three-dimensional CT images, wherein the three-dimensional CT images are obtained based on at least one first two-dimensional CT image, each of the at least one first two-dimensional CT image includes the first object, and the acquisition time of each of the at least one first two-dimensional CT image is the planned time.

[0132] The monitoring device can determine an image including the first rib from at least one first two-dimensional CT image as a second two-dimensional CT image. Then, based on the position of the region corresponding to the first rib in the second two-dimensional CT image within the two-dimensional CT image, and the position of the two-dimensional CT image within the three-dimensional CT image, the position of the first rib at the planned time is obtained. Similarly, based on the position of the region corresponding to the clavicle of the first object in the three-dimensional CT image within the three-dimensional CT image, and the position of the two-dimensional CT image within the three-dimensional CT image, the position of the clavicle of the first object at the planned time is obtained.

[0133] In one optional implementation, the position of the first rib at the planning time is obtained based on the position of the region corresponding to the first rib in the second two-dimensional CT image and the position of the two-dimensional CT image in the three-dimensional CT image. This includes: obtaining the position of the first rib in the three-dimensional CT image based on the position of the region corresponding to the first rib in the second two-dimensional CT image and the position of the two-dimensional CT image in the three-dimensional CT image; and converting the position of the first rib in the three-dimensional CT image into a position in the optical coordinate system based on a first transformation relationship to obtain the position of the first rib at the planning time. The first transformation relationship is used to convert the position in the three-dimensional CT image into a position in the optical coordinate system.

[0134] Based on the location of the region corresponding to the clavicle of the first object in the 3D CT image and the location of the region in the 2D CT image within the 3D CT image, the position of the clavicle of the first object at the planning time is obtained. This includes: based on the location of the region corresponding to the clavicle of the first object in the 2D CT image and the location of the region in the 2D CT image within the 3D CT image, the position of the clavicle of the first object in the 3D CT image is obtained. Based on a first transformation relationship, the position of the clavicle of the first object in the 3D CT image is converted to its position in the optical coordinate system to obtain the position of the clavicle of the first object at the planning time.

[0135] This ensures that the position of the first object's clavicle at the planning time and the position of the first rib at the planning time are both in the optical coordinate system, and thus the fourth difference can be the difference in position in the optical coordinate system.

[0136] The respiratory pattern of the first subject is obtained based on at least one third difference and at least one acquisition time, including: converting at least one third difference into at least one fifth difference based on a second transformation relationship, wherein the second transformation relationship is used to convert the position in the first two-dimensional ultrasound image into the position in the optical coordinate system. The respiratory pattern of the first subject is obtained based on at least one fifth difference and at least one acquisition time. This ensures that the position in the respiratory pattern is in the optical coordinate system, which is beneficial for obtaining the target time based on the fourth difference and the respiratory pattern of the first subject.

[0137] To better understand the rib-based respiratory monitoring method provided in this application, the embodiments of this application will be described in detail with reference to the accompanying drawings and the following examples.

[0138] First, acquire at least one first two-dimensional CT image acquired at the planned time, wherein each of the at least one first two-dimensional CT image includes the first object. For example, Figure 2a , Figure 2b , Figure 2c These are all schematic diagrams of a first two-dimensional CT image provided in the embodiments of this application, wherein, Figure 2a The first two-dimensional CT image shown is a cross-section of the lung of the first subject. Figure 2b The first two-dimensional CT image shown is the coronal plane of the lung of the first object, and the first two-dimensional CT image shown in 2a is the sagittal plane of the lung of the first object. Organs and bones in at least one first two-dimensional CT image are segmented to obtain at least one segmentation result of at least one first two-dimensional CT image, wherein the first two-dimensional CT image and the segmentation result correspond one-to-one.

[0139] A 3D CT image is obtained by reconstructing a 3D image based on at least one first 2D CT image, and a target path is planned based on the 3D CT image. The 3D CT image is a 3D image in an optical coordinate system, the starting point of the target path is a point in the skin region of the first object, and the ending point of the target path is a lesion in an organ of the first object. For example, Figure 3 This is a schematic diagram of a three-dimensional CT image provided in an embodiment of this application. Figure 3 The 3D CT image shown includes the bones and organs (such as) of the first object. Figure 3 (Lungs) and tissues. For example, Figure 4 This is a schematic diagram of a target path provided in an embodiment of this application. Specifically, Figure 4 The target path is shown in the first two-dimensional ultrasound image. Figure 5 This is a schematic diagram illustrating another target path provided in an embodiment of this application. Specifically, Figure 5 The target path in the 3D CT image is shown. Figure 6 This is a schematic diagram of yet another target path provided in an embodiment of this application. Figure 6 The diagram shows a 3D CT image, a target path, and a 3D model of a guide. The 3D model of the guide is a 3D model constructed based on the guide, which is used to fix the target object so that the target object moves along the target path from the skin region of the first object to the tissue of the first object.

[0140] A reference optical marker (such as an optical marker sphere) is fixed onto the ultrasound probe. After applying a coupling agent to the ultrasound probe, the probe is moved to the vicinity of the clavicle of the first object and along the clavicle to the scapula, so that the ultrasound probe can obtain n two-dimensional ultrasound images of the clavicle, including a cross-section of the clavicle, optionally n=50. For example, Figure 7 This is a schematic diagram illustrating the movement of an ultrasound probe along the clavicle, as provided in an embodiment of this application. Figure 7 As shown, the scanning area of ​​the ultrasound probe moves from one end of the clavicle of the first object to the other end of the clavicle, and n two-dimensional ultrasound images of the clavicle are obtained through scanning during the movement.

[0141] Based on the tracking results of the optical tracking device on the reference optical marker and the relative positional relationship between the reference optical marker and the ultrasound probe, n two-dimensional ultrasound images of the clavicle are converted to the optical coordinate system to obtain n converted two-dimensional ultrasound images.

[0142] Optionally, the ultrasound probe is a high-frequency linear array ultrasound probe, which can improve the clarity of the two-dimensional ultrasound image obtained by the ultrasound probe scanning, and thus help to determine the skeletal regions in the two-dimensional ultrasound image, such as the region corresponding to the clavicle and the region corresponding to the ribs.

[0143] A three-dimensional ultrasound image is obtained by reconstructing n converted two-dimensional ultrasound images. The three-dimensional ultrasound image and the three-dimensional CT image are then registered to align the clavicle in the three-dimensional ultrasound image and the clavicle in the three-dimensional CT image. The three-dimensional information is then displayed, including the registered three-dimensional ultrasound image and the registered three-dimensional CT image. Optionally, the displayed three-dimensional information includes at least one segmentation result from at least one first two-dimensional CT image, thereby allowing the display of the relative positional relationship between the organs and bones of the first object.

[0144] Based on the displayed 3D information, the user can determine the position of the ribs of the first object. Then, based on the rib position in the 3D information, the user can place the ultrasound probe near the first rib of the first object, so that the first rib is within the scanning area of ​​the ultrasound probe. The ultrasound probe's pose is then fixed, and at least one first 2D ultrasound image is obtained by scanning with the ultrasound probe at at least one moment, wherein different images in the at least one first 2D ultrasound image are acquired at different times. For example, Figure 8 This is a schematic diagram of a first two-dimensional ultrasound image provided in an embodiment of this application. Figure 8 In the first two-dimensional ultrasound image shown, the first rib area is a black area (i.e., Figure 8 The black area in the lower left corner, i.e., the area of ​​the first rib, has a relatively large gray value. Figure 9 A schematic diagram of another first two-dimensional ultrasound image provided in the embodiments of this application, in Figure 9In the first two-dimensional ultrasound image shown, the first rib region is darker in color, while the first pleura region is lighter in color; that is, the grayscale value of the first rib region is larger, and the grayscale value of the first pleura region is smaller. Furthermore, in the first two-dimensional ultrasound image, the first pleura region is located above the first rib region, i.e., it is obtained by scanning the first object using an ultrasound probe. Figure 9 In the first two-dimensional ultrasound image shown, the distance between the pleura of the first subject and the ultrasound probe is greater than the distance between the ribs of the first subject and the ultrasound probe.

[0145] Optionally, the first rib is one of the following: the rib that is 3rd farthest from the clavicle in the body of the first object, the 4th farthest from the clavicle in the body of the first object, the 5th farthest from the clavicle in the body of the first object, or the 6th farthest from the clavicle in the body of the first object.

[0146] At least one second rib region is obtained based on at least one first two-dimensional ultrasound image (the specific implementation process can be found in steps 101 to 103 described above). Based on the position of at least one second rib region in at least one first two-dimensional ultrasound image and at least one acquisition time, the respiratory pattern of the first subject is obtained (the specific implementation process can be found in step 104 described above). Optionally, based on the position of at least one second rib region in at least one first two-dimensional ultrasound image and at least one acquisition time, the motion curve of the first rib is obtained. Since the motion period in the motion curve of the first rib is the same as the respiratory period of the first subject, the motion curve of the first rib can be used as the respiratory pattern of the first subject.

[0147] A second 2D CT image, including the first rib, is determined from at least one first 2D CT image. Then, based on the position of the region corresponding to the first rib in the second 2D CT image within the 2D CT image, and the position of the 2D CT image within the 3D CT image, the position of the first rib at the planning time is obtained. Similarly, based on the position of the region corresponding to the clavicle of the first object in the 3D CT image within the 3D CT image, and the position of the 2D CT image within the 3D CT image, the position of the clavicle of the first object at the planning time is obtained.

[0148] The fourth difference is obtained by determining the difference between the position of the first subject's clavicle and the position of the first rib at the planning time. This fourth difference, along with the breathing pattern of the first subject, yields the target time. For example, Figure 10 This is a schematic diagram illustrating the relationship between a fourth difference and respiratory pattern, provided as an embodiment of this application. Figure 10 A coordinate system is shown, with the horizontal axis representing time and the vertical axis representing the difference between the position of the clavicle and the position of the first rib. The unit of the vertical axis is millimeters. Figure 10 The breathing pattern of the first subject was also shown (i.e. Figure 10(The curve in the image) and a reference horizontal line, where the ordinate of the reference horizontal line is the fourth difference value. Specifically, in... Figure 10 In the diagram, the vertical coordinate of the reference horizontal line is 116.80 mm, meaning the fourth difference is 116.80 mm. Based on... Figure 10 The target time can be obtained by relating the breathing pattern to the reference horizontal line. Optionally, the vertical coordinate corresponding to the target time in the breathing pattern is 116.89 mm, meaning the difference between the fourth difference and the vertical coordinate corresponding to the target time is -0.09 mm. Optionally, the target time can also be obtained based on the horizontal coordinate corresponding to the intersection of the reference horizontal line and the breathing pattern.

[0149] Therefore, after obtaining the target time, the target path can be used to move from the skin region of the first object to the tissue of the first object based on the target path at the target time, which can improve the accuracy of reaching the end point of the target path.

[0150] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0151] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, while using clear signs / information to inform users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0152] The methods of the embodiments of this application have been described in detail above, and the apparatus of the embodiments of this application is provided below.

[0153] Please see Figure 11 , Figure 11 This is a schematic diagram of a rib-based respiratory monitoring device provided in an embodiment of this application. The rib-based respiratory monitoring device 1 includes: an acquisition unit 11 and a processing unit 12, wherein:

[0154] Acquisition unit 11 is used to acquire at least one first two-dimensional ultrasound image, wherein the image in the at least one first two-dimensional ultrasound image includes a first object;

[0155] Processing unit 12 is used to determine the region corresponding to the rib of the first object from the at least one first two-dimensional ultrasound image to obtain at least one first rib region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region in the at least one first rib region.

[0156] The processing unit 12 is further configured to obtain at least one second rib region based on the at least one first rib region, wherein the at least one second rib region corresponds to the first rib of the first object, and the region in the at least one second rib region corresponds one-to-one with the image in the at least one first two-dimensional ultrasound image;

[0157] The processing unit 12 is further configured to obtain the breathing pattern of the first object based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time, wherein the time in the at least one acquisition time is the time when the image in the at least one first two-dimensional ultrasound image is acquired, and the time in the at least one acquisition time corresponds one-to-one with the image in the at least one first two-dimensional ultrasound image.

[0158] In conjunction with any embodiment of this application, the processing unit 12 is further configured to:

[0159] The region corresponding to the pleura of the first object is determined from the at least one first two-dimensional ultrasound image to obtain at least one first pleural region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region among the at least one first pleural region;

[0160] From the at least one first two-dimensional ultrasound image, a region with a gray value greater than or equal to a gray value threshold is determined to obtain at least one rib candidate region, wherein one of the at least one first two-dimensional ultrasound images corresponds to at least one region among the at least one rib candidate region;

[0161] Based on the at least one first pleural region, the at least one first rib region is determined from the at least one rib candidate region.

[0162] In any embodiment of this application, the at least one first two-dimensional ultrasound image is obtained by scanning the first object using an ultrasound probe; the pixels in the ultrasound image obtained by scanning the object using the ultrasound probe are ultrasound pixels, and the coordinates of the ultrasound pixels are negatively correlated with the distance from the scanned object corresponding to the ultrasound pixel to the ultrasound probe; when obtaining the at least one first two-dimensional ultrasound image using an ultrasound probe, the distance from the ribs of the first object to the ultrasound probe is smaller than the distance from the pleura of the first object to the ultrasound probe;

[0163] The processing unit 12 is further configured to:

[0164] At least one pleural location is obtained based on the coordinates of the at least one first pleural region in the at least one first two-dimensional ultrasound image;

[0165] Based on the coordinates of the at least one rib candidate region in the at least one first two-dimensional ultrasound image, at least one rib candidate position is obtained;

[0166] The difference between each of the at least one candidate rib location and each of the at least one pleural location is determined to obtain at least one first difference value;

[0167] At least one second difference is determined from the at least one first difference, each of the at least one second difference being greater than or equal to a first threshold and less than or equal to a second threshold, wherein the first threshold is a positive number;

[0168] Based on the at least one second difference, at least one first rib position is determined from the at least one rib candidate position, wherein the at least one second difference is obtained based on the at least one first rib position;

[0169] The at least one first rib region is obtained based on the region corresponding to the position of the at least one first rib in at least one rib candidate region.

[0170] In any embodiment of this application, the at least one first two-dimensional ultrasound image is obtained by scanning while the pose of the ultrasound probe remains unchanged;

[0171] The processing unit 12 is further configured to:

[0172] Clustering is performed on at least one first rib position in the at least one first rib region to obtain at least one rib position class, wherein the difference between any two first rib positions in any one rib position class is less than or equal to a third threshold.

[0173] The at least one second rib region is determined from the at least one first rib region based on the first rib position in any one of the at least one rib position classes.

[0174] In conjunction with any embodiment of this application, the processing unit 12 is further configured to:

[0175] Determine at least one second two-dimensional ultrasound image from the at least one first two-dimensional ultrasound image that includes the at least one second rib region;

[0176] Determine the location of the region corresponding to the clavicle of the first object in the at least one second two-dimensional ultrasound image to obtain at least one clavicle location;

[0177] Determine the position of the at least one second rib region in the at least one second two-dimensional ultrasound image to obtain the position of at least one second rib.

[0178] Determine the difference between the position of the at least one clavicle and the position of the at least one second rib to obtain at least one third difference;

[0179] Based on the at least one third difference and the at least one acquisition time, the breathing pattern of the first object is obtained.

[0180] Optionally, obtaining the breathing pattern of the first object based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time includes: obtaining the breathing pattern of the first object based on the position of the center of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time.

[0181] In conjunction with any embodiment of this application, the acquisition unit 11 is further configured to acquire a fourth difference, the fourth difference being the difference between the position of the first object's clavicle at the planning time and the position of the first rib at the planning time, the planning time being the time when the target path is planned, the starting point of the target path being a point in the skin region of the first object, and the ending point of the target path being a point in the tissue of the first object.

[0182] The processing unit 12 is further configured to obtain a target time based on the fourth difference and the breathing pattern of the first object, wherein the difference between the breathing pattern and the target time is the target difference, the absolute value of the difference between the target difference and the fourth difference is less than or equal to a third threshold, and the target time is the time when the object moves from the skin area of ​​the first object to the tissue of the first object based on the target path.

[0183] In this embodiment, at least one first two-dimensional ultrasound image includes a first object. After acquiring at least one first two-dimensional ultrasound image, the rib-based respiratory monitoring device determines the region corresponding to the ribs of the first object from the at least one first two-dimensional ultrasound image, obtaining at least one first rib region. Then, based on the at least one first rib region, at least one second rib region is obtained, wherein the at least one second rib region corresponds to the first ribs of the first object, and the regions within the at least one second rib region correspond one-to-one with the images in the at least one first two-dimensional ultrasound image. Based on the position of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time, the respiratory pattern of the first object is obtained. This allows the respiratory pattern of the object to be obtained without using an optical tracking device to determine the position information of optical markers, thereby improving the accuracy of the respiratory pattern and reducing damage to the object's skin area.

[0184] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0185] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, memory 22, input device 23, and output device 24 are coupled together via connectors, which include various interfaces, transmission lines, or buses, etc., and are not limited in this embodiment. It should be understood that in the various embodiments of this application, coupling refers to mutual connection in a specific way, including direct connection or indirect connection through other devices, such as through various interfaces, transmission lines, buses, etc.

[0186] The processor 21 can be one or more graphics processing units (GPUs). If the processor 21 is a GPU, the GPU can be a single-core GPU or a multi-core GPU. Optionally, the processor 21 can be a processor group composed of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Optionally, the processor can also be other types of processors, etc., which are not limited in this embodiment.

[0187] The memory 22 can be used to store computer program instructions, as well as various types of computer program code, including program code for executing the scheme of this application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0188] Input device 23 is used to input data and / or signals, and output device 24 is used to output data and / or signals. Input device 23 and output device 24 can be independent devices or an integrated device.

[0189] It is understood that in this embodiment of the application, the memory 22 can be used not only to store related instructions, but also to store related data. This embodiment of the application does not limit the specific data stored in the memory.

[0190] Understandable, Figure 12 This is merely a simplified design of an electronic device. In practical applications, the electronic device may also include other necessary components, including, but not limited to, any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of this application are within the protection scope of this application.

[0191] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0192] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will also readily understand that the various embodiments of this application have different focuses, and for the sake of convenience and brevity, the same or similar parts may not be repeated in different embodiments. Therefore, parts not described or not described in detail in one embodiment can be referred to the descriptions in other embodiments.

[0193] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0194] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0195] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0196] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0197] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A rib-based respiration monitoring method, characterized by, The method comprises: acquiring at least one first two-dimensional ultrasound image, the images in the at least one first two-dimensional ultrasound image comprising a first object; determining, from the at least one first two-dimensional ultrasound image, regions corresponding to ribs of the first object to obtain at least one first rib region, one image in the at least one first two-dimensional ultrasound image corresponding to at least one region in the at least one first rib region; based on the at least one first rib region, obtaining at least one second rib region corresponding to first ribs of the first object, the regions in the at least one second rib region corresponding to the images in the at least one first two-dimensional ultrasound image one by one; based on positions of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time, obtaining a breathing rule of the first object, the times in the at least one acquisition time being times of acquiring the images in the at least one first two-dimensional ultrasound image, the times in the at least one acquisition time corresponding to the images in the at least one first two-dimensional ultrasound image one by one; the obtaining of the breathing rule of the first object based on the positions of the at least one second rib region in the at least one first two-dimensional ultrasound image and the at least one acquisition time comprises: determining, from the at least one first two-dimensional ultrasound image, at least one second two-dimensional ultrasound image comprising the at least one second rib region; determining positions of regions corresponding to clavicles of the first object in the at least one second two-dimensional ultrasound image to obtain at least one clavicle position; determining positions of the at least one second rib region in the at least one second two-dimensional ultrasound image to obtain at least one second rib position; determining difference values of the at least one clavicle position and the at least one second rib position to obtain at least one third difference value; and obtaining the breathing rule of the first object based on the at least one third difference value and the at least one acquisition time.

2. The method of claim 1, wherein, the determining of the at least one first rib region from the at least one first two-dimensional ultrasound image comprises: determining, from the at least one first two-dimensional ultrasound image, regions corresponding to pleura of the first object to obtain at least one first pleura region, one image in the at least one first two-dimensional ultrasound image corresponding to at least one region in the at least one first pleura region; determining, from the at least one first two-dimensional ultrasound image, regions with a gray value greater than or equal to a gray threshold value to obtain at least one rib candidate region, one image in the at least one first two-dimensional ultrasound image corresponding to at least one region in the at least one rib candidate region; determining, from the at least one rib candidate region, the at least one first rib region based on the at least one first pleura region.

3. The method of claim 2, wherein, The at least one first two-dimensional ultrasound image is obtained by scanning the first object by using an ultrasound probe; pixels in an ultrasound image obtained by scanning an object by using the ultrasound probe are ultrasound pixels, coordinates of the ultrasound pixels are negatively correlated with distances from the scanning object corresponding to the ultrasound pixels to the ultrasound probe; when the at least one first two-dimensional ultrasound image is obtained by scanning by using the ultrasound probe, a distance from a rib of the first object to the ultrasound probe is smaller than a distance from a pleura of the first object to the ultrasound probe; The determining the at least one first rib region from the at least one rib candidate region based on the at least one first pleura region comprises: obtaining at least one pleura position based on coordinates of the at least one first pleura region in the at least one first two-dimensional ultrasound image; obtaining at least one rib candidate position based on coordinates of the at least one rib candidate region in the at least one first two-dimensional ultrasound image; determining a difference between each position in the at least one rib candidate position and each position in the at least one pleura position respectively to obtain at least one first difference value; determining at least one second difference value from the at least one first difference value, each difference value in the at least one second difference value is greater than or equal to a first threshold value and less than or equal to a second threshold value, the first threshold value is a positive number; determining at least one first rib position from the at least one rib candidate position based on the at least one second difference value, the at least one second difference value is obtained based on the at least one first rib position; obtaining the at least one first rib region based on a region corresponding to the at least one first rib position in the at least one rib candidate region.

4. The method of claim 3, wherein, The at least one first two-dimensional ultrasound image is obtained in a case where a pose of the ultrasound probe is unchanged; The obtaining the at least one second rib region based on the at least one first rib region comprises: clustering the at least one first rib position in the at least one first rib region to obtain at least one rib position class, a difference value between any two first rib positions in any rib position class in the at least one rib position class is less than or equal to a third threshold value; determining the at least one second rib region from the at least one first rib region based on the first rib position in any rib position class in the at least one rib position class.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: obtaining a fourth difference value, the fourth difference value is a difference value between a position of a clavicle of the first object at a planning time and a position of the first rib at the planning time, the planning time is a time of planning a target path, a starting point of the target path is a point in a skin region of the first object, and an end point of the target path is a point in a tissue of the first object; obtaining a target time point based on the fourth difference value and the breathing rule of the first object, a difference value corresponding to the target time point in the breathing rule being a target difference value, an absolute value of a difference between the target difference value and the fourth difference value being less than or equal to a third threshold value, the target time point being based on the target path moving from a skin region of the first object to a tissue of the first object.

6. A rib-based respiration monitoring device, characterized by The rib-based respiration monitoring device comprises: an acquisition unit configured to acquire at least one first two-dimensional ultrasound image, an image in the at least one first two-dimensional ultrasound image comprising a first object; a processing unit configured to determine, from the at least one first two-dimensional ultrasound image, a region corresponding to a rib of the first object to obtain at least one first rib region, one image in the at least one first two-dimensional ultrasound image corresponding to at least one region in the at least one first rib region; the processing unit is further configured to obtain, based on the at least one first rib region, at least one second rib region, the at least one second rib region corresponding to the first rib of the first object, a region in the at least one second rib region corresponding to one image in the at least one first two-dimensional ultrasound image; the processing unit is further configured to obtain, based on positions of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time point, a breathing rule of the first object, a time point in the at least one acquisition time point being a time point at which an image in the at least one first two-dimensional ultrasound image is acquired, a time point in the at least one acquisition time point corresponding to one image in the at least one first two-dimensional ultrasound image; the processing unit is further configured to obtain, based on positions of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time point, a breathing rule of the first object, a time point in the at least one acquisition time point being a time point at which an image in the at least one first two-dimensional ultrasound image is acquired, a time point in the at least one acquisition time point corresponding to one image in the at least one first two-dimensional ultrasound image; 7. A surgical robot, characterised in that, the processing unit is further configured to obtain, based on positions of the at least one second rib region in the at least one first two-dimensional ultrasound image and at least one acquisition time point, a breathing rule of the first object, a time point in the at least one acquisition time point being a time point at which an image in the at least one first two-dimensional ultrasound image is acquired, a time point in the at least one acquisition time point corresponding to one image in the at least one first two-dimensional ultrasound image; 8. An electronic device, comprising: The rib-based respiration monitoring device comprises: a processor and a memory, the memory being configured to store computer program code comprising computer instructions, the electronic device being configured to execute the method according to any one of claims 1 to 5 when the processor executes the computer instructions.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program includes program instructions. The program instructions, when executed by a processor, cause the processor to perform the method in any one of claims 1 to 5.

Images