Method for determining the relationship between respiratory displacement, puncture method and related products

Abstract

The present application discloses a method for determining the relationship of respiratory displacement, a puncture method and related products. The method is used to determine the target displacement relationship between the respiratory displacements of different points in the target organ of the object to be punctured, and the method includes: acquiring a first image and a second image, both of which include the target organ; determining a first displacement between a first pixel in the target organ in the first image and a second pixel in the target organ in the second image, the first pixel and the second pixel being image points of the same name; determining a second displacement between a third pixel in the target organ in the first image and a fourth pixel in the target organ in the second image, the third pixel and the fourth pixel being image points of the same name; based on the first displacement and the second displacement, determining the target displacement relationship. The respiratory displacements of different points in the target organ can be determined by this method, and then the relationship between the respiratory displacements of different points in the target organ can be determined.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a method for determining the relationship between respiratory displacement, a puncture method and related products. Background Art

[0002] The target organ will move under the influence of breathing, which will cause the points in the target organ to produce respiratory displacement. Moreover, different points in the target organ are affected differently by breathing, and the respiratory displacements of different points in the target organ are also different.

[0003] The current method is usually to build a motion model that can characterize the target organ's motion with respiration, and then determine the respiratory displacement of points in the target organ based on the motion model. However, the amount of data processing required to build the motion model is large, resulting in a large amount of data processing required for this method.

[0004] Considering that there is usually a relationship between the respiratory displacements of different points in the target organ, when the respiratory displacement of a point in the target organ is determined, the respiratory displacement of any point in the target organ can be determined based on the relationship. Therefore, how to determine the relationship between the respiratory displacements of different points in the target organ is of great significance for determining the respiratory displacement of a point in the target organ. Summary of the invention

[0005] The present application provides a method for determining a relationship between respiratory displacements, a puncture method and related products, so as to determine a target displacement relationship between respiratory displacements at different points in a target organ of a subject to be punctured.

[0006] In a first aspect, a method for determining a relationship of respiratory displacement is provided, the method being used to determine a target displacement relationship between respiratory displacements of different points in a target organ of a subject to be punctured, the respiratory displacement being a displacement caused by the breathing of the subject to be punctured, the method comprising:

[0007] Acquiring a first image and a second image, wherein both the first image and the second image include the target organ, the first image and the second image are acquired during the breathing process of the object to be punctured, and the respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image;

[0008] Determining a first displacement between a first pixel in the target organ in the first image and a second pixel in the target organ in the second image, the first pixel and the second pixel being image points of the same name;

[0009] determining a second displacement between a third pixel in the target organ in the first image and a fourth pixel in the target organ in the second image, the third pixel and the fourth pixel being image points of the same name, and the third pixel being different from the first pixel;

[0010] The target displacement relationship is determined based on the first displacement and the second displacement.

[0011] In combination with any implementation manner of the present application, determining the target displacement relationship based on the first displacement and the second displacement includes:

[0012] determining a first position of the first pixel within the target organ in the first image;

[0013] determining a second position of the third pixel within the target organ in the first image;

[0014] determining first data based on the first position and the first displacement;

[0015] determining second data based on the second position and the second displacement;

[0016] The target displacement relationship is obtained by fitting the first data and the second data.

[0017] In combination with any implementation manner of the present application, the step of obtaining the target displacement relationship by fitting the first data and the second data includes:

[0018] Using at least two candidate fitting methods, respectively fitting the first data and the second data to obtain at least two candidate displacement relationships, wherein the candidate displacement relationships represent the relationship between the position of the point in the target organ in the target organ and the respiratory displacement of the point in the target organ, and the candidate fitting methods correspond to the candidate displacement relationships in a one-to-one manner;

[0019] Determine a displacement corresponding to a third position based on each of the at least two candidate displacement relations, respectively, to obtain at least two third displacements, the third position corresponding to a position of the fifth pixel in the target organ in the first image, and the third displacements corresponding to the candidate displacement relations one by one;

[0020] determining at least two errors of the at least two candidate displacement relationships based on a difference between the fourth displacement and the at least two third displacements, the third displacements corresponding to the errors one-to-one, the fourth displacement being a displacement between the fifth pixel and a sixth pixel in the target organ in the second image, the fifth pixel and the sixth pixel being image points of the same name;

[0021] The target displacement relationship is determined from the at least two candidate displacement relationships based on the at least two errors.

[0022] In combination with any embodiment of the present application, the determining the target displacement relationship from the at least two candidate displacement relationships based on the at least two errors includes:

[0023] determining a minimum of the at least two errors;

[0024] A displacement relationship corresponding to the minimum value is determined from the at least two candidate displacement relationships as the target displacement relationship.

[0025] In combination with any embodiment of the present application, after determining the first position of the first pixel in the target organ in the first image and before determining first data based on the first position and the first displacement, the method further includes:

[0026] determining a size of the target organ in the first image;

[0027] determining a fourth position based on a quotient of the first position and the size, the fourth position representing a relative position of the first pixel within the target organ;

[0028] The determining first data based on the first position and the first displacement includes:

[0029] Based on the fourth position and the first displacement, first data is determined.

[0030] In combination with any embodiment of the present application, determining a first position of the first pixel in the target organ in the first image includes:

[0031] Segmenting the target organ from the first image to obtain an organ image;

[0032] The position of the first pixel in the organ image is determined to obtain the first position.

[0033] In a second aspect, a puncture method is provided, the method comprising:

[0034] Acquire a target displacement relationship for characterizing respiratory displacements of different points in a target organ of the object to be punctured, wherein the target displacement relationship is obtained by the first aspect and any one of its embodiments, and the respiratory displacement is a displacement generated by the breathing of the object to be punctured;

[0035] Acquire a reference displacement of a reference point in the target organ, wherein the reference displacement is the respiratory displacement of the reference point at a target time;

[0036] Based on the target displacement relationship and the reference displacement, a target displacement of a target point in the target organ is determined, where the target displacement is the respiratory displacement of the target point at the target moment.

[0037] In combination with any embodiment of the present application, after determining the target displacement of the target point in the target organ based on the target displacement relationship and the reference displacement, the method further includes:

[0038] Determining a target position of the target organ at a target time based on the reference displacement and the target displacement;

[0039] Based on the target position, a puncture path for a lesion in the target organ is determined.

[0040] In a third aspect, a device for determining a relationship of respiratory displacement is provided, the device for determining a relationship of respiratory displacement is used to determine a target displacement relationship between respiratory displacements of different points in a target organ of a subject to be punctured, the respiratory displacement being a displacement caused by the breathing of the subject to be punctured, the device for determining a relationship of respiratory displacement comprising:

[0041] an acquisition unit, configured to acquire a first image and a second image, wherein the first image and the second image both include the target organ, the first image and the second image are both acquired during the breathing process of the object to be punctured, and the respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image;

[0042] a determining unit, configured to determine a first displacement between a first pixel in the target organ in the first image and a second pixel in the target organ in the second image, the first pixel and the second pixel being image points of the same name;

[0043] The determining unit is further configured to determine a second displacement between a third pixel in the target organ in the first image and a fourth pixel in the target organ in the second image, the third pixel and the fourth pixel being image points of the same name, and the third pixel being different from the first pixel;

[0044] The determining unit is further configured to determine the target displacement relationship based on the first displacement and the second displacement.

[0045] In combination with any implementation manner of the present application, the determining unit is specifically configured to:

[0046] determining a first position of the first pixel within the target organ in the first image;

[0047] determining a second position of the third pixel within the target organ in the first image;

[0048] determining first data based on the first position and the first displacement;

[0049] determining second data based on the second position and the second displacement;

[0050] The target displacement relationship is obtained by fitting the first data and the second data.

[0051] In combination with any implementation manner of the present application, the determining unit is specifically configured to:

[0052] Using at least two candidate fitting methods, respectively fitting the first data and the second data to obtain at least two candidate displacement relationships, wherein the candidate displacement relationships represent the relationship between the position of the point in the target organ in the target organ and the respiratory displacement of the point in the target organ, and the candidate fitting methods correspond to the candidate displacement relationships in a one-to-one manner;

[0053] Determine a displacement corresponding to a third position based on each of the at least two candidate displacement relations, respectively, to obtain at least two third displacements, the third position corresponding to a position of the fifth pixel in the target organ in the first image, and the third displacements corresponding to the candidate displacement relations one by one;

[0054] determining at least two errors of the at least two candidate displacement relationships based on a difference between the fourth displacement and the at least two third displacements, the third displacements corresponding to the errors one-to-one, the fourth displacement being a displacement between the fifth pixel and a sixth pixel in the target organ in the second image, the fifth pixel and the sixth pixel being image points of the same name;

[0055] The target displacement relationship is determined from the at least two candidate displacement relationships based on the at least two errors.

[0056] In combination with any implementation manner of the present application, the determining unit is specifically configured to:

[0057] determining a minimum of the at least two errors;

[0058] A displacement relationship corresponding to the minimum value is determined from the at least two candidate displacement relationships as the target displacement relationship.

[0059] In combination with any implementation manner of the present application, the determining unit is further configured to:

[0060] determining a size of the target organ in the first image;

[0061] determining a fourth position based on a quotient of the first position and the size, the fourth position representing a relative position of the first pixel within the target organ;

[0062] Based on the fourth position and the first displacement, first data is determined.

[0063] In combination with any implementation manner of the present application, the determining unit is specifically configured to:

[0064] Segmenting the target organ from the first image to obtain an organ image;

[0065] The position of the first pixel in the organ image is determined to obtain the first position.

[0066] In a fourth aspect, the present application further provides a puncture device, the puncture device comprising:

[0067] an acquisition unit, configured to acquire a target displacement relationship between respiratory displacements of different points in a target organ of the object to be punctured, the target displacement relationship being obtained through the first aspect and any one of its embodiments, the respiratory displacement being a displacement generated by the breathing of the object to be punctured;

[0068] The acquisition unit is further used to acquire a reference displacement of a reference point in the target organ, wherein the reference displacement is the respiratory displacement of the reference point at a target moment;

[0069] A determination unit is used to determine a target displacement of a target point in the target organ based on the target displacement relationship and the reference displacement, wherein the target displacement is the respiratory displacement of the target point at the target moment.

[0070] In combination with any implementation manner of the present application, the determining unit is further configured to:

[0071] Determining a target position of the target organ at a target time based on the reference displacement and the target displacement;

[0072] Based on the target position, a puncture path for a lesion in the target organ is determined.

[0073] In a fifth aspect, an electronic device is provided, comprising: a processor and a memory, the memory being used to store computer program code, the computer program code comprising computer instructions, and when the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation thereof, or the electronic device executes the method as described in the second aspect above and any possible implementation thereof.

[0074] In a sixth aspect, another electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation thereof, or the electronic device executes the method as described in the second aspect above and any possible implementation thereof.

[0075] In the seventh aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute a method such as the first aspect and any possible implementation thereof, or the processor is caused to execute a method such as the second aspect and any possible implementation thereof.

[0076] In an eighth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the method of the above-mentioned first aspect and any possible implementation thereof, or the computer is caused to execute the method of the above-mentioned second aspect and any possible implementation thereof.

[0077] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0078] In the present application, both the first image and the second image include the target organ, and both the first image and the second image are acquired during the breathing process of the object to be punctured, and the respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image. After acquiring the first image and the second image, the device for determining the relationship between respiratory displacements determines the first displacement between the first pixel in the target organ in the first image and the second pixel in the target organ in the second image, wherein the first pixel and the second pixel are image points of the same name, and the first displacement can characterize the respiratory displacement of the first pixel in the target organ from the first moment to the second moment, wherein the first moment is the moment corresponding to the acquisition moment of the first image in the respiratory cycle, and the second moment is the moment corresponding to the acquisition moment of the second image in the respiratory cycle. Determine the second displacement between the third pixel in the target organ in the first image and the fourth pixel in the target organ in the second image, wherein the third pixel and the fourth pixel are image points of the same name, and the third pixel is different from the first pixel, and the second displacement can characterize the respiratory displacement of the third pixel in the target organ from the first moment to the second moment. The device for determining the relationship between respiratory displacements can determine the relationship between the respiratory displacements of different points in the target organ based on the first displacement and the second displacement, which is the target displacement relationship. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0080] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0081] Figure 1 A flow chart of a method for determining a relationship between respiratory displacements provided in an embodiment of the present application;

[0082] Figure 2 A schematic diagram of a target lateral displacement relationship of the left lung provided in an embodiment of the present application;

[0083] Figure 3 A schematic diagram of a target longitudinal displacement relationship of the left lung provided in an embodiment of the present application;

[0084] Figure 4 A schematic diagram of a target vertical displacement relationship of the left lung provided in an embodiment of the present application;

[0085] Figure 5 A schematic diagram of a target lateral displacement relationship of the right lung provided in an embodiment of the present application;

[0086] Figure 6 A schematic diagram of a target longitudinal displacement relationship of the right lung provided in an embodiment of the present application;

[0087] Figure 7 A schematic diagram of a target vertical displacement relationship of the right lung provided in an embodiment of the present application;

[0088] Figure 8a A schematic diagram of comparison between a target organ in a first image and a target organ in a second image provided in an embodiment of the present application;

[0089] Figure 8b A method for providing a method for Figure 8a Schematic diagram of the target organ position after compensation;

[0090] Fig. 9 A schematic diagram of a puncture method provided in an embodiment of the present application;

[0091] Fig.10 A schematic diagram of the structure of a device for determining the relationship between respiratory displacements provided in an embodiment of the present application;

[0092] Fig.11A schematic diagram of the structure of a puncture device provided in an embodiment of the present application;

[0093] Fig.12 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0094] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0095] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0096] Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one (item)" means one or more, "multiple" means two or more, and "at least two (items)" means two or three and more.

[0097] When puncturing a lesion in a target organ of a puncture subject, it is necessary to first plan a puncture path for the lesion, and then puncture the lesion based on the puncture path. For example, the puncture subject is a human, the target organ is a lung in a human body, and the lesion is a lesion in the lung. However, since the time of determining the puncture path is different from the time of puncturing based on the puncture path, and the breathing of the puncture subject may cause the target organ to move, the position of the target organ when planning the puncture path may be different from the position of the target organ during actual puncture, and the position of the lesion when planning the puncture path may be different from the position of the lesion during actual puncture, which leads to poor effect of puncturing the lesion based on the puncture path.

[0098] The current method is usually to build a motion model that can characterize the target organ's motion with breathing, and then determine the position of the target organ and the position of the lesion at the time of actual puncture based on the motion model, and then perform puncture based on the position of the target organ and the position of the lesion at the time of actual puncture. However, the amount of data processing required to build the motion model is large, resulting in a large amount of data processing required for this method.

[0099] Considering that different points in the target organ are affected differently by respiration, the respiratory displacements of different points in the target organ are also different, wherein the respiratory displacement is the displacement caused by the respiration of the object to be punctured. For example, the diaphragm in the lungs is more affected by respiration than the tissues in the lungs other than the diaphragm, so the respiratory displacement of the diaphragm is greater than the respiratory displacement of the tissues other than the diaphragm. For another example, in the diaphragm, the position close to the heart is more affected by respiration than the position far from the heart, so the respiratory displacement of the point close to the heart is greater than the respiratory displacement of the point far from the heart. However, there is usually a relationship between the respiratory displacements of different points in the target organ. For example, the target organ includes point A and point B, wherein the respiratory displacement of point A is greater than the respiratory displacement of point B by s1. Therefore, in the case of determining the respiratory displacement of a point in the target organ during actual puncture, based on the relationship between the respiratory displacement of the point and the respiratory displacement of any point in the target organ, the respiratory displacement of any point in the target organ during actual puncture can be determined. Thus, the position of the target organ during actual puncture and the position of the lesion during actual puncture can be determined.

[0100] Based on this, an embodiment of the present application provides a method for determining the relationship of respiratory displacement, based on which the target displacement relationship between the respiratory displacements of different points in the target organ of the object to be punctured can be determined. The execution subject of the method for determining the relationship of respiratory displacement is a device for determining the relationship of respiratory displacement (hereinafter referred to as a determination device). Optionally, the determination device can be one of the following: a computer, a server.

[0101] It should be understood that the method embodiment of the present application can also be implemented by a processor executing a computer program code. The following describes the embodiment of the present application in conjunction with the drawings in the embodiment of the present application. Figure 1 , Figure 1 A flowchart of a method for determining the relationship of respiratory displacement provided in an embodiment of the present application.

[0102] 101. Acquire a first image and a second image, wherein both the first image and the second image include the target organ, wherein both the first image and the second image are acquired during the breathing process of the object to be punctured, and the respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image.

[0103] In the embodiment of the present application, the target organ may be any organ, for example, the target organ is a lung. The first image and the second image may be any two images including the target organ, for example, the first image and the second image are two computed tomography (CT) images including the target organ. Optionally, both the first image and the second image are three-dimensional CT images.

[0104] The first image and the second image are both acquired during the breathing process of the object to be punctured, indicating that the target organ in the first image and the target organ in the second image both have respiratory displacement. It should be understood that the respiratory displacement in the embodiment of the present application does not limit the displacement caused by the continuous breathing, nor does it limit the displacement caused by which stage in the respiratory cycle. Since breathing can cause displacement, the respiratory displacement in the embodiment of the present application can be the displacement caused by breathing of any duration, or it can be the displacement caused by any stage in the respiratory cycle. For example, the breathing cycle of the object to be punctured is called a respiratory cycle. The respiratory displacement can be the displacement of the position of the lung at 1 / 2 of the respiratory cycle compared to the position at the start of the same respiratory cycle. The respiratory displacement can also be the displacement of the position of the lung at 3 / 4 of the respiratory cycle compared to the position at 1 / 4 of the same respiratory cycle. The respiratory displacement can also be the displacement of the position of the lung at 1 / 2 of the first respiratory cycle compared to the position at 1 / 4 of the second respiratory cycle, wherein the first respiratory cycle is the respiratory cycle after the second respiratory cycle.

[0105] The respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image, indicating that the acquisition moment of the first image and the acquisition moment of the second image correspond to different moments of the respiratory cycle, wherein the moment in the respiratory cycle is determined based on the start moment of the respiratory cycle. Different moments of the respiratory cycle include different moments in the same respiratory cycle and two moments in different respiratory cycles, and the start moment of the cycle to which the two periods belong is different. For the convenience of expression, the moment corresponding to the acquisition moment of the first image in the respiratory cycle is referred to as the first moment, and the moment corresponding to the acquisition moment of the second image in the respiratory cycle is referred to as the second moment.

[0106] For example, the acquisition time of the first image is moment t1, and the acquisition time of the second image is moment t2. Moments t1 and t2 are two different moments in the same respiratory cycle. When the starting time of the respiratory cycle is 0, moment t1 is the first moment, and moment t2 is the second moment. At this time, the moment corresponding to the acquisition time of the first image in the respiratory cycle is the first moment, and the moment corresponding to the acquisition time of the second image in the respiratory cycle is the second moment. For another example, the acquisition time of the first image is moment t1, and the acquisition time of the second image is moment t2, wherein moment t1 is a moment in respiratory cycle a, and moment t2 is a moment in respiratory cycle b, and respiratory cycle a and respiratory cycle b are different respiratory cycles. When the starting time of respiratory cycle a is 0, moment t1 is the first moment, and the moment corresponding to the acquisition time of the first image in the respiratory cycle is the first moment. When the starting time of respiratory cycle b is 0, moment t2 is the second moment, and the moment corresponding to the acquisition time of the second image in the respiratory cycle is the second moment.

[0107] Since there is also respiratory displacement between different moments of the respiratory cycle, the target organ in the first image also has respiratory displacement compared to the target organ in the second image. Specifically, the respiratory displacement of the target organ in the first image compared to the target organ in the second image is the respiratory displacement of the target organ from the first moment to the second moment. For example, the respiratory displacement of the target organ in the first image is d1, and the respiratory displacement of the target organ in the second image is d2, (d1-d2) is the respiratory displacement of the target organ in the first image compared to the target organ in the second image, that is, (d1-d2) is the respiratory displacement of the target organ from the first moment to the second moment.

[0108] In an implementation of acquiring the first image and the second image, the determining device receives the first image and the second image input by the user through an input component. The input component includes: a keyboard, a mouse, a touch screen, a touch pad, and an audio input device.

[0109] In another implementation of acquiring the first image and the second image, the determining device receives the first image and the second image sent by the terminal. Optionally, the terminal may be any one of the following: a mobile phone, a computer, a tablet computer, a server, and a wearable device.

[0110] It should be understood that the step of determining that the device acquires the first image and the step of acquiring the second image may be performed separately or simultaneously, and this application does not limit this.

[0111] 102. Determine a first displacement between a first pixel in the target organ in the first image and a second pixel in the target organ in the second image, wherein the first pixel and the second pixel are image points with the same name.

[0112] In an embodiment of the present application, two pixels with the same name correspond to the same object point. Specifically, a pixel in the target organ in the first image and a pixel in the target organ in the second image are image points with the same name, then a pixel in the target organ in the first image and a pixel in the target organ in the second image correspond to the same object point.

[0113] It should be understood that in the target organ of the first image and the target organ of the second image, the number of image points with the same name can be greater than 1, each image point with the same name includes a pixel in the target organ in the first image and a pixel in the target organ in the second image, and the two pixels in any image point with the same name correspond to the same object point.

[0114] The first pixel and the second pixel are image points of the same name. The displacement between the first pixel and the second pixel is a first displacement, and the first pixel and the second pixel can be aligned based on the first displacement. For example, the position of the first pixel in the first image can be moved according to the first displacement to obtain the position of the second pixel in the second image, or the position of the second pixel in the second image can be moved according to the first displacement to obtain the position of the first pixel in the first image. In a possible implementation, the determination device can determine the first displacement between the first pixel and the second pixel by aligning the target organ in the first image with the target organ in the second image.

[0115] As described in step 101, the respiratory displacement of the target organ in the first image compared to the target organ in the second image is the respiratory displacement of the target organ from the first moment to the second moment, and the first displacement is the displacement between the first pixel in the first image and the second pixel in the second image. Therefore, the first displacement can represent the respiratory displacement of the first pixel in the first image compared to the second pixel in the second image. Specifically, the first displacement can represent the respiratory displacement of the first pixel in the target organ from the first moment to the second moment, or can represent the respiratory displacement of the second pixel in the target organ from the first moment to the second moment. It should be understood that the first pixel and the second pixel correspond to the same object point in the target organ, so the respiratory displacement of the first pixel from the first moment to the second moment is the same as the respiratory displacement of the second pixel from the first moment to the second moment.

[0116] The first displacement may represent the respiratory displacement of the second pixel in the second image compared to the first pixel in the first image. Specifically, the first displacement may represent the respiratory displacement of the first pixel in the target organ from the second moment to the first moment, or may represent the respiratory displacement of the second pixel in the target organ from the second moment to the first moment. It should be understood that the first pixel and the second pixel correspond to the same object point in the target organ, so the respiratory displacement of the first pixel from the second moment to the first moment is the same as the respiratory displacement of the second pixel from the second moment to the first moment.

[0117] 103. Determine a second displacement between a third pixel in the target organ in the first image and a fourth pixel in the target organ in the second image, wherein the third pixel and the fourth pixel are image points of the same name, and the third pixel is different from the first pixel.

[0118] In an embodiment of the present application, the third pixel is different from the first pixel, that is, the first pixel and the third pixel are two different pixels in the target organ of the first image. The third pixel and the fourth pixel are image points of the same name. The displacement between the third pixel and the fourth pixel is the second displacement, and the third pixel and the fourth pixel can be aligned based on the second displacement. For example, the position of the third pixel in the first image can be moved according to the second displacement to obtain the position of the fourth pixel in the second image, or the position of the fourth pixel in the second image can be moved according to the second displacement to obtain the position of the third pixel in the first image. In one possible implementation method, the determination device can determine the second displacement between the third pixel and the fourth pixel by aligning the target organ in the first image with the target organ in the second image.

[0119] As described in step 101, the respiratory displacement of the target organ in the first image compared to the target organ in the second image is the respiratory displacement of the target organ from the first moment to the second moment, and the second displacement is the displacement between the third pixel in the first image and the fourth pixel in the second image. Therefore, the second displacement can represent the respiratory displacement of the third pixel in the first image compared to the fourth pixel in the second image. Specifically, the second displacement can represent the respiratory displacement of the third pixel in the target organ from the first moment to the second moment, or can represent the respiratory displacement of the fourth pixel in the target organ from the first moment to the second moment. It should be understood that the third pixel and the fourth pixel correspond to the same object point in the target organ, so the respiratory displacement of the third pixel from the first moment to the second moment is the same as the respiratory displacement of the fourth pixel from the first moment to the second moment.

[0120] The second displacement may represent the respiratory displacement of the fourth pixel in the second image compared to the third pixel in the first image. Specifically, the second displacement may represent the respiratory displacement of the third pixel in the target organ from the second moment to the first moment, or may represent the respiratory displacement of the fourth pixel in the target organ from the second moment to the first moment. It should be understood that the third pixel and the fourth pixel correspond to the same object point in the target organ, so the respiratory displacement of the third pixel from the second moment to the first moment is the same as the respiratory displacement of the fourth pixel from the second moment to the first moment.

[0121] 104. Determine the target displacement relationship based on the first displacement and the second displacement.

[0122] The first pixel and the third pixel are two different pixels in the target organ of the first image, that is, the first pixel and the third pixel are two pixels corresponding to different points in the target organ. The first displacement can represent the respiratory displacement of the first pixel in the target organ from the first moment to the second moment, and the second displacement can represent the respiratory displacement of the third pixel in the target organ from the first moment to the second moment. Therefore, based on the first displacement and the second displacement, the relationship between the respiratory displacements of different points in the target organ can be determined.

[0123] In one possible implementation, the determining device determines a first position of a first pixel in a target organ in a first image. Determines a second position of a third pixel in a target organ in the first image. Determines first data based on the first position and the first displacement. Determines second data based on the second position and the second displacement. Obtains a target displacement relationship by fitting the first data and the second data.

[0124] Optionally, the determination device implements "obtaining the target displacement relationship by fitting the first data and the second data" by executing the following steps: using at least two candidate fitting methods, respectively fitting the first data and the second data to obtain at least two candidate displacement relationships, wherein the candidate displacement relationship represents the relationship between the position of a point in the target organ in the target organ and the respiratory displacement of the point in the target organ, and the candidate fitting methods correspond to the candidate displacement relationships one-to-one. Based on each of the at least two candidate displacement relationships, respectively, a displacement corresponding to a third position is determined to obtain at least two third displacements, wherein the third displacement corresponds one-to-one to the candidate displacement relationship, and the third position corresponds to the position of the fifth pixel in the target organ in the first image. Specifically, the third position represents the position of the fifth pixel in the target organ, and the fifth pixel is different from the first pixel and the third pixel. Based on the difference between the fourth displacement and the at least two third displacements, at least two errors of the at least two candidate displacement relationships are determined, wherein the third displacement corresponds one-to-one to the error, the fourth displacement is the displacement between the fifth pixel and the sixth pixel in the target organ in the second image, and the fifth pixel and the sixth pixel are image points of the same name. Optionally, the error includes a mean squared error (MSE). Based on the at least two errors, a target displacement relationship is determined from at least two candidate displacement relationships.

[0125] The candidate fitting methods include: a linear function, a quadratic function, a cubic function, and a quartic function. That is, when at least two candidate fitting methods include a linear function, the determining device may fit the first data and the second data based on the linear function to obtain a candidate displacement relationship, and the candidate displacement relationship may be represented by a linear function. Similarly, when at least two candidate fitting methods include a quadratic function, the candidate displacement relationship may be represented by a quadratic function, when at least two candidate fitting methods include a cubic function, the candidate displacement relationship may be represented by a cubic function, and when at least two candidate fitting methods include a quartic function, the candidate displacement relationship may be represented by a quartic function.

[0126] Optionally, the determination device can determine the displacement between two pixels in each of the same-name image points in the first image and the second image by registering the target organ in the first image with the target organ in the second image. Then the same-name image points in the first image and the second image are divided into a training set and a test set, wherein the ratio of the number of same-name image points in the training set to the number of same-name image points in the test set is a predetermined ratio, and optionally, the predetermined ratio is 8:2. The determination device determines the data for fitting based on the same-name image points in the training set, and obtains the candidate displacement relationship by fitting the data. The determination device determines the error based on the same-name image points in the test set, that is, the first pixel, the second pixel, the third pixel and the fourth pixel all belong to the training set, and the fifth pixel and the sixth pixel all belong to the test set.

[0127] Optionally, the determining device implements "determining the target displacement relationship from at least two candidate displacement relationships based on at least two errors" by performing the following steps: determining a minimum value of the at least two errors. Determining the displacement relationship corresponding to the minimum value from the at least two candidate displacement relationships as the target displacement relationship, thereby improving the accuracy of the target displacement relationship.

[0128] As an optional manner, the above positions (including the first position, the second position and the third position) include the horizontal coordinate in the organ coordinate system, the vertical coordinate in the organ coordinate system, and the vertical coordinate in the organ coordinate system, wherein the organ coordinate system is a coordinate system constructed based on the target organ, the horizontal coordinate is the coordinate in the horizontal axis direction of the organ coordinate system, the vertical coordinate is the coordinate in the vertical axis direction of the organ coordinate system, and the vertical coordinate is the coordinate in the vertical axis direction of the organ coordinate system. Optionally, the origin of the organ coordinate system is the center of the target organ. At this time, the first position includes the first horizontal coordinate, the first vertical coordinate and the first vertical coordinate, and the second position includes the second horizontal coordinate, the second vertical coordinate and the second vertical coordinate. The displacement between two pixels in the same-name image point includes a horizontal displacement along the horizontal axis direction, a vertical displacement along the vertical axis direction, and a vertical displacement along the vertical axis direction. At this time, the first displacement includes a first horizontal displacement, a first vertical displacement and a first vertical displacement, and the second displacement includes a second horizontal displacement, a second vertical displacement and a second vertical displacement.

[0129] The determination device determines data based on the coordinates in each direction and the displacement in each direction respectively, and obtains the target displacement relationship corresponding to each direction by fitting the data corresponding to each direction.

[0130] Specifically, the determining device performs the following steps in the process of determining the first data based on the first position and the first displacement: determining the first horizontal data based on the first horizontal coordinate and the first horizontal displacement, determining the first vertical data based on the first vertical coordinate and the first vertical displacement, and determining the first vertical data based on the first vertical coordinate and the first vertical displacement. The determining device performs the following steps in the process of determining the second data based on the second position and the second displacement: determining the second horizontal data based on the second horizontal coordinate and the second horizontal displacement, determining the second vertical data based on the second vertical coordinate and the second vertical displacement, and determining the second vertical data based on the second vertical coordinate and the second vertical displacement. The determining device performs the following steps in the process of obtaining the target displacement relationship by fitting the first data and the second data: obtaining the target horizontal displacement relationship by fitting the first horizontal data and the second horizontal data, obtaining the target vertical displacement relationship by fitting the first vertical data and the second vertical data, and obtaining the target vertical displacement relationship by fitting the first vertical data and the second vertical data. At this time, the target displacement relationship includes the target horizontal displacement relationship, the target vertical displacement relationship, and the target vertical displacement relationship.

[0131] Optionally, the determining device obtains candidate displacement relations corresponding to each direction by fitting the data corresponding to each direction, and determines the target displacement relations corresponding to each direction based on the candidate displacement relations corresponding to each direction.

[0132] Optionally, the target organ includes the left lung, the target transverse displacement relationship can be expressed by a quadratic function, the target longitudinal displacement relationship can be expressed by a linear function, and the target vertical displacement relationship can be expressed by a cubic function.

[0133] Optionally, the target organ includes the right lung, the target transverse displacement relationship can be expressed by a quadratic function, the target longitudinal displacement relationship can be expressed by a quartic function, and the target vertical displacement relationship can be expressed by a quartic function.

[0134] Optionally, the determining device determines the first position of the first pixel in the first image by performing the following steps: segmenting the target organ from the first image to obtain an organ image. Determining the position of the first pixel in the organ image to obtain the first position. Optionally, the determining device determines the second position of the third pixel in the first image by performing the following steps: segmenting the target organ from the first image to obtain an organ image. Determining the position of the third pixel in the organ image to obtain the second position.

[0135] Optionally, after determining the first position of the first pixel in the target organ in the first image, the determining device further performs the following steps before determining the first data based on the first position and the first displacement: determining the size of the target organ in the first image. Optionally, the size of the target organ in the first image includes the horizontal size of the target organ in the horizontal axis direction of the organ coordinate system, the vertical size of the target organ in the vertical axis direction of the organ coordinate system, and the vertical size of the target organ in the vertical axis direction of the organ coordinate system, wherein the organ coordinate system is a coordinate system constructed based on the target organ. Based on the quotient of the first position and the size, a fourth position is determined, wherein the fourth position represents the relative position of the first pixel in the target organ. Optionally, in the case where the first position includes the first horizontal coordinate, the first vertical coordinate and the first vertical coordinate, the fourth horizontal coordinate is determined based on the quotient of the first horizontal coordinate and the horizontal size, the fourth vertical coordinate is determined based on the quotient of the first vertical coordinate and the vertical size, and the fourth vertical coordinate is determined based on the quotient of the first vertical coordinate and the vertical size, and the fourth position includes the fourth horizontal coordinate, the fourth vertical coordinate and the fourth vertical coordinate. After obtaining the fourth position, the determining device determines the first data by performing the following steps: determining the first data based on the fourth position and the first displacement.

[0136] The fourth position represents the relative position of the first pixel in the target organ, rather than the absolute position of the first pixel in the first image. For example, if the target organ is a lung, the fourth position represents the middle position of the lung from the left boundary to the right boundary in the horizontal direction, and the fourth position may represent the position of one quarter of the lung from the lower boundary to the upper boundary in the vertical direction. Accordingly, the first displacement may represent the respiratory displacement of a point in the target organ whose relative position is the fourth position from the first moment to the second moment.

[0137] Therefore, the first data determined based on the fourth position and the first displacement includes the following information: the respiratory displacement of the point whose relative position in the target organ is the fourth position from the first moment to the second moment. Then, by fitting the first data and the second data, a target displacement relationship can be obtained, wherein the target displacement relationship can characterize the respiratory displacement of the point whose relative position in the target organ is any position from the first moment to the second moment, that is, the respiratory displacement of the point whose relative position in the target organ is any position from the first moment to the second moment can be determined based on the target displacement relationship.

[0138] Although different points in the target organ have different respiratory displacements at different moments in the respiratory cycle, the relationship between the respiratory displacements of different points in the target organ is the same between any two moments in the respiratory cycle. For example, for point A and point B in the target organ, from the first moment to the second moment, the respiratory displacement of point A is s1 greater than the respiratory displacement of point B. From the second moment to the third moment, the respiratory displacement of point A is also s1 greater than the respiratory displacement of point B. Therefore, when the relationship between the respiratory displacements of points with different relative positions in the target organ from the first moment to the second moment can be determined based on the target relationship, the relationship between the respiratory displacements of points with different relative positions in the target organ can be determined based on the target displacement relationship.

[0139] Although the sizes of target organs of different objects may be different, the respiratory displacements of points at the same relative position in target organs of different sizes are usually not much different. Therefore, in this implementation, after determining the fourth position and the second position, the determination device determines the first data based on the fourth position and the first displacement. The second data is determined based on the second position and the second displacement. By fitting the first data and the second data to obtain the target displacement relationship, the influence of individual differences of the objects on the relationship between the respiratory displacements of different points in the target organ can be reduced, thereby improving the accuracy of the target displacement relationship.

[0140] Optionally, after determining the second position of the third pixel in the target organ in the first image, and before determining the second data based on the second position and the second displacement, the determination device further performs the following steps: determining the size of the target organ in the first image. Based on the quotient of the second position and the size, a fifth position is determined, wherein the fifth position represents the relative position of the third pixel in the target organ. After obtaining the fifth position, the determination device determines the second data by performing the following steps: determining the first data based on the fifth position and the second displacement. The second position represents the relative position of the third pixel in the target organ, rather than the position of the third pixel in the first image. Accordingly, the second displacement can represent the respiratory displacement of a point in the target organ whose relative position is the second position from the first moment to the second moment. The second data determined based on the second position and the second displacement includes the following information: the respiratory displacement of a point in the target organ whose relative position is the second position from the first moment to the second moment.

[0141] It should be understood that the first data and the second data are the description objects selected for the purpose of briefly describing the technical solution, and it should not be understood that the determination device only determines two data, and only obtains the target displacement relationship by fitting the two data. In practical applications, the determination device can determine n data in the same manner as the first data, where n is data greater than or equal to 2, and then obtain the target displacement relationship by fitting the n data.

[0142] In one possible scenario, the target organ includes the left lung. Figure 2 A schematic diagram of a target lateral displacement relationship of the left lung provided in an embodiment of the present application. Figure 2 In the figure, the horizontal axis represents the horizontal coordinate of the point in the left lung in the organ coordinate system, and the vertical axis represents the horizontal displacement of the point in the left lung. Figure 2 The relationship between the transverse data and the target transverse displacement is shown in the figure, wherein the transverse data is data determined based on the transverse coordinate of a point in the left lung in the organ coordinate system and the transverse displacement of the point in the left lung, such as the first transverse data or the second transverse data described above. Figure 2 The target lateral displacement relationship in is obtained by Figure 2 It is obtained by fitting the horizontal data in . Figure 3 A schematic diagram of a target longitudinal displacement relationship of the left lung provided in an embodiment of the present application. Figure 3 In the figure, the horizontal axis represents the vertical coordinate of the point in the left lung in the organ coordinate system, and the vertical axis represents the vertical displacement of the point in the left lung. Figure 3 The relationship between the longitudinal data and the target longitudinal displacement is shown in the figure, wherein the longitudinal data is data determined based on the longitudinal coordinate of a point in the left lung in the organ coordinate system and the longitudinal displacement of the point in the left lung, such as the first longitudinal data and the second longitudinal data described above. Figure 3 The target longitudinal displacement relationship in Figure 3 It is obtained by fitting the vertical data in . Figure 4 A schematic diagram of a target vertical displacement relationship of the left lung provided in an embodiment of the present application. Figure 4 In the figure, the horizontal axis represents the vertical coordinate of the point in the left lung in the organ coordinate system, and the vertical axis represents the vertical displacement of the point in the left lung. Figure 4 The relationship between vertical data and target vertical displacement is shown in FIG. 1 , wherein the vertical data is data determined based on the vertical coordinate of a point in the left lung in the organ coordinate system and the vertical displacement of the point in the left lung, such as the first vertical data and the second vertical data described above. Figure 4 The target vertical displacement relationship in Figure 4 It is obtained by fitting the vertical data in .

[0143] Optionally, the left lung in this implementation scenario is the target organ in the first image, and the displacement (including lateral displacement, longitudinal displacement and vertical displacement) is the displacement of a point in the left lung in the first image compared to a point in the left lung in the second image. Figure 2 The target lateral displacement relationship in Figure 3 The target vertical displacement relationship in Figure 4The position of the point in the left lung in the first image is compensated according to the target vertical displacement relationship, and a compensated image can be obtained. The root mean square of the displacement of the point in the left lung in the first image compared to the point in the left lung in the second image is called the total displacement before compensation, and the root mean square of the displacement of the point in the left lung in the compensated image compared to the point in the left lung in the second image is called the total displacement after compensation, wherein the total displacement before compensation includes the lateral displacement before compensation, the longitudinal displacement before compensation, and the vertical displacement before compensation, and the total displacement after compensation includes the lateral displacement after compensation, the longitudinal displacement after compensation, and the vertical displacement after compensation. A comparison of the total displacement before the compensator and the total displacement after compensation can be seen in Table 1 below.

[0144] Table 1

[0145]

[0146] In another possible scenario, the target organ includes the right lung. Figure 5 A schematic diagram of a target transverse displacement relationship of the right lung provided in an embodiment of the present application. Figure 5 In the figure, the horizontal axis represents the horizontal coordinate of the point in the right lung in the organ coordinate system, and the vertical axis represents the horizontal displacement of the point in the right lung. Figure 5 The relationship between the transverse data and the target transverse displacement is shown in the figure, wherein the transverse data is data determined based on the transverse coordinate of a point in the right lung in the organ coordinate system and the transverse displacement of the point in the right lung, such as the first transverse data or the second transverse data described above. Figure 5 The target lateral displacement relationship in is obtained by Figure 5 It is obtained by fitting the horizontal data in . Figure 6 A schematic diagram of a target longitudinal displacement relationship of the right lung provided in an embodiment of the present application. Figure 6 In the figure, the horizontal axis represents the vertical coordinate of the point in the right lung in the organ coordinate system, and the vertical axis represents the vertical displacement of the point in the right lung. Figure 6 The relationship between the longitudinal data and the target longitudinal displacement is shown in the figure, wherein the longitudinal data is data determined based on the longitudinal coordinate of a point in the right lung in the organ coordinate system and the longitudinal displacement of the point in the right lung, such as the first longitudinal data and the second longitudinal data described above. Figure 6 The target longitudinal displacement relationship in Figure 6 It is obtained by fitting the vertical data in . Figure 7 A schematic diagram of a target vertical displacement relationship of the right lung provided in an embodiment of the present application. Figure 7 In the figure, the horizontal axis represents the vertical coordinate of the point in the right lung in the organ coordinate system, and the vertical axis represents the vertical displacement of the point in the right lung. Figure 7 The relationship between vertical data and target vertical displacement is shown in FIG. 1 , wherein the vertical data is data determined based on the vertical coordinate of a point in the right lung in the organ coordinate system and the vertical displacement of the point in the right lung, such as the first vertical data and the second vertical data described above. Figure 7The target vertical displacement relationship in Figure 7 It is obtained by fitting the vertical data in .

[0147] Optional, Figure 8a A schematic diagram of a comparison between a target organ in a first image and a target organ in a second image provided in an embodiment of the present application, Figure 8a In the figure, the target organs include the left lung and the right lung, wherein the left lung in the first image and the left lung in the second image are displayed overlappingly, and the right lung in the first image and the right lung in the second image are also displayed overlappingly. Since the time at which the lung in the first image is located in the respiratory cycle is different from the time at which the lung in the second image is located in the respiratory cycle, the position of the point in the lung in the first image is different from the position of the point in the lung in the second image, which results in the volume of the lung in the first image being different from the volume of the lung in the second image. Specifically, the volume of the lung in the second image is larger than the volume of the lung in the first image. Therefore, the lung in the second image will have an extra part compared to the lung in the first image, such as Figure 8a As shown, both the left and right lungs include non-overlapping areas (i.e. Figure 8a medium to light grey areas).

[0148] After determining the target displacement relationship based on the method for determining the respiratory displacement relationship described above, and determining the respiratory displacement of a point in the left lung from a first moment (i.e., the moment in the respiratory cycle of the lung in the first image) to a second moment (i.e., the moment in the respiratory cycle of the lung in the second image), the respiratory displacement of any point in the left lung from the first moment to the second moment can be determined based on the target displacement relationship and the respiratory displacement. Based on the respiratory displacement of each point in the left lung in the first image from the first moment to the second moment, the position of each point in the left lung in the first image is compensated. Similarly, the position of each point in the right lung in the first image can be compensated. Please refer to Figure 8b , Figure 8b A method for providing a method for Figure 8a Schematic diagram of the target organ position after compensation. Figure 8b As shown, after compensating the positions of the points in the left lung in the first image and the right lung in the first image, the non-overlapping area (i.e. Figure 8b The area of ​​the light gray area is Figure 8a This means that based on the target displacement relationship and the respiratory displacement of a point in the lung from the first moment to the second moment, the respiratory displacement of any point in the lung from the first moment to the second moment can be determined more accurately.

[0149] After obtaining the target displacement relationship based on the method for determining the relationship of respiratory displacement described above, the target displacement relationship can be used to estimate the respiratory displacement of the point in the target organ, and then puncture can be performed based on the estimated result. Based on this, the embodiment of the present application also provides a puncture method, the execution subject of the method is a puncture device (hereinafter referred to as the puncture device). Optionally, the puncture device can be one of the following: a computer, a server. Please refer to Fig. 9 , Fig. 9 A schematic diagram of a puncture method provided in an embodiment of the present application.

[0150] 901. Obtain a target displacement relationship for characterizing respiratory displacements of different points in a target organ of a subject to be punctured.

[0151] The target displacement relationship in this step is obtained by the method for determining the relationship of respiratory displacement as described above.

[0152] 902. Obtain a reference displacement of a reference point in the target organ, wherein the reference displacement is a respiratory displacement of the reference point at a target time.

[0153] In the embodiment of the present application, the reference point may be any point in the target organ. Optionally, the relative position of the reference point in the target organ may be any position. The target time may be any time after the target displacement relationship is obtained.

[0154] Optionally, the reference displacement is the respiratory displacement of the reference point from the start time to the target time, wherein the start time may be any time before the target time.

[0155] 903. Determine a target displacement of a target point in the target organ based on the target displacement relationship and the reference displacement, wherein the target displacement is a respiratory displacement of the target point at a target time.

[0156] In the embodiment of the present application, the target point is a point in the target organ that is different from the reference point. Optionally, the number of target points is greater than 1 or equal to 1. As described above, the relationship between the respiratory displacements of points with different relative positions in the target organ can be determined based on the target displacement relationship, so the puncture device can determine the relationship between the respiratory displacement of the reference point and the respiratory displacement of the target point based on the target displacement relationship, and then determine the respiratory displacement of the target point at the target moment based on the relationship and the respiratory displacement of the reference point at the target moment.

[0157] In an embodiment of the present application, after obtaining the target displacement relationship and the reference displacement of the reference point, the puncture device can determine the target displacement of the target point in the target organ based on the target displacement relationship and the reference displacement, thereby estimating the respiratory displacement of the target point based on the target displacement relationship.

[0158] As an optional implementation, after executing step 903, the puncture device further performs the following steps: determining the target position of the target organ at the target time based on the reference displacement and the target displacement, and determining the puncture path for the lesion in the target organ based on the target position.

[0159] In this embodiment, after determining the reference displacement of the reference point in the target organ and the target displacement of the target point in the target organ, the puncture device can determine the target position of the target organ at the target moment. Optionally, the reference displacement is the respiratory displacement of the reference point from the starting moment to the target moment, and the puncture device can determine the target position based on the starting position, reference displacement and target displacement of the target organ at the starting moment. Specifically, the puncture device can determine the position of the reference point at the starting moment and the position of the target point at the starting moment based on the starting position of the target organ at the starting moment. Then, based on the position of the reference point at the starting moment and the reference displacement, the position of the reference point at the target moment can be determined, and based on the position of the target point at the starting moment and the target displacement, the position of the target point at the target moment can be determined. Finally, based on the position of the reference point at the target moment and the position of the target point at the target moment, the target position of the target organ at the target moment can be determined.

[0160] After determining the target position of the target organ at the target time, the puncture path for the lesion in the target organ is determined based on the target position, which can improve the accuracy of the puncture path at the target time. Optionally, the target time is the time when the puncture object is punctured, so that the position of the target organ at the time of puncture can be estimated based on the target displacement relationship, and the puncture path can be planned based on the position, thereby improving the accuracy of the puncture path.

[0161] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0162] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among which, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0163] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

[0164] See also Fig.10 , Fig.10 A schematic diagram of a structure of a device for determining a relationship of respiratory displacement provided in an embodiment of the present application, wherein the device 1 for determining a relationship of respiratory displacement is used to determine a target displacement relationship between respiratory displacements of different points in a target organ of an object to be punctured, wherein the respiratory displacement is a displacement caused by the breathing of the object to be punctured, and the device 1 for determining a relationship of respiratory displacement comprises: an acquisition unit 11 and a determination unit 12, specifically:

[0165] an acquisition unit 11, configured to acquire a first image and a second image, wherein the first image and the second image both include the target organ, the first image and the second image are both acquired during the breathing process of the object to be punctured, and the respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image;

[0166] A determination unit 12, configured to determine a first displacement between a first pixel in the target organ in the first image and a second pixel in the target organ in the second image, the first pixel and the second pixel being image points of the same name;

[0167] The determining unit 12 is further configured to determine a second displacement between a third pixel in the target organ in the first image and a fourth pixel in the target organ in the second image, the third pixel and the fourth pixel being image points of the same name, and the third pixel being different from the first pixel;

[0168] The determining unit 12 is further configured to determine the target displacement relationship based on the first displacement and the second displacement.

[0169] In combination with any implementation manner of the present application, the determining unit 12 is specifically configured to:

[0170] determining a first position of the first pixel within the target organ in the first image;

[0171] determining a second position of the third pixel within the target organ in the first image;

[0172] determining first data based on the first position and the first displacement;

[0173] determining second data based on the second position and the second displacement;

[0174] The target displacement relationship is obtained by fitting the first data and the second data.

[0175] In combination with any implementation manner of the present application, the determining unit 12 is specifically configured to:

[0176] Using at least two candidate fitting methods, respectively fitting the first data and the second data to obtain at least two candidate displacement relationships, wherein the candidate displacement relationships represent the relationship between the position of the point in the target organ in the target organ and the respiratory displacement of the point in the target organ, and the candidate fitting methods correspond to the candidate displacement relationships in a one-to-one manner;

[0177] Determine a displacement corresponding to a third position based on each of the at least two candidate displacement relations, respectively, to obtain at least two third displacements, the third position corresponding to a position of the fifth pixel in the target organ in the first image, and the third displacements corresponding to the candidate displacement relations one by one;

[0178] determining at least two errors of the at least two candidate displacement relationships based on a difference between the fourth displacement and the at least two third displacements, the third displacements corresponding to the errors one-to-one, the fourth displacement being a displacement between the fifth pixel and a sixth pixel in the target organ in the second image, the fifth pixel and the sixth pixel being image points of the same name;

[0179] The target displacement relationship is determined from the at least two candidate displacement relationships based on the at least two errors.

[0180] In combination with any implementation manner of the present application, the determining unit 12 is specifically configured to:

[0181] determining a minimum of the at least two errors;

[0182] A displacement relationship corresponding to the minimum value is determined from the at least two candidate displacement relationships as the target displacement relationship.

[0183] In combination with any implementation manner of the present application, the determining unit 12 is further configured to:

[0184] determining a size of the target organ in the first image;

[0185] determining a fourth position based on a quotient of the first position and the size, the fourth position representing a relative position of the first pixel within the target organ;

[0186] Based on the fourth position and the first displacement, first data is determined.

[0187] In combination with any implementation manner of the present application, the determining unit 12 is specifically configured to:

[0188] Segmenting the target organ from the first image to obtain an organ image;

[0189] The position of the first pixel in the organ image is determined to obtain the first position.

[0190] In the embodiment of the present application, the first image and the second image both include the target organ, the first image and the second image are both acquired during the breathing process of the object to be punctured, and the respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image. After acquiring the first image and the second image, the device for determining the relationship between respiratory displacements determines the first displacement between the first pixel in the target organ in the first image and the second pixel in the target organ in the second image, wherein the first pixel and the second pixel are image points of the same name, and the first displacement can characterize the respiratory displacement of the first pixel in the target organ from the first moment to the second moment, wherein the first moment is the moment corresponding to the acquisition moment of the first image in the respiratory cycle, and the second moment is the moment corresponding to the acquisition moment of the second image in the respiratory cycle. Determine the second displacement between the third pixel in the target organ in the first image and the fourth pixel in the target organ in the second image, wherein the third pixel and the fourth pixel are image points of the same name, and the third pixel is different from the first pixel, and the second displacement can characterize the respiratory displacement of the third pixel in the target organ from the first moment to the second moment. The device for determining the relationship between respiratory displacements can determine the relationship between the respiratory displacements of different points in the target organ based on the first displacement and the second displacement, which is the target displacement relationship.

[0191] See also Fig.11 , Fig.11 The schematic diagram of the structure of a puncture device provided in an embodiment of the present application is as follows. The puncture device 2 includes: an acquisition unit 21 and a determination unit 22. Specifically:

[0192] An acquisition unit 21 is used to acquire a target displacement relationship between respiratory displacements of different points in a target organ of the object to be punctured, the target displacement relationship being obtained by the first aspect and any one of its embodiments, the respiratory displacement being a displacement generated by the breathing of the object to be punctured;

[0193] The acquisition unit 21 is further used to acquire a reference displacement of a reference point in the target organ, wherein the reference displacement is the respiratory displacement of the reference point at a target moment;

[0194] The determination unit 22 is configured to determine a target displacement of a target point in the target organ based on the target displacement relationship and the reference displacement, wherein the target displacement is the respiratory displacement of the target point at the target moment.

[0195] In combination with any implementation manner of the present application, the determining unit 22 is further configured to:

[0196] Determining a target position of the target organ at a target time based on the reference displacement and the target displacement;

[0197] Based on the target position, a puncture path for a lesion in the target organ is determined.

[0198] In an embodiment of the present application, after obtaining the target displacement relationship and the reference displacement of the reference point, the puncture device can determine the target displacement of the target point in the target organ based on the target displacement relationship and the reference displacement, thereby estimating the respiratory displacement of the target point based on the target displacement relationship.

[0199] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0200] Fig.12 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 3 includes a processor 31 and a memory 32. Optionally, the electronic device 3 also includes an input device 33 and an output device 34. The processor 31, the memory 32, the input device 33 and the output device 34 are coupled via a connector, and the connector includes various interfaces, transmission lines or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, it can be connected through various interfaces, transmission lines, buses, etc.

[0201] The processor 31 may be one or more graphics processing units (GPUs). When the processor 31 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Optionally, the processor 31 may be a processor group consisting of multiple GPUs, and the multiple processors are coupled to each other via one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in the embodiments of the present application.

[0202] The memory 32 can be used to store computer program instructions and various computer program codes including the program code for executing the program code of the present application. Optionally, the memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read only memory (CD-ROM), which is used for related instructions and data.

[0203] The input device 33 is used to input data and / or signals, and the output device 34 is used to output data and / or signals. The input device 33 and the output device 34 can be independent devices or an integrated device.

[0204] It can be understood that in the embodiment of the present application, the memory 32 can be used not only to store relevant instructions, but also to store relevant data. The embodiment of the present application does not limit the specific data stored in the memory.

[0205] Understandably, Fig.12 Only a simplified design of an electronic device is shown. In practical applications, the electronic device may also include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0206] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0207] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. Those skilled in the art can also clearly understand that the descriptions of the various embodiments of the present application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, refer to the records of other embodiments.

[0208] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0211] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0212] A person skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: a read-only memory (ROM) or a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

Claims

1. A method for determining the relationship between respiratory displacement, characterized in that The method is used to determine a target displacement relationship between respiratory displacements of different points in a target organ of a subject to be punctured, wherein the respiratory displacement is a displacement caused by the breathing of the subject to be punctured, and the method comprises: Acquiring a first image and a second image, wherein both the first image and the second image include the target organ, the first image and the second image are acquired during the breathing process of the object to be punctured, and the respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image; Determining a first displacement between a first pixel in the target organ in the first image and a second pixel in the target organ in the second image, the first pixel and the second pixel being image points of the same name; determining a second displacement between a third pixel in the target organ in the first image and a fourth pixel in the target organ in the second image, the third pixel and the fourth pixel being image points of the same name, and the third pixel being different from the first pixel; Determining the target displacement relationship based on the first displacement and the second displacement; The determining of the target displacement relationship based on the first displacement and the second displacement includes: determining a first position of the first pixel in the target organ in the first image; determining a size of the target organ in the first image; determining a fourth position based on a quotient of the first position and the size, the fourth position representing a relative position of the first pixel in the target organ; determining a second position of the third pixel in the target organ in the first image; determining first data based on the first position and the first displacement; the determining of first data based on the first position and the first displacement includes: determining first data based on the fourth position and the first displacement; determining second data based on the second position and the second displacement; and obtaining the target displacement relationship by fitting the first data and the second data.

2. The method according to claim 1, characterized in that The step of obtaining the target displacement relationship by fitting the first data and the second data includes: Using at least two candidate fitting methods, respectively fitting the first data and the second data to obtain at least two candidate displacement relationships, wherein the candidate displacement relationships represent the relationship between the position of the point in the target organ in the target organ and the respiratory displacement of the point in the target organ, and the candidate fitting methods correspond to the candidate displacement relationships in a one-to-one manner; Determine a displacement corresponding to a third position based on each of the at least two candidate displacement relations, respectively, to obtain at least two third displacements, wherein the third position is a position of the fifth pixel in the target organ in the first image, and the third displacements correspond one-to-one to the candidate displacement relations; determining at least two errors of the at least two candidate displacement relationships based on a difference between the fourth displacement and the at least two third displacements, the third displacements corresponding to the errors one-to-one, the fourth displacement being a displacement between the fifth pixel and a sixth pixel in the target organ in the second image, the fifth pixel and the sixth pixel being image points of the same name; The target displacement relationship is determined from the at least two candidate displacement relationships based on the at least two errors.

3. The method according to claim 2, characterized in that The determining the target displacement relationship from the at least two candidate displacement relationships based on the at least two errors includes: determining a minimum of the at least two errors; A displacement relationship corresponding to the minimum value is determined from the at least two candidate displacement relationships as the target displacement relationship.

4. A method for determining respiratory displacement, characterized in that The method comprises: Acquiring a target displacement relationship for characterizing respiratory displacements of different points in a target organ of the object to be punctured, wherein the target displacement relationship is obtained by the method of any one of claims 1 to 3, wherein the respiratory displacement is a displacement caused by the breathing of the object to be punctured; Acquire a reference displacement of a reference point in the target organ, wherein the reference displacement is the respiratory displacement of the reference point at a target time; Based on the target displacement relationship and the reference displacement, a target displacement of a target point in the target organ is determined, where the target displacement is the respiratory displacement of the target point at the target moment.

5. A device for determining the relationship between respiratory displacement, characterized in that The device for determining the relationship of respiratory displacement is used to determine the target displacement relationship between the respiratory displacements of different points in the target organ of the object to be punctured, wherein the respiratory displacement is the displacement caused by the breathing of the object to be punctured, and the device for determining the relationship of respiratory displacement includes: an acquisition unit, configured to acquire a first image and a second image, wherein the first image and the second image both include the target organ, the first image and the second image are both acquired during the breathing process of the object to be punctured, and the respiratory displacement of the target organ in the first image is different from the respiratory displacement of the target organ in the second image; a determining unit, configured to determine a first displacement between a first pixel in the target organ in the first image and a second pixel in the target organ in the second image, the first pixel and the second pixel being image points of the same name; The determining unit is further configured to determine a second displacement between a third pixel in the target organ in the first image and a fourth pixel in the target organ in the second image, the third pixel and the fourth pixel being image points of the same name, and the third pixel being different from the first pixel; The determining unit is further configured to determine the target displacement relationship based on the first displacement and the second displacement; The determination unit is specifically used to: determine the target displacement relationship based on the first displacement and the second displacement, including: determine a first position of the first pixel in the target organ in the first image; determine the size of the target organ in the first image; determine a fourth position based on a quotient of the first position and the size, the fourth position representing the relative position of the first pixel in the target organ; determine a second position of the third pixel in the target organ in the first image; determine first data based on the first position and the first displacement; determine first data based on the first position and the first displacement, including: determine first data based on the fourth position and the first displacement; determine second data based on the second position and the second displacement; obtain the target displacement relationship by fitting the first data and the second data.

6. A puncture device, characterized in that: The puncture device comprises: an acquisition unit, configured to acquire a target displacement relationship between respiratory displacements of different points in a target organ of the object to be punctured, the target displacement relationship being obtained by the method of any one of claims 1 to 3, the respiratory displacement being a displacement generated by the breathing of the object to be punctured; The acquisition unit is further used to acquire a reference displacement of a reference point in the target organ, wherein the reference displacement is the respiratory displacement of the reference point at a target moment; A determination unit is used to determine a target displacement of a target point in the target organ based on the target displacement relationship and the reference displacement, wherein the target displacement is the respiratory displacement of the target point at the target moment.

7. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program code, wherein the computer program code includes computer instructions, and when the processor executes the computer instructions, the electronic device executes the method as claimed in any one of claims 1 to 3, or executes the method as claimed in claim 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 3, or execute the method according to claim 4.

Images