Biopsy puncture position positioning method and device and storage medium

Abstract

The present application is suitable for the technical field of image processing, and provides a biopsy puncture position positioning method and device and a storage medium, the method comprising: performing feature extraction on an X-ray image containing a sheath body and a plurality of developing rings to obtain a sheath feature map and a developing ring feature map; performing detection on the sheath feature map and the developing ring feature map to obtain a first coordinate set and a second coordinate set respectively; mapping the coordinates of the pixel points of the first coordinate set and the second coordinate set to a coordinate system in which the X-ray image is located to obtain a sheath body coordinate in the X-ray image and a developing ring coordinate of each developing ring; calculating the distance between each developing ring and the sheath body according to the coordinates; and sorting each developing ring according to the distance, so as to determine the accurate position information of the sheath body and each developing ring. Then, the accurate position information obtained can be used to position each biopsy device passing through the sheath, so that the position information of the biopsy device can be conveniently obtained.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and particularly relates to a method, device, and storage medium for locating the site of a biopsy puncture. Background Technology

[0002] BTPNA (Bronchoscopic Transparenchymal Nodule Access), also known as the tunneling technique, involves creating a tunnel in the bronchial wall through a perforation. This allows access to the nodule within the lung parenchyma, bypassing the natural bronchial lumen and theoretically enabling "whole-lung access" to the nodule. Upon reaching the nodule, a biopsy is performed. Biopsy is a crucial clinical diagnostic tool for understanding the nature of lesions. The most commonly biopsied organs include the liver, breast, kidney, thyroid, and lungs. The biopsy procedure generally follows: First, Doppler ultrasound or CT scans are used to guide the location of the lesion. Sometimes, the lesion is deep or close to large blood vessels, requiring repeated adjustments by the patient to achieve the optimal puncture position. Next, local infiltration anesthesia is administered to the puncture site. Then, a needle and biopsy gun are used to directly extract the specimen from the lesion. The collected specimen is soaked in formalin solution and immediately sent to pathology for examination.

[0003] Existing CT or ultrasound-guided puncture biopsy techniques are difficult to accurately locate some smaller lesions, leading to repeated punctures of the diseased organ during the biopsy process. Repeated punctures can cause a series of complications to the organ, causing more pain to the patient. Summary of the Invention

[0004] The purpose of this invention is to provide a biopsy puncture location positioning method, device, and storage medium, aiming to solve the problem of low accuracy in existing biopsy puncture location positioning methods.

[0005] On one hand, the present invention provides a method for locating the site of a biopsy puncture, the method comprising:

[0006] Feature extraction is performed on the input X-ray image to obtain the sheath feature map and the imaging ring feature map corresponding to the X-ray image. The X-ray image includes the sheath body and several imaging rings.

[0007] The sheath body and the developing ring feature map are detected by the sheath body and the developing ring, respectively, to obtain the first set of coordinates of the pixels corresponding to the sheath body in the sheath feature map and the second set of coordinates of the pixels corresponding to the developing ring in the developing ring feature map.

[0008] The coordinates of the pixels in the first coordinate set and the second coordinate set are respectively mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image.

[0009] Based on the coordinates of the sheath body and the coordinates of the imaging rings, the distance between each imaging ring and the sheath body is calculated, and the imaging rings contained in the X-ray image are sorted according to the calculated distances.

[0010] On the other hand, the present invention provides a biopsy puncture location positioning device, the device comprising:

[0011] The feature extraction unit is used to extract features from the input X-ray image to obtain the sheath feature map and the imaging ring feature map corresponding to the X-ray image. The X-ray image includes the sheath body and several imaging rings.

[0012] The detection unit is used to detect the sheath body and the developing ring in the sheath feature map and the developing ring feature map, so as to obtain the first coordinate set of the pixel points corresponding to the sheath body in the sheath feature map and the second coordinate set of the pixel points corresponding to the developing ring in the developing ring feature map, respectively.

[0013] A mapping unit is configured to map the coordinates of pixels in the first coordinate set and the second coordinate set to the coordinate system of the X-ray image, respectively, to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image; and

[0014] The sorting unit is used to calculate the distance between each developing ring and the sheath body based on the coordinates of the sheath body and the developing ring coordinates, and to sort the developing rings contained in the X-ray image according to the calculated distances.

[0015] On the other hand, the present invention also provides an electronic device, including: a memory and a processor;

[0016] The memory stores executable program code;

[0017] The processor coupled to the memory calls the executable program code stored in the memory to execute the biopsy puncture location method provided in the above embodiments.

[0018] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when run by a processor, implements the biopsy puncture location positioning method provided in the above embodiments.

[0019] This invention extracts features from an X-ray image containing a sheath body and several radiopaque rings, obtaining a sheath feature map and a radiopaque ring feature map corresponding to the X-ray image. The sheath body and radiopaque rings are detected in the sheath feature map and radiopaque ring feature map to obtain a first coordinate set and a second coordinate set, respectively. The coordinates of the pixels in the first coordinate set and the second coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each radiopaque ring in the X-ray image. Based on the coordinates of the sheath body and the radiopaque rings, the distance between each radiopaque ring and the sheath body is calculated. The radiopaque rings in the X-ray image are sorted according to the calculated distances, thereby determining the accurate position information of the sheath body and each radiopaque ring. This accurate position information can then be used to locate the biopsy devices that subsequently pass through the sheath, thus conveniently obtaining the position information of the biopsy devices, effectively avoiding trial adjustments of the biopsy devices during operation, and significantly improving the accuracy of the operation. Attached Figure Description

[0020] Figure 1 A flowchart illustrating the implementation of a biopsy puncture location positioning method according to an embodiment of this application;

[0021] Figure 2A A flowchart illustrating the implementation of a biopsy puncture location positioning method according to an embodiment of this application;

[0022] Figure 2B This is a schematic diagram of the target detection network in a biopsy puncture location method provided in an embodiment of this application;

[0023] Figure 2C A schematic diagram showing an example of the segmentation of the sheath body and the imaging ring provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the biopsy puncture location positioning device provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the biopsy puncture location positioning device provided in an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The specific implementation of the present invention will be described in detail below with reference to specific embodiments:

[0029] See Figure 1 An embodiment of the present invention provides an implementation flow of a biopsy puncture location positioning method. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below:

[0030] In step S101, feature extraction is performed on the input X-ray image to obtain the sheath feature map and the imaging ring feature map corresponding to the X-ray image. The X-ray image includes the sheath body and several imaging rings.

[0031] The embodiments of the present invention are applicable to electronic devices, which may be mobile phones, tablet computers, wearable devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of this application do not impose any restrictions on the specific type of electronic device.

[0032] In this embodiment of the invention, the X-ray image is an image of the lesion with a sheath body and contrast rings, wherein the sheath comprises a sheath body and several contrast rings, the contrast rings being located at the distal end of the sheath. During interventional treatment, an entry point is typically created using puncture techniques, and then the sheath is inserted into the corresponding entry point, thereby establishing a channel from the entry point to the lesion location. The sheath can be considered an interventional delivery device; puncture needles or catheters can be delivered through the sheath to the corresponding location for treatment. The X-ray image can be an image acquired by medical imaging equipment, such as an X-ray machine, CT equipment (e.g., a C-arm machine), magnetic resonance imaging system, and ultrasound imaging equipment. The X-ray image may include images acquired by medical imaging equipment during sheath intervention in the lungs, liver, or kidneys, etc., and this specification does not limit this to such images.

[0033] In this embodiment of the invention, when extracting features from X-ray images, features are extracted at multiple scales, including shallow and deep scales. The feature map of the radiopaque ring, belonging to the shallow scale, undergoes fewer convolutions and has a smaller receptive field, making it suitable for detecting small objects such as radiopaque rings. Therefore, the feature map of the radiopaque ring at the shallow scale typically contains more texture features of the radiopaque ring in the X-ray image. Conversely, the feature map of the sheath, belonging to the deep scale, undergoes more convolutions and has a larger receptive field, making it suitable for detecting large objects such as the sheath body. Therefore, the feature map of the sheath at the deep scale typically contains more semantic information about the sheath in the X-ray image.

[0034] In a specific embodiment of this application, features can be extracted from X-ray images using a U-Net convolutional neural network. The feature map output from the shallow layer of the U-Net convolutional neural network can be used as the imaging ring feature map, while the feature map output from the deep layer of the U-Net convolutional neural network can be used as the sheath feature map. Preferably, the downsampling layer and the upsampling layer in the U-Net network are connected by a skip connection, so that the features extracted by the downsampling layer can be directly passed to the upsampling layer, thereby fusing feature information of different scales together and realizing feature sharing among layers.

[0035] In step S102, the sheath tube feature map and the developing ring feature map are subjected to sheath tube body and developing ring detection, so as to obtain the first coordinate set of the pixel points corresponding to the sheath tube body in the sheath tube feature map and the second coordinate set of the pixel points corresponding to the developing ring in the developing ring feature map.

[0036] In this embodiment of the invention, by performing sheath body detection on the sheath feature map, a first set of coordinates of the pixels corresponding to the sheath body within the sheath feature map can be obtained. By performing development ring detection on the development ring feature map, a second set of coordinates of the pixels corresponding to the corresponding development ring within the development ring feature map can be obtained.

[0037] In step S103, the coordinates of the pixels in the first coordinate set and the second coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image.

[0038] In this embodiment of the invention, the coordinates of the pixels of the sheath body in the first coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image. The coordinates of the pixels of the imaging rings in the second coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of each imaging ring in the X-ray image, thereby obtaining the positions of the sheath body and imaging rings in the X-ray image.

[0039] In step S104, the distance between each developing ring and the sheath body is calculated based on the coordinates of the sheath body and the developing ring coordinates, and the developing rings in the X-ray image are sorted according to the calculated distances.

[0040] In this embodiment of the invention, the distance between each developing ring and the sheath body is calculated based on the coordinates of the sheath body and the developing ring in the coordinate system of the X-ray image. The developing rings in the X-ray image are then sorted according to the calculated distances. The distance can represent the straight-line distance from each developing ring to the distal end of the sheath body.

[0041] This invention extracts features from an X-ray image containing a sheath body and several imaging rings to obtain a sheath feature map and imaging ring feature map corresponding to the X-ray image. The sheath body and imaging rings are then detected in the sheath feature map and imaging ring feature map to obtain a first coordinate set corresponding to the pixels of the sheath body and a second coordinate set corresponding to the pixels of the imaging rings. The coordinates of the pixels in the first and second coordinate sets are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body and the imaging rings in the X-ray image. Based on the coordinates of the sheath body and the imaging rings, the distance between each imaging ring and the sheath body is calculated. The imaging rings in the X-ray image are then sorted according to the calculated distances to determine the accurate position information of the sheath body and each imaging ring. This accurate position information can then be used to locate the biopsy devices that subsequently pass through the sheath, thus conveniently obtaining the position information of the biopsy devices, effectively avoiding trial adjustments during operation, and significantly improving the accuracy of the operation.

[0042] See Figure 2A An embodiment of the present invention provides an implementation flow of a biopsy puncture location positioning method. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below:

[0043] In step S201, feature extraction is performed on the input X-ray image to obtain the sheath feature map and the imaging ring feature map corresponding to the X-ray image. The X-ray image includes the sheath body and several imaging rings.

[0044] In this embodiment of the invention, the X-ray image is a lesion image with a sheath body and several radiopaque rings. The sheath includes a sheath body and several radiopaque rings, which are located at the distal end of the sheath. When extracting features from the X-ray image, features are extracted at multiple scales, including shallow and deep scales. Radiopaque ring feature maps at the shallow scale undergo fewer convolutions and have a smaller receptive field; therefore, they typically contain more textural features of the radiopaque rings in the X-ray image. Sheath feature maps at the deep scale undergo more convolutions and have a larger receptive field; therefore, they typically contain more semantic information about the sheath in the X-ray image.

[0045] In a specific embodiment of this application, features can be extracted from X-ray images using a U-Net convolutional neural network. Preferably, the downsampling layer and the upsampling layer in the U-Net network are connected by a skip connection, so that the features extracted by the downsampling layer can be directly passed to the upsampling layer, thereby fusing feature information of different scales together and realizing feature sharing among layers.

[0046] In step S202, the sheath body and the developing ring feature map are detected to obtain the first set of coordinates of the pixels corresponding to the sheath body in the sheath feature map and the second set of coordinates of the pixels corresponding to the developing ring in the developing ring feature map.

[0047] In this embodiment of the invention, by performing sheath body detection on the sheath feature map, a first set of coordinates of the pixels corresponding to the sheath body within the sheath feature map can be obtained. The first set of coordinates may include the coordinates of the pixels corresponding to the outline shape of the sheath body, or the coordinates of the pixels inside the sheath body, or both the outline shape and the internal coordinates of the pixels inside the sheath body. By performing development ring detection on the development ring feature map, a second set of coordinates of the pixels corresponding to the corresponding development ring within the development ring feature map can be obtained. The second set of coordinates may include the coordinates of the pixels corresponding to the outline shape of each development ring, or the coordinates of the pixels inside each development ring, or both the outline shape and the internal coordinates of the pixels inside the development ring.

[0048] In a preferred embodiment of the present invention, when detecting the sheath body and the developing ring in the sheath feature map and the developing ring feature map, a target detection network is used to identify the sheath feature map and the developing ring feature map respectively to obtain a sheath positioning feature map and a sheath classification feature map corresponding to the sheath body, and a developing ring positioning feature map and a developing ring classification feature map corresponding to the corresponding developing ring. Based on the sheath positioning feature map and the developing ring positioning feature map, detection boxes containing the sheath body and detection boxes containing the corresponding developing ring are determined. The target detection network can be a neural network built based on YOLOv5, etc., and this specification does not limit this. Edge detection is performed on the detection boxes containing the sheath body and the detection boxes containing the corresponding developing ring, respectively, to obtain a first coordinate set and a second coordinate set. The sheath classification feature map indicates whether the sheath body is included in the sheath classification feature map, the developing ring classification feature map indicates whether the developing ring is included in the developing ring classification feature map, the sheath positioning feature map indicates the position of the sheath body in the sheath positioning feature map, and the developing ring positioning feature map indicates the position of the developing ring in the developing ring feature map. Furthermore, the sheath positioning feature map and the imaging ring positioning feature map may include the coordinate positions of the detection frame corresponding to the sheath body and the detection frame corresponding to the imaging ring. The coordinate positions may include information about the center point of the detection frame, the width of the detection frame, and the height of the detection frame.

[0049] In this embodiment of the invention, a target detection network is used to identify the sheath feature map and the developing ring feature map, respectively, to obtain a sheath positioning feature map and a sheath classification feature map corresponding to the sheath body, and a developing ring positioning feature map and a developing ring classification feature map corresponding to the developing ring. Then, the sheath classification feature map is used to determine whether the sheath body is included. If the sheath body is included, the detection box containing the sheath body is determined based on the sheath positioning feature map. If the sheath body is not included, no further processing of the corresponding sheath positioning feature map is required, thereby improving processing efficiency and significantly reducing computational resource consumption. Similarly, the developing ring classification feature map is used to determine whether a developing ring is included. If a developing ring is included, the number of developing rings can be further determined, facilitating accurate sorting of multiple developing rings. The detection box containing the developing ring is then determined based on the developing ring positioning feature map. If the developing ring classification feature map is not included, no further processing of the developing ring positioning feature map is required, further improving processing efficiency and reducing computational resource consumption.

[0050] In specific implementation, the target detection network may include a sheath network and a developing ring network. The sheath network and developing ring network have identical structures, each including a fully convolutional module, a dilated convolutional module, a self-attention module, and a region extraction module. Specifically, when recognizing the sheath feature map and the developing ring feature map, the fully convolutional modules in the sheath network and the developing ring network are used to perform fully convolutional operations on the sheath feature map and the developing ring feature map respectively, to obtain the first feature map and the second feature map. The dilated convolutional modules in the sheath network and the developing ring network are used to add holes to the first feature map and the second feature map respectively, to obtain the third feature map and the fourth feature map respectively. The self-attention modules in the sheath network and the developing ring network are used to determine the pixel weights of the sheath body and the developing ring in the third feature map and the fourth feature map respectively, to obtain the fifth feature map and the sixth feature map respectively. The region extraction modules in the sheath network and the developing ring network are used to perform region extraction operations on the fifth feature map and the sixth feature map respectively, to obtain the sheath classification feature map, the sheath localization feature map, the developing ring classification feature map, and the developing ring localization feature map respectively. Among them, such as Figure 2B The diagram shown illustrates how the sheath network processes the sheath feature map.

[0051] In practical implementation, when using the fully convolutional modules in the sheath network and the imaging ring network to perform fully convolutional operations on the sheath feature map and the imaging ring feature map respectively, the fully convolutional modules can be constructed based on the FCN (Fully Convolutional Networks). The fully convolutional modules constructed using the FCN network can perform fully convolutional operations on the deep-scale sheath feature map and the shallow-scale imaging ring feature map of any size obtained after feature extraction, without having to convert the size of the sheath feature map and the imaging ring feature map, thereby improving the recognition efficiency of the sheath and imaging ring in X-ray images.

[0052] In specific implementation, when adding holes to the first feature map and the second feature map using the dilated convolution modules in the sheath network and the imaging ring network, the dilation method is used to expand the feature image of the first feature map and the second feature map to increase the receptive field of the feature map, so that the output of each convolutional layer contains a larger range of information, thereby improving the resolution of the image and recovering the information lost during the downsampling process of feature extraction.

[0053] In specific implementation, the self-attention module includes three convolutional layers, an inner product layer, and a normalization layer. When using the self-attention modules in the sheath network and the developing ring network to determine the pixel weights of the sheath body and the developing ring in the third and fourth feature maps, respectively, the three convolutional layers in the sheath network perform convolution operations on the third feature map to obtain the first, second, and third values ​​of the third feature map. The inner product layer performs an inner product of the first and second values ​​of the third feature map, and then performs an inner product of the result of the first and second values ​​with the third value to obtain the attention weight of each pixel in the third feature map. Then, the normalization layer normalizes the attention weights to obtain the attention weight coefficients of each pixel in the third feature map, thereby determining the pixel weights of the sheath body in the third feature map. Combining the attention weight coefficients and the third feature map for calculation and processing can highlight features belonging to the sheath body in the third feature map and suppress features not belonging to the sheath body. The steps for determining the pixel weights of the developing ring in the fourth feature map using the self-attention module in the developing ring network are the same as the steps for determining the pixel weights of the sheath body in the third feature map using the self-attention module in the sheath network, and will not be repeated here.

[0054] In specific implementation, the region extraction module includes a region extraction layer, a pooling layer, and a fully connected layer. When performing region extraction operations on the fifth and sixth feature maps using the region extraction modules in the sheath network and the imaging ring network, respectively, the region extraction layer performs convolution and activation operations on the fifth feature map to obtain a region probability feature map. Then, a convolution operation is performed on the region probability feature map to obtain the original features, the extracted regions of the original features, and the features of the extracted regions. Here, the original features refer to the features on the fifth feature map that have undergone region probability calculation without losing the original feature information. The extracted regions of the original features refer to the pixel region range corresponding to the sheath body obtained based on the original features. Next, the pooling layer performs pooling operations on the original features, the extracted regions of the original features, and the features of the extracted regions. Finally, the pooled original features, the extracted regions of the original features, and the features of the extracted regions are fully connected to obtain the sheath classification feature map and the sheath localization feature map. The steps for performing region extraction operations on the sixth feature map using the region extraction module in the imaging ring network are the same as the steps for performing region extraction operations on the fifth feature map using the region extraction module in the sheath network, and will not be repeated here.

[0055] In practical implementation, since the HU value of the sheath body in the sheath positioning feature map differs from its surroundings, and the HU value of the developing ring in the developing ring positioning feature map also differs from its surroundings, edge detection can be performed on the sheath positioning feature map and the developing ring positioning feature map to obtain a first set of coordinates corresponding to the sheath body and a second set of coordinates corresponding to the developing ring. Taking edge detection on the sheath positioning feature map as an example, noise in the sheath positioning feature map can first be removed using a Gaussian filter to effectively prevent false detections of the sheath body and remove other interfering information besides the sheath body. Then, using the Sobel operator and a set threshold corresponding to the sheath body, the sheath positioning feature map after Gaussian filter processing is processed to extract a clear image corresponding to the contour shape of the sheath body, thus obtaining a first set of coordinates containing the pixel coordinates of the contour shape of the sheath body. The process of edge detection on the developing ring positioning feature map is similar to the process of edge detection on the sheath positioning feature map described above, and will not be repeated here.

[0056] In another preferred embodiment of the present invention, when detecting the sheath body and the developing rings using the sheath feature map and the developing ring feature map, the sheath feature map and the developing ring feature map are respectively input into a pre-trained segmentation network for instance segmentation. This yields instance segmentation bounding boxes corresponding to the sheath body, instance segmentation bounding boxes corresponding to each developing ring, a first coordinate set, and a second coordinate set output by the segmentation network. This allows for direct detection of the sheath body and the developing rings through the pre-trained segmentation network, simplifying the detection process and improving detection efficiency. However, compared to the training process of the target detection network, the training process of the segmentation network is more complex. Specifically, the instance segmentation bounding box corresponding to the sheath body may include the outline shape of the sheath body, and the instance segmentation bounding box corresponding to each developing ring may include the outline shape of the corresponding developing ring. The segmentation network can be a neural network built based on SOLOv1 (SOLO, SegmentingObjectsbyLocations) or SOLOv2, etc.

[0057] In step S203, the coordinates of the pixels in the first coordinate set and the second coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image.

[0058] In this embodiment of the invention, the coordinates of the pixels of the sheath body in the first coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image. Similarly, the coordinates of the pixels of the imaging rings in the second coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of each imaging ring in the X-ray image. In a specific embodiment, when mapping the coordinates of the pixels in the first and second coordinate sets to the coordinate system of the X-ray image, according to a first transformation relationship between the sheath feature map and the X-ray image, the coordinates of the pixels in the first coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body. The first transformation relationship includes a translation matrix and a rotation matrix between the coordinate system of the sheath feature map and the coordinate system of the X-ray image. According to a second transformation relationship between the imaging ring feature map and the X-ray image, the coordinates of the pixels in the second coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of the imaging rings. The second transformation relationship includes a translation matrix and a rotation matrix between the coordinate system of the imaging ring feature map and the coordinate system of the X-ray image.

[0059] In step S204, the distance between each developing ring and the sheath body is calculated based on the coordinates of the sheath body and the developing ring coordinates, and the developing rings in the X-ray image are sorted according to the calculated distances.

[0060] In this embodiment of the invention, the distance between each developing ring and the sheath body is calculated based on the coordinates of the sheath body and the developing rings in the coordinate system of the X-ray image. The developing rings in the X-ray image are then sorted according to the calculated distances. For example, such as... Figure 2C As shown, the distance between each developing ring and the sheath body can represent the straight-line distance from each developing ring to the distal end of the sheath body. When sorting, the rings can be sorted in ascending or descending order based on the magnitude of the straight-line distance. Each developing ring corresponds to an identification number, which can be a number, letter, or Chinese name.

[0061] In practical implementation, when calculating the distance between each developing ring and the sheath body based on the coordinates of the sheath body and the developing ring in the coordinate system of the X-ray image, the coordinates of the geometric center pixel of the sheath body are calculated using the coordinate values ​​of each pixel contained in the sheath body, and the coordinates of the geometric center pixel of the developing ring are calculated using the coordinate values ​​of each pixel contained in the developing ring. Based on the coordinates of the geometric center pixels of the sheath body and the developing ring, the straight-line distance from the developing ring to the sheath body is determined, and the developing rings are sorted and numbered according to the magnitude of the straight-line distance. Since both the sheath body and the developing ring belong to simply connected regions, the geometric center of each simply connected region can be calculated using the OpenCV algorithm, thus obtaining the coordinates of the geometric center pixels of the sheath body and the developing ring. Of course, other methods can also be used to calculate the coordinates of the geometric center pixels of the sheath body and the developing ring; this specification does not limit this method.

[0062] In practical implementation, when calculating the distance between each developing ring and the sheath body based on the coordinates of the sheath body and the developing ring in the coordinate system of the X-ray image, the position of the sheath body can be represented by the coordinate value of any pixel in the sheath body. Then, based on the coordinate value of this arbitrary pixel and the coordinates of the geometric center pixel of the developing ring, the distance from the developing ring to the sheath body is determined. The developing rings are then sorted and numbered according to the determined distance. Of course, other methods can also be used to determine the distance between the sheath body and the developing ring, and this specification does not limit this.

[0063] In step S205, multiple X-ray images acquired by the image acquisition device from different angles are obtained, and the coordinates of the sheath body and the imaging ring contained in each X-ray image are determined in sequence, and the numbering result obtained by sorting each imaging ring is obtained.

[0064] In this embodiment of the invention, the multiple X-ray images at different angles are X-ray images at other angles corresponding to the X-ray image input in step S201, that is, lesion images with sheaths and contrast rings acquired from different angles. The coordinates of the sheath body and the contrast ring in each X-ray image at different angles can be obtained through steps S201 to S203, and the method for obtaining the number of the contrast ring in the sheath body of each X-ray image can be obtained through step S204, which will not be described in detail here.

[0065] In step S206, based on the angle at which the image acquisition device acquires different X-ray images and the coordinates of the pixels corresponding to the same number of the developing ring in the different X-ray images, 3D reconstruction is performed on each developing ring to obtain a three-dimensional model of the developing ring.

[0066] In this embodiment of the invention, by acquiring the angle at which the image acquisition device acquires different X-ray images and the coordinates of the pixels corresponding to the same number of the developing ring in the different X-ray images, 3D reconstruction of each developing ring is performed to obtain a three-dimensional model of the developing ring, thereby determining the pose information of the developing ring.

[0067] By combining the determined pose information of each contrast ring with the actual lesion location obtained from CT images, the three-dimensional spatial relationship between the contrast rings and the actual lesion location can be determined. This spatial relationship can include the distance scalar and position vector between each contrast ring and the actual lesion location. Therefore, during interventional treatment, the pose information of the biopsy device passing through the sheath can be determined based on the pose information of each contrast ring on the sheath. This allows for a quick determination of the position reached by the biopsy device, facilitating accurate attainment of the actual lesion location and avoiding multiple trial adjustments. The biopsy device can include a puncture needle, biopsy forceps, etc.

[0068] In this embodiment of the invention, feature extraction is performed on an X-ray image containing a sheath body and several imaging rings to obtain a sheath feature map and imaging ring feature map corresponding to the X-ray image. The sheath body and imaging rings are then detected in the sheath feature map and imaging ring feature map to obtain a first coordinate set and a second coordinate set, respectively. The coordinates of the pixels in the first and second coordinate sets are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image. Based on the coordinates of the sheath body and the imaging rings, the distance between each imaging ring and the sheath body is calculated. The imaging rings in the X-ray image are then sorted according to the calculated distances to determine the accurate position information of the sheath body and each imaging ring. This accurate position information can then be used to locate the biopsy devices that subsequently pass through the sheath, thus conveniently obtaining the position information of the biopsy devices, effectively avoiding trial adjustments of the biopsy devices during operation, and significantly improving the accuracy of the operation.

[0069] See Figure 3 The structure of the biopsy puncture location positioning device provided in one embodiment of the present invention is shown only for the purposes of explanation, including the parts relevant to the embodiment of the present invention, which include:

[0070] The feature extraction unit 31 is used to extract features from the input X-ray image to obtain the sheath feature map and the imaging ring feature map corresponding to the X-ray image. The X-ray image includes the sheath body and several imaging rings.

[0071] The detection unit 32 is used to detect the sheath body and the developing ring in the sheath feature map and the developing ring feature map, so as to obtain the first coordinate set of the pixel points corresponding to the sheath body in the sheath feature map and the second coordinate set of the pixel points corresponding to the developing ring in the developing ring feature map, respectively.

[0072] Mapping unit 33 is used to map the coordinates of pixels in the first coordinate set and the second coordinate set to the coordinate system of the X-ray image, respectively, to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image; and

[0073] The sorting unit 34 is used to calculate the distance between each developing ring and the sheath body based on the coordinates of the sheath body and the developing ring, and to sort the developing rings contained in the X-ray image according to the calculated distance.

[0074] In this embodiment of the invention, feature extraction is performed on an X-ray image containing a sheath body and several imaging rings to obtain a sheath feature map and imaging ring feature map corresponding to the X-ray image. The sheath body and imaging rings are then detected in the sheath feature map and imaging ring feature map to obtain a first coordinate set and a second coordinate set, respectively. The coordinates of the pixels in the first and second coordinate sets are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image. Based on the coordinates of the sheath body and the imaging rings, the distance between each imaging ring and the sheath body is calculated. The imaging rings in the X-ray image are then sorted according to the calculated distances to determine the accurate position information of the sheath body and each imaging ring. This accurate position information can then be used to locate the biopsy devices that subsequently pass through the sheath, thus conveniently obtaining the position information of the biopsy devices, effectively avoiding trial adjustments of the biopsy devices during operation, and significantly improving the accuracy of the operation.

[0075] See Figure 4 The structure of the biopsy puncture location positioning device provided in one embodiment of the present invention is shown only for the purposes of explanation, including the parts relevant to the embodiment of the present invention, which include:

[0076] The feature extraction unit 41 is used to extract features from the input X-ray image to obtain the sheath feature map and the imaging ring feature map corresponding to the X-ray image. The X-ray image includes the sheath body and several imaging rings.

[0077] Detection unit 42 is used to detect the sheath body and the developing ring in the sheath feature map and the developing ring feature map, so as to obtain the first coordinate set of the pixel points corresponding to the sheath body in the sheath feature map and the second coordinate set of the pixel points corresponding to the developing ring in the developing ring feature map.

[0078] The mapping unit 43 is used to map the coordinates of the pixels in the first coordinate set and the second coordinate set to the coordinate system of the X-ray image, so as to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image.

[0079] The sorting unit 44 is used to calculate the distance between each developing ring and the sheath body based on the coordinates of the sheath body and the developing ring, and to sort the developing rings contained in the X-ray image according to the calculated distance.

[0080] The numbering acquisition unit 45 is used to acquire multiple X-ray images acquired by the image acquisition device from different angles, and sequentially determine the coordinates of the sheath body and the imaging ring contained in each X-ray image, as well as acquire the numbering result obtained by sorting each imaging ring; and

[0081] The 3D reconstruction unit 46 is used to perform 3D reconstruction of each developing ring based on the angle at which the image acquisition device acquires different X-ray images and the coordinates of the pixels corresponding to the same number of developing ring in different X-ray images, so as to obtain a three-dimensional model of the developing ring.

[0082] In one specific embodiment, the detection unit 42 includes:

[0083] The feature map acquisition unit is used to identify the sheath feature map and the imaging ring feature map through the target detection network, respectively, to obtain the sheath positioning feature map and sheath classification feature map corresponding to the sheath body, and the imaging ring positioning feature map and imaging ring classification feature map corresponding to the corresponding imaging ring;

[0084] The detection frame determination unit is used to determine the detection frame containing the sheath body and the detection frame containing the corresponding imaging ring based on the sheath positioning feature map and the imaging ring positioning feature map.

[0085] The set acquisition unit is used to perform edge detection on the detection frame containing the sheath body and the detection frame containing the corresponding imaging ring, respectively, to obtain a first coordinate set and a second coordinate set.

[0086] Specifically, the object detection network includes a sheath network and a developmental loop network. Both the sheath network and the developmental loop network include a fully convolutional module, a dilated convolutional module, a self-attention module, and a region extraction module. In this case, the feature map acquisition unit may include:

[0087] The fully convolutional unit is used to perform fully convolutional operations on the sheath feature map and the developing ring feature map respectively using the fully convolutional modules in the sheath network and the developing ring network to obtain the first feature map and the second feature map respectively.

[0088] The hole-adding unit is used to add holes to the first feature map and the second feature map respectively using the hole convolution modules in the sheath network and the imaging ring network, so as to obtain the third feature map and the fourth feature map respectively.

[0089] The self-attention unit is used to determine the pixel weights of the sheath body and the developing ring in the third and fourth feature maps respectively using the self-attention modules in the sheath network and the developing ring network, so as to obtain the fifth and sixth feature maps accordingly.

[0090] The region extraction unit is used to perform region extraction operations on the fifth feature map and the sixth feature map respectively using the region extraction modules in the sheath network and the developing ring network, so as to obtain the sheath classification feature map, sheath positioning feature map, developing ring classification feature map and developing ring positioning feature map respectively.

[0091] In one specific embodiment, the detection unit 42 includes:

[0092] The instance segmentation unit is used to input the sheath feature map and the imaging ring feature map into a pre-trained segmentation network for instance segmentation, and obtain the instance segmentation annotation box corresponding to the main body of the sheath, the instance segmentation annotation box corresponding to each imaging ring, the first coordinate set, and the second coordinate set output by the segmentation network.

[0093] In one specific embodiment, the mapping unit 43 includes:

[0094] The first mapping unit is used to map the coordinates of the pixels contained in the first coordinate set to the coordinate system of the X-ray image according to the first transformation relationship between the sheath feature map and the X-ray image, so as to obtain the coordinates of the sheath body.

[0095] The second mapping unit is used to map the coordinates of the pixels contained in the second coordinate set to the coordinate system of the X-ray image according to the second transformation relationship between the development ring feature map and the X-ray image, so as to obtain the development ring coordinates.

[0096] In one specific embodiment, the sorting unit 44 includes:

[0097] The first coordinate calculation unit is used to calculate the coordinates of the geometric center pixel of the sheath body using the coordinates of the sheath body.

[0098] The second coordinate calculation unit is used to calculate the coordinates of the geometric center pixel of any developing ring using the developing ring coordinates corresponding to any developing ring.

[0099] The sorting subunit is used to determine the distance from each developing ring to the sheath based on the coordinates of the geometric center pixel of the sheath body and the geometric center pixel coordinates of each developing ring, and to sort the developing rings contained in the X-ray image based on the determined distances.

[0100] In this embodiment of the invention, each unit or module of the biopsy puncture location positioning device can be implemented by corresponding hardware or software units or modules. Each unit or module can be an independent hardware or software unit or module, or it can be integrated into a single hardware or software unit or module, which is not intended to limit the invention. Specific implementation methods for each unit or module of the biopsy puncture location positioning device can be found in the description of the foregoing method embodiments, and will not be repeated here.

[0101] See Figure 5 The present application provides a schematic diagram of the hardware structure of an electronic device according to an embodiment.

[0102] For example, the electronic device can be any of various types of computer system devices that are non-movable or movable or portable and perform wireless or wired communication. Specifically, the electronic device can be a desktop computer, server, mobile phone or smartphone (e.g., iPhone™-based, Android™-based phone), portable gaming device (e.g., Nintendo DS™, PlayStation Portable™, Gameboy Advance™, iPhone™), laptop computer, PDA, portable internet device, portable medical device, smart camera, music player and data storage device, other handheld devices and such as watches, headphones, pendants, etc. The electronic device can also be other wearable devices (e.g., such as electronic glasses, electronic clothing, electronic bracelets, electronic necklaces and other head-mounted devices (HMDs)).

[0103] like Figure 5 As shown, the electronic device 5 may include a control circuit, which may include a storage and processing circuit 51. The storage and processing circuit 51 may include a memory, such as a hard disk drive, a non-volatile memory (e.g., flash memory or other electronically programmable erasable memory used to form a solid-state drive), or a volatile memory (e.g., static or dynamic random access memory), etc., which are not limited in this embodiment. The processing circuit in the storage and processing circuit 51 can be used to control the operation of the electronic device 5. This processing circuit may be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application-specific integrated circuits (ASICs), display driver integrated circuits, etc.

[0104] The storage and processing circuit 51 can be used to run software in the electronic device 5, such as internet browsing applications, Voice over Internet Protocol (VoIP) telephone calling applications, email applications, media playback applications, operating system functions, etc. This software can be used to perform various control operations, such as image acquisition based on a camera, ambient light measurement based on an ambient light sensor, proximity sensor measurement based on a proximity sensor, information display functions based on status indicators such as LED status lights, touch event detection based on a touch sensor, functions associated with displaying information on multiple (e.g., layered) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals, control operations associated with collecting and processing button press event data, and other functions in the electronic device 5, etc., which are not limited in the embodiments of this application.

[0105] Furthermore, the memory stores executable program code, and a processor coupled to the memory calls the executable program code stored in the memory to execute the biopsy puncture location method as described in the foregoing embodiments, for example: Figure 1 The method described in steps S101-S104.

[0106] The executable program code includes various units or modules of the biopsy puncture location positioning device as described in the foregoing embodiments, for example: Figure 3 Modules 31-34 are included. The specific processes by which the above-mentioned units or modules implement their respective functions, as described in the relevant descriptions of the above-described biopsy puncture location positioning device embodiments, will not be repeated here.

[0107] Furthermore, this application embodiment also provides a non-transitory computer-readable storage medium, which can be configured in the server in the above embodiments. The non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the biopsy puncture location positioning method described in the aforementioned biopsy puncture location positioning method embodiments.

[0108] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0109] Those skilled in the art will recognize that the modules / units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0110] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0114] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for locating the site of a biopsy puncture, characterized in that, The method includes: The input X-ray image is subjected to feature extraction at both deep and shallow scales. The feature extraction at the deep scale yields the sheath feature map corresponding to the X-ray image, and the feature extraction at the shallow scale yields the imaging ring feature map corresponding to the X-ray image. The X-ray image contains the sheath body and several imaging rings. The sheath body and the developing ring feature map are detected by the sheath body and the developing ring, respectively, to obtain the first set of coordinates of the pixels corresponding to the sheath body in the sheath feature map and the second set of coordinates of the pixels corresponding to the developing ring in the developing ring feature map. The coordinates of the pixels in the first coordinate set and the second coordinate set are respectively mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image. Based on the coordinates of the sheath body and the coordinates of the imaging rings, the distance between each imaging ring and the sheath body is calculated, and the imaging rings contained in the X-ray image are sorted according to the calculated distances.

2. The method as described in claim 1, characterized in that, The method further includes: Multiple X-ray images acquired from different angles by an image acquisition device are obtained, and the coordinates of the sheath body and the imaging ring contained in each X-ray image are determined in sequence. The numbering results obtained by sorting each imaging ring are also obtained. Based on the angle at which the image acquisition device acquires different X-ray images and the coordinates of the pixels corresponding to the same numbered development ring in the different X-ray images, 3D reconstruction is performed on each development ring to obtain a three-dimensional model of the development ring.

3. The method as described in claim 1, characterized in that, The step of detecting the sheath body and the radiopaque ring in the sheath feature map and radiopaque ring feature map includes: The target detection network is used to identify the sheath feature map and the imaging ring feature map respectively, so as to obtain the sheath positioning feature map and sheath classification feature map corresponding to the sheath body, and the imaging ring positioning feature map and imaging ring classification feature map corresponding to the corresponding imaging ring; Based on the sheath positioning feature map and the imaging ring positioning feature map, a detection frame containing the sheath body and a detection frame containing the corresponding imaging ring are determined. Edge detection is performed on the detection frame containing the sheath body and the detection frame containing the corresponding imaging ring to obtain the first coordinate set and the second coordinate set.

4. The method as described in claim 3, characterized in that, The target detection network includes a sheath network and a developing loop network. Both the sheath network and the developing loop network include a fully convolutional module, a dilated convolutional module, a self-attention module, and a region extraction module. The step of identifying the sheath feature map and the radiopaque feature map through a target detection network to obtain the sheath positioning feature map and sheath classification feature map corresponding to the sheath body, and the radiopaque positioning feature map and radiopaque classification feature map corresponding to the corresponding radiopaque ring, includes: The sheath feature map and the developing ring feature map are subjected to full convolution operations using the fully convolution modules in the sheath network and the developing ring network, respectively, to obtain the first feature map and the second feature map accordingly. The first and second feature maps are respectively dilated using the hole convolution modules in the sheath network and the imaging ring network to obtain the third and fourth feature maps. The pixel weights of the sheath body and the developing ring in the third and fourth feature maps are determined by the self-attention modules in the sheath network and the developing ring network, respectively, so as to obtain the fifth and sixth feature maps accordingly. The region extraction modules in the sheath network and the developing ring network are used to perform region extraction operations on the fifth feature map and the sixth feature map respectively, so as to obtain the sheath classification feature map, sheath positioning feature map, developing ring classification feature map and developing ring positioning feature map respectively.

5. The method as described in claim 1, characterized in that, The steps for detecting the sheath body and the radiopaque ring based on the sheath feature map and radiopaque ring feature map include: The sheath feature map and the imaging ring feature map are respectively input into a pre-trained segmentation network for instance segmentation, and the instance segmentation annotation boxes corresponding to the sheath body, the instance segmentation annotation boxes corresponding to each imaging ring, the first coordinate set, and the second coordinate set are obtained from the output of the segmentation network.

6. The method as described in claim 1, characterized in that, The step of calculating the distance between each imaging ring and the sheath body based on the coordinates of the sheath body and the imaging rings, and sorting the imaging rings in the X-ray image according to the calculated distances, includes: Using the coordinates of the sheath body, calculate the coordinates of the geometric center pixel of the sheath body; Using the coordinates of the developing ring corresponding to any developing ring, calculate the coordinates of the geometric center pixel of any developing ring; Based on the coordinates of the geometric center pixel of the sheath body and the coordinates of the geometric center pixel of each imaging ring, the distance from each imaging ring to the sheath body is determined, and based on the determined distance, the imaging rings contained in the X-ray image are sorted.

7. The method as described in claim 1, characterized in that, The step of mapping the coordinates of pixels in the first coordinate set and the second coordinate set to the coordinate system of the X-ray image to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image includes: Based on the first transformation relationship between the sheath feature map and the X-ray image, the coordinates of the pixels contained in the first coordinate set are mapped to the coordinate system of the X-ray image to obtain the coordinates of the sheath body. Based on the second transformation relationship between the imaging ring feature map and the X-ray image, the coordinates of the pixels contained in the second coordinate set are mapped to the coordinate system of the X-ray image to obtain the imaging ring coordinates.

8. A biopsy puncture location positioning device, characterized in that, The device includes: The feature extraction unit is used to extract features at both deep and shallow scales from the input X-ray image. The feature extraction at the deep scale yields a sheath feature map corresponding to the X-ray image, and the feature extraction at the shallow scale yields a radiopaque feature map corresponding to the X-ray image. The X-ray image contains the sheath body and several radiopaque rings. The detection unit is used to detect the sheath body and the developing ring in the sheath feature map and the developing ring feature map, so as to obtain the first coordinate set of the pixel points corresponding to the sheath body in the sheath feature map and the second coordinate set of the pixel points corresponding to the developing ring in the developing ring feature map, respectively. A mapping unit is configured to map the coordinates of pixels in the first coordinate set and the second coordinate set to the coordinate system of the X-ray image, respectively, to obtain the coordinates of the sheath body in the X-ray image and the coordinates of each imaging ring in the X-ray image; and The sorting unit is used to calculate the distance between each developing ring and the sheath body based on the coordinates of the sheath body and the developing ring coordinates, and to sort the developing rings contained in the X-ray image according to the calculated distances.

9. An electronic device, comprising a memory and a processor; The memory stores executable program code; The processor coupled to the memory invokes the executable program code stored in the memory to perform the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images