Noise reduction method, device, terminal and computer-readable storage medium for perspective video

By calculating feature maps and spatial transformation, combined with U-Net neural network and ORB algorithm, the noise problem caused by low-dose shooting in X-ray fluoroscopy is solved, and effective noise reduction for fluoroscopy videos and image clarity is improved. Clear display of guidewire and catheter during the operation.

CN114677299BActive Publication Date: 2024-11-05SUZHOU IND PARK ZHIZAITIANXIA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210302478.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-24
Publication Date
2024-11-05
Estimated Expiration
2042-03-24

AI Technical Summary

Technical Problem

In X-ray fluoroscopy, due to low dose shooting, the traditional noise reduction method will cause blur during the operation, making it difficult to effectively reduce noise while maintaining the clarity of fine structures.

Method used

By obtaining video frames from X-ray fluoroscopic video, calculating feature maps and spatial transformations, acquiring feature points using U-Net neural network and ORB algorithms, calculating distance maps and weight maps, and performing noise reduction processing to reduce noise.

Benefits of technology

It effectively reduces the noise of the perspective video, improves the clarity of the image and observability of the guidewire and catheter, reduces blurring, and is suitable for the clear display of fine objects during surgery.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114677299B_ABST
    Figure CN114677299B_ABST
Patent Text Reader

Abstract

The present invention discloses a denoising method, device, terminal and computer-readable storage medium for perspective video, the denoising method comprising: obtaining a plurality of video frames arranged in time sequence from the perspective video obtained by X-ray fluorescence perspective, obtaining a feature map of each video frame, then performing a registration operation, and then performing a denoising operation, so that the perspective video can be denoised.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of X-ray images, and in particular to a denoising method, device, terminal and computer-readable storage medium for perspective videos. Background Art

[0002] X-ray fluoroscopy refers to the use of an X-ray system to continuously take projection images of human body parts at low doses and short time intervals. Multiple projection images constitute a fluoroscopic video. This technology can display X-ray images in real time with high temporal resolution, and the frame rate range is usually between 3-60FPS (Frames Per Second). Due to the widespread use of flat-panel detectors, this technology has a wide imaging field of view and high spatial resolution. Fluoroscopy is widely used in clinical practice, for example, during electrophysiological examinations (for arrhythmias) and during coronary catheterization to determine whether the catheter is correctly placed in the heart; when using X-ray opaque contrast agents (such as barium, usually taken orally), it is used to evaluate the gastrointestinal tract; during the evaluation of musculoskeletal injuries, it is used to observe the movement of bones and joints.

[0003] Since a fluoroscopic examination requires continuous collection of a large number of images (more than several hundred), and X-rays can cause ionizing radiation hazards to the human body, we can only use extremely low exposure doses to collect images. The dose of each frame in a fluoroscopic examination is about one-five-hundredth of that of an X-ray film. However, low-dose shooting leads to a lot of noise in the image. Time dimension smoothing can restore clear images very well. In interventional treatment, doctors need to use fluoroscopic images to observe some delicate objects, such as guidewires and catheters. During the operation, these objects move violently, and traditional noise reduction methods will cause blur.

[0004] Therefore, how to reduce the noise of perspective video while maintaining the clarity of fine structures becomes an urgent problem to be solved. Summary of the invention

[0005] In view of this, the main purpose of the present invention is to provide a noise reduction method, device, terminal and computer-readable storage medium for perspective video.

[0006] To achieve the above object, the technical solution of the present invention is implemented as follows: a method for reducing noise of a fluoroscopic video obtained by X-ray fluoroscopy, comprising the following steps: obtaining N video frames I1, I2, ..., I2 arranged in time sequence from the fluoroscopic video obtained by X-ray fluoroscopy; N , video frames I1,I2,…,I N The lengths and widths of the video frames I1, I2, …, I NHave the same pixel coordinate system C1, N is a natural number, N ≥ 2; Get video frame I j The corresponding feature map F j , feature map F j The length and width of the video frame I j The length and width of the feature map F j Set the pixel coordinate system C1; in the feature map F j The pixel value Value of each pixel DF in the video frame I is used to represent: j , the probability value of the pixel DI at the same coordinate as the pixel DF belonging to the guidewire image or the catheter image, 0≤Value≤1, j=1,2,...,N; Get the video frame I k To video frame I k+1 The spatial transformation T k→k+1 , get video frame I k Align to I N The video frame I′ k , get the feature map F k Align to F N The feature map F′ k , video frame I′ k The length and width are equal to I N The length and width of the feature map F′ k The length and width are equal to F N The length and width of the video frame I′ k and feature map F′ k Set the pixel coordinate system C1, k = 1, 2, ..., N-1, let F′ N =F N , I′ N =I N ; Calculation I j ′ and I′ N The first distance map between Calculate F′ j With F N The second distance map between According to the first distance map and the second distance map Calculate the weight graph w j , Wherein, j=1, 2, ..., N, σ1 and σ2 are both constants, x is any coordinate in the pixel coordinate system C1, I′ j (x), I′ N (x), F′ j (x) and F′ N (x) are the coordinates x in the first distance map Second distance map Video frame I′j, video frame I′N, feature map F′j and feature map F′ N The pixel value of the corresponding pixel in the video frame I, abs() is the absolute value function; N The corresponding denoised frame is in, and I′ t (x) are the coordinates x in the denoised frame and video frame I′ t The pixel value of the corresponding pixel in .

[0007] As an improvement of the embodiment of the present invention, the “obtaining video frame I j The corresponding feature map F j "Specifically include: Based on the U-Net neural network, obtain the video frame I j The corresponding feature map F j When training the U-Net neural network, obtain several video frames I" in the perspective video obtained by X-ray fluoroscopy, and the feature annotation map F corresponding to each video frame I" gt , feature annotation graph F gt is a binary image, feature annotation image F gt The length and width of the video frame I' are equal to the length and width of the feature annotation figure F gt and video frame I' have the same pixel coordinate system C2; feature annotation map F gt Used to characterize: When the pixel F gt When the pixel value of (y) = 1, the pixel I" (y) belongs to the guidewire image or catheter image. gt When the pixel value of (y) = 0, the pixel I" (y) does not belong to the guidewire image, nor does it belong to the catheter image, where y is any coordinate in the pixel coordinate system C2, and F gt (y) and I”(y) are the coordinates of y in the feature annotation map F gt and the corresponding pixels in the video frame I”; the feature annotation map F corresponding to each video frame I” gt The input is into the U-Net neural network to train the U-Net neural network.

[0008] As an improvement of the embodiment of the present invention, the “obtaining video frame I k To video frame I k+1 The spatial transformation T k→k+1 "Specifically includes: based on the feature point method, obtaining the video frame I k To video frame I k+1 The spatial transformation T k→k+1 .

[0009] As an improvement of the embodiment of the present invention, the “obtaining video frame I k To video frame I k+1 The spatial transformation T k→k+1 "Specifically include: Based on the ORB algorithm, from the video frame I k Get multiple FAST feature pixels to form a feature point set P k , from video frame I k+1 Get multiple FAST feature pixels to form a feature point set P k+1 ; Get feature point set P k The BRIEF descriptor of each feature point in the feature point set P is obtained k+1 The BRIEF descriptor of each feature point in the feature point set P k Each element E k With the feature point set P k+1 The element E in k+1 Pairing obtains multiple paired points, and the distance between each point pair is F(E k ,E k+1 ), and satisfy: F(E k ,E k+1 )=min(F(E k , E)|E∈feature point set P k+1 ), F(x1, x2) is the Hamming distance between the BRIEF descriptor of x1 and the BRIEF descriptor of x2; a preset ratio of point pairs with the shortest distance is selected from multiple point pairs to form multiple target point pairs, and the spatial transformation T is calculated according to the positional relationship between the two paired elements in each target point. k→k+1 , where 0<preset ratio≤1.

[0010] As an improvement to the embodiment of the present invention, the preset ratio=90%.

[0011] As an improvement of the embodiment of the present invention, the feature point set P k And the feature point set P k+1 Each contains 5000 FAST feature pixels.

[0012] As an improvement to the embodiment of the present invention, σ1=0.02, σ2=0.25.

[0013] As an improvement of the embodiment of the present invention, the “obtaining N video frames I1, I2, …, I3 arranged in time sequence from the perspective video obtained by X-ray fluoroscopy” N Specifically, it includes: obtaining N video frames I1, I2, ..., I1 arranged in time sequence from the perspective video obtained by X-ray fluorescence perspective. N , for video frames I1,I2,…,I NEdge-preserving spatial noise reduction is performed on both.

[0014] The embodiment of the present invention further provides a denoising device for a perspective video of X-ray fluoroscopy, comprising the following modules: a video acquisition module, for acquiring N video frames I1, I2, ..., I1 arranged in time sequence from the perspective video obtained by X-ray fluoroscopy; N , video frames I1,I2,…,I N The lengths and widths of the video frames I1, I2, …, I N have the same pixel coordinate system C1, N is a natural number, N ≥ 2; feature acquisition module, used to obtain the video frame I j The corresponding feature map F j , feature map F j The length and width of the video frame I j The length and width of the feature map F j Set the pixel coordinate system C1; in the feature map F j The pixel value Value of each pixel DF in the video frame I is used to represent: j The probability value of the pixel DI at the same coordinate as the pixel DF belonging to the guidewire image or the catheter image, 0≤Value≤1, j=1,2,...,N; the registration module is used to obtain the video frame I k To video frame I k+1 The spatial transformation T k→k+1 , get video frame I k Align to I N The video frame I′ k , get the feature map F k Align to F N The feature map F′ k , video frame I′ k The length and width are equal to I N The length and width of the feature map F′ k The length and width are equal to F N The length and width of the video frame I′ k and feature map F′ k Set the pixel coordinate system C1, k = 1, 2, ..., N-1, let F′ N =F N , I′ N =I N ; Denoising module, used to calculate I j ′ and I′ N The first distance map between Calculate F j ′ and F′ N The second distance map between According to the first distance map and the second distance map Calculate the weight graph w j , Wherein, j=1, 2, ..., N, σ1 and σ2 are both constants, x is any coordinate in the pixel coordinate system C1, I j ′(x), I′ N (x), F j ′(x) and F′ N (x) are the coordinates x in the first distance map Second distance map Video frame I′ j , video frame I′ N , feature map F′ j and feature map F′ N The pixel value of the corresponding pixel in the video frame I is abs(), which is an absolute value function; the processing module is used for N The corresponding denoised frame is in, and I′ t (x) are the coordinates x in the denoised frame and video frame I′ t The pixel value of the corresponding pixel in .

[0015] This embodiment further provides a terminal, including: a memory, used to store a computer program; and a processor, used to implement the steps of the above-mentioned denoising method when executing the computer program.

[0016] This embodiment further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned denoising method are implemented.

[0017] The denoising method, device, terminal and computer-readable storage medium of the perspective video provided by the embodiment of the present invention have the following advantages: The embodiment of the present invention discloses a denoising method, device, terminal and computer-readable storage medium of the perspective video, the denoising method comprises: obtaining a plurality of video frames arranged in time sequence from the perspective video obtained by X-ray fluoroscopy, obtaining a feature map of each video frame, then performing a registration operation, and then performing a denoising operation. Thus, the denoising process can be performed on the perspective video. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a denoising method according to an embodiment of the present invention;

[0019] Figure 2A, Figure 2B , Figure 3 , Figure 4 and Figure 5 This is the experimental result of the denoising method. DETAILED DESCRIPTION

[0020] The present invention will be described in detail below in conjunction with the embodiments shown in the accompanying drawings. However, the embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on the embodiments are all within the protection scope of the present invention.

[0021] The following description and accompanying drawings fully illustrate the specific embodiments of this article so that those skilled in the art can practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments of this article includes the entire scope of the claims, as well as all available equivalents of the claims. Herein, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the structure, device or equipment including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such structure, device or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the structure, device or equipment including the elements. Each embodiment is described in a progressive manner herein, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0022] The terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. in this document indicate the orientation or position relationship based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing this document and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description of this document, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, it can also be the internal communication of two elements, it can be a direct connection, or it can be an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0023] The embodiment of the present invention provides a method for reducing noise of a fluoroscopic video for X-ray fluoroscopy, such as Figure 1 As shown, the following steps are included:

[0024] Step 101: Obtain N video frames I1, I2, ..., I3 arranged in time sequence from the perspective video obtained by X-ray fluoroscopy. N , video frames I1,I2,…,I N The lengths and widths of the video frames I1, I2, …, I N Have the same pixel coordinate system C1, N is a natural number, N ≥ 2; here, it can be assumed that the length H and width W of each video frame are, then the video frame can be expressed as I∈R H*W , R H*W is a two-dimensional real number space, optional, the generation time of I1 < the generation time of I2 <... N The time of generation.

[0025] Step 102: Get video frame I j The corresponding feature map F j , feature map F j The length and width of the video frame I j The length and width of the feature map F j Set the pixel coordinate system C1; in the feature map F j The pixel value Value of each pixel DF in the video frame I is used to represent: j , the probability value of the pixel DI at the same coordinate as the pixel DF belonging to the guidewire image or the catheter image, 0≤Value≤1, j=1,2,...,N; Here, it can be understood that when the pixel value Value of the pixel DF is larger, the video frame I j The greater the probability that the pixel DI belongs to the guidewire image or the catheter image. For each pixel, its position in the length direction is obtained, and then its position in the width direction is obtained. Therefore, the position in the length direction and the position in the width direction constitute the coordinate x of the pixel (including the position in the length direction and the position in the width direction). Then, the pixel coordinate system is generated with the coordinate. For example, in the image, the coordinate of the pixel in the upper left corner is (0,0). In the right direction, the width position gradually increases, and in the downward direction, the length position gradually increases.

[0026] Here, it can be understood that in the perspective video, the guide wire and the catheter both present low-intensity features and are linear structures. Therefore, based on this feature, the video frame I can be extracted. j The corresponding feature map F j .

[0027] ​Here, if Figure 2A shows a video frame, Figure 2A There is a curved catheter in the vertical direction, a section of catheter in the horizontal direction, and a thinner guide wire extending from the catheter. Figure 2B For the corresponding feature map, the pixel response related to the guidewire catheter is very high, while the response to other information is very small.

[0028] Step 103: Get video frame I k To video frame I k+1 The spatial transformation T k→k+1 , get video frame I k Align to I N The video frame I′ k , get the feature map F k Align to F N The feature map F′ k , video frame I′ k The length and width are equal to I N The length and width of the feature map F′ k The length and width are equal to F N The length and width of the video frame I′ k and feature map F′ k Set the pixel coordinate system C1, k = 1, 2, ..., N-1, let F′ N =F N , I′ N =I N Here, in the perspective imaging process, the imaging device is fixed, and the imaging object is often translated to achieve the purpose of observing different parts. The input of the image registration module is two adjacent video frames I k and I k+1 , the output is I k To I k+1 The spatial transformation T k→k+1 .

[0029] The video frame I k With video frame I N Align (i.e. motion compensation), and transform the feature map F k With the feature map F N Alignment (i.e. motion compensation) is performed so that from a global perspective, the content information described at the same position in all images is corresponding. Calculation is performed to align to the target frame I k A set of perspective images:

[0030] I′1=T N-1→N (T N-2→N-1 (…T 1→2 (I1))),

[0031] …,

[0032] I′ k =T N-1→N (T N-2→N-1 (…T k→k+1 (I k ))),

[0033] …,

[0034] I′ N-1 =T N-1→N (I N-1 ).

[0035] Calculate the registration to the target frame I N A set of feature maps:

[0036] F′1=T N-1→N (T N-2→N-1 (…T 1→2 (F1))),

[0037] …,

[0038] F′ k =T N-1→N (T N-2→N-1 (…T k→k+1 (F k ))),

[0039] …,

[0040] F′ N-1 =T N-1→N (F N-1 ).

[0041] here, Figure 3 is the process of image registration. Figure 3 In the figure, the left and right images are the 105th and 106th video frames in a perspective video, respectively. The figure shows the matched feature points extracted from these two images. The circles are the positions of the feature points, and the horizontal line segments connect a pair of feature points. It can be seen that the position offsets of different matching point pairs in the left and right images are consistent, so the spatial transformation calculated based on these point pairs is effective.

[0042] Step 104: Calculate I′ j with I′ N The first distance map between Calculate F′ j and F′ N The second distance map between According to the first distance map and the second distance map Calculate the weight graph w j , Wherein, j=1, 2, ..., N, σ1 and σ2 are both constants, x is any coordinate in the pixel coordinate system C1, I′ j (x), I′ N (x), F′ j (x) and F′ N (x) are the coordinates x in the first distance map Second distance map Video frame I′ j , video frame I′ N , feature map F′ j and feature map F′ N The pixel value of the corresponding pixel in , abs() is the absolute value function.

[0043] Step 105: Video Frame I N The corresponding denoised frame is in, and I′ t (x) are the coordinates x in the denoised frame and video frame I′ t The pixel value of the corresponding pixel in .

[0044] Calculate video frame I j ′ and reference frame I′ N The first and second distance maps between the two are calculated based on the pixel value difference at the same coordinates for each video frame I j ′ and reference frame I′ N The first distance map Where j = 1, 2, ..., N, indicating that when the video frame I j ′ and reference frame I′ N The difference between the pixel values ​​at the same coordinate is greater, indicating greater content differences. Each feature map F′ is calculated based on the degree of feature consistency at the same coordinate. j With the reference feature map F′ N The second distance map When the pixel values ​​at the same position of the feature map are all 0, it indicates that they all belong to the non-guidewire catheter area, and when they are all 1, it indicates that they all belong to the guidewire catheter area. The greater the difference in pixel values, the smaller the probability that they belong to the same area.

[0045] The weight image w for each image relative to the reference frame j , the weight image is obtained by weighting the first distance map and the second distance map. Parameters σ1 and σ2 are the standard deviations acting on the first distance map and the second distance map respectively. The larger the standard deviation, the smaller the weight difference, and the less emphasis is placed on the distance difference.

[0046] Figure 4The denoising effect diagrams without and with feature information are shown in Figure 4 In the figure, the upper and lower rows show the renderings of the 44th and 45th video frames respectively. By observing the perspective image without noise reduction, it is found that the middle horizontal guide wire moves from bottom to top. In general, the noise reduction images without and with feature information have a certain noise reduction effect, the noise of anatomical structures such as the spine is reduced, and the contrast is improved. In the noise reduction effect image without feature information, the horizontal guide wire of the 45th video frame is blurred. At the same time, two very faint shadows of the guide wire are observed from the image, one of which is the residual image of the guide wire in the 44th video frame. In the noise reduction image using feature information, the existence of the horizontal guide wire can still be clearly observed. This is because the positions of the horizontal guide wires in the 44th and 45th video frames are different, and the feature responses of the pixels where the guide wires are located in different frames are greatly different. The calculated second distance map becomes larger, which reduces the fusion weight of the corresponding pixels in the weighted average, so that the guide wire of the 45th video frame is retained in the image and the residual image of the guide wire of the 44th video frame is not introduced.

[0047] Figure 5 The denoising effect diagram is shown without and with motion compensation. Figure 5 In the figure, the upper and lower rows show the renderings of the 21st and 22nd video frames respectively. Observing the perspective image without noise reduction, the imaging content of the 22nd video frame is translated to the left by a distance relative to the 21st video frame. The middle column shows the rendering without motion compensation. It can be seen that due to the misalignment of the information at the same position of the previous and next frames, a large range of blurring occurs in the noise reduction image. After using motion compensation, the blurring disappears.

[0048] In this embodiment, the “obtaining video frame I j The corresponding feature map F j "Specifically include: Based on the U-Net neural network, obtain the video frame I j The corresponding feature map F j When training the U-Net neural network, obtain several video frames I" in the perspective video obtained by X-ray fluoroscopy, and the feature annotation map F corresponding to each video frame I" gt , feature annotation graph F gt is a binary image, feature annotation image F gt The length and width of the video frame I' are equal to the length and width of the feature annotation figure F gt and video frame I' have the same pixel coordinate system C2; feature annotation map F gt Used to characterize: When the pixel F gt When the pixel value of (y) = 1, the pixel I" (y) belongs to the guidewire image or catheter image. gtWhen the pixel value of (y) = 0, the pixel I" (y) does not belong to the guidewire image, nor does it belong to the catheter image, where y is any coordinate in the pixel coordinate system C2, and F gt (y) and I”(y) are the coordinates of y in the feature annotation map F gt and the corresponding pixels in the video frame I”; the feature annotation map F corresponding to each video frame I” gt The U-Net neural network is input to train the U-Net neural network. Here, each video frame I' can be manually annotated, so that its corresponding feature annotation map F can be obtained. gt .

[0049] In this embodiment, the “obtaining video frame I k To video frame I k+1 The spatial transformation T k→k+1 "Specifically includes: based on the feature point method, obtaining the video frame I k To video frame I k+1 The spatial transformation T k→k+1 .

[0050] In this embodiment, the “obtaining video frame I k To video frame I k+1 The spatial transformation T k→k+1 Specifically include:

[0051] Step 1: Based on the ORB (Oriented Fast and Rotated Brief) algorithm, extract the image from the video frame I k Get multiple FAST feature pixels to form a feature point set P k , from video frame I k+1 Get multiple FAST feature pixels to form a feature point set P k+1 Here, the steps can be: (1) Select a pixel P from the image and first set its brightness value to I P ; (2) Set a suitable threshold t. (3) Consider a discretized Bresenham circle with a radius of 3 pixels centered on the pixel P. There are 16 pixels on the boundary of this circle;

[0052] (4) Now, if there are n consecutive pixels on this circle of size 16 pixels, their pixel values ​​are either greater than I P +t is larger than I P -t is small, then the pixel P is a FAST feature pixel.

[0053] Step 2: Get the feature point set P kThe BRIEF descriptor of each feature point in the feature point set P is obtained k+1 BRIEF descriptor for each feature point in; here, for the feature point set P k and P k+1 Calculate the BRIEF descriptor of the feature point. The BRIEF descriptor is a binary descriptor (optional, a 128-bit binary string, etc.). It is calculated by randomly selecting 128 point pairs around the feature point. For each pair of points, if the pixel value of the previous point is greater than that of the next point, then take 1, otherwise take 0.

[0054] Step 3: Set the feature point set P k Each element E k With the feature point set P k+1 The element E in k+1 Pairing obtains multiple paired points, and the distance between each point pair is F(E k ,E k+1 ), and satisfy: F(E k ,E k+1 )=min(F(E k , E)|E∈feature point set P k+1 ), F(x1, x2) is the Hamming distance between the BRIEF descriptor of x1 and the BRIEF descriptor of x2; a preset ratio of point pairs with the shortest distance is selected from multiple point pairs to form multiple target point pairs, and the spatial transformation T is calculated according to the positional relationship between the two paired elements in each target point. k→k+1 , where 0<preset ratio≤1.

[0055] In this embodiment, the preset ratio=90%.

[0056] In this embodiment, the feature point set P k And the feature point set P k+1 Each contains 5000 FAST feature pixels.

[0057] In this embodiment, σ1=0.02, σ2=0.25.

[0058] In this embodiment, the “obtaining N video frames I1, I2, ..., I3 arranged in time sequence from the perspective video obtained by X-ray fluoroscopy” N Specifically, it includes: obtaining N video frames I1, I2, ..., I1 arranged in time sequence from the perspective video obtained by X-ray fluorescence perspective. N , for video frames I1,I2,…,I N The edge-preserving spatial noise reduction process is performed to obtain the denoised image.

[0059] Embodiment 2 of the present invention provides a noise reduction device for X-ray fluoroscopic video, comprising the following modules:

[0060] The video acquisition module is used to acquire N video frames I1, I2, ..., I1 arranged in time sequence from the perspective video obtained by X-ray fluorescence perspective. N , video frames I1,I2,…,I N The lengths and widths of the video frames I1, I2, …, I N Have the same pixel coordinate system C1, N is a natural number, N ≥ 2;

[0061] Feature acquisition module, used to obtain video frame I j The corresponding feature map F j , feature map F j The length and width of the video frame I j The length and width of the feature map F j Set the pixel coordinate system C1; in the feature map F j The pixel value Value of each pixel DF in the video frame I is used to represent: j , the probability value of the pixel DI at the same coordinate as the pixel DF belonging to the guidewire image or the catheter image, 0≤Value≤1, j=1,2,...,N;

[0062] Registration module, used to obtain video frame I k To video frame I k+1 The spatial transformation T k→k+1 , get video frame I k Align to I N The video frame I′ k , get the feature map F k Align to F N The feature map F′ k , video frame I′ k The length and width are equal to I N The length and width of the feature map F′ k The length and width are equal to F N The length and width of the video frame I′ k and feature map F′ k Set the pixel coordinate system C1, k = 1, 2, ..., N-1, let F′ N =F N , I′ N =I N ;

[0063] Denoising module, used to calculate I′ j with I′ N The first distance map between Calculate F′ j and F′ N The second distance map between According to the first distance map and the second distance map Calculate the weight graph w j , Wherein, j=1, 2, ..., N, σ1 and σ2 are both constants, x is any coordinate in the pixel coordinate system C1, I′ j (x), I′ N (x), F′ j (x) and F′ N (x) are the coordinates x in the first distance map Second distance map Video frame I′ j , video frame I′ N , feature map F′ j and feature map F′ N The pixel value of the corresponding pixel in , abs() is the absolute value function;

[0064] Processing module for video frame I N The corresponding denoised frame is in, and I′ t (x) are the coordinates x in the denoised frame and video frame I′ t The pixel value of the corresponding pixel in .

[0065] Embodiment 3 of the present invention provides a terminal, comprising: a memory for storing a computer program; and a processor for implementing the steps of the denoising method in Embodiment 1 when executing the computer program.

[0066] Embodiment 4 of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the denoising method in embodiment 1 are implemented.

[0067] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0068] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for noise reduction of fluoroscopic video for X-ray fluoroscopy, It is characterized in that The following steps are involved: From the perspective video obtained by X-ray fluoroscopy, N video frames I are obtained in time sequence. 1 ,I 2 ,…,I N , video frame I 1 ,I 2 ,…,I N The lengths and widths of the video frames I 1 ,I 2 ,…,I N Have the same pixel coordinate system C1, N is a natural number, N ≥ 2; Get video frame I j The corresponding feature map F j , feature map F j The length and width of the video frame I j The length and width of the feature map F j Set the pixel coordinate system C1; in the feature map F j The pixel value Value of each pixel DF in the video frame I is used to represent: j , the probability value of the pixel DI at the same coordinate as the pixel DF belonging to the guidewire image or the catheter image, 0≤Value≤1, j=1,2,...,N; Get video frame I k To video frame I k+1 The spatial transformation T k→k+1 , get video frame I k Align to I N The video frame I′ k , get the feature map F k Align to F N The feature map F′ k , video frame I′ k The length and width are equal to I N The length and width of the feature map F′ k The length and width are equal to F N The length and width of the video frame I′ k and feature map F′ k Set the pixel coordinate system C1, k = 1, 2, ..., N-1, let F′ N =F N , I′ N =I N ; Calculate I′ j with I′ N The first distance map between Calculate F′ j and F′ N The second distance map between According to the first distance map and the second distance map Calculate the weight graph w j , Where j = 1, 2, ..., N, σ 1 and σ 2 are all constants, x is any coordinate in the pixel coordinate system C1, I′ j (x), I′ N (x), F′ j (x) and F′ N (x) are the coordinates x in the first distance map Second distance map Video frame I′ j , video frame I′ N , feature map F′ j and feature map F′ N The pixel value of the corresponding pixel in , abs() is the absolute value function; Video Frame I N The corresponding denoised frame is in, and I′ t (x) are the coordinates x in the denoised frame and video frame I′ t The pixel value of the corresponding pixel in .

2. The noise reduction method according to claim 1, It is characterized in that The "obtaining video frame I j The corresponding feature map F j Specifically include: Based on U-Net neural network, get video frame I j The corresponding feature map F j ; When training the U-Net neural network, a plurality of video frames I" in the perspective video obtained by X-ray fluorescence perspective and a feature annotation map F corresponding to each video frame I" are obtained. gt , feature annotation graph F gt is a binary image, feature annotation image F gt The length and width of the video frame I' are equal to the length and width of the feature annotation figure F gt and video frame I' have the same pixel coordinate system C2; feature annotation map F gt Used to characterize: When the pixel F gt When the pixel value of (y) = 1, the pixel I" (y) belongs to the guidewire image or catheter image. gt When the pixel value of (y) = 0, the pixel I" (y) does not belong to the guidewire image, nor does it belong to the catheter image, where y is any coordinate in the pixel coordinate system C2, and F gt (y) and I”(y) are the coordinates of y in the feature annotation map F gt and the corresponding pixels in the video frame I”; the feature annotation graph F corresponding to each video frame I” gt The input is into the U-Net neural network to train the U-Net neural network.

3. The noise reduction method according to claim 1, It is characterized in that The "obtaining video frame I k To video frame I k+1 The spatial transformation T k→k+1 Specifically include: Based on the feature point method, obtain the video frame I k To video frame I k+1 The spatial transformation T k→k+1 .

4. The noise reduction method according to claim 1, It is characterized in that The "obtaining video frame I k To video frame I k+1 The spatial transformation T k→k+1 Specifically include: Based on the ORB algorithm, from the video frame I k Get multiple FAST feature pixels to form a feature point set P k , from video frame I k+1 Get multiple FAST feature pixels to form a feature point set P k+1 ; Get feature point set P k The BRIEF descriptor of each feature point in the feature point set P is obtained k+1 BRIEF descriptor for each feature point in; The feature point set P k Each element E k With the feature point set P k+1 The element E in k+1 Pairing obtains multiple paired points, and the distance between each point pair is F(E k ,E k+1 ), and satisfy: F(E k ,E k+1 )=min(F(E k , E)|E∈feature point set P k+1 ), F(x1, x2) is the Hamming distance between the BRIEF descriptor of x1 and the BRIEF descriptor of x2; a preset ratio of point pairs with the shortest distance is selected from multiple point pairs to form multiple target point pairs, and the spatial transformation T is calculated according to the positional relationship between the two paired elements in each target point. k→k+1 , where 0<preset ratio≤1.

5. The noise reduction method according to claim 4, Features: Default ratio = 90%.

6. The noise reduction method according to claim 4, Features: The feature point set P k And the feature point set P k+1 Each contains 5000 FAST feature pixels.

7. The noise reduction method according to claim 1, Features: s 1 =0.02,σ 2 = 0.

25.

8. The noise reduction method according to claim 1, It is characterized in that The method of "obtaining N video frames arranged in time sequence from the perspective video obtained by X-ray fluoroscopy" 1 ,I 2 ,…,I N Specifically include: From the perspective video obtained by X-ray fluoroscopy, N video frames I are obtained in time sequence. 1 ,I 2 ,…,I N , for video frame I 1 ,I 2 ,…,I N Edge-preserving spatial noise reduction is performed on both.

9. A noise reduction device for fluoroscopic video of X-ray fluoroscopy, It is characterized in that Includes the following modules: The video acquisition module is used to acquire N video frames I arranged in time order from the perspective video obtained by X-ray fluorescence perspective. 1 ,I 2 ,…,I N , video frame I 1 ,I 2 ,…,I N The lengths and widths of the video frames I 1 ,I 2 ,…,I N Have the same pixel coordinate system C1, N is a natural number, N ≥ 2; Feature acquisition module, used to obtain video frame I j The corresponding feature map F j , feature map F j The length and width of the video frame I j The length and width of the feature map F j Set the pixel coordinate system C1; in the feature map F j The pixel value Value of each pixel DF in the video frame I is used to represent: j , the probability value of the pixel DI at the same coordinate as the pixel DF belonging to the guidewire image or the catheter image, 0≤Value≤1, j=1,2,...,N; Registration module, used to obtain video frame I k To video frame I k+1 The spatial transformation T k→k+1 , get video frame I k Align to I N The video frame I′ k , get the feature map F k Align to F N The feature map F′ k , video frame I′ k The length and width are equal to I N The length and width of the feature map F′ k The length and width are equal to F N The length and width of the video frame I′ k and feature map F′ k Set the pixel coordinate system C1, k = 1, 2, ..., N-1, let F′ N =F N , I′ N =I N ; Denoising module, used to calculate I′ j with I′ N The first distance map between Calculate F′ j and F′ N The second distance map between According to the first distance map and the second distance map Calculate the weight graph w j , Where j = 1, 2, ..., N, σ 1 and σ 2 are all constants, x is any coordinate in the pixel coordinate system C1, I′ j (x), I′ N (x), F′ j (x) and F′ N (x) are the coordinates x in the first distance map Second distance map Video frame I′ j , video frame I′ N , feature map F′ j and feature map F′ N The pixel value of the corresponding pixel in , abs() is the absolute value function; Processing module for video frame I N The corresponding denoised frame is in, and I′ t (x) are the coordinates x in the denoised frame and video frame I′ t The pixel value of the corresponding pixel in .

10. A terminal, It is characterized in that include: Memory for storing computer programs; A processor, configured to implement the steps of the denoising method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the denoising method according to any one of claims 1 to 8 are implemented.