Spatiotemporal noise reduction methods, devices, electronic equipment, and storage media

Abstract

This invention provides a spatiotemporal combined noise reduction method, apparatus, electronic device, and storage medium. The spatiotemporal combined noise reduction method first uses a real-time perspective image to be superimposed on an image to be superimposed and a current frame image; then, based on the difference between the current frame image and the image to be superimposed, it obtains a first weight for the current frame image and a second weight for the image to be superimposed; next, based on the grayscale levels of the current frame image and the image to be superimposed, it obtains a first relative noise level value for the current frame image and a second relative noise level value for the image to be superimposed; then, based on the first relative noise level value, the second relative noise level value, the first weight, the second weight, and a preset target noise level, it obtains a compensation noise level; finally, based on the compensation noise level and a preset spatial domain noise reduction strategy, it obtains spatial domain noise reduction parameters. This invention enables the noise level of real-time perspective images to become more consistent, improving image quality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a spatiotemporal noise reduction method, apparatus, electronic device, and storage medium. Background Technology

[0002] X-ray imaging is one of the most widely used techniques in medical imaging examinations. For real-time fluoroscopic images (i.e., X-ray images), multi-frame image stacking (temporal domain noise reduction) is commonly used to reduce image noise. However, when the subject being imaged (e.g., various voluntary and involuntary movements of the subject) or the imaging equipment itself moves, multi-frame stacking can introduce significant motion artifacts, while insufficient or no stacking can result in substantial noise in the image. Therefore, existing techniques often use different stacking weights in different regions of the image based on the motion to reduce motion artifacts. However, different stacking weights often lead to variations in noise levels across different regions of an image, requiring further improvement in image quality.

[0003] It should be noted that the information disclosed in the background section of this invention is intended only to enhance the understanding of the general background of this invention, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to address the problem in the prior art where the noise level varies in different regions of the same image when denoising real-time perspective images. This invention provides a spatiotemporal combined noise reduction method, apparatus, electronic device, and storage medium to make the noise level of real-time perspective images more consistent and improve image quality.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a spatiotemporal noise reduction method, comprising:

[0006] Obtain the image to be overlaid and the current frame image from the real-time perspective image;

[0007] Based on the difference between the current frame image and the image to be superimposed, obtain the first weight of the current frame image and the second weight of the image to be superimposed;

[0008] Based on the grayscale levels of the current frame image and the image to be superimposed, obtain the first relative value of the noise level of the current frame image and the second relative value of the noise level of the image to be superimposed;

[0009] Based on the first relative noise level value, the second relative noise level value, the first weight, and the second weight, the predicted noise level of the current frame image and the image to be superimposed is obtained; and based on the predicted noise level and the preset target noise level, the compensation noise level is obtained.

[0010] Based on the compensation noise level and the preset spatial noise reduction strategy, spatial noise reduction parameters are obtained.

[0011] Optionally, the spatiotemporal combined noise reduction method further includes:

[0012] Motion compensation is performed on the image to be superimposed, and the motion-compensated image is used as the superimposed image.

[0013] Optionally, the spatiotemporal combined noise reduction method includes:

[0014] Based on the difference between the pixel value of each pixel in the current frame image and the pixel value of the corresponding pixel in the image to be superimposed, a first weight of each pixel in the current frame image is calculated; and based on the first weight, a second weight of each pixel in the image to be superimposed is calculated.

[0015] or

[0016] Based on the image similarity between the current frame image and the image to be superimposed, a first weight of the current frame image is determined; and based on the first weight, a second weight of the image to be superimposed is determined.

[0017] Optionally, the spatiotemporal combined noise reduction method further includes normalizing the first weight to a value between 0 and 1; and calculating the second weight according to the following formula:

[0018] wa = 1 - wb

[0019] In the formula, wb is the first weight and wa is the second weight;

[0020] Obtain the grayscale regions in the current frame image where the image grayscale is less than a preset grayscale threshold; and reduce the first weight corresponding to the grayscale region based on the image grayscale of the grayscale region.

[0021] Optionally, the spatiotemporal noise reduction method further includes: obtaining a motion region in the current frame image whose difference from the image to be superimposed is greater than a preset difference threshold; and increasing the first weight corresponding to the motion region based on the difference between the motion region in the current frame image and the corresponding region in the image to be superimposed.

[0022] Optionally, the spatiotemporal combined noise reduction method includes:

[0023] Based on the grayscale levels of the current frame image and the image to be superimposed, a noise model is used to obtain the first relative value of the noise level of the current frame image and the second relative value of the noise level of the image to be superimposed.

[0024] Optionally, the compensation noise level is calculated using the following formula:

[0025] nt = ndst - nc

[0026] In the formula, nt is the compensated noise level, ndst is the preset target noise level, and nc is the predicted noise level.

[0027] To achieve the above objectives, the present invention also provides a spatiotemporal noise reduction device, the spatiotemporal noise reduction device comprising:

[0028] The image frame acquisition unit is configured to acquire the image to be overlaid in the real-time perspective image and the current frame image, and to acquire the image based on the current frame image;

[0029] The image weight acquisition unit is configured to acquire a first weight of the current frame image and a second weight of the image to be superimposed based on the difference between the current frame image and the image to be superimposed.

[0030] The noise level relative value acquisition unit is configured to acquire a first noise level relative value of the current frame image and a second noise level relative value of the image to be superimposed based on the grayscale levels of the current frame image and the image to be superimposed.

[0031] The noise compensation level acquisition unit is configured to acquire the predicted noise level of the current frame image and the image to be superimposed after superposition based on the first relative noise level value, the second relative noise level value, the first weight, and the second weight; and to acquire the noise compensation level based on the predicted noise level and the preset target noise level.

[0032] The airspace noise reduction parameter acquisition unit is configured to acquire airspace noise reduction parameters based on the compensation noise level and the preset airspace noise reduction strategy.

[0033] To achieve the above objectives, the present invention also provides an electronic device, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the spatiotemporal combination noise reduction method described above.

[0034] To achieve the above objectives, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the spatiotemporal denoising method described in any of the above claims.

[0035] Compared with existing technologies, the spatiotemporal noise reduction method, apparatus, electronic device, and storage medium provided by this invention have the following advantages:

[0036] The spatiotemporal denoising method provided by this invention first acquires the image to be superimposed and the current frame image of a real-time perspective image; then, based on the difference between the current frame image and the image to be superimposed, it acquires a first weight of the current frame image and a second weight of the image to be superimposed; next, based on the grayscale levels of the current frame image and the image to be superimposed, it acquires a first relative noise level of the current frame image and a second relative noise level of the image to be superimposed; then, based on the first relative noise level, the second relative noise level, the first weight, and the second weight, it acquires the predicted noise level after superimposing the current frame image and the image to be superimposed; and based on the predicted noise level and a preset target noise level, it acquires a compensated noise level; finally, based on the compensated noise level and a preset spatial denoising strategy, it acquires spatial denoising parameters. Therefore, the spatiotemporal denoising method provided by this invention estimates the required spatial denoising intensity based on the noise levels of different regions after temporal denoising, thereby compensating for the differences in noise levels caused by different weights in different regions of temporal denoising, enabling the noise level of the real-time perspective image to become more consistent and improving image quality.

[0037] Furthermore, the spatiotemporal combined denoising method provided by the present invention further includes reducing the first weight corresponding to the grayscale region based on the image grayscale of the grayscale region, and / or increasing the first weight corresponding to the motion region based on the difference between the motion region in the current frame image and the corresponding region of the image to be superimposed. Therefore, the spatiotemporal combined denoising method provided by the present invention, by further weighting the superimposition weights according to grayscale and motion conditions, can reduce motion blur, improve image contrast, and thus further enhance image quality.

[0038] Since the spatiotemporal noise reduction combined device, electronic device and storage medium provided by the present invention belong to the same inventive concept as the spatiotemporal noise reduction combined method provided by the present invention, the spatiotemporal noise reduction combined device, electronic device and storage medium provided by the present invention have at least all the advantages of the spatiotemporal noise reduction combined method provided by the present invention. For more detailed content, please refer to the description of the spatiotemporal noise reduction combined method above, which will not be repeated here. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the overall process of the spatiotemporal noise reduction method provided in Embodiment 1 of the present invention;

[0040] Figure 2 This is a specific example diagram illustrating the calculation of a first weight based on image similarity in one embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the relationship curve (reduction function) between image grayscale and first weight in a spatiotemporal combined noise reduction method provided in one embodiment of the present invention.

[0042] Figure 4 This is a schematic diagram of the relationship curve (increasing function) between image difference and first weight in a spatiotemporal combined noise reduction method provided in one embodiment of the present invention.

[0043] Figure 5 This is a schematic diagram of the spatiotemporal noise reduction device provided in Embodiment 2 of the present invention;

[0044] Figure 6 This is a block diagram of the electronic device provided in Embodiment 3 of the present invention.

[0045] The accompanying figure is labeled as follows:

[0046] Image frame acquisition unit-110, image weight acquisition unit-120, noise level relative value acquisition unit-130, compensation noise level acquisition unit-140, spatial domain noise reduction parameter acquisition unit-150;

[0047] Processor-210, communication interface-220, memory-230, communication bus-240. Detailed Implementation

[0048] The spatiotemporal noise reduction method, apparatus, electronic device, and storage medium proposed in this invention will be further described in detail below with reference to the accompanying drawings. The advantages and features of this invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the embodiments of this invention. For the purposes, features, and advantages of this invention to be more apparent and understandable, please refer to the accompanying drawings. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the implementation conditions of this invention. Any modifications to the structure, changes in proportions, or adjustments to the size, provided that the effects and purposes achieved by this invention are the same or similar, should still fall within the scope of the technical content disclosed in this invention. Specific design features of the invention disclosed herein, including, for example, specific dimensions, orientations, positions, and shapes, will be determined in part by the specific application and usage environment. Furthermore, in the embodiments described below, the same reference numerals are sometimes used across different drawings to denote the same parts or parts having the same function, and repeated descriptions are omitted. In this specification, similar reference numerals and letters are used to denote similar items; therefore, once an item is defined in one figure, it need not be discussed further in subsequent figures. Furthermore, if the methods described herein involve a series of steps, and the order of these steps presented herein is not necessarily the only possible order in which they can be performed, some of the described steps may be omitted and / or other steps not described herein may be added to the method.

[0049] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The singular forms “a,” “an,” and “the” include plural objects. The term “or” is generally used to mean “and / or,” the term “several” is generally used to mean “at least one,” and the term “at least two” is generally used to mean “two or more.” Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0050] It should be noted that the spatiotemporal noise reduction method provided by the present invention can be applied to the electronic device provided by the present invention. The electronic device can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a mobile phone, a tablet computer, or other hardware device with various operating systems.

[0051] Example 1

[0052] This embodiment provides a spatiotemporal combined noise reduction method. For details, please refer to... Figure 1 The diagram illustrates the overall process of the spatiotemporal noise reduction method provided by an embodiment of the present invention. Figure 1 As can be seen, the spatiotemporal noise reduction method provided in this embodiment includes:

[0053] S100: Obtain the image to be overlaid and the current frame image of the real-time perspective image;

[0054] S200: Based on the difference between the current frame image and the image to be superimposed, obtain the first weight of the current frame image and the second weight of the image to be superimposed;

[0055] S300: Based on the grayscale levels of the current frame image and the image to be superimposed, obtain the first relative value of the noise level of the current frame image and the second relative value of the noise level of the image to be superimposed;

[0056] S400: Based on the first relative noise level value, the second relative noise level value, the first weight, and the second weight, obtain the predicted noise level after superimposing the current frame image and the image to be superimposed; and based on the predicted noise level and the preset target noise level, obtain the compensation noise level;

[0057] S500: Obtain spatial noise reduction parameters based on the compensation noise level and the preset spatial noise reduction strategy.

[0058] Therefore, the spatiotemporal denoising method provided in this embodiment estimates the required spatial denoising intensity based on the noise level of different regions after temporal denoising, thereby compensating for the differences in noise levels caused by different weights in different regions of temporal denoising. This enables the noise level of real-time perspective images to become more consistent and improves image quality.

[0059] Specifically, in step S100, the image to be superimposed and the current frame image are images acquired based on a single scan of the same subject (e.g., a patient), such as images acquired sequentially and superimposed images. In some embodiments, the current frame image and the image to be superimposed can be images acquired continuously or with very short intervals, such as two consecutive frames, two frames separated by one or two frames, etc. In some embodiments, the current frame image may include the latest acquired frame image, and the image to be superimposed may include the frame image preceding the current frame. In some embodiments, the current frame image may include the latest acquired frame image, and the image to be superimposed may include a superimposed image obtained by superimposing multiple acquired frames preceding the current frame. In some embodiments, the current frame image and the image to be superimposed can be acquired by scanning the subject using medical imaging equipment. In some embodiments, the current frame image and the image to be superimposed can be acquired by storage devices or other means, which is not limited by the present invention. Furthermore, in some embodiments, the current frame image and the image to be superimposed may include X-ray images, such as DSA images. In some embodiments, the current frame image and the image to be superimposed may reflect the vascular information of the subject. It is understood that the above are merely illustrative examples and not limitations of the present invention.

[0060] To facilitate understanding and explanation of the present invention, the spatiotemporal denoising method provided by the present invention will be illustrated using the example of obtaining a real-time perspective image through registration and denoising of two frames. Specifically, image A represents the image to be superimposed, and image B represents the current frame image.

[0061] As one preferred embodiment, the spatiotemporal noise reduction method further includes:

[0062] Motion compensation is performed on the image to be superimposed, and the motion-compensated image is used as the superimposed image. With this configuration, the spatiotemporal denoising method provided in this embodiment can further reduce motion artifacts and improve image quality by using the motion-compensated image to be superimposed. Between step S100 and step S200, the current frame image is still image B, and the image to be superimposed is obtained by registration and transformation from image A to the motion-compensated image A'.

[0063] It should be noted that this invention does not limit the specific motion compensation method. When implementing this invention, the motion compensation methods that can be used include, but are not limited to, methods based on deep learning / machine learning, optical flow methods, and block matching methods. For more detailed information on motion compensation methods, please refer to the prior art known to those skilled in the art.

[0064] In some preferred embodiments, step S200, based on the difference between the current frame image and the image to be superimposed, obtains a first weight of the current frame image and a second weight of the image to be superimposed, specifically including:

[0065] Based on the difference between the pixel value of each pixel in the current frame image and the pixel value of the corresponding pixel in the image to be overlaid, a first weight is calculated for each pixel in the current frame image; and based on the first weight, a second weight is calculated for each pixel in the image to be overlaid. Therefore, the spatiotemporal noise reduction method provided by this embodiment, by considering the difference between the pixel value of each pixel in the current frame image and the pixel value of the corresponding pixel in the image to be overlaid—that is, the greater the difference between the pixel value of each pixel in the current frame image and the pixel value of the corresponding pixel in the image to be overlaid—results in a larger first weight for the current pixel in the current frame image, thereby further reducing motion artifacts caused by the overlay of the current frame image and the image to be overlaid.

[0066] In some other preferred embodiments, step S200, based on the difference between the current frame image and the image to be superimposed, obtains a first weight of the current frame image and a second weight of the image to be superimposed, specifically including:

[0067] Based on the image similarity between the current frame image and the image to be superimposed, a first weight of the current frame image is determined; and based on the first weight, a second weight of the image to be superimposed is determined. Thus, the spatiotemporal denoising method provided by this embodiment can balance denoising effect and denoising processing efficiency. Specifically, the image similarity includes, but is not limited to, the structural similarity between the current frame image and the image to be superimposed. For example, the current frame image and the image to be superimposed can be divided into several sub-image blocks (e.g., 100*100), and then based on the difference between each sub-image block of the current frame image and each corresponding sub-image block of the image to be superimposed, a first weight of each sub-image block of the current frame image is calculated.

[0068] Specifically, please see Figure 2 The diagram illustrates a specific example of calculating the first weight based on image similarity in one embodiment of this work. Figure 2 As can be seen, in this embodiment, image B is divided into four sub-image blocks B11, B12, B21, and B22. Correspondingly, image A' is divided into four sub-image blocks A'11, A'12, A'21, and A'22. More specifically, sub-image block B11 corresponds to sub-image block A'11. Based on the difference between sub-image block B11 and sub-image block A'11, a first weight of sub-image block B11 is calculated. Then, based on the first weight of sub-image block B11, a second weight of sub-image block A'11 is calculated. This process is repeated to obtain the first weights of sub-image blocks B12, B21, and B22, and the second weights of their corresponding sub-image blocks A'12, A'21, and A'22. Therefore, based on the first weights of each sub-image block B11, B12, B21 and B22 of image B, the first weight of image B can be obtained, and based on the second weights of each sub-image block A'11, A'12, A'21 and A'22 of image A', the second weight of image A' can be obtained.

[0069] It should be noted that the first weight / second weight corresponding to each pixel or sub-image block can be the same or different, and this invention does not impose any limitations on this.

[0070] In one preferred embodiment, step S200 further includes normalizing the first weight to a value between 0 and 1; and calculating the second weight according to the following formula:

[0071] wa = 1 - wb

[0072] In the formula, wb is the first weight and wa is the second weight.

[0073] Therefore, by normalizing the first weight of the current frame image to between 0 and 1, it is easier to calculate the second weight of the image to be superimposed.

[0074] Thus, the first weight wb of image B is obtained, and correspondingly, the second weight wa of image A' is obtained.

[0075] In one preferred embodiment, before step S300, the spatiotemporal combined noise reduction method further includes: acquiring grayscale regions in the current frame image whose image grayscale is less than a preset grayscale threshold; and reducing the first weight corresponding to the grayscale region based on the image grayscale of the grayscale region. Therefore, the spatiotemporal combined noise reduction method provided in this embodiment, by acquiring grayscale regions in the current frame image whose image grayscale is less than a preset grayscale threshold and reducing the first weight corresponding to the grayscale region, can further reduce motion blur, improve image contrast, and thus improve the quality of real-time perspective images.

[0076] Specifically, in real-time fluoroscopic images (such as those obtained by DSA equipment), areas with lower grayscale levels are often the areas where doctors focus more on interventional instruments. Therefore, it is necessary to reduce motion blur in these areas to improve image contrast. Assuming that the function F,q is a decreasing function of image grayscale, the first weight corresponding to image B can be adjusted by the following formula:

[0077] wb=Fg(B)*wb′

[0078] In the formula, wb′ is the first weight of the adjusted image B, and B is the gray value of the pixel corresponding to the first weight wb′ before adjustment (i.e., the first weight after normalization in step S200). It can be understood that, correspondingly, the second weight wa is calculated using the adjusted first weight wb.

[0079] For example, please see Figure 3 The diagram illustrates the relationship curve (reduction function) between image grayscale and the first weight in one embodiment of the spatiotemporal denoising method provided in this example. Figure 3 It can be seen that the larger the gray value of the pixel in image B, the smaller the value of Fg(B), and the smaller the adjusted first weight value of that pixel.

[0080] It should be noted that the present invention does not limit the specific value of the preset grayscale threshold. For example, the preset grayscale threshold can be any value in [1000, 2000].

[0081] In one preferred embodiment, before step S300, the spatiotemporal denoising method further includes: acquiring motion regions in the current frame image whose difference from the image to be superimposed is greater than a preset difference threshold; and increasing the first weight corresponding to the motion region based on the difference between the motion region in the current frame image and the corresponding region in the image to be superimposed. Therefore, the spatiotemporal denoising method provided in this embodiment, by acquiring motion regions in the current frame image and increasing the first weight corresponding to the motion regions, can further reduce motion blur, improve image contrast, and thus improve the quality of real-time perspective images.

[0082] Specifically, the motion region is typically the edge region of interventional devices and organ tissues in real-time fluoroscopic images. Therefore, to reduce motion blur and improve contrast in this region, its weight needs to be increased to enhance image contrast. Assuming the function Fm is an increasing function of image differences, or motion probabilities, the first weight corresponding to image B can be adjusted using the following formula:

[0083] wb = Fm(diff(A′,B)) * wb″

[0084] In the formula, wb is the first weight of the adjusted image B, and diff(A′, B) is the difference between the pixels of image A′ and image B corresponding to the first weight wb″ before adjustment using the increasing function. Correspondingly, the second weight wa uses the new first weight wb. Unless otherwise specified below, the first weight wb is the adjusted first weight wb′, and the second weight wa is calculated using the adjusted first weight wb.

[0085] For example, please see Figure 4 The diagram illustrates the relationship curve (increasing function) between image difference and first weight in a spatiotemporal denoising method provided in one embodiment of the present invention. Figure 4 It can be seen that the larger the value of diff(A′, B) (i.e., the difference between image A′ and image B), the larger the corresponding adjusted first weight value.

[0086] It should be noted that, as those skilled in the art will understand, the further adjustment of the first weight with respect to the grayscale region and the motion region is merely an illustrative example and not a limitation of the present invention. Only one of them may be performed, or both may be performed. When both are performed, the order of execution is not limited, but the later execution is based on the result of the earlier execution.

[0087] In one preferred embodiment, step S300 specifically includes: obtaining a first relative noise level of the current frame image and a second relative noise level of the image to be superimposed based on the grayscale levels of the current frame image and the image to be superimposed using a noise model. For details regarding the use of a noise model to obtain the first relative noise level of the current frame image and the second relative noise level of the image to be superimposed, please refer to existing technologies known to those skilled in the art; further explanation is omitted here due to space limitations. However, it should be noted that the present invention does not limit the noise model, which includes, but is not limited to, Gaussian noise models, Poisson noise distributions, etc.

[0088] In one preferred embodiment, step S400 obtains the predicted noise level of the current frame image and the image to be superimposed based on the first relative noise level value, the second relative noise level value, the first weight, and the second weight. Specifically, this includes calculating the predicted noise level using the following formula:

[0089] nc=sqrt((wa*wa*na*na+wb*wb*nb*nb))

[0090] In the formula, wa is the value of the second weight, na is the relative value of the second noise level, wb is the value of the first weight, and nb is the relative value of the first noise level.

[0091] In one preferred embodiment, the compensation noise level is calculated using the following formula:

[0092] nt = ndst - nc

[0093] In the formula, nt is the compensated noise level, ndst is the preset target noise level, and nc is the predicted noise level.

[0094] Therefore, the spatiotemporal denoising method provided in this embodiment estimates the required spatial denoising intensity based on the noise level of different regions after temporal denoising, thereby compensating for the differences in noise levels caused by different weights in different regions of temporal denoising. This enables the noise level of real-time perspective images to become more consistent and improves image quality.

[0095] Furthermore, as those skilled in the art will understand, the current frame image and the image to be superimposed can be registered and superimposed based on the first weight of the current frame image, the second weight of the image to be superimposed, and the compensation noise level to obtain a new image to be superimposed. By analogy, a real-time perspective image with a uniform noise level can be obtained.

[0096] Example 2

[0097] This embodiment provides a spatiotemporal noise reduction device. For details, please refer to [link to relevant documentation]. Figure 5 The diagram illustrates the structure of the spatiotemporal noise reduction device provided in this embodiment. Figure 5 As can be seen, the spatiotemporal noise reduction device provided in this embodiment includes an image frame acquisition unit 110, an image weight acquisition unit 120, a noise level relative value acquisition unit 130, a compensation noise level acquisition unit 140, and a spatial domain noise reduction parameter acquisition unit 150.

[0098] Specifically, the image frame acquisition unit 110 is configured to acquire the image to be superimposed and the current frame image of the real-time perspective image. The image weight acquisition unit 120 is configured to acquire a first weight of the current frame image and a second weight of the image to be superimposed based on the difference between the current frame image and the image to be superimposed. The noise level relative value acquisition unit 130 is configured to acquire a first noise level relative value of the current frame image and a second noise level relative value of the image to be superimposed based on the grayscale levels of the current frame image and the image to be superimposed. The noise compensation level acquisition unit 140 is configured to acquire the predicted noise level of the superimposed image and the image to be superimposed based on the first noise level relative value, the second noise level relative value, the first weight, and the second weight; and acquire the compensation noise level based on the predicted noise level and a preset target noise level. The spatial domain denoising parameter acquisition unit 150 is configured to acquire spatial domain denoising parameters based on the compensation noise level and a preset spatial domain denoising strategy.

[0099] Therefore, the spatiotemporal noise reduction device provided in this embodiment estimates the required spatial noise reduction intensity based on the noise level of different regions after temporal noise reduction, thereby compensating for the differences in noise levels caused by different weights in different regions of temporal noise reduction. This enables the noise level of real-time perspective images to become more consistent and improves image quality.

[0100] Specifically, since the spatiotemporal noise reduction device provided in this embodiment is similar in principle to the spatiotemporal noise reduction method provided in the various embodiments of the above embodiment 1, in order to avoid redundancy, it will not be described in detail here. For more detailed information about the spatiotemporal noise reduction device provided in this embodiment, please refer to the relevant content of the spatiotemporal noise reduction method provided in the above embodiment 1 for an adaptive understanding.

[0101] In some application scenarios, image denoising systems may include processing equipment and medical imaging equipment. The image denoising system can implement the methods and / or processes disclosed in this specification through processing equipment to achieve motion compensation of images acquired by medical imaging equipment, thereby removing motion artifacts in the process of multi-frame superposition, effectively reducing noise in the image, improving image quality, and enhancing diagnostic results.

[0102] Example 3

[0103] This embodiment provides an electronic device; please refer to [reference needed]. Figure 6 The diagram illustrates the block structure of the electronic device provided in this embodiment. Figure 6 As shown, the electronic device provided in this embodiment includes a processor 210 and a memory 230. The memory 230 stores a computer program. When the computer program is executed by the processor 210, it implements the spatiotemporal denoising method described above. Since the electronic device provided in this embodiment belongs to the same inventive concept as the spatiotemporal denoising method provided in the various embodiments of Embodiment 1 above, the electronic device provided in this embodiment has at least all the advantages of the spatiotemporal denoising method provided in the various embodiments of Embodiment 1 above, and will not be elaborated further here.

[0104] like Figure 6 As shown, the electronic device also includes a communication interface 220 and a communication bus 240, wherein the processor 210, the communication interface 220, and the memory 230 communicate with each other via the communication bus 240. The communication bus 240 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 240 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 220 is used for communication between the aforementioned electronic device and other devices.

[0105] The processor 210 referred to in this invention can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 210 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines.

[0106] The memory 230 can be used to store the computer program. The processor 210 implements various functions of the electronic device by running or executing the computer program stored in the memory 230 and calling the data stored in the memory 230.

[0107] The memory 230 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0108] Example 4

[0109] This embodiment provides a readable storage medium storing a computer program. When executed by a processor, the computer program can implement the spatiotemporal denoising method described above. Since the readable storage medium provided in this embodiment belongs to the same inventive concept as the spatiotemporal denoising method in Embodiment 1, the readable storage medium provided in this embodiment possesses at least all the advantages of the spatiotemporal denoising method provided in each embodiment of Embodiment 1. Further details will not be elaborated here.

[0110] The readable storage medium of embodiments of the present invention can be any combination of one or more computer-readable media. The readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer hard disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0111] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0112] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] In summary, compared with the prior art, the spatiotemporal noise reduction method, apparatus, electronic device, and storage medium provided by the present invention have the following advantages:

[0114] Compared with existing technologies, the spatiotemporal noise reduction method, apparatus, electronic device, and storage medium provided by this invention have the following advantages:

[0115] The spatiotemporal denoising method provided by this invention estimates the required spatial denoising intensity based on the noise levels of different regions after temporal denoising, thereby compensating for the differences in noise levels caused by different weights in different regions of temporal denoising. This enables the noise levels of real-time perspective images to become more consistent, improving image quality. Furthermore, the spatiotemporal denoising method also includes reducing the first weight corresponding to the grayscale region based on its image grayscale value, and / or increasing the first weight corresponding to the motion region based on the difference between the motion region in the current frame image and the corresponding region of the image to be superimposed. Therefore, the spatiotemporal denoising method provided by this invention, by further weighting the superimposition weights based on grayscale and motion, can reduce motion blur, improve image contrast, and thus further enhance image quality.

[0116] Since the spatiotemporal noise reduction combined device, electronic device and storage medium provided by the present invention belong to the same inventive concept as the spatiotemporal noise reduction combined method provided by the present invention, the spatiotemporal noise reduction combined device, electronic device and storage medium provided by the present invention have at least all the advantages of the spatiotemporal noise reduction combined method provided by the present invention. For more detailed content, please refer to the description of the spatiotemporal noise reduction combined method above, which will not be repeated here.

[0117] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0118] In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0119] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A spatiotemporal combination noise reduction method, characterized in that, The method comprises: obtaining a to-be-stacked image and a current frame image of a real-time perspective image; obtaining a first weight of the current frame image and a second weight of the to-be-stacked image according to a difference between the current frame image and the to-be-stacked image; obtaining a first noise level relative value of the current frame image and a second noise level relative value of the to-be-stacked image according to a grayscale level of the current frame image and the to-be-stacked image; obtaining a predicted noise level after the current frame image and the to-be-stacked image are stacked by the following formula; and obtaining a compensation noise level according to the predicted noise level and a preset target noise level; ； wherein is a value of the second weight, is the second noise level relative value, is a value of the first weight, is the first noise level relative value; obtaining a spatial domain noise reduction parameter according to the compensation noise level and a preset spatial domain noise reduction strategy.

2. The spatio-temporal combination noise reduction method of claim 1, wherein, The to-be-stacked image further comprises: motion compensation is performed on the to-be-stacked image, and the to-be-stacked image after motion compensation is used as the to-be-stacked image.

3. The spatio-temporal combination noise reduction method of claim 1, wherein, The method comprises: calculating a first weight of each pixel point of the current frame image according to a difference between a pixel value of each pixel point of the current frame image and a pixel value of a pixel point at a corresponding position of the to-be-stacked image; and calculating a second weight of each pixel point of the to-be-stacked image according to the first weight; or determining a first weight of the current frame image according to an image similarity between the current frame image and the to-be-stacked image; and determining a second weight of the to-be-stacked image according to the first weight.

4. The spatio-temporal combination noise reduction method of claim 3, wherein, The method further comprises normalizing the first weight to between 0 and 1; and calculating the second weight according to the following formula: ； wherein is the first weight, is the second weight; obtaining a grayscale region in the current frame image whose image grayscale is less than a preset grayscale threshold; and reducing the first weight corresponding to the grayscale region according to the image grayscale of the grayscale region.

5. The spatio-temporal combination noise reduction method of claim 4, wherein, The method further comprises: obtaining a motion region in the current frame image whose difference with the to-be-stacked image is greater than a preset difference threshold; and increasing the first weight corresponding to the motion region according to a difference between the motion region in the current frame image and a corresponding region of the to-be-stacked image.

6. The spatio-temporal combination noise reduction method of claim 1, wherein, The spatio-temporal combined noise reduction method comprises: obtaining a first noise level relative value of the current frame image and a second noise level relative value of the to-be-stacked image by using a noise model according to a grayscale level of the current frame image and the to-be-stacked image.

7. The spatio-temporal combination noise reduction method of claim 1, wherein, The compensation noise level is calculated by the following formula: ； In the formula, is the compensation noise level, is the preset target noise level, is the predicted noise level.

8. A space-time combined noise reduction device, characterized in that, The method comprises: an image frame obtaining unit configured to obtain a to-be-stacked image and a current frame image of a real-time perspective image; an image weight obtaining unit configured to obtain a first weight of the current frame image and a second weight of the to-be-stacked image according to a difference between the current frame image and the to-be-stacked image; a noise level relative value obtaining unit configured to obtain a first noise level relative value of the current frame image and a second noise level relative value of the to-be-stacked image according to a grayscale level of the current frame image and the to-be-stacked image; a compensation noise level obtaining unit configured to obtain a predicted noise level after the current frame image and the to-be-stacked image are stacked by the following formula; and obtain a compensation noise level according to the predicted noise level and a preset target noise level; ； wherein is a value of the second weight, is the second noise level relative value, is a value of the first weight, is the first noise level relative value; The spatial domain noise reduction parameter obtaining unit is configured to obtain a spatial domain noise reduction parameter according to the compensated noise level and a preset spatial domain noise reduction strategy.

9. An electronic device, comprising: The device comprises a processor and a memory, and the memory stores a computer program which, when executed by the processor, implements the spatio-temporal combination noise reduction method of any one of claims 1 to 7.

10. A readable storage medium, characterized by, The readable storage medium stores a computer program which, when executed by a processor, implements the spatio-temporal combination noise reduction method of any one of claims 1 to 7.

Images