Image freezing method, image processing unit and endoscope system

Abstract

The embodiment of the present application provides an image freezing method, an image processing unit and an endoscope system, which can be used in the field of image processing technology. The method includes: in response to receiving an image freezing instruction, acquiring a first reference frame image and multiple first related frame images; performing motion estimation based on the first reference frame image and multiple first related frame images to determine the motion amount information corresponding to each first related frame image; the motion amount information is used to represent the motion amount of the first related frame image relative to the first reference frame image; based on the motion amount information corresponding to each first related frame image, performing image fusion on the first reference frame image and multiple first related frame images to obtain a frozen image. The present application improves the clarity of the frozen image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image freezing method, an image processing unit and an endoscope system. Background Art

[0002] The endoscope system has an image freezing function, which plays an important role in endoscopy. The image freezing function allows the examiner to fix the image acquired by the endoscope during the inspection process, making it easier for the examiner to observe and analyze the details of the image.

[0003] One image freezing method in the related art is to directly freeze the image at the current moment, but the image may be blurred due to the movement of the endoscope or the person being examined; another image freezing method is to select the image with the highest clarity from the images within a period of time before and after the current moment to freeze, but the clarity of multiple adjacent images is often not much different, so the clarity of the final frozen image is not high, which is difficult to meet the needs of medical examinations. Summary of the invention

[0004] The image freezing method, image processing unit and endoscope system provided in the embodiments of the present application are used to improve the clarity of the frozen image.

[0005] In a first aspect, an embodiment of the present application provides an image freezing method, comprising:

[0006] In response to receiving an image freezing instruction, acquiring a first reference frame image and a plurality of first related frame images;

[0007] Performing motion estimation based on the first reference frame image and the plurality of first related frame images to determine motion amount information corresponding to each of the first related frame images; the motion amount information is used to represent the motion amount of the first related frame image relative to the first reference frame image;

[0008] Based on the motion information respectively corresponding to each of the first related frame images, the first reference frame image and a plurality of the first related frame images are fused to obtain a frozen image.

[0009] In a possible implementation manner, performing motion estimation based on the first reference frame image and the plurality of first related frame images to determine motion amount information corresponding to each of the first related frame images includes:

[0010] Performing smoothing processing on the first reference frame image and the plurality of first related frame images respectively to obtain a second reference frame image and a plurality of second related frame images;

[0011] Motion estimation is performed on a plurality of the second related frame images respectively to obtain motion amount information corresponding to each of the first related frame images.

[0012] In a possible implementation manner, the motion amount information corresponding to the first related frame image includes the motion amount value of each pixel in the first related frame image; and performing motion estimation on the plurality of second related frame images respectively to obtain the motion amount information corresponding to each of the first related frame images respectively includes:

[0013] For each of the second related frame images, the following operations are performed respectively, and the following operations include:

[0014] For each pixel in the second related frame image, determine an absolute difference between a value of the pixel and a value of a pixel at the same position in the second reference frame image;

[0015] For each pixel in the second related frame image, the motion value of the pixel is determined based on the absolute difference of the pixel, and the motion value of the pixel is determined as the motion value of the pixel at the same position in the first related frame image.

[0016] In a possible implementation manner, determining the motion value of the pixel point based on the absolute difference value of the pixel point includes:

[0017] The product of the absolute difference of the pixel point and the first preset value is rounded to obtain the motion value of the pixel point.

[0018] In a possible implementation manner, the step of respectively smoothing the first reference frame image and the plurality of first related frame images to obtain a second reference frame image and a plurality of second related frame images includes:

[0019] Performing edge filling on the first reference frame image and the plurality of first related frame images respectively to obtain a third reference frame image and a plurality of third related frame images;

[0020] filtering the third reference frame image and the plurality of third related frame images respectively to obtain a fourth reference frame image and a plurality of fourth related frame images;

[0021] The fourth reference frame image and multiple fourth related frame images are edge cropped respectively to obtain the second reference frame image and multiple second related frame images; the size of the second reference frame image is the same as that of the first reference frame image, and the size of the second related frame image is the same as that of the first related frame image.

[0022] In a possible implementation manner, the performing image fusion on the first reference frame image and the plurality of first related frame images based on the motion information respectively corresponding to each of the first related frame images to obtain a frozen image includes:

[0023] Determine fusion weight information corresponding to each of the first related frame images based on a preset motion amount threshold and motion amount information corresponding to each of the first related frame images;

[0024] Based on the fusion weight information corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image, the first reference frame image and multiple first related frame images are fused to obtain the frozen image.

[0025] In a possible implementation manner, the motion amount information corresponding to the first related frame image includes the motion amount value of each pixel in the first related frame image; the fusion weight information corresponding to the first related frame image includes the fusion weight of each pixel in the first related frame image;

[0026] The determining, based on a preset motion amount threshold and motion amount information corresponding to each of the first related frame images, fusion weight information corresponding to each of the first related frame images respectively includes:

[0027] For each of the first related frame images, the following operations are performed respectively, and the following operations include:

[0028] For each pixel in the first related frame image, the difference between the preset motion threshold and the motion value corresponding to the pixel is determined, and the maximum value between the second preset value and the difference is determined as the fusion weight of the pixel.

[0029] In a possible implementation, the performing image fusion on the first reference frame image and the plurality of first related frame images based on the fusion weight information respectively corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image to obtain the frozen image includes:

[0030] The first reference frame image and the plurality of first related frame images are respectively converted from a YUV format to an RGB format to obtain a fifth reference frame image and a plurality of fifth related frame images; the fifth reference frame image and the fifth related frame image each include image data of three channels, and the image data include component values ​​of each pixel point in the image on the channels;

[0031] Based on the fusion weight information corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image, the fifth reference frame image and multiple fifth related frame images are fused to obtain the frozen image.

[0032] In a possible implementation, the frozen image includes fused image data of three channels, and the fused image data includes fused component values ​​of each pixel in the frozen image on the channels; the fusion weight information corresponding to the first related frame image includes the fusion weight of each pixel in the first related frame image, and the fusion weight information corresponding to the first reference frame image includes the fusion weight of each pixel in the first reference frame image;

[0033] The step of fusing the fifth reference frame image and a plurality of the fifth related frame images based on the fusion weight information respectively corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image to obtain the frozen image comprises:

[0034] For each pixel point, the following operations are performed respectively, and the following operations include:

[0035] For each of the channels, weighted summing is performed on component values ​​of the pixel points at the same position in the fifth reference frame image and a plurality of the fifth related frame images on the channel according to the fusion weights of the pixel points at the same position in each of the first related frame images and the fusion weights of the pixel points at the same position in the first reference frame image, so as to obtain a first component value of the pixel points on the channel;

[0036] For each of the channels, an offset process is performed on the first component value of the pixel point on the channel to obtain a fused component value of the pixel point on the channel.

[0037] In a possible implementation, performing an offset process on the first component value of the pixel point on the channel to obtain a fused component value of the pixel point on the channel includes:

[0038] Determine a sum of a first component value of the pixel point on the channel and a first weight sum; the first weight sum is a weight sum of a first preset multiple, and the weight sum is a sum of a fusion weight of a pixel point at the same position in each of the first related frame images and a fusion weight of a pixel point at the same position in the first reference frame image;

[0039] The ratio of the sum value to the second weighted sum is rounded to obtain the fusion component value of the pixel point on the channel; the second weighted sum is the weighted sum of a second preset multiple.

[0040] In a second aspect, an embodiment of the present application provides an image freezing device, comprising:

[0041] An acquisition module, configured to acquire a first reference frame image and a plurality of first related frame images in response to receiving an image freezing instruction;

[0042] a motion estimation module, configured to perform motion estimation based on the first reference frame image and the plurality of first related frame images, so as to determine motion amount information corresponding to each of the first related frame images; the motion amount information is used to represent the motion amount of the first related frame image relative to the first reference frame image;

[0043] A fusion module is used to perform image fusion on the first reference frame image and multiple first related frame images based on the motion information respectively corresponding to each of the first related frame images to obtain a frozen image.

[0044] In a third aspect, an embodiment of the present application provides an image processing unit, including: a processor and a memory connected to the processor in communication;

[0045] The memory stores computer-executable instructions;

[0046] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0048] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0049] In a sixth aspect, an embodiment of the present application provides an endoscope system, comprising: an endoscope, a command input unit, an image processing unit and a display unit as described in the third aspect; the image processing unit is connected to the endoscope, the command input unit, and the display unit respectively;

[0050] The endoscope is used to collect images;

[0051] The instruction input unit is used to input an image freezing instruction;

[0052] The image processing unit is used for, in response to receiving the image freezing instruction, freezing the image based on the image collected by the endoscope to obtain a frozen image;

[0053] The display unit is used to display the frozen image.

[0054] The image freezing method, image processing unit and endoscope system provided in the embodiment of the present application can identify the amount of motion of each first related frame image relative to the first reference frame image by acquiring a first reference frame image and multiple first related frame images and performing motion estimation. This allows the image blurring caused by the movement of the endoscope or the person being examined to be effectively compensated during the image fusion process, thereby improving the clarity of the frozen image. On this basis, by using image fusion technology, the information of multiple images is integrated into one frozen image, which can retain more detailed information and help doctors make more accurate diagnoses. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0056] Figure 1 A schematic diagram of the structure of the endoscope system provided in this application;

[0057] Figure 2 Schematic diagram of the image freezing method provided in this application Figure 1 ;

[0058] Figure 3 Schematic diagram of the image freezing method provided in this application Figure 2 ;

[0059] Figure 4 A schematic diagram of the edge filling process provided for this application;

[0060] Figure 5 A schematic diagram of the filter core provided for this application;

[0061] Figure 6 A schematic diagram of the structure of the image freezing device provided in this application;

[0062] Figure 7 This is a schematic diagram of the structure of the image processing unit provided in this application.

[0063] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0065] The endoscope system has an image freezing function, which plays an important role in endoscopy. The image freezing function allows the examiner to fix the image acquired by the endoscope during the inspection process, making it easier for the examiner to observe and analyze the details of the image.

[0066] One image freezing method in the related art is to directly freeze the image at the current moment, but the image may be blurred due to the movement of the endoscope or the person being examined; another image freezing method is to select the image with the highest clarity from the images within a period of time before and after the current moment to freeze, but the clarity of multiple adjacent images is often not much different, so the clarity of the final frozen image is not high, which is difficult to meet the needs of medical examinations.

[0067] The endoscope system collects images through the image sensor set on the endoscope. The inventor of this application found that the random shot noise of some image sensors is relatively serious. The human eye has a certain smoothing effect on shot noise, so it does not affect the visual experience of the human eye in the video. However, when the image freeze function is triggered, severe shot noise will reduce the clarity of the frozen image.

[0068] The image freezing method, image processing unit and endoscope system provided by the present application are intended to solve the above technical problems. During the endoscopic examination, the movement of the patient and the endoscope is inevitable. The method of the present application can better adapt to these dynamic changes and provide stable image quality through motion estimation and image fusion.

[0069] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0070] Figure 1 This is a schematic diagram of the structure of the endoscope system provided in this application. Figure 1 As shown, the endoscope system 10 includes an endoscope 101, a command input unit 102, an image processing unit 50 and a display unit 103; the image processing unit 50 is connected to the endoscope 101, the command input unit 102, and the display unit 103 respectively.

[0071] The endoscope 101 is used to collect images;

[0072] The instruction input unit 102 is used to input an image freezing instruction;

[0073] The image processing unit 50 is used for, in response to receiving an image freezing instruction, freezing the image based on the image acquired by the endoscope 101 to obtain a frozen image;

[0074] The display unit 103 is used to display the frozen image.

[0075] In one scenario, the examiner inserts the endoscope 101 into the corresponding part of the examinee, so that the endoscope 11 can collect an image of the examined part through an image sensor disposed thereon.

[0076] Optionally, the command input unit 102 may be a button or other control arranged on the handle of the endoscope, so that the examiner can input the image freezing command. For example, the examiner can press a button to trigger the image freezing command.

[0077] In the present application, the instruction input unit 102 sends the image freezing instruction input by the examiner to the image processing unit 50, so that the image processing unit 50 executes the image freezing method provided in the present application, obtains a frozen image, and sends the frozen image to the display unit 103, which displays the frozen image.

[0078] The image freezing method provided in the present application is performed by an image freezing device, which is disposed in the image processing unit 50. The image freezing method, the image freezing device and the image processing unit 50 are described in detail below.

[0079] Figure 2 Schematic diagram of the image freezing method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:

[0080] S201 . In response to receiving an image freezing instruction, acquiring a first reference frame image and a plurality of first related frame images.

[0081] Optionally, the images collected by the endoscope are stored in a double data rate synchronous dynamic random access memory (DDR). Exemplarily, DDR caches N frames of continuous images according to the first-in, first-out principle. The size of N can be set according to actual needs, and this application does not limit this. For example, N is 10, 20 or 30. After receiving the image freeze instruction, the image processing unit obtains the cached M images from DDR. The size of M can be set according to actual needs, and this application does not limit this. For example, M is 5, 6 or 7.

[0082] Among them, the M images include a first reference frame image and multiple first related frame images. Optionally, the first reference frame image can be the image with the highest definition among the cached N frame images, or the first reference frame image can also be the image captured at the moment of receiving the image freeze instruction. Optionally, the multiple first related frame images can be continuous M-1 frame images captured before the first reference frame image, or continuous M-1 frame images captured after the first reference frame image, or continuous A frame images captured before the first reference frame image and continuous B frame images captured after the first reference frame image, and A+B=M-1, A is a positive integer greater than or equal to 1, and B is a positive integer greater than or equal to 1. Among them, the sizes of A and B can be set according to actual needs, and this application does not limit this, for example, M=5, A=2, B=2.

[0083] Optionally, the image collected by the endoscope is in YUV format, and the image is stored in the form of image data. YUV format refers to a pixel format in which brightness parameters and chromaticity parameters are expressed separately. "Y" represents brightness (Luminance or Luma), which is a grayscale value; while "U" and "V" represent chromaticity (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.

[0084] For example, the M images include yuv0, yuv1, yuv2, yuv3 and yuv4, wherein yuv2 is the first reference frame image, yuv0 and yuv1 are the first related frame images collected before the first reference frame image, and yuv3 and yuv4 are the first related frame images collected after the first reference frame image.

[0085] The calculation method of the clarity can be implemented according to relevant technical means, and this application does not limit this.

[0086] S202, performing motion estimation based on the first reference frame image and multiple first related frame images to determine motion amount information corresponding to each first related frame image; the motion amount information is used to represent the motion amount of the first related frame image relative to the first reference frame image.

[0087] The motion information is used to describe the relative motion between images. By performing motion estimation on the acquired first reference frame image and multiple first related frame images, the motion information corresponding to each first related frame image is obtained, thereby providing a basis for subsequent image fusion to reduce image blur caused by the movement of the endoscope or the examinee.

[0088] S203 . Based on the motion information respectively corresponding to each of the first related frame images, perform image fusion on the first reference frame image and the plurality of first related frame images to obtain a frozen image.

[0089] The purpose of image fusion is to combine the information of multiple frames to reduce motion blur or improve the details and quality of the image. The frozen image is the result of reduced motion blur and can show the details more clearly.

[0090] The present application acquires a first reference frame image and multiple first related frame images and performs motion estimation to identify the amount of motion of each related frame image relative to the reference frame image. This allows the image blur caused by the movement of the endoscope or the person being examined to be effectively compensated during the image fusion process, thereby improving the clarity of the frozen image. On this basis, through image fusion technology, the information of multiple images is integrated into one frozen image, which can retain more detailed information and help doctors make more accurate diagnoses.

[0091] Figure 3 Schematic diagram of the image freezing method provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment, the image freezing method is described in detail, and the method includes:

[0092] S301 . In response to receiving an image freezing instruction, acquiring a first reference frame image and a plurality of first related frame images.

[0093] In this embodiment, the implementation of step S301 is the same as that of step S201 and will not be described in detail herein.

[0094] In this embodiment, after step S301, motion estimation is performed based on the first reference frame image and the plurality of first related frame images to determine motion amount information corresponding to each first related frame image, that is, step S202 is executed.

[0095] In some embodiments, before the motion estimation, the image is first smoothed to reduce the influence of noise in the image on the motion estimation. Step S202 specifically includes the following steps S302 to S303.

[0096] S302 : Smoothing the first reference frame image and the plurality of first related frame images respectively to obtain a second reference frame image and a plurality of second related frame images.

[0097] In practical applications, images captured by endoscopes may contain various noises. When performing motion estimation, the noise in the images may lead to unstable calculation results. Therefore, by smoothing each acquired image separately, the noise component can be reduced, the impact of noise can be reduced, and more stable and accurate data support can be provided for motion estimation.

[0098] In some embodiments, the smoothing process is implemented by filtering. Step S302 specifically includes the following steps S3021 to S3023.

[0099] S3021. Perform edge filling on a first reference frame image and a plurality of first related frame images respectively to obtain a third reference frame image and a plurality of third related frame images.

[0100] Among them, edge filling is to add extra pixels in the border area of ​​the image so that in the subsequent filtering process, the filter can completely cover the edge of the image. This processing can avoid information loss or edge effects caused by insufficient edge pixels during filtering, thereby improving the integrity and accuracy of the filtering results. Optionally, the edge filling method can be filling with 0, mirror filling or copy filling, etc. These methods can be selected according to the specific application scenario and image characteristics.

[0101] Optionally, the M images are all in YUV format, and edge filling refers to edge filling of the Y component of each image. The filling radius is pad_r, the size of the Y component before filling is H×W, H is the height, W is the width, and the size of the Y component after filling is (H+pad_r×2)×(W+pad_r×2). The filling radius is determined based on the size of the filter kernel, and the size of the filter kernel can be set according to actual needs. This application does not limit this. For example, if the size of the filter kernel is 7×7, the filling radius is 3. For example, if the size of the filter kernel is 5×5, the filling radius is 2.

[0102] Exemplarily, taking the copying and filling of the first reference frame image as an example, one copying and filling method is: first copy the edge data of the Y component of the image in the horizontal direction pad_r times to obtain a new Y component, and then copy the edge data of the new Y component in the vertical direction pad_r times. Another copying and filling method is: first copy the edge data of the Y component of the image in the horizontal direction once to obtain a new Y component, and then copy the edge data of the new Y component in the vertical direction once; and so on, repeating pad_r times. The edge filling method of each first related frame image is the same as the edge filling method of the first reference frame image, which will not be repeated here.

[0103] For example, see Figure 4 In the edge filling process shown, pad_r=3, the edge data of the original Y component in the horizontal direction is first copied once to obtain a new Y component, and then the edge data of the new Y component in the vertical direction is copied once, and so on, and the cycle is repeated 3 times until the Y component after edge filling is obtained.

[0104] S3022. Filter the third reference frame image and the plurality of third related frame images respectively to obtain a fourth reference frame image and a plurality of fourth related frame images.

[0105] The filtering aims to reduce noise and unnecessary details in the image. By filtering, the overall quality of the image can be enhanced, and the filtered images (the fourth reference frame image and the plurality of fourth related frame images) have higher visual quality and less noise.

[0106] Optionally, the M images are all in YUV format, and filtering refers to filtering the Y component of each image. Optionally, the filtering method can be mean filtering, Gaussian filtering, or other filtering methods. For example, a mean filter with a filter kernel size of 7×7 is designed, and the mean filter is defined as ft. See Figure 5 , which shows a filter kernel of a mean filter, the coefficients of the four corners of the filter kernel are corrected so that the sum of the filter coefficients is 1024. The image is mean filtered using the mean filter, and the corresponding filtering formula is:

[0107]

[0108] Among them, y_smo(i,j) represents the Y component value of the filtered image at the pixel point (i,j); y(i+k,j+l) represents the Y component value of the image before filtering at the pixel point (i+k,j+l); ft(k+ft_r,l+ft_r) represents the coefficient of the filter kernel, ft_r represents the radius of the filter kernel, and for a 7×7 kernel, ft_r=3. Among them, pad_r≤i≤H+pad_r, 0≤j≤W+pad_r.

[0109] S3023 . Perform edge cropping on the fourth reference frame image and the plurality of fourth related frame images respectively to obtain a second reference frame image and a plurality of second related frame images.

[0110] Among them, edge cropping is to restore the filtered image to its original size to ensure that the size of the output image is consistent with the input image. Correspondingly, the size of the second reference frame image is the same as the size of the first reference frame image, and the size of the second related frame image is the same as the size of the first related frame image.

[0111] The size of the cropping radius is equal to the size of the filling radius. Exemplarily, the fourth reference frame image and the plurality of fourth related frame images are edge cropped so that the size of the image is restored to H×W, thereby obtaining: y0_smo, y1_smo, y2_smo, y3_smo and y4_smo.

[0112] This embodiment achieves the effect of significantly improving image quality without changing the image size through steps such as edge filling, filtering and cropping. The obtained second reference frame image and multiple second related frame images not only have the same size as the original image, but also have reduced noise and improved image quality after smoothing.

[0113] S303 , respectively perform motion estimation on a plurality of second related frame images to obtain motion amount information corresponding to each first related frame image.

[0114] This embodiment performs smoothing on the first reference frame image and multiple first related frame images before performing motion estimation, which can effectively reduce noise and detail interference in the image. By combining smoothing and motion estimation, an effective means is provided to obtain high-quality motion information.

[0115] The motion amount information corresponding to the first related frame image is obtained by performing motion estimation on the corresponding second related frame image.

[0116] In some embodiments, the motion information corresponding to the first related frame image includes the motion value of each pixel in the first related frame image, and accordingly, the motion value of the pixel is obtained by performing motion estimation on the pixel at the same position in the corresponding second related frame image. Accordingly, step S303 specifically includes: for each second related frame image, performing the following operations respectively, and the following operations specifically include the following steps S3031-S3032.

[0117] S3031. For each pixel in the second related frame image, determine the absolute difference between the value of the pixel and the value of the pixel at the same position in the second reference frame image.

[0118] The pixel value refers to the Y component of the pixel, and the absolute difference refers to the absolute value of the difference.

[0119] S3032. For each pixel in the second related frame image, determine the motion value of the pixel based on the absolute difference of the pixel, and determine the motion value of the pixel as the motion value of the pixel at the same position in the corresponding first related frame image.

[0120] On the one hand, this embodiment can quickly identify motion changes by comparing the Y components of corresponding pixels in the second related frame image and the second reference frame image, and the absolute difference calculation is simple and has a small amount of calculation, which is suitable for real-time or efficient image processing requirements. On the other hand, by performing motion estimation at the pixel level and determining the motion magnitude using the absolute difference, high-precision and high-robustness motion magnitude information can be provided.

[0121] In some embodiments, determining the motion value of the pixel based on the absolute difference of the pixel in S3032 specifically includes: rounding the product of the absolute difference of the pixel and a first preset value to obtain the motion value of the pixel.

[0122] Optionally, the first preset value is 128.

[0123] Optionally, step S303 is implemented using the following formula:

[0124] move_msg20(i,j)=round(128*abs(y2_smo(i,j)-y0_smo(i,j)));

[0125] move_msg21(i,j)=round(128*abs(y2_smo(i,j)-y1_smo(i,j)));

[0126] move_msg23(i,j)=round(128*abs(y2_smo(i,j)-y3_smo(i,j)));

[0127] move_msg24(i,j)=round(128*abs(y2_smo(i,j)-y4_smo(i,j)));

[0128] Wherein, y2_smo(i,j) represents the Y component value of the pixel point (i,j) in the second reference frame image, y0_smo(i,j) represents the Y component value of the pixel point (i,j) in the second related frame image y0_smo, move_msg20(i,j) represents the motion value of the pixel point (i,j) in the second related frame image y0_smo, y1_smo(i,j) represents the Y component value of the pixel point (i,j) in the second related frame image y1_smo, move_msg21(i,j) represents the motion value of the pixel point (i,j) in the second related frame image y1_smo, y3_smo(i ,j) represents the Y component value of the pixel point (i,j) in the second related frame image y3_smo, move_msg23(i,j) represents the motion value of the pixel point (i,j) in the second related frame image y3_smo, y4_smo(i,j) represents the Y component value of the pixel point (i,j) in the second related frame image y4_smo, move_msg24(i,j) represents the motion value of the pixel point (i,j) in the second related frame image y4_smo, and 0≤i≤H-1, 0≤j≤W-1abs means taking the absolute value, * means multiplication, and round(x) means rounding x to the nearest integer.

[0129] In this embodiment, after step S303, based on the motion information corresponding to each first related frame image, the first reference frame image and the plurality of first related frame images are fused to obtain a frozen image, that is, step S203 is executed.

[0130] In some embodiments, step S203 specifically includes the following steps S304-S305.

[0131] S304: Determine fusion weight information corresponding to each first related frame image based on a preset motion amount threshold and motion amount information corresponding to each first related frame image.

[0132] The preset motion threshold (mv_thr) indicates a pre-set possible maximum motion. The preset motion threshold can be set according to actual needs, and this application does not limit this. Optionally, the preset motion threshold is positively correlated with the total number of M images. The larger M is, the larger mv_thr is; conversely, the smaller M is, the smaller mv_thr is.

[0133] In some embodiments, the motion information corresponding to the first related frame image includes the motion value of each pixel in the first related frame image; the fusion weight information corresponding to the first related frame image includes the fusion weight of each pixel in the first related frame image; accordingly, step S304 specifically includes: for each first related frame image, the following operations are performed respectively, and the following operations include: for each pixel in the first related frame image, determining the difference between the preset motion threshold and the motion value corresponding to the pixel, and determining the maximum value between the second preset value and the difference as the fusion weight of the pixel.

[0134] Optionally, the second preset value is 0.

[0135] Optionally, step S304 is implemented using the following formula:

[0136] wt0(i,j)=max(0,mv_thr-move_msg20(i,j));

[0137] wt1(i,j)=max(0,mv_thr-move_msg21(i,j));

[0138] wt3(i,j)=max(0,mv_thr-move_msg23(i,j));

[0139] wt4(i,j)=max(0,mv_thr-move_msg24(i,j));

[0140] Among them, max(x,y) means taking the maximum value of x and y, wt0(i,j) means the fusion weight of the pixel point (i,j) in the first related frame image yuv0, wt1(i,j) means the fusion weight of the pixel point (i,j) in the first related frame image yuv1, wt3(i,j) means the fusion weight of the pixel point (i,j) in the first related frame image yuv3, and wt4(i,j) means the fusion weight of the pixel point (i,j) in the first related frame image yuv4.

[0141] S305 . Based on the fusion weight information corresponding to each first related frame image and the fusion weight information corresponding to the first reference frame image, perform image fusion on the first reference frame image and the plurality of first related frame images to obtain a frozen image.

[0142] The fusion weight information corresponding to the first reference frame image includes the fusion weight of each pixel in the first reference frame image. Optionally, a preset motion threshold is determined as the fusion weight of each pixel in the first reference frame image.

[0143] In some embodiments, step S305 specifically includes the following steps S3051-S3052.

[0144] S3051. Convert the first reference frame image and the plurality of first related frame images from the YUV format to the RGB format respectively to obtain the fifth reference frame image and the plurality of fifth related frame images.

[0145] The fifth reference frame image and the fifth related frame image both include image data of three channels, and the image data of one channel includes the component value of each pixel point in the image on the channel.

[0146] Optionally, a preset conversion matrix is ​​used to convert yuv0, yuv1, yuv2, yuv3, yuv4 into rgb0, rgb1, rgb2, rgb3, rgb4 respectively. The preset conversion matrix yuv2rgbCoef is expressed as follows:

[0147]

[0148] Accordingly, the converted image is:

[0149]

[0150] Among them, rgb(i,j,1) represents the component value of pixel (i,j) in the R channel, rgb(i,j,2) represents the component value of pixel (i,j) in the G channel, rgb(i,j,3) represents the component value of pixel (i,j) in the B channel, 0≤i≤H-1, 0≤j≤W-1.

[0151] S3052: Based on the fusion weight information corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image, perform image fusion on the fifth reference frame image and the plurality of fifth related frame images to obtain a frozen image.

[0152] In some embodiments, the frozen image includes fused image data of three channels, and the fused image data of one channel includes the fused component value of each pixel in the frozen image on the channel; the fused weight information corresponding to the first related frame image includes the fused weight of each pixel in the first related frame image, and the fused weight information corresponding to the first reference frame image includes the fused weight of each pixel in the first reference frame image. Accordingly, step S3052 specifically includes: for each pixel, performing the following operations respectively, the following operations include:

[0153] For each channel, weighted summing is performed on the component values ​​of the pixel at the same position in the fifth reference frame image and the plurality of fifth related frame images on the channel according to the fusion weights of the pixel at the same position in each first related frame image and the fusion weights of the pixel at the same position in the first reference frame image, so as to obtain a first component value of the pixel on the channel;

[0154] For each channel, an offset process is performed on the first component value of the pixel point on the channel to obtain a fused component value of the pixel point on the channel.

[0155] Since the image is composed of multiple color channels, processing each channel separately can better preserve the color information. On this basis, for each channel, the component values ​​of the pixel points at the same position in multiple frames of images on the channel are weighted and summed, which can accurately fuse the image information from different frames. This weighted summation utilizes the fusion weight of each pixel point, so that the fusion result can better reflect the importance and contribution of each frame, which can effectively reduce the noise and instability in the single-frame image and improve the quality and clarity of the frozen image.

[0156] In some embodiments, the first component value of the pixel point on the channel is offset to obtain the fused component value of the pixel point on the channel, specifically including: determining the sum of the first component value of the pixel point on the channel and the first weight sum; rounding the ratio of the sum to the second weight sum to obtain the fused component value of the pixel point on the channel.

[0157] The first weight sum is the weight sum of the first preset multiple, and the second weight sum is the weight sum of the second preset multiple. The weight sum is the sum of the fusion weights of the pixels at the same position in each first related frame image and the fusion weights of the pixels at the same position in the first reference frame image. Exemplarily, the calculation formula of the weight sum is as follows:

[0158] wtSum(i,j)=wt0(i,j)+wt1(i,j)+wt2(i,j)+wt3(i,j)+wt4(i,j)

[0159] Among them, wtSum(i,j) is the weight sum of pixel point (i,j); wt2(i,j) is the fusion weight of pixel point (i,j) in the first reference frame.

[0160] Exemplarily, step S3052 is implemented based on the following method to obtain the frozen image rgbOut:

[0161]

[0162]

[0163]

[0164] Among them, rgbOut(i,j,1) represents the fused component value of the pixel point (i,j) in the frozen image rgbOut on the R channel, rgbOut(i,j,2) represents the fused component value of the pixel point (i,j) in the frozen image rgbOut on the G channel, rgbOut(i,j,3) represents the fused component value of the pixel point (i,j) in the frozen image rgbOut on the B channel; floor represents the floor function; rgb0(i,j,1) represents the component value of the pixel point (i,j) in the fifth related frame image rgb0 on the R channel, rgb0( rgb0(i,j,2) represents the component value of the pixel point (i,j) in the fifth related frame image rgb0 in the G channel, rgb0(i,j,3) represents the component value of the pixel point (i,j) in the fifth related frame image rgb0 in the B channel; rgb1(i,j,1) represents the component value of the pixel point (i,j) in the fifth related frame image rgb1 in the R channel, rgb1(i,j,2) represents the component value of the pixel point (i,j) in the fifth related frame image rgb1 in the G channel, rgb1(i,j,3) represents the component value of the pixel point (i,j) in the fifth related frame image rgb1 in the B channel rgb2(i,j,1) represents the component value of the pixel point (i,j) in the fifth reference frame image rgb2 in the R channel, rgb2(i,j,2) represents the component value of the pixel point (i,j) in the fifth reference frame image rgb2 in the G channel, rgb2(i,j,3) represents the component value of the pixel point (i,j) in the fifth reference frame image rgb2 in the B channel; rgb3(i,j,1) represents the component value of the pixel point (i,j) in the fifth related frame image rgb3 in the R channel, rgb3(i,j,2) represents the component value of the pixel point (i,j) in the fifth related frame image rgb3 in the The component value of the pixel point (i, j) in the G channel, rgb3(i, j, 3) represents the component value of the pixel point (i, j) in the fifth related frame image rgb3 in the B channel; rgb4(i, j, 1) represents the component value of the pixel point (i, j) in the fifth related frame image rgb4 in the R channel, rgb4(i, j, 2) represents the component value of the pixel point (i, j) in the fifth related frame image rgb4 in the G channel, rgb4(i, j, 3) represents the component value of the pixel point (i, j) in the fifth related frame image rgb4 in the B channel; 0≤i≤H-1, 0≤j≤W-1.

[0165] In this embodiment, the value of each pixel in the frozen image is obtained by weighted summing the corresponding pixels of multiple first related frame images and the first reference frame image. The formula processes the three channels of R, G, and B respectively, ensuring that the details and color features of each color channel are retained during the fusion process. After the weighted summation, the result is normalized by dividing by the weight sum to ensure that the fused pixel value is within a reasonable range. The offset processing is to achieve a rounding effect and ensure the accuracy of integer calculations. This processing helps to reduce the image quality degradation caused by rounding errors. The use of the floor function ensures that the final pixel value is an integer, preventing non-integer values ​​caused by floating-point operations, thereby avoiding problems when displaying or storing images.

[0166] Figure 6 The schematic diagram of the structure of the image freezing device provided in this application is as follows: Figure 6 As shown, the image freezing device 40 provided in this embodiment includes:

[0167] An acquisition module 401 is used for acquiring a first reference frame image and a plurality of first related frame images in response to receiving an image freezing instruction;

[0168] The motion estimation module 402 is used to perform motion estimation based on the first reference frame image and the plurality of first related frame images to determine motion amount information corresponding to each of the first related frame images; the motion amount information is used to represent the motion amount of the first related frame image relative to the first reference frame image;

[0169] The fusion module 403 is used to perform image fusion on the first reference frame image and the plurality of first related frame images based on the motion information respectively corresponding to each first related frame image, so as to obtain a frozen image.

[0170] In a possible implementation, the motion estimation module 402 is specifically configured to:

[0171] Performing smoothing processing on the first reference frame image and the plurality of first related frame images respectively to obtain a second reference frame image and a plurality of second related frame images;

[0172] Motion estimation is performed on a plurality of second related frame images respectively to obtain motion amount information corresponding to each first related frame image.

[0173] In a possible implementation, the motion amount information corresponding to the first related frame image includes the motion amount value of each pixel in the first related frame image; the motion estimation module 402, when performing motion estimation on the plurality of second related frame images respectively to obtain the motion amount information corresponding to each first related frame image respectively, is specifically used to:

[0174] For each second related frame image, the following operations are performed respectively, and the following operations include:

[0175] For each pixel in the second related frame image, determine the absolute difference between the value of the pixel and the value of the pixel at the same position in the second reference frame image;

[0176] For each pixel in the second related frame image, a motion magnitude of the pixel is determined based on the absolute difference of the pixel, and the motion magnitude of the pixel is determined as the motion magnitude of the pixel at the same position in the first related frame image.

[0177] In a possible implementation, the motion estimation module 402, when determining the motion value of the pixel based on the absolute difference of the pixel, is specifically configured to:

[0178] The product of the absolute difference of the pixel point and the first preset value is rounded to obtain the motion value of the pixel point.

[0179] In a possible implementation manner, the motion estimation module 402, when performing smoothing processing on the first reference frame image and the plurality of first related frame images respectively to obtain the second reference frame image and the plurality of second related frame images, is specifically configured to:

[0180] Performing edge filling on the first reference frame image and the plurality of first related frame images respectively to obtain a third reference frame image and a plurality of third related frame images;

[0181] filtering the third reference frame image and the plurality of third related frame images respectively to obtain a fourth reference frame image and a plurality of fourth related frame images;

[0182] The fourth reference frame image and multiple fourth related frame images are edge cropped respectively to obtain a second reference frame image and multiple second related frame images; the size of the second reference frame image is the same as that of the first reference frame image, and the size of the second related frame image is the same as that of the first related frame image.

[0183] In a possible implementation, the fusion module 403 is specifically configured to:

[0184] Determine fusion weight information corresponding to each first related frame image based on a preset motion amount threshold and motion amount information corresponding to each first related frame image;

[0185] Based on the fusion weight information corresponding to each first related frame image and the fusion weight information corresponding to the first reference frame image, the first reference frame image and the plurality of first related frame images are fused to obtain a frozen image.

[0186] In a possible implementation, the motion information corresponding to the first related frame image includes the motion value of each pixel in the first related frame image; the fusion weight information corresponding to the first related frame image includes the fusion weight of each pixel in the first related frame image;

[0187] The fusion module 403, when determining the fusion weight information corresponding to each first related frame image based on the preset motion amount threshold and the motion amount information corresponding to each first related frame image, is specifically used to:

[0188] For each first related frame image, the following operations are performed respectively, and the following operations include:

[0189] For each pixel in the first related frame image, the difference between the preset motion threshold and the motion value corresponding to the pixel is determined, and the maximum value between the second preset value and the difference is determined as the fusion weight of the pixel.

[0190] In a possible implementation, the fusion module 403, when performing image fusion on the first reference frame image and the plurality of first related frame images based on the fusion weight information respectively corresponding to each first related frame image and the fusion weight information corresponding to the first reference frame image to obtain a frozen image, is specifically configured to:

[0191] The first reference frame image and the plurality of first related frame images are respectively converted from a YUV format to an RGB format to obtain a fifth reference frame image and a plurality of fifth related frame images; the fifth reference frame image and the fifth related frame image each include image data of three channels, and the image data include component values ​​of each pixel point in the image on the channel;

[0192] Based on the fusion weight information corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image, the fifth reference frame image and the plurality of fifth related frame images are fused to obtain a frozen image.

[0193] In a possible implementation, the frozen image includes fused image data of three channels, the fused image data includes fused component values ​​of each pixel in the frozen image on the channel; the fusion weight information corresponding to the first related frame image includes the fusion weight of each pixel in the first related frame image, and the fusion weight information corresponding to the first reference frame image includes the fusion weight of each pixel in the first reference frame image;

[0194] The fusion module 403, when performing image fusion on the fifth reference frame image and the plurality of fifth related frame images based on the fusion weight information respectively corresponding to the first related frame images and the fusion weight information corresponding to the first reference frame image to obtain a frozen image, is specifically used to:

[0195] For each pixel, the following operations are performed respectively, including:

[0196] For each channel, weighted summing is performed on the component values ​​of the pixel points at the same position in the fifth reference frame image and the plurality of fifth related frame images according to the fusion weights of the pixel points at the same position in each first related frame image and the fusion weights of the pixel points at the same position in the first reference frame image, so as to obtain a first component value of the pixel points on the channel;

[0197] For each channel, an offset process is performed on the first component value of the pixel point on the channel to obtain a fused component value of the pixel point on the channel.

[0198] In a possible implementation manner, the fusion module 403, when performing an offset process on the first component value of the pixel point on the channel to obtain the fused component value of the pixel point on the channel, is specifically configured to:

[0199] Determine the sum of the first component value of the pixel point on the channel and the first weight sum; the first weight sum is a weight sum of a first preset multiple, and the weight sum is the sum of the fusion weight of the pixel point at the same position in each first related frame image and the fusion weight of the pixel point at the same position in the first reference frame image;

[0200] The ratio of the sum value to the second weighted sum is rounded to obtain the fusion component value of the pixel point on the channel; the second weighted sum is the weighted sum of the second preset multiple.

[0201] The image freezing device 40 provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0202] Figure 7 This is a schematic diagram of the structure of the image processing unit provided in this application. Figure 7 As shown, the image processing unit 50 provided in this embodiment includes: a processor 501 and a memory 502 communicatively connected to the processor 501 .

[0203] The memory 502 stores computer-executable instructions; the processor 501 executes the computer-executable instructions stored in the memory 502 to implement the image freezing method provided in the present application.

[0204] The memory 502 and the processor 501 are connected via a bus. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, and the like.

[0205] The components, their connections and relationships, and their functions shown herein are only examples and are not intended to limit the implementation of the present application described and / or required herein. The various components are interconnected using different buses and can be installed on a common motherboard or in other ways as needed.

[0206] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0207] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0208] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0209] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0210] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0211] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0212] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components. Unless otherwise specified, the memory can be implemented by any type of volatile or non-volatile storage device or a combination of them, such as a USB flash drive, random-access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), enhanced dynamic random-access memory (EDRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), high-bandwidth memory (HBM) or hybrid memory cube (HMC) and other media that can store program code.

[0213] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0214] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0215] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An image freezing method, characterized in that: include: In response to receiving an image freezing instruction, acquiring a first reference frame image and a plurality of first related frame images; Performing motion estimation based on the first reference frame image and the plurality of first related frame images to determine motion amount information corresponding to each of the first related frame images; The motion amount information is used to represent the motion amount of the first related frame image relative to the first reference frame image; Based on the motion information respectively corresponding to each of the first related frame images, the first reference frame image and a plurality of the first related frame images are fused to obtain a frozen image.

2. The method according to claim 1, characterized in that: The performing motion estimation based on the first reference frame image and the plurality of first related frame images to determine motion amount information respectively corresponding to each of the first related frame images includes: Performing smoothing processing on the first reference frame image and the plurality of first related frame images respectively to obtain a second reference frame image and a plurality of second related frame images; Motion estimation is performed on a plurality of the second related frame images respectively to obtain motion amount information corresponding to each of the first related frame images.

3. The method according to claim 2, characterized in that The motion information corresponding to the first related frame image includes the motion value of each pixel in the first related frame image; and the motion estimation is performed on the plurality of second related frame images respectively to obtain the motion information corresponding to each of the first related frame images respectively, including: For each of the second related frame images, the following operations are performed respectively, and the following operations include: For each pixel in the second related frame image, determine an absolute difference between a value of the pixel and a value of a pixel at the same position in the second reference frame image; For each pixel in the second related frame image, the motion value of the pixel is determined based on the absolute difference of the pixel, and the motion value of the pixel is determined as the motion value of the pixel at the same position in the first related frame image.

4. The method according to claim 3, characterized in that: The determining the motion value of the pixel point based on the absolute difference of the pixel point includes: The product of the absolute difference of the pixel point and the first preset value is rounded to obtain the motion value of the pixel point.

5. The method according to claim 2, characterized in that: The step of respectively smoothing the first reference frame image and the plurality of first related frame images to obtain a second reference frame image and a plurality of second related frame images comprises: Performing edge filling on the first reference frame image and the plurality of first related frame images respectively to obtain a third reference frame image and a plurality of third related frame images; filtering the third reference frame image and the plurality of third related frame images respectively to obtain a fourth reference frame image and a plurality of fourth related frame images; The fourth reference frame image and multiple fourth related frame images are edge cropped respectively to obtain the second reference frame image and multiple second related frame images; the size of the second reference frame image is the same as that of the first reference frame image, and the size of the second related frame image is the same as that of the first related frame image.

6. The method according to claim 1, characterized in that The step of fusing the first reference frame image and the plurality of first related frame images based on the motion information respectively corresponding to the first related frame images to obtain a frozen image includes: Determine fusion weight information corresponding to each of the first related frame images based on a preset motion amount threshold and motion amount information corresponding to each of the first related frame images; Based on the fusion weight information corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image, the first reference frame image and multiple first related frame images are fused to obtain the frozen image.

7. The method according to claim 6, characterized in that The motion amount information corresponding to the first related frame image includes the motion amount value of each pixel in the first related frame image; the fusion weight information corresponding to the first related frame image includes the fusion weight of each pixel in the first related frame image; The determining, based on a preset motion amount threshold and motion amount information corresponding to each of the first related frame images, fusion weight information corresponding to each of the first related frame images respectively includes: For each of the first related frame images, the following operations are performed respectively, and the following operations include: For each pixel in the first related frame image, the difference between the preset motion threshold and the motion value corresponding to the pixel is determined, and the maximum value between the second preset value and the difference is determined as the fusion weight of the pixel.

8. The method according to claim 6, characterized in that The step of fusing the first reference frame image and a plurality of the first related frame images based on the fusion weight information respectively corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image to obtain the frozen image includes: The first reference frame image and the plurality of first related frame images are respectively converted from a YUV format to an RGB format to obtain a fifth reference frame image and a plurality of fifth related frame images; the fifth reference frame image and the fifth related frame image each include image data of three channels, and the image data include component values ​​of each pixel point in the image on the channels; Based on the fusion weight information corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image, the fifth reference frame image and multiple fifth related frame images are fused to obtain the frozen image.

9. The method according to claim 8, characterized in that The frozen image includes fused image data of three channels, and the fused image data includes fused component values ​​of each pixel in the frozen image on the channels; the fusion weight information corresponding to the first related frame image includes the fusion weight of each pixel in the first related frame image, and the fusion weight information corresponding to the first reference frame image includes the fusion weight of each pixel in the first reference frame image; The step of fusing the fifth reference frame image and a plurality of the fifth related frame images based on the fusion weight information respectively corresponding to each of the first related frame images and the fusion weight information corresponding to the first reference frame image to obtain the frozen image comprises: For each pixel point, the following operations are performed respectively, and the following operations include: For each of the channels, weighted summing is performed on component values ​​of the pixel points at the same position in the fifth reference frame image and a plurality of the fifth related frame images on the channel according to the fusion weights of the pixel points at the same position in each of the first related frame images and the fusion weights of the pixel points at the same position in the first reference frame image, so as to obtain a first component value of the pixel points on the channel; For each of the channels, an offset process is performed on the first component value of the pixel point on the channel to obtain a fused component value of the pixel point on the channel.

10. The method according to claim 9, characterized in that The performing an offset process on the first component value of the pixel point on the channel to obtain a fused component value of the pixel point on the channel includes: Determine a sum of a first component value of the pixel point on the channel and a first weight sum; the first weight sum is a weight sum of a first preset multiple, and the weight sum is a sum of a fusion weight of a pixel point at the same position in each of the first related frame images and a fusion weight of a pixel point at the same position in the first reference frame image; The ratio of the sum value to the second weighted sum is rounded to obtain the fusion component value of the pixel point on the channel; the second weighted sum is the weighted sum of a second preset multiple.

11. An image processing unit, characterized in that: comprising a processor and a memory in communication with said processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the image freezing method according to any one of claims 1 to 10.

12. An endoscope system, characterized in that: include: An endoscope, a command input unit, an image processing unit and a display unit according to claim 11; the image processing unit is connected to the endoscope, the command input unit and the display unit respectively; The endoscope is used to collect images; The instruction input unit is used to input an image freezing instruction; The image processing unit is used for, in response to receiving the image freezing instruction, freezing the image based on the image collected by the endoscope to obtain a frozen image; The display unit is used to display the frozen image.

Images