Image processing method and device and visual inspection system
By acquiring the time domain noise information and airspace characteristics of the image acquisition device, combined with the time domain noise processing technology, the problem of poor image quality improvement effect in the prior art is solved, efficient image noise reduction is achieved, and signal-to-noise ratio is improved. Clarity.
Patent Information
- Application Number
- CN202411086632.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-12-03
AI Technical Summary
In the prior art, in image processing, image correction is poor in improving image quality by setting fixed parameters, especially in terms of signal-to-noise ratio and clarity.
By acquiring the image and time domain noise information collected by the target object, the image acquisition device extracts the low-frequency and high-frequency characteristics of the airspace, and combines the time domain noise information to determine the time-space noise, perform real-time noise reduction processing, and improve the signal-to-noise ratio of the image , clarity and fidelity.
Real-time noise reduction processing of the image is realized, which significantly improves the signal-to-noise ratio, clarity and fidelity, improves the image quality, and effectively suppresses the inhomogeneity caused by random noise and temperature changes.
Smart Images

Figure CN119067875A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and in particular, relates to an image processing method, device and visual inspection system. Background Art
[0002] With the development of artificial intelligence (AI) technology, the quality requirements for image information are getting higher and higher. Especially in the field of machine vision, the quality requirements for image information are mainly concentrated on aspects such as signal-to-noise ratio and clarity.
[0003] Currently, the image quality is improved mainly by correcting the image by setting fixed parameters. However, this method has the defect of poor image quality improvement effect. Summary of the invention
[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes an image processing method, device and visual inspection system, which can effectively improve image quality.
[0005] In a first aspect, the present application provides an image processing method, the method comprising:
[0006] Acquire an n-th frame of an image captured by an image acquisition device on a target object, and temporal noise information of the n-th frame of the image, wherein the temporal noise information is used to characterize noise characteristics of the image captured by the image acquisition device on the target object in the temporal domain;
[0007] Extracting spatial low-frequency features and spatial high-frequency features from the n-th frame image;
[0008] The spatiotemporal noise of the n-th frame image is determined according to the temporal noise information, the spatial low-frequency characteristics and the spatial high-frequency characteristics, so as to perform noise reduction processing on the n-th frame image.
[0009] According to the image processing method of the present application, through the spatial low-frequency characteristics and spatial high-frequency characteristics of the spatial domain, combined with the temporal noise information of the time domain, the image can be denoised in real time, thereby effectively improving the signal-to-noise ratio, clarity and fidelity, and improving the quality of the image.
[0010] According to an embodiment of the present application, determining the spatiotemporal noise of the nth frame image according to the temporal noise information, the spatial low-frequency features, and the spatial high-frequency features includes:
[0011] Determining a noise correction item of the n-th frame image according to the spatial low-frequency feature, wherein the noise correction item is used to characterize a texture suppression degree of the n-th frame image;
[0012] Based on the temporal noise information, the noise correction term and the spatial high-frequency feature, the temporal and spatial noise of the n-th frame image is obtained.
[0013] According to an embodiment of the present application, determining the noise correction item of the n-th frame image according to the spatial low-frequency feature includes:
[0014] Extracting edge features from the spatial low-frequency features to obtain a texture reference value of each pixel in the n-th frame image, wherein the texture reference value is used to characterize the texture intensity of the pixel;
[0015] The noise correction term is constructed based on the texture reference value.
[0016] According to an embodiment of the present application, extracting edge features from the spatial low-frequency features to obtain a texture reference value for each pixel in the n-th frame image includes:
[0017] Extract edge features from the spatial low-frequency features to determine texture area pixels of the target object in the n-th frame image, where the texture area pixels include at least one of edge area pixels, transition area pixels, flat area pixels, and background area pixels;
[0018] The texture reference value of each pixel in the n-th frame image is determined based on the pixel brightness of the pixels in the texture area.
[0019] According to an embodiment of the present application, constructing the noise correction term based on the texture reference value includes:
[0020] Based on the texture reference value, a monotonically decreasing function is constructed to obtain the noise correction term.
[0021] According to an embodiment of the present application, obtaining the spatiotemporal noise of the nth frame image based on the temporal noise information, the noise correction term and the spatial high-frequency feature includes:
[0022] Determining a noise weight matrix of pixels in the n-th frame image according to the noise correction term;
[0023] Determining the spatial noise term of the n-th frame image according to the noise weight matrix and the spatial high-frequency feature;
[0024] Determining the time domain noise term of the n-th frame image according to the time domain noise information and the noise weight matrix;
[0025] Based on the spatial domain noise term and the temporal domain noise term, the spatial and temporal domain noise of the n-th frame image is determined.
[0026] According to an embodiment of the present application, extracting spatial low-frequency features and spatial high-frequency features from the n-th frame image includes:
[0027] Performing smoothing filtering on the n-th frame image to obtain the spatial domain low-frequency features;
[0028] The n-th frame image is subtracted from the spatial low-frequency feature to obtain the spatial high-frequency feature.
[0029] In a second aspect, the present application provides an image processing device, the device comprising:
[0030] An acquisition module, used to acquire an n-th frame of an image acquired by an image acquisition device on a target object, and temporal noise information of the n-th frame of the image, wherein the temporal noise information is used to characterize noise characteristics of the image acquired by the image acquisition device on the target object in the temporal domain;
[0031] A first processing module is used to extract spatial low-frequency features and spatial high-frequency features from the n-th frame image;
[0032] The second processing module is used to determine the spatiotemporal noise of the n-th frame image according to the temporal noise information, the spatial low-frequency characteristics and the spatial high-frequency characteristics, so as to perform noise reduction processing on the n-th frame image.
[0033] In a third aspect, the present application provides a visual inspection system, the system comprising:
[0034] An image acquisition device, wherein the image acquisition device is used to acquire multiple frames of images of a target object;
[0035] A data processing device, the data processing device is electrically connected to the image acquisition device, and the data processing device is used to execute the image processing method described in the first aspect above.
[0036] According to one embodiment of the present application, the data processing device includes:
[0037] A texture extraction module is used to extract spatial low-frequency features and spatial high-frequency features from the n-th frame image;
[0038] A noise correction module, used to determine a noise correction item of the n-th frame image according to the spatial low-frequency feature, wherein the noise correction item is used to characterize the texture suppression degree of the n-th frame image;
[0039] A spatiotemporal noise calculation module, configured to obtain the spatiotemporal noise of the n-th frame image based on the spatiotemporal noise information of the n-th frame image, the noise correction term and the spatial high-frequency feature;
[0040] The image quality enhancement processing module is used to perform noise reduction processing on the n-th frame image based on the spatiotemporal noise of the n-th frame image.
[0041] In a fourth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image processing method as described in the first aspect above when executing the computer program.
[0042] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the image processing method as described in the first aspect above is implemented.
[0043] In a sixth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the image processing method as described in the first aspect.
[0044] In a seventh aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the image processing method as described in the first aspect above.
[0045] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0047] Figure 1 is a flowchart of an image processing method provided in an embodiment of the present application;
[0048] Figure 2 is a schematic diagram of the numerical relationship between the texture reference value and the noise correction term provided in an embodiment of the present application;
[0049] Figure 3 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application;
[0050] Figure 4 is a schematic diagram of the structure of a visual inspection system provided in an embodiment of the present application;
[0051] Figure 5 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;
[0052] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0054] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0055] The image processing method, image processing device, visual inspection system, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0056] The image processing method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.
[0057] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).
[0058] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.
[0059] The image processing method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the image processing method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The image processing method provided in the embodiment of the present application is described below using the electronic device as an example of the execution subject.
[0060] like Figure 1 As shown, the image processing method includes: step 110 to step 130.
[0061] Step 110: Obtain the nth frame image of the target object captured by the image acquisition device and the time domain noise information of the nth frame image, where the time domain noise information is used to characterize the noise characteristics of the image captured by the image acquisition device on the target object in the time domain.
[0062] The embodiments of the present application are applicable to a scenario where an image acquisition device acquires multiple frames of images of a target object in the same scene, and acts on the image acquisition device in real time to achieve real-time processing of the acquired n frames of images.
[0063] It is understandable that the time domain noise information can be used to characterize the noise characteristics contained in the image that vary with time, and can be obtained by performing noise analysis on any one or more frames of the first n-1 frames of the image.
[0064] For example, the time domain noise information may be random or non-random changes introduced when an image acquisition device acquires an image of a target object and transmits or stores the acquired image.
[0065] Wherein, since the nth frame image has temporal noise information, n is an integer greater than 1.
[0066] Step 120: extracting spatial low-frequency features and spatial high-frequency features from the n-th frame image.
[0067] Among them, frequency is the rate at which the nth frame image changes in the spatial domain. The spatial low-frequency feature corresponds to the slowly changing part of the nth frame image, and can be used to characterize the large background color or main structure in the nth frame image. The spatial high-frequency feature corresponds to the rapidly changing details or edge parts in the nth frame image, and can be used to characterize the texture of the nth frame image.
[0068] In this step, components of different frequencies in the n-th frame image can be analyzed in the spatial domain, and spatial low-frequency features and spatial high-frequency features can be obtained through low-frequency components and high-frequency components.
[0069] Alternatively, the nth frame image can be subjected to spatial domain analysis based on a filter, and the high-frequency components can be filtered out by a low-pass filter, and the low-frequency components can be retained to obtain spatial domain low-frequency features, and the low-frequency components can be filtered out by a high-pass filter, and the high-frequency components can be retained to obtain spatial domain high-frequency features.
[0070] Step 130: Determine the temporal and spatial noise of the n-th frame image according to the temporal noise information, the spatial low-frequency characteristics and the spatial high-frequency characteristics, so as to perform noise reduction processing on the n-th frame image.
[0071] The spatiotemporal noise may be obtained by fusing the noise of the n-th frame image in the spatial domain and the temporal domain, which can minimize the influence of the noise on the quality of the n-th frame image.
[0072] In this step, the spatial high-frequency features representing the texture are corrected secondary at the pixel level through the spatial low-frequency features, and the corrected spatial high-frequency features are combined with the temporal noise information to obtain the spatiotemporal noise of the n-th frame image, and a corresponding filter is constructed to reduce the noise of the n-th frame image through the filter.
[0073] In the nth frame image after denoising, the scene edges are clear, there is no obvious ghosting of moving objects, there is no stratification or blocking as a whole, the spatial and temporal noise are significantly suppressed, and the subjective visual experience is good. Compared with conventional denoising methods such as mean square error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM), it has higher fidelity.
[0074] In related technologies, there are two ways to reduce the noise of an image:
[0075] First, non-real-time correction, such as the flat-field correction commonly used in visible light cameras and the two-point correction commonly used in infrared cameras.
[0076] Second, the fixed pattern noise of the image is corrected in real time through two-dimensional spatial filtering.
[0077] The above methods all have the defect of poor suppression effect on random noise.
[0078] The image processing method of the present application can effectively improve the random noise suppression effect and suppress new non-uniformity caused by random noise and temperature changes.
[0079] According to the image processing method provided in the embodiment of the present application, through the spatial low-frequency characteristics and spatial high-frequency characteristics of the spatial domain, combined with the temporal noise information of the time domain, real-time noise reduction processing can be performed on the image, thereby effectively improving the signal-to-noise ratio, clarity and fidelity, and improving the quality of the image.
[0080] In some embodiments, extracting spatial low-frequency features and spatial high-frequency features from the nth frame image includes:
[0081] Perform smoothing filtering on the n-th frame image to obtain the low-frequency features in the spatial domain;
[0082] Subtract the n-th frame image from the spatial low-frequency features to obtain the spatial high-frequency features.
[0083] The smoothing filter may include one or more combinations of filtering methods such as bilateral filtering, guided filtering, side window filtering, non-local mean filtering, etc.
[0084] In actual implementation, the nth frame image x(n) is smoothed by a smoothing filtering algorithm to obtain the spatial low-frequency feature x SL (n), the nth frame image x(n) and the spatial low-frequency feature x SL (n) is subtracted to obtain a high-frequency residual image, which is used as the spatial high-frequency feature x SH (n).
[0085] In this embodiment, by subtracting the nth frame image from the spatial low-frequency features, the obtained spatial low-frequency features and spatial high-frequency features can retain the complete information in the nth frame image, providing a basis for improving the image processing effect of the nth frame image.
[0086] In some embodiments, determining the spatiotemporal noise of the nth frame image according to the temporal noise information, the spatial low-frequency characteristics and the spatial high-frequency characteristics includes:
[0087] Determine a noise correction item of the n-th frame image according to the low-frequency characteristics of the spatial domain, and the noise correction item is used to characterize the texture suppression degree of the n-th frame image;
[0088] Based on the temporal noise information, the noise correction term and the spatial high-frequency features, the temporal and spatial noise of the n-th frame image is obtained.
[0089] It is understandable that the noise correction term can be represented in a matrix form or in other mathematical forms. In the following, the noise correction term is taken as the noise correction matrix M c (n) is used as an example to illustrate the solution of the present application.
[0090] The degree of texture suppression is used to describe the degree of weakening or retaining of image texture details in image processing. Texture processing can improve the visualization effect of the n-th frame image x(n), enhance the visual information of a specific area, and filter the noise of the n-th frame image x(n).
[0091] In the n-th frame image, the low-frequency part usually has a larger amplitude, while the high-frequency part is relatively smaller.
[0092] In actual implementation, by analyzing the low-frequency features x in the spatial domain SL The energy or average value in (n) can be used to evaluate the texture suppression degree in the n-th frame image x(n) to obtain the noise correction term M c (n).
[0093] Pass-by noise correction term M c (n), enlarge or reduce the spatial high-frequency feature x SHThe amplitude of (n) can adjust the details and texture of the n-th frame image x(n) to correct the high-frequency features in the spatial domain and achieve better image correction effects.
[0094] The noise correction term M c (n) Corrected spatial high-frequency features x SH By combining the temporal noise information with the time domain noise information, the spatiotemporal noise of the n-th frame image x(n) can be obtained.
[0095] In this embodiment, the passing noise correction term M c (n) Extract the spatial high-frequency features x representing the texture information SH The strong edges and weak textures in (n) are suppressed to different degrees to achieve secondary correction of the extracted texture, which can effectively remove noise and artifacts in the n-th frame image, enhance details and clarity, and further improve image quality.
[0096] In some embodiments, determining the noise correction item of the nth frame image according to the spatial low-frequency feature includes:
[0097] Extract edge features from low-frequency features in the spatial domain to obtain a texture reference value for each pixel in the n-th frame image. The texture reference value is used to characterize the texture intensity of the pixel.
[0098] Based on the texture reference value, a noise correction term is constructed.
[0099] Among them, the texture intensity of a pixel can refer to the degree of change of its grayscale value or characteristics in the local area around a single pixel. The texture reference value does not require a preset threshold for judgment. The larger the texture reference value q(n) of the pixel, the stronger the texture of the pixel.
[0100] The texture reference value q(n) may be a matrix constructed based on the texture reference values and position coordinates corresponding to all pixels.
[0101] In some embodiments, edge features are extracted from spatial low-frequency features to obtain a texture reference value for each pixel in the n-th frame image, including:
[0102] Extract edge features from the spatial low-frequency features to determine texture area pixels of the target object in the nth frame image, where the texture area pixels include at least one of edge area pixels, transition area pixels, flat area pixels, and background area pixels;
[0103] Based on the pixel brightness of the pixels in the texture area, a texture reference value of each pixel in the n-th frame image is determined.
[0104] It should be noted that, when the pixel point is closer to the edge of the scene or target object in the nth frame image x(n), the texture reference value corresponding to the pixel point is larger, and the position of the pixel can be preliminarily judged based on the texture reference value. For example, the edge, transition area, flat area or background area of the target object corresponds to edge area pixels, transition area pixels, flat area pixels and background area pixels, respectively.
[0105] In actual implementation, the Sobel operator and other edge detection algorithms are used to detect the low-frequency features x in the spatial domain. SL (n) Edge extraction is performed, a corresponding local area is defined for each pixel, and each pixel is calculated by methods such as local gradient or gray level co-occurrence matrix to obtain a texture reference value q(n) corresponding to each pixel.
[0106] Alternatively, the texture reference value q(n) of each pixel is obtained by using the local variance of pixel brightness, the Euclidean distance between pixels, and the like.
[0107] According to the texture reference value q(n) of each pixel, the corresponding correction factor is selected to construct the noise correction term M c (n).
[0108] In this embodiment, a noise correction term is constructed through texture reference values to correct or adjust texture features to achieve accurate noise reduction for the n-th frame image.
[0109] In some embodiments, constructing a noise correction term based on the texture reference value includes:
[0110] Based on the texture reference value, a monotonically decreasing function is constructed to obtain the noise correction term.
[0111] It can be understood that, in the monotonically decreasing function, the noise correction term decreases as the texture reference value increases.
[0112] like Figure 2 As shown, the texture reference value q(n) and the noise correction term M c (n) are all values of (0,1). In the texture reference value q(n), the texture intensity increases with the increase of the value. c In (n), the texture suppression degree decreases as the value increases, and the monotonically decreasing function can be represented by curve 1 or curve 2.
[0113] In this embodiment, the noise correction term M is c (n) is designed as a monotonically decreasing function of the texture reference value q(n), satisfying that the larger the texture reference value q(n), the larger the noise correction term M c (n) is smaller.
[0114] For example, the noise correction term M c (n) is:
[0115] M c (n) = e -αq(n)
[0116] Among them, α is an adjustable suppression parameter.
[0117] The noise correction term M obtained c (n), used for high-frequency features x in the spatial domain SH (n) Perform noise correction. c The pixel position whose value in (n) is close to 0 acts on the spatial high-frequency feature x SH (n) will form a strong suppression, and the pixel position whose value is close to 1 will have a strong suppression on the spatial high-frequency feature x SH The inhibitory effect of (n) is weaker.
[0118] In the related art, high-frequency images are directly judged as image noise to achieve filtering and noise reduction.
[0119] However, high-frequency images contain scene information such as edges and textures. If high-frequency images are directly removed as noise, it will cause loss of scene information in the spatial domain and cause the ghosting of moving targets in the recursive calculation in the time domain.
[0120] In this embodiment, by constructing a monotonic decreasing function between the texture reference value and the noise correction term, the obtained noise correction term can suppress the pixels in the spatial high-frequency features to varying degrees. The suppression effect smoothly transitions with the texture size, and there is no threshold division, thereby avoiding the obvious unnatural transition in the spatial scene caused by threshold division and the appearance of moving target ghosting in the time domain.
[0121] In some embodiments, the temporal and spatial noise of the nth frame image is obtained based on the temporal noise information, the noise correction term and the spatial high-frequency feature, including:
[0122] Determine the noise weight matrix of pixels in the n-th frame image according to the noise correction term;
[0123] Determine the spatial noise term of the nth frame image according to the noise weight matrix and the spatial high-frequency characteristics;
[0124] Determine the time domain noise item of the n-th frame image according to the time domain noise information and the noise weight matrix;
[0125] Based on the spatial domain noise term and the temporal domain noise term, the temporal and spatial domain noise of the n-th frame image is determined.
[0126] In actual implementation, with the noise correction matrix M c (n) The associated noise weight matrix W S (n) is:
[0127]
[0128] Among them, K is an adjustable parameter and can be set to a positive integer greater than 1. The size of K depends on W. S (n) is related to the convergence speed.
[0129] The spatiotemporal noise f(n) of the nth frame image x(n) can be a recursive filter:
[0130]
[0131] Among them, W S (n)*x SH (n) is the spatial noise term of the n-th frame image x(n), (1-W S (n))*f(n-1) is the time domain noise term of the n-th frame image x(n), and f(n-1) is the time domain noise information, which can be the spatiotemporal noise of the image of the n-1th frame.
[0132] Remove the spatiotemporal noise f(n) from the n-th frame image x(n) to obtain the output image y(n) with improved imaging quality:
[0133] y(n)=x(n)-f(n)
[0134] In related technologies, three-dimensional spatiotemporal noise reduction methods use multiple frames of images for motion estimation, weighted averaging and motion compensation, or divide moving and static areas and use filtering of different intensities and types, which easily introduces new problems such as block effects, detail loss, ghosting, and contour artifacts.
[0135] In this embodiment, based on feature information such as the spatial high-frequency features represented by texture and the spatial low-frequency features represented by brightness in the n-th frame image, the degree of suppression of spatiotemporal noise is corrected in real time, which can retain scene information to the greatest extent, improve the image signal-to-noise ratio, and avoid introducing new problems such as block effects, loss of details, ghosting, and contour artifacts.
[0136] An embodiment is described below.
[0137] The nth frame image x(n) is smoothed by the smoothing filtering algorithm to obtain the spatial low-frequency feature x SL (n), the nth frame image x(n) and the spatial low-frequency feature x SL (n) is subtracted to obtain a high-frequency residual image, which is used as the spatial high-frequency feature x SH (n).
[0138] Through edge detection algorithms such as Sobel operator, the spatial low-frequency features x SL(n) Edge extraction is performed, a corresponding local area is defined for each pixel, and each pixel is calculated by methods such as local gradient or gray level co-occurrence matrix to obtain a texture reference value q(n) corresponding to each pixel.
[0139] Alternatively, the texture reference value q(n) of each pixel is obtained by using the local variance of pixel brightness, the Euclidean distance between pixels, and the like.
[0140] The noise correction term M c (n) is designed as a monotonically decreasing function of the texture reference value q(n), satisfying that the larger the texture reference value q(n), the larger the noise correction term M c (n) is smaller.
[0141] For example, the noise correction term M c (n) is:
[0142] M c (n) = e -αq(n)
[0143] Among them, α is an adjustable suppression parameter.
[0144] and the noise correction matrix M c (n) The associated noise weight matrix W S (n) is:
[0145]
[0146] Among them, K is an adjustable parameter and can be set to a positive integer greater than 1. The size of K depends on W. S (n) is related to the convergence speed.
[0147] The spatiotemporal noise f(n) of the nth frame image x(n) can be a recursive filter:
[0148]
[0149] Among them, W S (n)*x SH (n) is the spatial noise term of the n-th frame image x(n), (1-W S (n))*f(n-1) is the time domain noise term of the n-th frame image x(n), and f(n-1) is the time domain noise information, which can be the spatiotemporal noise of the image of the n-1th frame.
[0150] Remove the spatiotemporal noise f(n) from the n-th frame image x(n) to obtain the output image y(n) with improved imaging quality:
[0151] y(n)=x(n)-f(n)
[0152] In this embodiment, based on feature information such as the spatial high-frequency features represented by texture and the spatial low-frequency features represented by brightness in the n-th frame image, the degree of suppression of spatiotemporal noise is corrected in real time, which can retain scene information to the greatest extent, improve the image signal-to-noise ratio, and avoid new problems caused by common methods such as motion partitioning, blocking, and inter-frame weighted averaging.
[0153] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.
[0154] The embodiment of the present application also provides an image processing device.
[0155] like Figure 3 As shown, the image processing device includes: an acquisition module 310 , a first processing module 320 and a second processing module 330 .
[0156] An acquisition module 310 is used to acquire the n-th frame image acquired by the image acquisition device on the target object, and the time domain noise information of the n-th frame image, where the time domain noise information is used to characterize the noise characteristics of the image acquired by the image acquisition device on the target object in the time domain;
[0157] The first processing module 320 is used to extract spatial low-frequency features and spatial high-frequency features from the n-th frame image;
[0158] The second processing module 330 is used to determine the temporal and spatial noise of the n-th frame image according to the temporal noise information, the spatial low-frequency characteristics and the spatial high-frequency characteristics, so as to perform noise reduction processing on the n-th frame image.
[0159] According to the image processing device provided in the embodiment of the present application, through the spatial low-frequency characteristics and spatial high-frequency characteristics of the spatial domain, combined with the temporal noise information of the time domain, real-time noise reduction processing can be performed on the image, thereby effectively improving the signal-to-noise ratio, clarity and fidelity, and improving the quality of the image.
[0160] In some embodiments, the second processing module 330 is further configured to:
[0161] Determine a noise correction item of the n-th frame image according to the low-frequency characteristics of the spatial domain, and the noise correction item is used to characterize the texture suppression degree of the n-th frame image;
[0162] Based on the temporal noise information, the noise correction term and the spatial high-frequency features, the temporal and spatial noise of the n-th frame image is obtained.
[0163] In some embodiments, the second processing module 330 is further configured to:
[0164] Extract edge features from low-frequency features in the spatial domain to obtain a texture reference value for each pixel in the n-th frame image. The texture reference value is used to characterize the texture intensity of the pixel.
[0165] Based on the texture reference value, a noise correction term is constructed.
[0166] In some embodiments, the second processing module 330 is further configured to:
[0167] Extract edge features from the spatial low-frequency features to determine texture area pixels of the target object in the nth frame image, where the texture area pixels include at least one of edge area pixels, transition area pixels, flat area pixels, and background area pixels;
[0168] Based on the pixel brightness of the pixels in the texture area, a texture reference value of each pixel in the n-th frame image is determined.
[0169] In some embodiments, the second processing module 330 is further configured to:
[0170] Based on the texture reference value, a monotonically decreasing function is constructed to obtain the noise correction term.
[0171] In some embodiments, the second processing module 330 is further configured to:
[0172] Determine the noise weight matrix of pixels in the n-th frame image according to the noise correction term;
[0173] Determine the spatial noise term of the nth frame image according to the noise weight matrix and the spatial high-frequency characteristics;
[0174] Determine the time domain noise item of the nth frame image according to the time domain noise information and the noise weight matrix;
[0175] Based on the spatial domain noise term and the temporal domain noise term, the temporal and spatial domain noise of the n-th frame image is determined.
[0176] In some embodiments, the first processing module 320 is further configured to:
[0177] Perform smoothing filtering on the n-th frame image to obtain the low-frequency features in the spatial domain;
[0178] Subtract the n-th frame image from the spatial low-frequency features to obtain the spatial high-frequency features.
[0179] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (Augmented Reality, AR) / virtual reality (Virtual Reality, VR) device, a robot, a wearable device, a super mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), a television (Television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.
[0180] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0181] The image processing device provided in the embodiment of the present application can achieve Figure 1 and Figure 2 To avoid repetition, the various processes implemented in the image processing method embodiment are not described here.
[0182] In some embodiments, Figure 4 As shown, the embodiment of the present application also provides a visual inspection system, including:
[0183] An image acquisition device 410, the image acquisition device 410 is used to acquire multiple frames of images of a target object;
[0184] The data processing device 420 is electrically connected to the image acquisition device 410. The data processing device 420 is used to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0185] The data processing device 420 may be a chip, CPU or other device in which the algorithm program of the image processing method is written.
[0186] like Figure 5 As shown, the data processing device includes:
[0187] A texture extraction module is used to extract spatial low-frequency features and spatial high-frequency features from the n-th frame image;
[0188] A noise correction module, used to determine a noise correction item of the n-th frame image according to the low-frequency characteristics of the spatial domain, and the noise correction item is used to characterize the texture suppression degree of the n-th frame image;
[0189] A spatiotemporal noise calculation module, used for obtaining the spatiotemporal noise of the nth frame image based on the spatiotemporal noise information, noise correction term and spatial high-frequency characteristics of the nth frame image;
[0190] The image quality enhancement processing module is used to perform noise reduction processing on the n-th frame image based on the spatiotemporal noise of the n-th frame image.
[0191] The n-th frame image x(n) is input into the texture extraction module. In the texture extraction module, the n-th frame image x(n) is smoothed by a smoothing filter algorithm to obtain the spatial domain low-frequency feature x SL (n), the nth frame image x(n) and the spatial low-frequency feature x SL (n) is subtracted to obtain a high-frequency residual image, which is used as the spatial high-frequency feature x SH (n), realize the extraction of texture information and output the spatial low-frequency feature x SL (n) to the noise correction module and output the spatial high-frequency feature x SH (n) To the spatiotemporal noise calculation module.
[0192] In the noise correction module, the spatial low-frequency feature x is detected by edge detection algorithms such as the Sobel operator. SL (n) Edge extraction is performed, a corresponding local area is defined for each pixel, and each pixel is calculated by methods such as local gradient or gray level co-occurrence matrix to obtain a texture reference value q(n) corresponding to each pixel.
[0193] Alternatively, the texture reference value q(n) of each pixel is obtained by using the local variance of pixel brightness, the Euclidean distance between pixels, and the like.
[0194] The noise correction term M c (n) is designed as a monotonically decreasing function of the texture reference value q(n), satisfying that the larger the texture reference value q(n), the larger the noise correction term M c (n) is smaller.
[0195] For example, the noise correction term M c (n) is:
[0196] M c (n) = e -αq(n)
[0197] Among them, α is an adjustable suppression parameter.
[0198] The noise correction module outputs the noise correction term M c (n) To the spatiotemporal noise calculation module.
[0199] In the spatiotemporal noise calculation module, the noise correction matrix M c (n) The associated noise weight matrix W S (n) is:
[0200]
[0201] Among them, K is an adjustable parameter and can be set to a positive integer greater than 1. The size of K depends on W. S (n) is related to the convergence speed.
[0202] The spatiotemporal noise f(n) of the nth frame image x(n) can be a recursive filter:
[0203]
[0204] Among them, W S (n)*x SH (n) is the spatial noise term of the n-th frame image x(n), (1-W S (n))*f(n-1) is the time domain noise term of the n-th frame image x(n), and f(n-1) is the time domain noise information, which can be the spatiotemporal noise of the image of the n-1th frame.
[0205] The spatiotemporal noise calculation module outputs the spatiotemporal noise f(n) of the nth frame image x(n) to the image quality improvement processing module, and can also use the spatiotemporal noise f(n) of the nth frame image x(n) as the spatiotemporal noise information of the n+1th frame image.
[0206] In the image quality improvement processing module, the spatiotemporal noise f(n) is removed from the n-th frame image x(n) to obtain the output image y(n) with improved imaging quality:
[0207] y(n)=x(n)-f(n)
[0208] The image quality enhancement processing module generates and outputs an output image y(n) of the n-th frame image x(n).
[0209] In some embodiments, Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0210] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
[0211] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0212] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0213] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned image processing method when executed by a processor.
[0214] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0215] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0216] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0217] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0218] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the image processing methods of various embodiments of the present application.
[0219] In the description of this application, "first feature" or "second feature" may include one or more of the features.
[0220] In the description of the present application, “plurality” means two or more.
[0221] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
[0222] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0223] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. An image processing method, It is characterized in that include: Acquire an n-th frame of an image captured by an image acquisition device on a target object, and temporal noise information of the n-th frame of the image, wherein the temporal noise information is used to characterize noise characteristics of the image captured by the image acquisition device on the target object in the temporal domain; Extracting spatial low-frequency features and spatial high-frequency features from the n-th frame image; The spatiotemporal noise of the n-th frame image is determined according to the temporal noise information, the spatial low-frequency characteristics and the spatial high-frequency characteristics, so as to perform noise reduction processing on the n-th frame image.
2. The image processing method according to claim 1, It is characterized in that The step of determining the spatiotemporal noise of the nth frame image according to the temporal noise information, the spatial low-frequency features, and the spatial high-frequency features includes: Determining a noise correction item of the n-th frame image according to the spatial low-frequency feature, wherein the noise correction item is used to characterize a texture suppression degree of the n-th frame image; Based on the temporal noise information, the noise correction term and the spatial high-frequency feature, the temporal and spatial noise of the n-th frame image is obtained.
3. The image processing method according to claim 2, It is characterized in that The step of determining the noise correction item of the n-th frame image according to the spatial low-frequency feature comprises: Extracting edge features from the spatial low-frequency features to obtain a texture reference value of each pixel in the n-th frame image, wherein the texture reference value is used to characterize the texture intensity of the pixel; The noise correction term is constructed based on the texture reference value.
4. The image processing method according to claim 3, It is characterized in that The step of extracting edge features from the spatial low-frequency features to obtain a texture reference value for each pixel in the n-th frame image includes: Extract edge features from the spatial low-frequency features to determine texture area pixels of the target object in the n-th frame image, where the texture area pixels include at least one of edge area pixels, transition area pixels, flat area pixels, and background area pixels; The texture reference value of each pixel in the n-th frame image is determined based on the pixel brightness of the pixels in the texture area.
5. The image processing method according to claim 3, It is characterized in that The step of constructing the noise correction term based on the texture reference value comprises: Based on the texture reference value, a monotonically decreasing function is constructed to obtain the noise correction term.
6. The image processing method according to claim 2, It is characterized in that The obtaining the spatiotemporal noise of the n-th frame image based on the temporal noise information, the noise correction term and the spatial high-frequency feature comprises: Determining a noise weight matrix of pixels in the n-th frame image according to the noise correction term; Determining the spatial noise term of the n-th frame image according to the noise weight matrix and the spatial high-frequency feature; Determining the time domain noise term of the n-th frame image according to the time domain noise information and the noise weight matrix; Based on the spatial domain noise term and the temporal domain noise term, the spatial and temporal domain noise of the n-th frame image is determined.
7. The image processing method according to any one of claims 1 to 6, It is characterized in that The step of extracting the spatial low-frequency features and the spatial high-frequency features from the n-th frame image includes: Performing smoothing filtering on the n-th frame image to obtain the spatial domain low-frequency features; The n-th frame image is subtracted from the spatial low-frequency feature to obtain the spatial high-frequency feature.
8. An image processing device, It is characterized in that include: An acquisition module, used to acquire an n-th frame of an image acquired by an image acquisition device on a target object, and temporal noise information of the n-th frame of the image, wherein the temporal noise information is used to characterize noise characteristics of the image acquired by the image acquisition device on the target object in the temporal domain; A first processing module is used to extract spatial low-frequency features and spatial high-frequency features from the n-th frame image; The second processing module is used to determine the spatiotemporal noise of the n-th frame image according to the temporal noise information, the spatial low-frequency characteristics and the spatial high-frequency characteristics, so as to perform noise reduction processing on the n-th frame image.
9. A visual inspection system, It is characterized in that include: An image acquisition device, wherein the image acquisition device is used to acquire multiple frames of images of a target object; A data processing device, wherein the data processing device is electrically connected to the image acquisition device, and the data processing device is used to execute the image processing method according to any one of claims 1 to 7.
10. The visual inspection system according to claim 9, It is characterized in that The data processing device comprises: A texture extraction module is used to extract spatial low-frequency features and spatial high-frequency features from the n-th frame image; A noise correction module, used to determine a noise correction item of the n-th frame image according to the spatial low-frequency feature, wherein the noise correction item is used to characterize the texture suppression degree of the n-th frame image; A spatiotemporal noise calculation module, configured to obtain the spatiotemporal noise of the n-th frame image based on the spatiotemporal noise information of the n-th frame image, the noise correction term and the spatial high-frequency feature; The image quality enhancement processing module is used to perform noise reduction processing on the n-th frame image based on the spatiotemporal noise of the n-th frame image.
