Image denoising method and system

Abstract

The invention discloses an image denoising method and system, and the method comprises the steps: receiving original image data through a microwave range finder at a camera end, demodulating the original image data to a baseband, and obtaining a point target model of the original image data; performing distance compression processing on the point target model to obtain a distance compression signal of the original image data; transforming data on each distance in the distance compression signal to a distance Doppler domain through azimuth FFT (Fast Fourier Transform) to obtain FFT image data; comparing the phase of the FFT image data with the benchmarking graph, and determining a phase compensation value according to a comparison result; performing phase compensation on the FFT image data based on the phase compensation value; and performing inverse FFT on the FFT image data after phase compensation to obtain defibrillated image data. Compared with the prior art, the Doppler effect is utilized to solve the problem of image blurring caused by inaccurate focusing at the camera end by compensating the phase of the image.

Description

Technical Field

[0001] The present invention relates to the field of image denoising, and in particular to an image denoising method and system. Background Art

[0002] With the rapid development of science and technology, the development of integrated circuit technology, and the increase in market demand, people have higher and higher requirements for chip packaging technology, among which the requirements for surface mount technology (SMT) in chip packaging are getting higher and higher. "SMT" refers to a process of mounting electronic components on a printed circuit board (PCB) or other substrate carrier. Its key actions are: the negative pressure nozzle on the patch head sucks up the electronic components, then visually locates the electronic components and the PCB circuit board, then corrects the product position, and finally accurately places it in the designated planned position.

[0003] However, although modern chip packaging technology is quite advanced, there are still some challenges in practical application. One of them is the shaking of the camera end of the chip mounter during the chip packaging process, which can easily lead to inaccurate focus and blurry and unclear chip images. Summary of the invention

[0004] The present invention provides an image denoising method and system for solving the technical problem that the camera jitter of a chip mounter during chip packaging easily leads to inaccurate focusing and blurred and unclear chip images.

[0005] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0006] An image denoising method, comprising:

[0007] The original image data is received by the microwave rangefinder at the camera end and demodulated to the baseband to obtain the point target model of the original image data;

[0008] Performing distance compression processing on the point target model to obtain a distance compression signal of the original image data;

[0009] Transforming the data at each range in the range compression signal into the range Doppler domain by azimuth FFT to obtain FFT transformed image data;

[0010] Compare the phase of the FFT transformed image data with the reference image, and determine the phase compensation value according to the comparison result;

[0011] Performing phase compensation on the FFT transformed image data based on the phase compensation value;

[0012] The phase-compensated FFT transformed image data is subjected to an inverse FFT transform to obtain image data after defibrillation.

[0013] Preferably, the point target model is:

[0014]

[0015]

[0016]

[0017] Where G0(s,t) is the point target model; c / 2 is the resolution dimension length; A is an arbitrary complex constant; s is the range time; t is the close-range azimuth time; t c is the beam center deviation time; is the instantaneous slant range; R0 is the shortest distance from the microwave rangefinder to the target; 2R(t) / c is the target range migration with azimuth; f0 is the image center frequency; K a Adjust the frequency for distance; is the distance envelope; T r is the transmission pulse duration; is the azimuth envelope; θ a is the beam squint angle; θ b is the beam width;

[0018] and / or

[0019] The distance compression process is implemented by the following formula:

[0020]

[0021] Among them, G rb (s,t) is the distance compression signal, p r [s] is the compressed pulse network; V b is the equivalent microwave rangefinder speed;

[0022] and / or

[0023] The FFT transformed image data is obtained by the following formula:

[0024]

[0025]

[0026] Among them, G1(s,f t ) is the FFT transformed image data, f t is the azimuth frequency, FFT t represents the azimuth FFT transform function, is a matched filter, Direction diagram The frequency domain form of is the same in shape, where is the Doppler center frequency, Rrd is the phase compensation in the range Doppler domain; K b The frequency is modulated in azimuth;

[0027] and / or

[0028] The phase compensation value is:

[0029]

[0030] Preferably, the inverse FFT transform is implemented by the following formula:

[0031]

[0032] Among them, G rc (s, t) is the image data after defibrillation G2(s, f t ) is the FFT transformed image data after phase compensation; IFFT t is the inverse FFT transform function, Z a is the amplitude of the azimuth impulse response.

[0033] Preferably, the method further includes an image clearing processing step:

[0034] Construct the afterimage annealing operator H and its pseudo-inverse matrix of the post-defibrillation image data;

[0035] Constructing a diffusion network for removing residual images, and reconstructing the distribution space of the post-defibrillation image data based on the diffusion network for removing residual images;

[0036] Obtain the content space of the image data after defibrillation, and obtain the target annealing image based on the reconstructed distribution space and content space

[0037] Preferably, the afterimage annealing operator H is:

[0038]

[0039] Among them, x represents a high-definition image without residual image, y represents a picture taken by a camera, H is the residual image annealing operator, V t is the description of the image optical flow field at time t; τ is the time delay value, t is the time, Δt represents the time range from the current to the future frame, V t+τ The optical flow field description of the image with time delay τ; σ v , σ t are motion time standard deviation parameters, which respectively characterize the influence of the object's moving speed on the afterimage effect and the effect of the effect decaying over time.

[0040] Preferably, the expression of the afterimage removal diffusion network is:

[0041]

[0042] Among them, DNN w (x) represents the output of the afterimage diffusion network, x represents the network input, n represents the number of network layers, w represents the weight matrix, L i is the denoised image output by each layer, 1≤i≤n, σ is the activation function, is a linear operation operator.

[0043] Preferably, the afterimage removal diffusion network is iterated by the following formula:

[0044]

[0045] Among them, L t-1 is the iteration value of order t-1, is the target image of the i-th layer σ t It is a scalar parameter used to control the noise intensity during the iteration process. It is used to adjust the noise level added to the image and is a set of parameters preset in advance. ε is the Gaussian noise obtained by sampling from the standard normal distribution.

[0046] Preferably, based on the reconstructed distribution space, content space, shadow annealing operator H and its pseudo-inverse matrix H * Get the target annealing image This is achieved through the following formula:

[0047]

[0048] Among them, H * y is the target annealing image Content space, (IH * H)L i|0 Anneal the target image The distribution space of y represents the picture taken by the camera, H * is the pseudo-inverse matrix H * , I represents the identity matrix; L i|0 Represents the image after the i-th denoising.

[0049] Preferably, the target annealing image is processed based on the de-reflection diffusion network. The distribution space is reconstructed, including:

[0050] The image after the i-th denoising is calculated using the following formula:

[0051]

[0052] in, represents the predicted noise of the i-th layer of the de-reflection diffusion network, L i|0For a gradually clearer and more residual image, 1≤i≤n, ω i The image weight after denoising is outputted by the i-th layer;

[0053] Will l i|0 Bring in the (IH * H)L i|0 The target annealing image is obtained from distribution space.

[0054] A computer system comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0055] The present invention has the following beneficial effects:

[0056] 1. The present invention provides an image denoising method and system, which receives the original image data through a microwave rangefinder on the camera end and demodulates it to the baseband to obtain a point target model of the original image data; performs distance compression processing on the point target model to obtain a distance compression signal of the original image data; transforms the data at each distance in the distance compression signal to the range Doppler domain through azimuth FFT to obtain FFT transformed image data; compares the phase of the FFT transformed image data with the reference image, and determines the phase compensation value according to the comparison result; performs phase compensation on the FFT transformed image data based on the phase compensation value; and performs FFT inverse transformation on the phase-compensated FFT transformed image data to obtain defibrillated image data. Compared with the prior art, the present invention uses the Doppler effect to compensate the phase of the image to solve the image blurring problem caused by inaccurate focus on the camera end.

[0057] 2. In the preferred embodiment, the present invention proposes an image sharpening model, which finally achieves the effect of sharpening the blurred image through a series of operations such as image inverse problem, target annealing image theoretical decomposition, construction of a de-afterimage diffusion network, image distribution space reconstruction, and iterative output of the target image.

[0058] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0060] Figure 1 A flow chart of an image denoising method in a preferred embodiment of the present invention is shown;

[0061] Figure 2 A flow chart showing the steps of Doppler effect dynamic defibrillation in a preferred embodiment of the present invention is shown;

[0062] Figure 3 A flow chart of an image sharpening model in a preferred embodiment of the present invention is shown. DETAILED DESCRIPTION

[0063] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0064] The present invention proposes an image denoising method, which includes two parts:

[0065] The first part is the Doppler effect dynamic defibrillation step: that is, at the object end (camera end), the Doppler effect is used to correct the phase angle, which solves the problem of inaccurate focus of the camera itself at the chip packaging machine end and blurred images.

[0066] The second part is the afterimage denoising step: that is, at the viewing end (the image after correction processing at the object end), it solves the afterimage problem existing in the high-speed video stream image itself after the camera takes the image.

[0067] The overall idea is Figure 1 As shown, the following steps are included:

[0068] S1: The original image data is received by the microwave rangefinder on the camera end and demodulated to the baseband to obtain the point target model of the original image data; the Doppler effect dynamic defibrillation algorithm is used to perform fast Fourier transform, and the phase and amplitude are proposed. According to the phase obtained from the reference image (a clear image without vibration), the phase angle is corrected, and then it is returned through fast inverse Fourier transform to finally output the processed image.

[0069] like Figure 2 As shown, the specific implementation scheme is as follows:

[0070] S11: Input the original image and demodulate the original image data to baseband so as to set the middle of the range frequency to zero. The point target model after demodulation is:

[0071]

[0072] A is an arbitrary complex constant; s is the distance time; t is the close-range azimuth time; t c is the beam center deviation time; is the distance envelope (rectangular window function), that is

[0073]

[0074] is the azimuth envelope (square function), that is

[0075]

[0076] T r is the transmission pulse width; θ a is the beam squint angle; θ b is the beam width.

[0077] f0 is the center frequency of the image, K a is the distance modulation frequency, is the instantaneous slant range. The target azimuth time t is referenced to the zero Doppler in the echo G0(s,t). For multiple targets, a common absolute time t should be given, such as the data capture start time t = 0. According to the translation property of the Fourier transform, the phase term will be carried in the subsequent azimuth FFT.

[0078] S12: Distance compression, let G0(f s ,t) is the distance Fourier transform of G0(s,t), H(f s ) is a matched filter, then the distance compression output is:

[0079]

[0080] Among them, the compressed pulse network p r [s] is the window function W r (f s ) is the inverse Fourier transform of .

[0081]

[0082] From the previous analysis (Formula 4), the distance compression signal is:

[0083]

[0084] The azimuth phase modulation can be clearly seen from the second exponential term. Since the phase is t 2 The signal has a linear frequency modulation characteristic, and the frequency modulation is (approximation at low oblique viewing angle):

[0085]

[0086] S13: Transform the data at each range into the range Doppler domain through azimuth FFT. For a given target, formula G rb The first exponential term in (s, t) is a constant, so only the second exponential term needs to be considered in the signal derivation in the range Doppler domain. Using the principle of stationary phase (POSP), the time-frequency relationship in azimuth is obtained and expressed as f t To indicate that f t =-Kb t, set t = -f t / K b Substitution G rb (s, t), the signal of azimuth FFT is:

[0087]

[0088] Direction diagram The frequency domain form of is the same in shape, where is the Doppler center frequency. The above formula contains two exponential terms. The first term is the inherent phase information of the target and has nothing to do with the image intensity. The second term is the frequency domain azimuth modulation with linear frequency modulation characteristics.

[0089] S14: Compare the phase of the image obtained by fast Fourier transform with the reference image. The phase compensation in the range Doppler domain can be defined as:

[0090]

[0091] The phase that needs to be compensated is

[0092] S15: After the image deviation phase is compensated, an inverse fast Fourier transform is performed. The formula is as follows:

[0093]

[0094] Among them, G2(s,f t ) is the signal after phase compensation; Z a is the amplitude of the azimuth impulse response.

[0095] S2: A deep network sharpening architecture model is proposed to sharpen the image after step 1 through black box operation to remove the image afterimage problem after the camera is taken. (1) The image is clarified to remove the image afterimage problem after shooting on the camera side.

[0096] First, the problem is mathematically modeled and the residual image annealing operator H is defined as the inverse matrix H of the residual image annealing operator. * ,The operator H simulates the image afterimage problem and Gaussian blur on the camera side during the shooting process. The afterimage and blurring process can be simulated by the formula y=Hx, where x represents a high-definition image without afterimage and y represents the picture taken by the camera. Therefore, the steps (1) The input image y is used to perform an image inverse problem, and an attempt is made to infer the target annealing image. The removal of residual images is completed.

[0097]

[0098] Formula (1) gives the obtained de-ghosted image The constraints and the expression of the afterimage annealing operator H are shown in Figure 2. In H, Δt represents the time range from the current frame to the future frame, V t+τ The image optical flow field description with time delay τ, σ v , σ t , the motion time standard deviation parameter, respectively characterizes the influence of the object's moving speed on the afterimage effect, and the effect of the effect decaying over time.

[0099] Based on the above modeling, the specific operations are as follows:

[0100] S21: First, the pseudo-inverse matrix H of the residual image annealing operator H is calculated * , where the matrix P is defined as the spectral value matrix of the afterimage annealing operator H.

[0101]

[0102] Among them, z i represents the i-th singular value of the operator H, Ψ,γ are both unitary matrices,

[0103] Ψ,γ satisfy the following constraints:

[0104]

[0105] S22: Get the pseudo-inverse matrix H of the afterimage annealing operator H * After that, the target annealing image Theoretical decomposition of the two constraints in formula (1) is as follows:

[0106]

[0107] Among them, H * y is defined as the target annealing image content space, (IH * H)x r Defined as the target annealing image distribution space.

[0108] S23: Content Space H * y can be obtained from the camera image. Next, we use the deep network to obtain x r So as to calculate the distribution space (IH * H)x r , De-shadowing Diffusion Network DNN w (x) can be expressed as:

[0109]

[0110] Where x represents the network input, n represents the number of network layers, w is the weight matrix, L i (1≤i≤n) is the denoising of the output of each layer, calculated according to formula (15), σ is the activation function, is a linear operation operator.

[0111]

[0112] in, represents the predicted noise of the i-th layer of the de-reflection diffusion network, L i|0 (1≤i≤n) is a gradually cleared and de-imaged image. Here, L i|0 As x in formula (14) r , so after bringing it in, we can get the target image at the i-th layer

[0113] In order to improve the stability of the sampling process of the afterimage removal diffusion network and avoid falling into the local optimal solution, the target of the i-th layer is obtained

[0114] image Add a weak noise as the network input L of the next layer t-1 :

[0115]

[0116] where σ t ε represents a weak Gaussian noise. We get L t-1 As the input of the next layer of the afterimage removal diffusion network, the iteration continues and the target annealed image is finally obtained. The goal of removing the image afterimage left after shooting by the camera can be achieved.

[0117] Specifically, Figure 3 As shown, the iterative steps of the afterimage removal diffusion network in the present invention are as follows:

[0118] S231: Initialization;

[0119] S232: Calculate the image L after denoising at the i-th layer i , and calculate L according to formula (16) i|0 ;

[0120] S233: L i|0 As x r Bring in the (IH * H)x r The target annealing image at the i-th layer is obtained The distribution space of

[0121] S234: Based on the content space at the i-th layer and the target annealing image at the i-th layer The target annealing image when calculating the i-th layer in the distribution space

[0122] S235: Calculate the network input L of the next layer based on formula (17) t-1 ;

[0123] The process of S232-5 is repeated until the annealing is completed to obtain the final annealed image.

[0124] In summary, the present invention takes into account that the camera end of the placement machine is prone to jitter, resulting in inaccurate focus and blurred images. The present invention utilizes the Doppler effect and obtains the phase and amplitude of the input image through fast Fourier transform, and then obtains the phase compensation value by comparing it with the phase of the benchmark image to perform phase compensation and finally output the image; after the image is output, the image itself has problems such as afterimage. The present invention proposes an image sharpening model, which adopts black box operation, and finally achieves the effect of clarifying the blurred image through a series of operations such as image inverse problem, target annealing image theoretical decomposition, construction of a de-afterimage diffusion network, image distribution space reconstruction, iterative output of the target image, and outputs the final clear image at the same time.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image denoising method, characterized in that: include: The original image data is received by the microwave rangefinder at the camera end and demodulated to the baseband to obtain the point target model of the original image data; Performing distance compression processing on the point target model to obtain a distance compression signal of the original image data; Transforming the data at each range in the range compression signal into the range Doppler domain by azimuth FFT to obtain FFT transformed image data; Compare the phase of the FFT transformed image data with the reference image, and determine the phase compensation value according to the comparison result; Performing phase compensation on the FFT transformed image data based on the phase compensation value; The phase-compensated FFT transformed image data is subjected to an inverse FFT transform to obtain image data after defibrillation.

2. The image denoising method according to claim 1, characterized in that: The point target model is: Where G0(s,t) is the point target model; c / 2 is the resolution dimension length; A is an arbitrary complex constant; s is the range time; t is the close-range azimuth time; t c is the beam center deviation time; is the instantaneous slant range; R0 is the shortest distance from the microwave rangefinder to the target; 2R(t) / c is the target range migration with azimuth; f0 is the image center frequency; K a Adjust the frequency for distance; is the distance envelope; T r is the transmission pulse duration; is the azimuth envelope; θ a is the beam squint angle; θ b is the beam width; and / or The distance compression process is implemented by the following formula: Among them, G rb (s,t) is the distance compression signal, p r [s] is the compressed pulse network; V b is the equivalent microwave rangefinder speed; and / or The FFT transformed image data is obtained by the following formula: Among them, G1(s,f t ) is the FFT transformed image data, f t is the azimuth frequency, FFT t represents the azimuth FFT transform function, is a matched filter, Direction diagram The frequency domain form of is the same in shape, where is the Doppler center frequency, R rd is the phase compensation in the range Doppler domain; K b The frequency is modulated in azimuth; and / or The phase compensation value is:

3. The image denoising method according to claim 2, characterized in that: The inverse FFT transform is implemented by the following formula: Among them, G rc (s, t) is the image data after defibrillation; G2(s, f t ) is the FFT transformed image data after phase compensation; IFFT t is the inverse FFT transform function, Z a is the amplitude of the azimuth impulse response.

4. The image denoising method according to claim 3, characterized in that: It also includes the image sharpening process: Construct the afterimage annealing operator H and its pseudo-inverse matrix of the post-defibrillation image data; Constructing a diffusion network for removing residual images, and reconstructing the distribution space of the post-defibrillation image data based on the diffusion network for removing residual images; Obtain the content space of the image data after defibrillation, and obtain the target annealing image based on the reconstructed distribution space and content space 5. The image denoising method according to claim 4, characterized in that: The afterimage annealing operator H is: Among them, x represents a high-definition image without residual image, y represents a picture taken by a camera, H is the residual image annealing operator, V t is the description of the image optical flow field at time t; τ is the time delay value, t is the time, Δt represents the time range from the current to the future frame, V t+τ The optical flow field description of the image with time delay τ; σ v , σ t are motion time standard deviation parameters, which respectively characterize the influence of the object's moving speed on the afterimage effect and the effect of the effect decaying over time.

6. The image denoising method according to claim 5, characterized in that: The expression of the anti-afterimage diffusion network is: Among them, DNN w (x) represents the output of the afterimage diffusion network, x represents the network input, n represents the number of network layers, w represents the weight matrix, L i is the denoised image output by each layer, 1≤i≤n, σ is the activation function, is a linear operation operator.

7. The image denoising method according to claim 6, characterized in that: The de-ghosting diffusion network is iterated by the following formula: Among them, L t-1 is the iteration value of order t-1, is the target image of the i-th layer σ t It is a scalar parameter used to control the noise intensity during the iteration process. It is used to adjust the noise level added to the image and is a set of parameters preset in advance. ε is the Gaussian noise obtained by sampling from the standard normal distribution.

8. The image denoising method according to claim 7, characterized in that: Based on the reconstructed distribution space, content space, shadow annealing operator H and its pseudo-inverse matrix H * Get the target annealing image This is achieved through the following formula: Among them, H * y is the target annealing image Content space, (IH * H)L i|0 Anneal the target image The distribution space of y represents the picture taken by the camera, H * is the pseudo-inverse matrix H * , I represents the identity matrix; L i|0 Represents the image after the i-th denoising.

9. The image denoising method according to claim 8, characterized in that: Based on the de-image diffusion network, the target annealing image The distribution space is reconstructed, including: The image after the i-th denoising is calculated using the following formula: in, represents the predicted noise of the i-th layer of the de-reflection diffusion network, L i|0 For a gradually clearer and more residual image, 1≤i≤n, ω i The weight of the denoised image output of the i-th layer; L i|0 Bring in the (IH * H)L i|0 The target annealing image is obtained from distribution space.

10. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images