A background processing method to highlight image differences
Through the two-dimensional fast Fourier transform and spectrum multiplication and division method, the problem of image background elimination is solved by image background elimination due to image light and dark changes, camera vibration and image misalignment, and efficient image background elimination and signal-to-noise ratio improvement are achieved.
Patent Information
- Application Number
- CN202111225555.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-21
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
Smart Images

Figure CN114004735B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer image processing, and in particular to a background processing method for highlighting image differences. Background Art
[0002] Among a large number of computer image processing technologies, such as the inspection and control of target images and the detection of weak targets, they can have great practical value in military and civilian applications. Among them, the optical flow method and some adaptive algorithms often use inter-frame image subtraction to reduce the amount of information of the image to be processed. Through image subtraction, the changes of the target image can be highlighted, so that different backgrounds can be distinguished, and the signal-to-noise ratio can be improved. In the process of image transmission, due to the large amount of information, various compression methods such as Fourier transform and cosine transform are used to discard some high-frequency information that does not occupy much energy and transmit it without affecting human eye observation. Among them, inter-frame image subtraction technology is also used. When encoding and transmitting, only the difference between two adjacent images is transmitted, which greatly reduces the transmission image information and also reduces the transmission time. Research on eliminating image background is receiving attention from image processing workers.
[0003] However, when recording images with a camera or camcorder, the intensity of the same scene recorded is different due to different conditions of recording images at different times, such as shooting time, lighting changes, etc. The subtraction result is affected by the overall exposure, and the difference in the image is affected by the same part of the image. In addition, when the camera shakes or the image is not aligned when processing the image, resulting in the image of the same scene shifting, the effect of eliminating the image background will also be affected.
[0004] In view of the above problems, the present invention provides a background processing method for highlighting image differences, performs Fourier transform on discrete data of the image, and then uses spectrum multiplication and division methods to eliminate the image background and the above defects. Summary of the invention
[0005] The purpose of the present invention is to overcome the problems existing in the prior art and provide a background processing method for highlighting image differences.
[0006] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:
[0007] A background processing method for highlighting image differences, the method comprising the following steps:
[0008] Step 1) First obtain a predicted image;
[0009] Step 2) Perform a two-dimensional fast Fourier transform FFT2 on the predicted image to obtain the frequency spectrum of the predicted image, which is stored in a computer as known information;
[0010] Step 3) when the intensity of a certain pixel of the predicted image changes, the changed image is used as the instant image to obtain the instant image;
[0011] Step 4) Perform two-dimensional fast Fourier transform FFT2 on the real-time image to obtain the spectrum of the real-time image;
[0012] Step 5) Divide each matrix element value of the spectrum matrix of the instantaneous image by the corresponding matrix element value of the spectrum matrix of the predicted image to obtain the quotient of the spectrum of the instantaneous image and the predicted image;
[0013] Step 6) Perform two-dimensional inverse fast Fourier transform IFFT2 on the spectrum of the instantaneous image and the predicted image, and obtain an image matrix in the spatial domain, in which the value of the central matrix element is much higher than the values of other matrix elements;
[0014] Step 7) setting the central matrix element value of the image matrix obtained in step 6) to 0, performing a two-dimensional fast Fourier transform FFT2 on it, and performing a dot multiplication of the obtained spectrum matrix with the spectrum matrix of the predicted image in step 2) to obtain a spectrum matrix of the difference image;
[0015] Step 8) Perform two-dimensional inverse fast Fourier transform IFFT2 on the spectrum matrix of the difference image to obtain the difference between the instant image and the predicted image, highlighting the difference between the two images.
[0016] Furthermore, the predicted image and the instantaneous image both use NN pixels, and the data of the predicted image uses a two-dimensional matrix: , to reflect, the instant image uses a two-dimensional matrix: , to reflect.
[0017] Furthermore, the value of N is 121, and after being read into the computer, the value of each pixel is represented by 8 bits.
[0018] Furthermore, in step 6), the Dirac delta function is used to make the value of the central matrix element (1, 1) at the upper left corner of the image matrix obtained in the spatial domain much higher than the values of other matrix elements.
[0019] Furthermore, in step 1) and step 3), a CCD camera is used to record images to obtain image data files.
[0020] The beneficial effects of the present invention are:
[0021] The method of the present invention is not affected by changes in image brightness, camera vibration, image misalignment, etc., and can completely eliminate the image background, further reducing the transmitted image information and the transmission time. In addition, since the frequency spectrum of the predicted image can be obtained in advance and Fourier transform also has mature software and hardware, it has high practical value and can be applied to the fields of detecting supernova explosions and searching for asteroids around the earth. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 is a flow chart of the processing method of the present invention;
[0023] Figure 2a The predicted image obtained in the embodiment of the present invention;
[0024] Figure 2b The real-time image obtained in the embodiment of the present invention;
[0025] Figure 2c A difference image between the instant image and the predicted image obtained by the method of the present invention;
[0026] Figure 3a It is an enhanced image after the brightness and darkness of the image changes;
[0027] Figure 3b This is the effect image after the image brightness changes using common technology processing;
[0028] Figure 3c This is a picture showing the effect of using the method of the present invention to process an image after the brightness and darkness of the image changes;
[0029] Figure 4a This is a street view image after camera shaking;
[0030] Figure 4b This is a street view of the target car after the camera shakes;
[0031] Figure 4c This is the result of using common techniques to process the camera shake.
[0032] Figure 4d The figure is a diagram showing the effect of using the method of the present invention to process camera vibration. DETAILED DESCRIPTION
[0033] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.
[0034] like Figure 1 As shown, a background processing method for highlighting image differences comprises the following steps:
[0035] Step 1) First obtain a predicted image;
[0036] Step 2) Perform a two-dimensional fast Fourier transform FFT2 on the predicted image to obtain the frequency spectrum of the predicted image, which is stored in a computer as known information;
[0037] Step 3) when the intensity of a certain pixel of the predicted image changes, the changed image is used as the instant image to obtain the instant image;
[0038] Step 4) Perform two-dimensional fast Fourier transform FFT2 on the real-time image to obtain the spectrum of the real-time image;
[0039] Step 5) Divide each matrix element value of the spectrum matrix of the instantaneous image by the corresponding matrix element value of the spectrum matrix of the predicted image to obtain the quotient of the spectrum of the instantaneous image and the predicted image;
[0040] Step 6) Perform two-dimensional inverse fast Fourier transform IFFT2 on the spectrum of the instantaneous image and the predicted image, and obtain an image matrix in the spatial domain, in which the value of the central matrix element is much higher than the values of other matrix elements;
[0041] Step 7) setting the central matrix element value of the image matrix obtained in step 6) to 0, performing a two-dimensional fast Fourier transform FFT2 on it, and performing a dot multiplication of the obtained spectrum matrix with the spectrum matrix of the predicted image in step 2) to obtain a spectrum matrix of the difference image;
[0042] Step 8) Perform two-dimensional inverse fast Fourier transform IFFT2 on the spectrum matrix of the difference image to obtain the difference between the instantaneous image and the predicted image, such as Figure 2c As shown, the difference between the two images is highlighted.
[0043] In this embodiment, the specific calculation process is as follows:
[0044] Image Functions The Fourier transform of is:
[0045] , where is the frequency spectrum of the image. Let’s compare the image functions taken at two different times. First, the function taken at the first moment as the predicted image , followed by the instantaneous image taken at the second moment when the intensity of a certain pixel of the image changes , their difference is:
[0046] , the spectra of the two image functions are and , the spectrum of their difference graph is , the method of the present invention mainly detects the difference between two images, so the spectra of the two images are mostly the same, and the intensity of the difference in the spectra is very small. Dividing the spectra of the two images can obtain:
[0047] , where the first term is a constant. After the quotient of the frequency spectra of the two images is subjected to two-dimensional inverse fast Fourier transform IFFT2, the image function is:
[0048] In this formula, a Dirac delta function appears. If we try to eliminate the Dirac delta function and then perform Fourier transform, the spectrum is:
[0049] , multiply this formula by the frequency spectrum of the predicted image , which is equivalent to filtering again, and then performing a two-dimensional inverse fast Fourier transform IFFT2, to obtain the difference between the two images. .
[0050] Meanwhile, in this embodiment, when the optical observation instrument monitors the changes of a certain scene, it is often required to compare the images of the same scene at different times, or to predict a certain image and find the difference between the instantaneous image of the scene at the time of observation and the predicted image. For example, the CCD camera used to record the image has a data file of the image. After being read into the computer, the color image is converted into a grayscale image. In this embodiment, Figure 2a As shown, the predicted image is taken from the space map taken by the Hubble telescope. There are a total of (NN, N=121) pixels, and the value of each pixel is represented by 8 bits. Figure 2a The data is presented as a two-dimensional matrix: , to reflect, first predict the image ( Figure 2a ) is transformed by FFT2 to obtain the frequency spectrum of the predicted image, which is stored in the computer as known information. Suppose that when the Hubble telescope is monitoring the space, an unidentified flying object appears, resulting in a bright spot in the observed scene, which causes the intensity of a certain pixel of the predicted image to change. The changed image is used as the real-time image, such as Figure 2b As shown, the instant image obtained is a two-dimensional matrix: , to reflect, it is transformed by FFT2 to obtain its spectrum, each matrix element value of the spectrum matrix of the instant image is divided by the corresponding matrix element value of the spectrum matrix of the predicted image, and the quotient of the two image spectra is transformed by IFFT2, and an image matrix is obtained in the spatial domain. The value of the central matrix element (1, 1) in the upper left corner is much higher than the values of other matrix elements. It is the characteristic of Dirac delta function. Let the value of this central matrix element be 0, and then after FFT2 transformation, the obtained spectrum matrix is multiplied with the spectrum matrix of the predicted image, eliminating the influence of the spectrum of the predicted image on the spectrum of the difference image. At this time, the spectrum matrix of the difference image is obtained, and then iFFT2 transformation is performed to obtain the difference between the two images, such as Figure 2c As shown, the difference between the two images is highlighted in this figure, with only one pixel being brighter.
[0051] The above steps can be used to process the image brightness change and camera shake in practice.
[0052] Take the post-processing of image brightness changes as an example: when a CCD camera records an image, the resulting image is the accumulation of light energy emitted by the object on the photosensitive element over a period of time: , where is the intensity of the recorded image, It is an instantaneous scene graph, which can better reflect the spatial distribution of objects at a certain moment. Due to different exposure times or shooting conditions, the intensity of the recorded image is different. Under the first-order approximation, the intensity of the recorded image is proportional to the time. Suppose the time of recording the predicted image is , the time to record the instant image is , then the intensity of the two images is:
[0053] ,
[0054] , where and It is an instantaneous scene graph, which reflects the distribution of objects that people are concerned about in space. The previous method of computer image subtraction is to subtract the intensity value of each pixel instead of the instantaneous scene. The result is:
[0055] ; In FIG3: a. Enhanced graph, b. Effect of conventional technical processing, c. Effect of the method of the present invention, wherein, Figure 3a Each pixel value is Figure 2a The image is increased by 10% to simulate the image after the exposure is increased, and the usual subtraction method is used to convert Figure 3a and Figure 2a Subtracting them, the result is Figure 3b It can be seen that the result of image subtraction is affected by the exposure time, the signal is still affected by the same part of the image, and it is still difficult to observe the signal. The instantaneous scene image subtraction is the ideal result of image subtraction. In the filtering method image subtraction, the intensity of the recorded image is Fourier transformed, and after the spectrum is divided, it can be obtained:
[0056] From this formula, we can get that the quotient of the spectra of two images with different exposure times is the quotient of the instantaneous scene image spectrum multiplied by a constant coefficient. In the quotient of the spectrum, the influence of exposure time is eliminated. Continuing to use the previous method to subtract the images, the result is the difference of the instantaneous scene. ,like Figure 3c As shown, Figure 3b and Figure 3cThe images are all displayed with the image intensity magnified 8 times. It can be seen that the influence of the same part of the image on the result is greatly reduced, which is the ideal result. The main purpose of filtering image subtraction is to stop the influence of the same part of the image information on naked eye observation, measurement, and calculation. If the changing part of the image is regarded as a signal and the same part of the image is regarded as useless noise, the signal-to-noise ratio of the image before and after subtraction can be compared. Figure 3a The changing part is a pixel of the image, the pixel value is 1, and it is buried in the noise. Without image subtraction, the signal-to-noise ratio can be expressed as:
[0057] ,
[0058] The method used in this embodiment Figure 2a The signal-to-noise ratio is 6.13. After subtraction, the image signal-to-noise ratio is calculated as:
[0059] ,
[0060] Subtracting in the usual way gives Figure 3b The signal-to-noise ratio is 61.0. If the difference in exposure increases, the signal-to-noise ratio will drop further. The image is subtracted using the filtering method. Figure 3c The signal-to-noise ratio is 1.78X10 3 Moreover, it is not affected by the difference in exposure amount. It can be seen that the image signal-to-noise ratio is improved. Therefore, the method of the present invention has a significant effect.
[0061] Similarly, taking the processing after camera shaking as an example, in FIG4: a street scene, b the target car appears, c the effect of conventional technical processing, d the effect of the processing method of the present invention, wherein, Figure 4a and Figure 4b Due to the influence of camera vibration, the image as a whole shifts relatively, and the background cannot be overlapped and eliminated, resulting in Figure 4c The result of subtraction using conventional techniques is affected by image translation, and the background cannot be completely eliminated. Figure 4d In the example, the background is completely eliminated and is not affected by camera shake. Similarly, if the image is not aligned when processing the image, the background can be completely eliminated by applying the method of the present application.
[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A background processing method for highlighting image differences, characterized in that: The method comprises the following steps: Step 1) First obtain a predicted image; Step 2) Perform a two-dimensional fast Fourier transform FFT2 on the predicted image to obtain the frequency spectrum of the predicted image, which is stored in a computer as known information; Step 3) when the intensity of a certain pixel of the predicted image changes, the changed image is used as an instant image to obtain the instant image; Step 4) Perform two-dimensional fast Fourier transform FFT2 on the real-time image to obtain the spectrum of the real-time image; Step 5) Divide each matrix element value of the spectrum matrix of the instantaneous image by the corresponding matrix element value of the spectrum matrix of the predicted image to obtain the quotient of the spectrum of the instantaneous image and the predicted image; Step 6) Perform two-dimensional inverse fast Fourier transform IFFT2 on the spectrum of the instantaneous image and the predicted image, and obtain an image matrix in the spatial domain, in which the value of the central matrix element is much higher than the values of other matrix elements; Step 7) setting the central matrix element value of the image matrix obtained in step 6) to 0, performing a two-dimensional fast Fourier transform FFT2 on it, and performing a dot multiplication of the obtained spectrum matrix with the spectrum matrix of the predicted image in step 2) to obtain a spectrum matrix of the difference image; Step 8) Perform two-dimensional inverse fast Fourier transform IFFT2 on the spectrum matrix of the difference image to obtain the difference between the instant image and the predicted image, highlighting the difference between the two images.
2. The background processing method for highlighting image differences according to claim 1, characterized in that: The predicted image and the instantaneous image both use NN pixels, and the data of the predicted image uses a two-dimensional matrix: , to reflect, the instant image uses a two-dimensional matrix: , to reflect.
3. The background processing method for highlighting image differences according to claim 2, characterized in that: The value of N is 121. After being read into the computer, the value of each pixel is represented by 8 bits.
4. The background processing method for highlighting image differences according to claim 3, characterized in that: In step 6), the Dirac delta function is used to make the value of the central matrix element (1, 1) at the upper left corner of the image matrix obtained in the spatial domain much higher than the values of other matrix elements.
5. The background processing method for highlighting image differences according to claim 1, characterized in that: In step 1) and step 3), a CCD camera is used to record images to obtain image data files.
