A method, acquisition and restoration device and monitoring system for restoring flutter-blurred images

Abstract

This invention discloses a method, acquisition and restoration device, and monitoring system for restoring flutter-blurred images. Based on structural similarity and entropy, the steps include: acquiring the flutter-blurred image; constructing a blur kernel using a preset binary encoding sequence and blur length; calculating the structural similarity index between the flutter-blurred image and all decoded clear images; determining the image information entropy value within a certain range near the image with the highest structural similarity; calculating the information entropy value of the reconstructed image; taking the minimum information entropy value as the blur length of the most ordered reconstructed decoded image; and using an inverse filtering method to achieve clear image restoration. This invention aims to completely protect the target image information and considers methods for removing flutter-blurred targets from both imaging and post-processing perspectives. It utilizes precoding to modulate the exposure process, preserving the original high-frequency information in the acquired image and decoding it during subsequent image restoration to obtain a clear image.

Description

Technical Field

[0001] This invention relates to the field of image restoration technology, and in particular to a method, acquisition and restoration device and monitoring system for restoring flutter-blurred images. Background Technology

[0002] With the development of digital storage media, consumer electronic devices such as digital cameras and smartphones have become the main sources of emerging image information acquisition. Image clarity is one of the primary factors ensuring the reliability and effectiveness of acquired information. In today's era of widespread digital cameras, image information is acquired anytime, anywhere, and each acquired image contains the characteristics of the target. An image is ideal when it accurately reflects the characteristics of the target area. When an acquired image is unclear or fails to reflect the specific characteristics of the target area at that time, it indicates the loss of effective information, resulting in image degradation. For example, video surveillance images differ significantly between day and night lighting, aerial and remote sensing images are easily affected by atmospheric conditions, and underwater imaging is subject to multipath interference such as refraction. Therefore, when image degradation occurs, effective information is lost or erroneous, severely impacting image quality. Protecting the original information of the image is a pressing issue that needs to be addressed in both front-end image acquisition and back-end computer image processing.

[0003] Encoded exposure imaging, proposed by Raskar et al. in 2006, is a computational imaging technique. Its core idea is to control the opening and closing of the camera shutter by pre-setting a specific binary encoded sequence during the camera exposure process. Compared to traditional camera exposure modes, this is equivalent to setting a broadband filter in the time domain. From a frequency domain perspective, this allows for the retention of as much mid-to-high frequency information as possible during image acquisition, eliminating zero-point portions in the frequency domain and achieving reversible restoration, significantly improving the ill-conditioned problems of blurred image restoration.

[0004] Image degradation is a comprehensive manifestation of image blurring, distortion, noise, etc., resulting in the loss of image information. The causes of image degradation are multifaceted; generally, images undergo processes such as acquisition, imaging, transmission, and storage, and each stage can potentially contribute to image degradation. Motion blur is a common problem encountered in optical imaging. It occurs because the relative displacement between the object being captured and the camera during camera exposure causes motion blur in the acquired image, reducing image resolution and significantly impacting image quality. Motion blur image restoration technology, without re-acquiring the target scene, utilizes existing motion-blurred images. Through modeling the imaging physical process and mathematical solutions, it restores the blurred image to a clear one, offering significant applications in civilian and military fields.

[0005] Chinese patent CN110097509B utilizes an coded exposure mode for blurred image acquisition, employs background subtraction for initial target extraction, and then integrates the motion blur superposition characteristics of coded exposure with prior motion information to achieve accurate extraction of the motion-blurred target region. Combined with the Student-T restoration algorithm, it performs accurate PSF estimation and restoration reconstruction. After 2-3 iterations, the restoration result is usually obtained. This invention requires input of a scene background image and a locally blurred image of the moving target. Chinese patent CN202011448781.2 uses deep learning to estimate the blur kernel, solving the problem of difficulty in estimating the blur kernel in traditional methods. It uses a genetic algorithm to search for the image block with the highest high-frequency information content. However, it does not solve the matching problem between the coded image and the motion blur length. Since the accuracy of blur length estimation plays a crucial role in image restoration and reconstruction, this invention utilizes a method based on joint estimation of blur length using image structural similarity (SSIM) and image information entropy (Entropy) to restore coded exposure images. Summary of the Invention

[0006] To effectively solve the above technical problems, this invention, based on structural similarity and entropy, adopts the following technical solution: a flutter blur image restoration method.

[0007] Step 1: Acquisition of flutter blur image;

[0008] Step 2: By setting the preset encoding length as binary encoded sequence and fuzzy length Constructing fuzzy kernels The binary encoded sequence The length, using the formula The reconstructed image is obtained by calculation, where For flutter blur image, To decode the image;

[0009] Step 3: Calculate the flutter blur image and all decoded images Structural similarity index Fuzzy length when searching for the maximum value The formula is: In the formula The search range is for a fuzzy length;

[0010] Step 4: Determine the image information entropy value within a certain range near the image with the highest structural similarity. The search interval is... ,in, are positive integers and , For the reason The defined total fuzzy length limit range, in Within the limits, according to the formula and Calculate the spatial entropy value of each decoded image. and the spectral entropy value after image DCT transformation (discrete cosine transform) In the formula, ,in For characteristic binary pairs Frequency of occurrence Represents the grayscale value of a pixel. This represents the average gray level of the neighborhood. The scale of the image; if the signal The DCT transform is , These are generalized frequency domain variables, and the normalized DCT transform coefficients are: ;

[0011] Step 5: Formula Calculate the information entropy value of the reconstructed image In the formula, Image spatial entropy value Weighting The spectral entropy value of the DCT (Discrete Cosine Transform) coefficients of the image. Weighting;

[0012] Step Six: Take The minimum value is determined as the most ordered reconstructed decoded image. fuzzy length The formula is ;

[0013] Step 7: Use inverse filtering to achieve clear image restoration. The formula is as follows: In the formula, To reconstruct the decoded image in the most ordered way, For flutter blur image, For fuzzy kernel.

[0014] Furthermore, in step one, acquiring the flutter blur image involves setting the exposure time. Exposure time Divide into equal parts Each time slot The number of time slots (i.e., the coding length), with each time slot lasting for [duration]. Each time slot After the individual charge outputs are collected, they are then superimposed to form a flutter blur image. The total image acquisition time required is [time value missing]. .

[0015] Furthermore, in step one, setting the exposure time... Exposure time Divide into equal parts There are 1 time slot, and the time of each time slot is 1. , The number of time slots determines the number of times the outward drive transmission process is completed in a single operation. .

[0016] Furthermore, in step two, a preset length of... binary encoded sequence and fuzzy length Constructing fuzzy kernels The method involves transforming the image convolution operation into a multiplication operation. To avoid altering the frequency information of the original signal, a zero vector of a specific length is added to the end of the original signal. An improved mathematical model is established to represent the relative displacement of the acquired image in the spatial domain, which is equivalent to the length of the blurred pixels projected onto the image plane by the actual target. Greater than the pre-coded length It is necessary to encode the sequence Add several zeros to make it equal to the length of the blurred pixels, forming a shape with a length of [missing information]. Flutter blur image In the formula The length of the blur after zero padding. For flutter blur image The length in the direction of movement; at this point, the fuzzy kernel used for reconstruction. Based on the fuzzy length after zero-padding The Toeplitz matrix is ​​formed by shifting in a single direction.

[0017] On the other hand, the present invention also provides a device for acquiring and restoring flutter-blurred images, using a flutter-blurred image restoration method to achieve flutter-blurred image acquisition and restoration, including: an image acquisition module, a core control module, a communication module, a storage module, a clock module, and a transmission and display module; the image acquisition module consists of a CCD image sensor, a timing drive circuit, and a signal conditioning and conversion circuit; the timing drive circuit is divided into a horizontal timing drive module and a vertical timing drive module, which provides a suitable drive level for the CCD image sensor; the signal conditioning and conversion circuit includes analog signal amplification and filtering and analog-to-digital conversion; a pre-coded signal is sent to the image acquisition module through the core control module to acquire the flutter-blurred image of motion; the image data and the pre-coded signal are stored in the storage module, and the entire circuit is protected by the clock module and the communication module; the CCD image sensor includes a horizontal shift register and a vertical shift register.

[0018] Furthermore, the core control module includes a timing module, a decoding and reconstruction module, an image reading module, a DDR control module, an FPGA gigabit network core, an exposure encoder, and a reference clock.

[0019] Furthermore, the driving level is a three-state level.

[0020] Furthermore, the image acquisition module also includes a laser generator and a laser receiver. The CCD image sensor receives the laser emitted by the laser generator through the laser receiver for automatic focusing, thereby solving the image blurring problem caused by inaccurate focusing.

[0021] Furthermore, the image acquisition module also includes an ultrasonic generator and an ultrasonic receiver. The CCD image sensor receives the ultrasonic waves emitted by the ultrasonic generator through the ultrasonic receiver for automatic focusing, thereby solving the image blurring problem caused by inaccurate focusing.

[0022] Furthermore, the image acquisition module also includes an infrared generator and an infrared receiver. The CCD image sensor receives the infrared light emitted by the infrared generator through the infrared receiver for automatic focusing, thereby solving the image blurring problem caused by inaccurate focusing.

[0023] Furthermore, a flutter blur image acquisition and restoration device also includes a three-dimensional model and a visualization module. The image acquisition module is connected to the three-dimensional model and the visualization module, and the image is dynamically displayed in three-dimensional visualization through the three-dimensional model and the visualization module.

[0024] Furthermore, the core control module also includes a target tracking module, which, when multiple flutter blur images are acquired, designates a specific flutter blur image as the target image.

[0025] On the other hand, the present invention also provides a monitoring system using several flutter blur image acquisition and restoration devices. The monitoring system further includes a remote data transmission module and a blockchain module. The blockchain module is divided into monitoring nodes and management nodes. The remote data transmission module is installed on each of the several flutter blur image acquisition and restoration devices, forming several monitoring nodes. Each monitoring node reconstructs and decodes the most ordered image generated by its own node. Add a timestamp, and simultaneously generate a public and private key pair. Reconstruct the decoded image from the most ordered set of timestamped data. The data is uploaded to the blockchain module, digitally signed using the private key, and the public key is sent to the management node. The management node is a central control system with the remote data transmission module installed. The management node receives the public key from the monitoring node to verify the private key and view the most ordered reconstructed decoded image. .

[0026] The beneficial effects of this invention are:

[0027] 1. This invention discloses a coded exposure imaging restoration method for blurring caused by target jitter in a single direction of motion relative to a camera.

[0028] 2. This invention aims to completely protect the target image information and is a method for removing blur from moving targets by simultaneously considering both imaging and post-processing. It utilizes precoding to modulate the exposure process, preserving the original high-frequency information in the acquired image, and then decoding it during subsequent image restoration to obtain a clear image.

[0029] 3. The method of this invention eliminates the time required for manual selection or the use of external measurement equipment, and breaks the image limitations imposed by natural image patterns. By utilizing a joint image restoration algorithm based on image structural similarity and entropy, the complexity of the equipment used in existing technologies is significantly reduced.

[0030] 4. The image acquisition and restoration device disclosed in this invention can solve the problem of restoring motion-blurred images, achieve clear restored images from high frame rate cameras at a lower cost, and, using no-reference image evaluation index testing, the average improvement of relevant indicators compared to normal exposure is about twice the value of relevant indicators of normal exposure.

[0031] 5. The monitoring system of the present invention uploads image information to the blockchain module through the remote data transmission and control module, and utilizes the advantages of blockchain technology such as decentralization, encryption and immutability, so that it can be widely used in industries such as traffic monitoring. Attached Figure Description

[0032] To better illustrate the technical solution of this invention, the following is a description of the invention with accompanying drawings:

[0033] Figure 1 This is a schematic diagram of the image structure similarity structure in Example 1;

[0034] Figure 2 This is a diagram showing the blur kernel construction of the blurred image from the pre-coded exposure in Example 1;

[0035] Figure 3 This is a schematic diagram illustrating the relationship between one-dimensional relative motion and convolution calculation in Example 1;

[0036] Figure 4 This is a diagram illustrating the flutter blur image acquisition and reconstruction decoding process in Example 1.

[0037] Figure 5 This is a structural diagram of the flutter blur image acquisition and restoration device in Example 3;

[0038] Figure 6This is a flowchart of the encoding and decoding program for the flutter blur image in Example 3;

[0039] Figure 7 This is a comparison chart of the experimental results of the present invention;

[0040] The following are the reference numerals: 1. Image acquisition module; 11. CCD image sensor; 12. Timing drive circuit; 13. Signal conditioning and conversion circuit; 2. Core control module; 21. Timing module; 22. Decoding and reconstruction module; 23. Image reading module; 24. DDR control module; 25. FPGA gigabit network core module; 26. Exposure encoding module; 27. Reference clock; 3. Communication module; 4. Storage module; 5. Clock module. Detailed Implementation

[0041] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0042] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0043] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0044] The working principle of this invention is as follows: a blur kernel is constructed using the Toeplitz matrix, and the image restoration problem under a single direction of motion of the target and the camera is solved based on the construction form of the blur kernel.

[0045] Encoded exposure will be a complete exposure time Divided into The time slots are evenly spaced, with the number of time slots matching the preset coding length. Whether each time slot is exposed corresponds to the codeword in the corresponding bit. Taking a one-dimensional signal as an example, if the acquisition length is... One-dimensional clear signal Then the signal has a length of Linear motion fuzzy kernel Convolution of (i.e., binary encoded sequences) can be represented as:

[0046]

[0047] From the perspective of signal relative relationships, the above formula is relative to a one-dimensional signal. Movement in one direction causes data misalignment and superposition. This formula can be understood as using the Toeplitz matrix to separate clear one-dimensional signals. With fuzzy kernel The matrix convolution relationship between them is transformed into a matrix multiplication relationship. If Each of the data in has When, it refers to the traditional overlay of continuous data; while when The data in is or When, it represents the superposition of discontinuous or discontinuous data, where .

[0048] From the perspective of shutter speed exposure, the difference between coded exposure and normal exposure is that the shutter does not remain open for a single period, but rather its opening and closing are controlled by a pre-set code at a certain frequency. That is, if... This indicates that the exposure occurred in that time slot; conversely, if... This indicates that the time slot will not be exposed.

[0049] like Figure 3 It establishes a static binary code sequence One-dimensional clear signal with time-division misalignment The mathematical model for image acquisition is presented. Taking the one-dimensional relative motion between the imaging system and the target object as an example, the coded exposure imaging and image restoration process is explained. If the image is blurred... The length in the direction of movement is One-dimensional clear signal Exposure is performed using an imaging system, and the exposure duration of this imaging system is related to whether the exposure is performed. The preset binary encoding sequence Maintain consistency. Figure 3 Lieutenant General The default encoding is set to If the two signals move relative to each other, it is equivalent to a sampling length of... signal This signal represents a convolutional model between the two. When At that time, Over time, it has accumulated The information in; and when hour, There was no new information. Therefore, the signal was collected. Does each of them accumulate with Whether it is 1 or not is related. During the restoration process, time-division misalignment encoding is utilized. Restore a relatively stationary one-dimensional signal The above-described acquisition, encoding, exposure, image reconstruction, and decoding process is as follows: Figure 4 As shown.

[0050] The same exposure encoding and different blur lengths can create different blur kernels, resulting in varying qualities of the reconstructed decoded images. To find a clear restored image among numerous reconstructed decoded images, structural similarity, an image quality evaluation function, is used as a comparison criterion to achieve high-quality image reconstruction. The system block diagram is as follows: Figure 1 As shown.

[0051] Figure 1 middle, A flutter-blurred image indicating flutter degradation. This represents the original, clear image. Similar to human visual perception, structural similarity will be determined from brightness... Contrast Structural The structural similarity between two images can be independently determined from three perspectives: [the three quantities are listed here, but are not translated here].

[0052] , , ;

[0053] The formula contains,

[0054] , , , , , , For flutter blur image and original clear image The average value of pixels; For flutter blur image and original clear image Their respective standard deviations; For flutter blur image and original clear image The cross-correlation function; to avoid the case where the numerator and denominator are zero, three very small positive numbers are defined, such as... , , In this embodiment, This refers to the range of pixel values ​​for a grayscale image. For example, an 8-bit image is a grayscale image. . , This is the default value.

[0055] Then the flutter blur image and original clear image The similarity function between them can be expressed as:

[0056]

[0057] In this formula, the parameters To adjust , , The proportions of the three parameters, when Sometimes, This formula represents the original sharp image. and flutter blur image Structural similarity. The range of values ​​is . The larger the value, the more similar the quality of the two images.

[0058] However, in actual experiments, clear images cannot be obtained; the images are blurred due to flutter. This is the only available data, and corresponding reconstructed images can be obtained through different blur lengths. Since the acquired coded exposure-blurred image originates from the target object, similarity evaluation can be performed between the reconstructed image and the coded exposure-blurred image. Therefore, the flutter-blurred image... and encoding, exposure, decoding images The structural similarity index is defined as However, flutter blurs the image. It is a degraded image, used to evaluate the decoded image. This can cause deviations in the structural similarity index, so it is also necessary to calculate the orderliness of the image to finally determine the restored image.

[0059] Structural similarity methods can only find the reconstructed image most similar to the coded, blurred image, but this image is not necessarily the most ordered image conforming to natural statistical laws. The original sharp image should generally be an ordered, natural image; therefore, orderliness should also be used as an indicator of image quality in the reconstructed image. To avoid the bias caused by using structural similarity alone, information entropy is introduced. Entropy is a measure of the amount of information. By combining structural similarity to determine the range of images most similar to the blurred image, and then finding the most ordered image information, the final restored image is determined.

[0060] The more ordered a system is, the lower its information entropy; conversely, the more chaotic the system, the higher its information entropy. Here, spatial entropy is used to calculate the degree of order among grayscale values ​​of image pixels within the search interval. Simultaneously, to evaluate the flatness of the reconstructed decoded signal, discrete cosine transform is used to transform the signal to the frequency domain and calculate its spectral entropy. The optimal blur length is estimated by adjusting the weights of spatial entropy and spectral entropy, thereby reconstructing and restoring the clear image. The formula for spatial entropy, which characterizes the degree of orderliness of an image, is: In the formula, Represents the maximum value within the grayscale range of the image; Represents the grayscale in the reconstructed image The possibility of this is unknown. This one-dimensional image entropy cannot reflect the spatial characteristics of the image's gray-level distribution; therefore, a two-dimensional image entropy needs to be introduced. Here, we use the spatial distribution formula of the neighborhood gray-level mean of the feature image as an example:

[0061] In the formula ,in For characteristic binary pairs Frequency of occurrence Represents the grayscale value of a pixel. This represents the average gray level of the neighborhood. The scale of the image.

[0062] To obtain an accurate solution, spectral entropy is introduced to detect the flatness of the signal spectrum. When a signal exhibits strong correlation in the spatial domain, its transformation to the frequency domain reveals a concentration in specific regions. Here, the Discrete Cosine Transform (DCT) is used to transform the spatial domain signal to the frequency domain. Since image pixel values ​​are real numbers, its DCT also involves real number operations, making it faster than the complex number operations in the Fourier Transform. If the signal... The DCT transform is , If the variable is a generalized frequency domain variable, then the normalized DCT coefficients are: Its spectral entropy can be expressed as .

[0063] Example 1 Figure 1-4 A method for restoring dizzy blurred images:

[0064] Step 1: Acquiring flutter-blurred images using CCD image sensor 11, setting the exposure time. Exposure time Divide into equal parts There are 1 time slot, and the time of each time slot is 1. , The number of time slots, each time slot After the individual charge outputs are collected, they are then superimposed to form a flutter blur image. The total image acquisition time required is [time value missing]. .

[0065] Step 2: By setting the preset encoding length as binary encoded sequence and fuzzy length Constructing fuzzy kernels The binary encoded sequence The length, using the formula The reconstructed image is obtained by calculation, where For flutter blur image, To decode clear images;

[0066] like Figure 2 By setting a preset length binary encoded sequence and fuzzy length Constructing fuzzy kernels The method involves transforming the image convolution operation into a multiplication operation and adjusting the blur kernel... By shifting the matrix diagonally downwards and to the right, a matrix similar to the Toeplitz matrix is ​​formed, such as... Figure 2 As shown in (a), the blur length of the target projected onto the image plane is equal to the coding length. However, in reality, it is difficult to guarantee that the blur length of the target projected onto the image plane is equal to the coding length. If the actual blur pixel length of the target projected onto the image plane is large, a clear restored image cannot be obtained using the above model.

[0067] In order to avoid altering the original one-dimensional signal Frequency information in the original one-dimensional signal A zero vector of a specific length was added to the tail. An improved mathematical model of the relative displacement of the acquired images in the spatial domain is established, such as... Figure 2 As shown in (b), this situation is equivalent to the length of the blurred pixels projected onto the image plane by the actual target being greater than the pre-coded length. It is necessary to pad the encoding with several zeros to make it equal to the length of the blurred pixels, forming a length of... Flutter blur image In the formula The length of the blur after zero padding. For flutter blur image The length in the direction of movement; at this point, the fuzzy kernel used for reconstruction. Based on the fuzzy length after zero-padding The shifts form a Toeplitz matrix.

[0068] One-dimensional signal and The process of relative shifting and superposition is equivalent to the convolution process of signals. When there is a jitter-blurred image... The length in the direction of movement is One-dimensional signal When displacement occurs, it interacts with another one-dimensional signal of length m. The convolution process will produce a length of Flutter blur image If at this time If the sequence is a binary encoded sequence, then one-dimensional signals will be selectively superimposed according to the encoding rules. The information in the image. Therefore, the imaging process of coded exposure is similar to the cumulative trailing effect during ordinary exposure, which is the accumulation of images exposed within multiple coded time slots.

[0069] Step 3: Calculate the flutter blur image And all decoded clear images Structural similarity index Fuzzy length when searching for the maximum value The formula is: In the formula The search range is for a fuzzy length;

[0070] Step 4: Determine the image information entropy value within a certain range near the image with the highest structural similarity. The search interval is... ,in, are positive integers and , For the reason The defined total fuzzy length limit range, in Within the limits, according to the formula and Calculate the spatial entropy value of each decoded image. and the spectral entropy value after image DCT transformation (discrete cosine transform) In the formula, ,in For characteristic binary pairs Frequency of occurrence Represents the grayscale value of a pixel. This represents the average gray level of the neighborhood. The scale of the image; if the signal The DCT transform is , These are generalized frequency domain variables, and the normalized DCT transform coefficients are: ;

[0071] Step 5: Formula Calculate the information entropy value of the reconstructed image In the formula, Image spatial entropy value Weighting The spectral entropy value of the DCT (Discrete Cosine Transform) coefficients of the image. Weighting;

[0072] Step Six: Take The minimum value is determined as the most ordered reconstructed decoded image. fuzzy length The formula is ;

[0073] Step 7: Use inverse filtering to achieve clear image restoration. The formula is as follows: In the formula, To reconstruct the decoded image in the most ordered way, For flutter blur image, For fuzzy kernel.

[0074] The difference between Example 2 and Example 1 is that the exposure time is set in step one. Exposure time Divide into equal parts There are 1 time slot, and the time of each time slot is 1. , The number of time slots determines the number of times the outward drive transmission process is completed in a single operation. .

[0075] This method uses only one charge-driven transfer time. This method also reduces the probability of electronic noise. The total exposure time remains consistent with the normal exposure time. That is, under the same conditions, the pixel displacement of the target and camera projection in the image plane is consistent with that of a normal camera. This method advances the "motion blur overlay" process, transforming the motion process into a charge accumulation process that conforms to a preset coding rule. Since the time slots are equal to the total exposure time, the image acquisition interval is the smallest among the above schemes, meaning that under the same conditions, the blur length is minimized, facilitating the restoration and reconstruction of a clear image.

[0076] Example 3: A flutter blur image acquisition and restoration device, such as... Figure 5-6 .like Figure 5The system comprises an image acquisition module 1, a core control module 2, a communication module 3, a storage module 4, a clock module 5, a 3D model and visualization module, and a transmission and display module (not shown in the figure). The image acquisition module 1 consists of a CCD image sensor 11, a timing drive circuit 12, and a signal conditioning and conversion circuit 13. The timing drive circuit 2 is divided into a horizontal timing drive module and a vertical timing drive module, providing a suitable drive level for the CCD image sensor 11; the drive level is a three-state level. The signal conditioning and conversion circuit 13 includes analog signal amplification and filtering, and analog-to-digital conversion. The CCD image sensor 11 includes a horizontal shift register and a vertical shift register (not shown in the figure). The core control module 2 includes a timing module 21, a decoding and reconstruction module 22, an image reading module 23, a DDR control module 24, an FPGA gigabit network core module 25, an exposure encoding module 26, and a reference clock 27. The image acquisition module 1 uses a 3D model and visualization module to dynamically display the image in 3D visualization. The pre-coded image is sent to the image acquisition module 1 via the core control module 2, which then acquires the motion flutter blur image. The image data and the pre-coded image are stored in the storage module 4. The entire circuit is protected by the clock module 5 and the communication module 3.

[0077] To resolve image blurring caused by inaccurate focusing, the image acquisition module 1 can use a generator and a receiver. The CCD image sensor 11 receives the signal emitted by the generator through the receiver for automatic focusing. The generator and receiver can be any commercially available product, including but not limited to laser generators and laser receivers, ultrasonic generators and ultrasonic receivers, and infrared generators and infrared receivers.

[0078] The core control module 2 can be implemented using the Xilinx Spartan 6 series XC6SLX45T-3FG484C; the reference clock 27 is selected from... The CCD image sensor 11 is driven by a crystal oscillator. The timing sequence required for driving the CCD image sensor 11 is generated by the timing module 21, and the timing drive signals are generated by the core processing module (in this embodiment, a field-programmable logic device (FPGA) is used). Since driving the CCD image sensor 11 requires both two-level and three-level signals, a dedicated chip, CXD3400N, is selected. The image reading module 23 reads image data and stores it in the DDR data memory 24 via the FPGA's DDR management core. The decoding and reconstruction module 22 in the core control module 2 is reserved; it can read the jitter-blurred image from the DDR data memory 24 and then use relevant restoration methods to perform on-chip decoding and reconstruction of a clear restored image. The FPGA gigabit network core module 25, used for data communication with the host computer, is a high-speed bidirectional transmission channel that reliably transmits image data and status commands. The physical layer transceiver chip is a physical layer gigabit Ethernet transceiver 88E1111; the CCD image sensor 11 uses a high-sensitivity, low-noise ICX204AL black and white image sensor; the signal conditioning and conversion circuit 13 uses the AD9949 chip. Other modules are all commercially available general-purpose components.

[0079] like Figure 6 Flowchart of the encoding and decoding program for flutter-blurred images. Preset binary encoding sequence. It is also used as a blur kernel in the decoding module when providing control signals for shutter changes to the substrate SUB. The construction of the image. Therefore, when the flutter blur image conforms to the preset encoding. After being captured, the decoding module uses an inverse filtering method. Obtain a clear decoded image. Meanwhile, as an optional module in the embedded coded camera, blurred images can also be restored by transmitting them to a host computer for processing or by downloading complex reconstruction methods.

[0080] Example 4: A monitoring system using several flutter-blurred image acquisition and restoration devices. The monitoring system also includes a remote data transmission module and a blockchain module. The blockchain module is divided into monitoring nodes and management nodes. Each of the several flutter-blurred image acquisition and restoration devices is equipped with the remote data transmission module, becoming several monitoring nodes. Each monitoring node reconstructs and decodes the most ordered image generated by its own node. Add a timestamp, and simultaneously generate a public and private key pair. Reconstruct the decoded image from the most ordered set of timestamped data. The data is uploaded to the blockchain module, digitally signed using the private key, and the public key is sent to the management node. The management node is a central control system with the remote data transmission module installed. The management node receives the public key from the monitoring node to verify the private key and view the most ordered reconstructed decoded image. .

[0081] Figure 7 The experimental results using other methods of the prior art are compared with the experimental results using the method and apparatus of the present invention.

[0082] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for restoring flutter-blurred images, characterized in that, include Step 1: Acquisition of flutter blur image; Step 2: Construct a fuzzy kernel K using a binary encoded sequence k of preset encoding length m and a fuzzy length r, where m is the length of the binary encoded sequence k. Then, use formula L... range =B / K is used to calculate the reconstructed image, where B is the flutter blur image and L is the reconstructed image. range To decode the image; Step 3: Calculate the flutter blurred image B and all decoded images L range Structural similarity index S B,L The fuzzy length r when searching for the maximum value s The formula is: r s =argmax r∈R (S B,L (r)), where R is the search range of the fuzzy length; Step 4: Determine the image information entropy value within a certain range near the image with the highest structural similarity; the search interval is r. s-opt ∈[r s -q,r s +q], where q is a positive integer and q≤r s ,r s-opt For S B,L The defined total fuzzy length limit range, in r s-opt Within the limits, according to the formula and Calculate the spatial entropy value H of each decoded image. s and the spectral entropy value H after image DCT transformation (discrete cosine transform) f In the formula, P ij =f ij / S 2 , where f ij Let f be the frequency of occurrence of the feature pair ij, where i represents the gray value of a pixel, j represents the average gray value of its neighborhood, and S is the image scale; if signal f c The DCT transform of (x,y) is u, v are generalized frequency domain variables, and the normalized DCT transform coefficients are: Step 5: Formula H(r) s-opt )=σ s H s (r s-opt )+σ f H f (r s-opt Calculate the information entropy value H(r) of the reconstructed image. s-opt ), where σ s Image spatial entropy value H s Weight, σ f H is the spectral entropy value of the DCT (Discrete Cosine Transform) coefficients of the image. f Weighting; Step Six: Take H(r) s-opt The minimum value is determined as the most ordered reconstructed decoded image L. opt fuzzy length r opt The formula is Step 7: Use inverse filtering to achieve clear image restoration, the formula is L opt =B / K, where L opt For the most ordered reconstructed decoded image, B is the flutter blur image, and K is the blur kernel.

2. The method for restoring a flutter-blurred image according to claim 1, characterized in that, In step one, acquiring the flutter blur image involves setting the exposure time t. b Exposure time t b Divide the time into m equal time slots, with each time slot having a duration of Δt = t. b / m, after separately outputting the charge acquired in each time slot Δt, they are then superimposed to form a flutter blur image. The total image acquisition time required is t=m(Δt+t p )=t b +mt p .

3. The method for restoring flutter-blurred images according to claim 1, characterized in that, In step one, setting the exposure time t b Exposure time t b Divide the time into m equal time slots, with each time slot having a duration of Δt = t. b / m, the outward drive transmission process is completed only once, i.e., t=mΔt+t p =t b +t p .

4. The method for restoring a flutter-blurred image according to claim 1, characterized in that, In step two, a blur kernel K is constructed using a binary encoding sequence k of preset length m and a blur length r. The method involves transforming the image convolution operation into a product operation, adding a zero vector 0 of a specific length to the end of the original signal, establishing an improved mathematical model of the relative displacement of the acquired image in the spatial domain, and forming a flutter blur image B of length (n+r-1), where r is the blur length after zero-padding and n is the length of the flutter blur image B in the direction of movement. At this point, the blur kernel K used for reconstruction is shifted according to the blur length r after zero-padding to form a Toeplitz matrix.

5. A flutter blur image acquisition and restoration device, characterized in that, The method for restoring flutter-blurred images according to any one of claims 1-4 is used to acquire flutter-blurred images, comprising: an image acquisition module, a core control module, a communication module, a storage module, a clock module, and a transmission and display module; the image acquisition module consists of a CCD image sensor, a timing drive circuit, and a signal conditioning and conversion circuit; the timing drive circuit is divided into a horizontal timing drive module and a vertical timing drive module to provide a suitable drive level for the CCD image sensor; the signal conditioning and conversion circuit includes analog signal amplification and filtering and analog-to-digital conversion; the CCD image sensor includes a horizontal shift register and a vertical shift register.

6. The flutter blur image acquisition and restoration device according to claim 5, characterized in that, The core control module includes a timing module, a decoding and reconstruction module, an image reading module, a DDR control module, an FPGA gigabit network core, an exposure encoder, and a reference clock.

7. The flutter blur image acquisition and restoration device according to claim 5, characterized in that, The driving level is a three-state level.

8. The flutter blur image acquisition and restoration device according to claim 5, characterized in that, It also includes a 3D model and a visualization module, with the image acquisition module connected to the 3D model and the visualization module.

9. The flutter blur image acquisition and restoration device according to claim 5, characterized in that, The core control module also includes a target tracking module.

10. A monitoring system, characterized in that, Using the flutter blur image restoration and acquisition device according to any one of claims 5-9, the monitoring system further includes a remote data transmission module and a blockchain module. The blockchain module is divided into monitoring nodes and management nodes. The remote data transmission module is installed on several of the flutter blur image acquisition and restoration devices, forming several monitoring nodes. Each monitoring node transmits the most ordered reconstructed decoded image L generated by its own node. opt Add a timestamp, and simultaneously generate a public and private key pair. Then, use the timestamped, most ordered reconstructed decoded image L. opt Uploaded to the blockchain module, digitally signed using the private key, and the public key sent to the management node; The management node is a central control system equipped with the remote data transmission module. The management node receives the public key from the monitoring node to verify the private key and view the most ordered reconstructed decoded image L. opt .

Images