A displacement image processing method and system based on optical sensors

Abstract

This invention discloses a displacement image processing method and system based on an optical sensor, belonging to the field of image processing technology. It acquires image parameters from at least one grayscale image collected by an optical sensor, generates an ultra-high resolution image corresponding to the grayscale image based on the image parameters, uses the ultra-high resolution image as a reference image, selects a reference point from the reference image, determines a target image of a preset size based on the reference point, constructs an image trajectory function for the target image, and determines the image coordinates of pixel displacements in the target image based on the image trajectory function. Multiple images to be processed are acquired, and displacement image data corresponding to the multiple images to be processed are output based on the image coordinates. The displacement image can be arbitrarily configured according to noise characteristics, improving the accuracy and applicability of evaluating various image defects.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and particularly relates to a displacement image processing method and system based on an optical sensor. Background Technology

[0002] Currently, the most direct way to acquire displacement images is by using optical sensors to directly collect images during the sensor's movement. The purpose of acquiring these images is to develop displacement algorithms, etc. However, while images directly acquired by optical sensors are closer to reality, they also have the following disadvantages: (1) The motion trajectory between two acquisitions cannot be precisely controlled, and the actual displacement trajectory between the previous frame image data and the next frame image data is unknown; (2) The characteristics of the sensor itself are uncontrollable. The sensor itself may have pixel defects, noise, etc. Each sensor will be different, and the same sensor will also be inconsistent under different states, and the differences may be huge, resulting in uncontrollable acquired images. Therefore, it is urgent to provide a displacement image processing method based on optical sensors to solve the above-mentioned technical problems. Summary of the Invention

[0003] In view of this, the present invention provides a displacement image processing method and system based on optical sensors, which generates displacement images with controllable motion trajectories and arbitrarily configurable noise characteristics at low cost, and also improves the accuracy and applicability of evaluating various image defects. The specific technical solution is as follows.

[0004] In a first aspect, the present invention provides a displacement image processing method based on an optical sensor, comprising the following steps: The image parameters of at least one grayscale image acquired by an optical sensor are obtained, and an ultra-high resolution image corresponding to the grayscale image is generated based on the image parameters, wherein the image parameters include contrast, grayscale range, and feature point size; Using the ultra-high resolution image as a reference image, a reference point is selected from the reference image, and a target image of a preset size is determined based on the reference point; Construct an image trajectory function for the target image, and determine the image coordinates of the pixel displacements in the target image based on the image trajectory function; Multiple images to be processed are acquired, and displacement image data corresponding to the multiple images to be processed are output according to the image coordinates.

[0005] As a preferred embodiment of the above technical solution, selecting a reference point from the reference image and determining a target image of a preset size based on the reference point includes: Obtain the resolution of the reference image, and select a reference point of the reference image based on the resolution; A second pixel in the first target image is generated based on the reference point using multiple first pixels in the reference image, wherein the number of multiple pixels in the reference image is determined by the sub-pixel level minimum displacement; The second target image is obtained by performing simulation analysis on the first target image corresponding to the generated second pixel.

[0006] As a preferred embodiment of the above technical solution, the second target image is obtained by simulation analysis of the first target image corresponding to the generated second pixel, including: Obtain the pixel size of the preset displacement image, and perform simulation processing on the first target image according to the pixel size to obtain the first image data. The first image data includes a first matrix and a second matrix. The first matrix represents the non-uniformity of the light response of the pixels of the preset displacement image, and the second matrix represents the background additive FPN template. Based on the first matrix and the second matrix, a traversal operation is performed on all pixels corresponding to the preset displacement image to obtain new second image data with FPN. .

[0007] As a preferred embodiment of the above technical solution, the expression for traversing all pixels corresponding to the preset displacement image based on the first matrix and the second matrix is ​​as follows: (1) in, This is the first image data. For the second image data, the background additive FPN template is generated once in one image trajectory.

[0008] As a preferred embodiment of the above technical solution, obtaining the pixel size of a preset displacement image, and performing simulation processing on the first target image based on the pixel size to obtain first image data, includes: The noise mean and noise standard deviation of the first image data are preset, and a Gaussian noise matrix is ​​generated by constructing a function based on the noise mean and the noise standard deviation; The Gaussian noise matrix is ​​used to perform traversal operations on all pixels in the second image data to obtain new third image data with FPN and Gaussian noise. A second target image corresponding to the first target image is obtained based on the second image data and the third image data.

[0009] As a preferred embodiment of the above technical solution, the expression for traversing all pixels in the second image data based on the Gaussian noise matrix is ​​as follows: (2) in, This represents the third image data. This represents the Gaussian noise matrix. The Gaussian noise matrix for each frame of the image needs to be re-randomized during each traversal operation.

[0010] As a preferred embodiment of the above technical solution, constructing an image trajectory function for the target image and determining the image coordinates of pixel displacements in the target image based on the image trajectory function includes: The speckle coordinates of a preset image to be processed are: The speckle radius is If the rotation angle of each frame is α and the cumulative rotation angle is sum_α, then the corresponding image trajectory function expression is: (3) Wherein, when the speckle motion of the image to be processed is variable speed, Replace with a function; Let the current frame count be k. Then Substituting the corresponding expression into formula (3) yields a circular trajectory of uniformly accelerated motion.

[0011] As a preferred embodiment of the above technical solution, acquiring image parameters of at least one grayscale image captured by an optical sensor, and generating an ultra-high resolution image corresponding to the grayscale image based on the image parameters, includes: A random phase screen is constructed, and a speckle image is obtained by controlling the image parameters of the grayscale image using a low-pass filter; The speckle field corresponding to the speckle image is obtained by inverse Fourier transform based on the speckle size of the speckle image; The ultra-high resolution image corresponding to the speckle image is determined based on the speckle field and the speckle size.

[0012] Secondly, the present invention also provides a displacement image processing system based on an optical sensor, applied to the aforementioned displacement image processing method based on an optical sensor, comprising: An image parameter acquisition unit is used to acquire image parameters of at least one grayscale image collected by an optical sensor, and generate an ultra-high resolution image corresponding to the grayscale image based on the image parameters, wherein the image parameters include contrast, grayscale range and feature point size; The target image determination unit is used to take the ultra-high resolution image as a reference image, select a reference point from the reference image, and determine a target image of a preset size based on the reference point; An image coordinate determination unit is used to construct an image trajectory function of the target image and determine the image coordinates of the pixel displacements in the target image based on the image trajectory function; The displacement image output unit is used to acquire multiple images to be processed and output displacement image data corresponding to the multiple images to be processed according to the image coordinates.

[0013] This invention provides a displacement image processing method and system based on an optical sensor. It acquires image parameters from at least one grayscale image captured by an optical sensor, generates an ultra-high resolution image corresponding to the grayscale image based on the image parameters, uses the ultra-high resolution image as a reference image, selects a reference point from the reference image, determines a target image of a preset size based on the reference point, constructs an image trajectory function for the target image, and determines the image coordinates of pixel displacements in the target image based on the image trajectory function. Multiple images to be processed are acquired, and displacement image data corresponding to the multiple images to be processed are output based on the image coordinates. The displacement image can be arbitrarily configured according to noise characteristics, and the accuracy and applicability of evaluating various image defects are improved. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart of the displacement image processing method based on an optical sensor provided by the present invention; Figure 2 This is a diagram illustrating the effect of outputting four sets of circular image trajectory data provided by the present invention. Figure 3 The structural block diagram of the displacement image processing system based on optical sensors provided by the present invention. Detailed Implementation

[0016] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0017] See Figure 1 In a first aspect, the present invention provides a displacement image processing method based on an optical sensor, comprising the following steps: S1: Obtain image parameters of at least one grayscale image acquired by an optical sensor, and generate an ultra-high resolution image corresponding to the grayscale image based on the image parameters, wherein the image parameters include contrast, grayscale range, and feature point size; S2: Using the ultra-high resolution image as a reference image, select a reference point from the reference image, and determine a target image of a preset size based on the reference point; S3: Construct the image trajectory function of the target image, and determine the image coordinates of the pixel displacement in the target image based on the image trajectory function; S4: Acquire multiple images to be processed, and output the displacement image data corresponding to the multiple images to be processed according to the image coordinates.

[0018] In this embodiment, acquiring image parameters of at least one grayscale image collected by an optical sensor and generating an ultra-high resolution image corresponding to the grayscale image based on the image parameters includes: constructing a random phase screen and using a low-pass filter to control the image parameters of the grayscale image to obtain a speckle image; using an inverse Fourier transform based on the speckle size of the speckle image to obtain the speckle field corresponding to the speckle image; and determining the ultra-high resolution image corresponding to the speckle image based on the speckle field and the speckle size. In other words, the image acquired by the optical sensor is a grayscale image, and the image parameters of the grayscale image mainly include contrast, grayscale range, and feature point size. First, a very large image dataset is generated, or a suitable ultra-high resolution image is found. The image resolution is set according to the desired motion trajectory. For example, if the generated trajectory is circular, then the aspect ratio is best at 1:1. The specific resolution is also based on the distance of the motion trajectory. For example, if an image with a displacement of 1000 pixels to the left is required, then the horizontal resolution of this large image dataset must be at least greater than 1000. Among them, ultra-large image data or ultra-high resolution images are speckle patterns. For example, a random phase screen is first created, and then the speckle size is controlled by a low-pass filter. The low-pass filter is applied to the random phase, and then the speckle field is obtained by inverse Fourier transform.

[0019] It should be noted that Super-Resolution (SR) refers to the process of recovering image details and other data information based on known image information using optics and related optical knowledge. Simply put, it increases the resolution of an image to prevent image quality degradation. "Suitable" in a suitable super-resolution image refers to the pattern; some images have textures that don't perfectly conform to natural laws, such as some AI-generated images, but using such images results in poor simulation effects. The process involves selecting a reference point from the reference image and determining a target image of a preset size based on the reference point, including: obtaining the resolution of the reference image and selecting a reference point from the reference image based on the resolution; generating a second pixel in the first target image using multiple first pixels in the reference image based on the reference point, wherein the number of multiple pixels in the reference image is determined by the sub-pixel minimum displacement; and performing simulation analysis on the first target image corresponding to the generated second pixel to obtain a second target image. The second target image is obtained by further processing the first target image, for example, by adding image noise to ensure that the generated second target image is closer to the real image.

[0020] Specifically, a reference point (origin or reference point) of the ultra-high resolution image is selected in the reference image to generate an image of the required size. Here, the ultra-high resolution image is denoted as X, and the image of the required size generated from the origin is defined as image A, which is a part of image X. For example, if the required displacement image is 16×16, then image A is 16×16. However, image A (the initial image) is not simply selected from the 16x16 range of data in image X (the ultra-high resolution image), as this would prevent the generation of sub-pixel displacement. Therefore, multiple pixels in image X are used to generate one pixel in image A. The number of multiple pixels is determined by the minimum required sub-pixel displacement distance. For example, if the minimum required displacement is 0.1 pixels, then the number of synthesized pixels is 10×10, i.e., (1 / 0.1) * (1 / 0.1). The synthesis method is to directly average the data within the 10×10 range in image X, thus ultimately obtaining a 16×16 pixel image A synthesized from 160×160 pixels in image X. The discrete image trajectory function can be the trajectory equation of a circle or an ellipse, etc., and the coordinates of each discrete pixel point satisfy the image trajectory function.

[0021] It should be understood that generating discrete image trajectory functions and using them as image coordinates after each displacement, and acquiring images B, C, D... according to S2 and S3 at each coordinate; merging A, B, C, D, etc. into a single file for output, yields a fully controllable displacement image data that can be used for algorithm development. By acquiring image parameters from at least one grayscale image collected by an optical sensor, and generating an ultra-high resolution image corresponding to the grayscale image based on the image parameters, using the ultra-high resolution image as a reference image, selecting a reference point from the reference image, and determining a target image of a preset size based on the reference point, constructing an image trajectory function for the target image, and determining the image coordinates of pixel displacements in the target image based on the image trajectory function, multiple images to be processed are acquired, and the displacement image data corresponding to the multiple images to be processed is output based on the image coordinates. The displacement image can be arbitrarily configured according to noise characteristics, which also improves the accuracy and applicability of evaluating various image defects.

[0022] Optionally, the target image is obtained by performing simulation analysis on the image corresponding to the generated second pixel, including: Obtain the pixel size of the preset displacement image, and perform simulation processing on the first target image according to the pixel size to obtain the first image data. The first image data includes a first matrix and a second matrix. The first matrix represents the non-uniformity of the light response of the pixels of the preset displacement image, and the second matrix represents the background additive FPN template. Based on the first matrix and the second matrix, a traversal operation is performed on all pixels corresponding to the preset displacement image to obtain new second image data with FPN. .

[0023] In this embodiment, the expression for traversing all pixels of the target image based on the first matrix M and the second matrix N is as follows: (1) in, This is the first image data. For the second image data, the background additive FPN template is generated once in one image trajectory.

[0024] It should be noted that Fixed Pattern Noise (FPN) is a common type of noise in image sensors. Its characteristic is that the noise location is fixed and does not change with image variations. FPN is divided into row FPN and column FPN based on its formation mechanism, manifesting as horizontal and vertical stripes in the image, respectively. The process involves obtaining the dimensions of a preset displacement image, and performing simulation processing on the first target image based on the pixel dimensions to obtain first image data. This includes: presetting the noise mean and standard deviation of the first image data, and generating a Gaussian noise matrix using a function constructed based on the noise mean and standard deviation; performing traversal operations on all pixels in the second image data using the Gaussian noise matrix to obtain new third image data with FPN and Gaussian noise; and obtaining a second target image corresponding to the first target image based on the second and third image data. In other words, further processing of image A, such as adding image noise, is performed. This step, depending on the requirements, can add random noise, FPN, pixel defects, etc., of any size to the first target image. If a perfect ideal image is required, this step can be skipped.

[0025] Specifically, the expression for traversing all pixels in the second image data based on the Gaussian noise matrix is ​​as follows: (2) in, This represents the third image data. This represents the Gaussian noise matrix. The Gaussian noise matrix for each frame of the image needs to be re-randomized during each traversal operation.

[0026] It should be noted that constructing the image trajectory function of the target image, and determining the image coordinates of the pixel displacements in the target image based on the image trajectory function, includes: The speckle coordinates of a preset image to be processed are: The speckle radius is If the rotation angle of each frame is α and the cumulative rotation angle is sum_α, then the corresponding image trajectory function expression is: (3) Wherein, when the speckle motion of the image to be processed is variable speed, Replace with a function; Let the current frame count be k. Then Substituting the corresponding expression into formula (6) yields a circular trajectory of uniformly accelerated motion. The purpose of this invention is to obtain image data suitable for simulation. Simulation analysis simulates image defects added by the sensor, such as noise, rather than removing these defects. Formula (3) above is for illustrative purposes only and can be flexibly adjusted according to actual needs; no special limitations are imposed here.

[0027] See Figure 2 , Figure 2 The diagram illustrates four sets of circular image trajectory data output using the method provided in this invention, and the resulting displacements after being processed by the same displacement algorithm. In reality, the four sets of generated circular image trajectories are all standard, ideal circles, such as... Figure 2 (a) shows that the trajectories reconstructed using the displacement algorithm in the latter three images are significantly worse than those in the latter three images. This is because different image defects were introduced when generating the trajectory data for the latter three sets of circular images, specifically as follows: Figure 2 As shown in (b), (c), and (d) in the diagram; in this way, we can determine the limits of various image defects that our displacement algorithm can withstand, and which of the composite defects has the greatest impact. This is very helpful for the development of displacement algorithms. Similarly, we can also generate circular trajectories at different speeds to determine the speed limits of the displacement algorithm.

[0028] See Figure 3 The present invention also provides a displacement image processing system based on an optical sensor, applied to the above-mentioned displacement image processing method based on an optical sensor, comprising: An image parameter acquisition unit is used to acquire image parameters of at least one grayscale image collected by an optical sensor, and generate an ultra-high resolution image corresponding to the grayscale image based on the image parameters, wherein the image parameters include contrast, grayscale range and feature point size; The target image determination unit is used to take the ultra-high resolution image as a reference image, select a reference point from the reference image, and determine a target image of a preset size based on the reference point; An image coordinate determination unit is used to construct an image trajectory function of the target image and determine the image coordinates of the pixel displacements in the target image based on the image trajectory function; The displacement image output unit is used to acquire multiple images to be processed and output displacement image data corresponding to the multiple images to be processed according to the image coordinates.

[0029] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.

[0030] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0031] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A displacement image processing method based on an optical sensor, characterized in that, Includes the following steps: The image parameters of at least one grayscale image acquired by an optical sensor are obtained, and an ultra-high resolution image corresponding to the grayscale image is generated based on the image parameters, wherein the image parameters include contrast, grayscale range, and feature point size; Using the ultra-high resolution image as a reference image, a reference point is selected from the reference image, and a target image of a preset size is determined based on the reference point; Construct an image trajectory function for the target image, and determine the image coordinates of the pixel displacements in the target image based on the image trajectory function; Multiple images to be processed are acquired, and displacement image data corresponding to the multiple images to be processed are output according to the image coordinates.

2. The optical sensor-based displacement image processing method according to claim 1, characterized in that, Selecting a reference point from the reference image and determining a target image of a preset size based on the reference point includes: Obtain the resolution of the reference image, and select a reference point of the reference image based on the resolution; A second pixel in the first target image is generated based on the reference point using multiple first pixels in the reference image, wherein the number of multiple pixels in the reference image is determined by the sub-pixel level minimum displacement; The second target image is obtained by performing simulation analysis on the first target image corresponding to the generated second pixel.

3. The optical sensor-based displacement image processing method of claim 1, wherein, The second target image is obtained by performing simulation analysis on the first target image corresponding to the generated second pixel, including: Obtain the pixel size of a preset displacement image, and perform simulation processing on the first target image based on the pixel size to obtain first image data. The first image data includes a first matrix and a second matrix. The first matrix represents the non-uniformity of the light response of the pixels of the preset displacement image, and the second matrix represents the background additive FPN template. According to the first matrix and the second matrix, all pixel points corresponding to the preset displacement image are iterated to obtain new second image data with FPN .

4. The optical sensor-based displacement image processing method according to claim 3, characterized in that, The expression for traversing all pixels corresponding to the preset displacement image based on the first matrix and the second matrix is ​​as follows: （1） wherein is the first image data, is the second image data, the base-additive FPN template is generated once in one image track.

5. The optical sensor-based displacement image processing method according to claim 4, characterized in that, Obtain the pixel dimensions of a preset displacement image, and perform simulation processing on the first target image based on the pixel dimensions to obtain first image data, including: The noise mean and noise standard deviation of the first image data are preset, and a Gaussian noise matrix is ​​generated by constructing a function based on the noise mean and the noise standard deviation; The Gaussian noise matrix is ​​used to perform traversal operations on all pixels in the second image data to obtain new third image data with FPN and Gaussian noise. A second target image corresponding to the first target image is obtained based on the second image data and the third image data.

6. The optical sensor-based displacement image processing method according to claim 5, characterized in that, The expression for traversing all pixels in the second image data based on the Gaussian noise matrix is ​​as follows: （2） wherein denotes third image data, denotes a Gaussian noise matrix, the Gaussian noise matrix corresponding to each frame of image being randomly re-generated at each iteration of the operation.

7. The displacement image processing method based on an optical sensor according to claim 1, characterized in that, Constructing an image trajectory function for the target image, and determining the image coordinates of pixel displacements in the target image based on the image trajectory function, includes: The speckle coordinates of a preset image to be processed are: The speckle radius is If the rotation angle of each frame is α and the cumulative rotation angle is sum_α, then the corresponding image trajectory function expression is: （3） Wherein, when the speckle motion of the image to be processed is variable speed, Replace with a function; Let the current frame count be k. Then Substituting the corresponding expression into formula (3) yields a circular trajectory of uniformly accelerated motion.

8. The displacement image processing method based on an optical sensor according to claim 1, characterized in that, Acquiring image parameters from at least one grayscale image captured by an optical sensor, and generating an ultra-high resolution image corresponding to the grayscale image based on the image parameters, including: A random phase screen is constructed, and a speckle image is obtained by controlling the image parameters of the grayscale image using a low-pass filter; The speckle field corresponding to the speckle image is obtained by inverse Fourier transform based on the speckle size of the speckle image; The ultra-high resolution image corresponding to the speckle image is determined based on the speckle field and the speckle size.

9. A displacement image processing system based on an optical sensor, characterized in that, The displacement image processing method based on optical sensors as described in claims 1-8 includes: An image parameter acquisition unit is used to acquire image parameters of at least one grayscale image collected by an optical sensor, and generate an ultra-high resolution image corresponding to the grayscale image based on the image parameters, wherein the image parameters include contrast, grayscale range and feature point size; The target image determination unit is used to take the ultra-high resolution image as a reference image, select a reference point from the reference image, and determine a target image of a preset size based on the reference point; An image coordinate determination unit is used to construct an image trajectory function of the target image and determine the image coordinates of the pixel displacements in the target image based on the image trajectory function; The displacement image output unit is used to acquire multiple images to be processed and output displacement image data corresponding to the multiple images to be processed according to the image coordinates.

Images