A laser radar imaging device and method based on focal plane pixel sparse array

Abstract

The application provides a laser radar imaging device and method based on a focal plane pixel sparse array, and the method comprises the following steps: a focal plane sparse laser diode array emits high-frequency laser, and the high-frequency laser is irradiated to a target object in a field of view; a reflected echo reflected by the target object is received by a sparse avalanche photodiode array, a signal is obtained through processing, and the signal is sent to a parallel multi-path synchronous timing circuit; the parallel multi-path synchronous timing circuit obtains a laser flight time according to the signal; a mass data acquisition unit determines distance information of the target object according to the laser flight time, converts the distance information into corresponding pixel values, obtains a sparse sampling image, and sends the sparse sampling image to an image restoration unit; and the image restoration unit restores and reconstructs the sparse sampling image to obtain a target image. The application improves the sampling efficiency of the laser radar.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a lidar imaging device and method based on a sparse array of focal plane pixels. Background Technology

[0002] LiDAR is a high-precision, highly concealed active detection device. The depth images it generates can provide rich 3D information about the target scene and have been widely used in many fields such as autonomous driving, 3D construction, and target tracking.

[0003] With the development of technology, people's demand for information and the speed of information acquisition are increasing day by day. For LiDAR, it needs to develop towards higher sampling rates, larger data volumes, and faster transmission and storage efficiency. However, traditional sampling strategies are constrained by the Nyquist sampling theorem, requiring the collection of a large amount of redundant information, which limits the sampling efficiency and depth image generation speed of LiDAR.

[0004] Therefore, improving the sampling efficiency of lidar has become a current research focus. Summary of the Invention

[0005] This application provides a lidar imaging method and apparatus based on a sparse array of focal plane pixels, which can be used to solve the technical problem of improving the sampling efficiency of lidar.

[0006] In a first aspect, embodiments of this application provide a lidar imaging method based on a sparse array of focal plane pixels, the method comprising:

[0007] A sparsely arranged array of laser diodes on the focal plane emits high-frequency laser light, which is directed at the target object in the field of view.

[0008] The reflected echo after being reflected by the target object is received by a sparsely arranged avalanche photodiode array, processed to obtain an echo signal, and the echo signal is sent to a parallel multi-channel synchronous timing circuit.

[0009] The parallel multi-channel synchronous timing circuit obtains the laser flight time based on the callback signal;

[0010] The massive data acquisition unit determines the distance information of the target object based on the laser flight time, converts the distance information into corresponding pixel values, obtains a sparsely sampled image, and sends it to the image restoration unit.

[0011] The image restoration unit restores and reconstructs the sparsely sampled image to obtain the target image.

[0012] In conjunction with the first aspect, in one possible implementation of the first aspect, the image restoration unit restores and reconstructs the sparsely sampled image to obtain a target image, including:

[0013] Each iterative module in the image restoration unit performs restoration and reconstruction processing on the sparsely sampled image, and uses the output of the last iterative module as the target image. The image restoration unit includes nine iterative modules with identical structures, and the output of any iterative module is determined by the following method:

[0014] r (k) =x (k-1) -ρΦ T (Φx (k-1) -y)

[0015] In the formula, r(k) is the intermediate quantity in the process of obtaining x(k) from x(k-1), x(k) is the output of the iteration module, x(k-1) is the input of the iteration module, k is the iteration index, the initial value is 1, k = k + 1 is executed after each iteration module, and the value range of k is 1 ≤ k ≤ 9; x(k-1) is the output result of the iteration module k-1; ρ is the step size of x(k-1) moving towards the negative gradient direction of the fidelity term in the (k-1)th iteration; Φ is the observation matrix, specifically the matrix Φ corresponding to the sparse array of focal plane pixels, which is a randomly generated (0,1) distribution of size 64×64 Gaussian random matrix, and y is the one-dimensional representation of the sparsely sampled image of the lidar.

[0016]

[0017] In the formula, x(k) is the output of the iterative module, λ is the non-zero parameter in the optimization problem, and Ψ is the sparse basis matrix in compressed sensing.

[0018] In conjunction with the first aspect, in one possible implementation of the first aspect, the parallel multi-channel synchronous timing circuit obtains the laser flight time based on the callback signal, and then further includes:

[0019] The multi-channel synchronous timing circuit synchronizes the laser flight time to the massive data acquisition unit.

[0020] In conjunction with the first aspect, in one possible implementation of the first aspect, the image restoration unit is trained using the following method:

[0021] 978 33×33 sub-images were extracted from the luminance component of each image in the training dataset to generate training data pairs. Used for training the network;

[0022] The loss function involved in training is determined as follows:

[0023]

[0024] In the formula, Nb is the number of images in the training dataset, which is 978; N is the size of each image in the training dataset, which is 1089; To obtain the output of the reconstructed image of the i-th image in the training dataset, i.e., the one-dimensional representation of the restored image, x i This is a one-dimensional representation of the i-th image sample;

[0025] The image restoration unit is trained based on the training data and the loss function until the training objective is met.

[0026] In conjunction with the first aspect, in one possible implementation of the first aspect, a sparsely arranged array of laser diodes on the focal plane emits a high-frequency laser beam to illuminate a target object in the field of view, further comprising:

[0027] The laser diode array is arranged in a sparsely arranged focal plane configuration; the laser diode array is arranged in the same way as the sparsely arranged avalanche photodiode array.

[0028] In conjunction with the first aspect, one possible implementation of the first aspect includes an array arrangement of laser diodes with a sparse focal plane, comprising:

[0029] Generate a Gaussian random matrix with a (0, 1) distribution and a size of 64×64, which will be used as the observation matrix; the proportion of 1 elements in the Gaussian random matrix is ​​set to 25%;

[0030] The positions of the sparsely arranged laser diode array and the observation matrix are matched one-to-one;

[0031] Laser diodes are placed at positions where the element is 1 in the observation matrix.

[0032] Secondly, embodiments of this application provide a lidar imaging device based on a sparse array of focal plane pixels, the device comprising:

[0033] The system includes a sparsely arranged laser diode array, an avalanche photodiode array, a parallel multi-channel synchronous timing circuit, a massive data acquisition unit, and an image restoration unit.

[0034] The sparsely arranged laser diode array on the focal plane is used to emit high-frequency lasers to illuminate the target object in the field of view;

[0035] The avalanche photodiode array is used to receive the reflected echo after being reflected by the target object, process it to obtain the echo signal, and send the echo signal to the parallel multi-channel synchronous timing circuit.

[0036] The parallel multi-channel synchronous timing circuit obtains the laser flight time based on the callback signal;

[0037] The massive data acquisition unit determines the distance information of the target object based on the laser flight time, converts the distance information into corresponding pixel values, obtains a sparsely sampled image, and sends it to the image restoration unit.

[0038] The image restoration unit restores and reconstructs the sparsely sampled image to obtain the target image.

[0039] In conjunction with the second aspect, in one possible implementation of the second aspect, the image restoration unit is specifically used for:

[0040] Each iterative module in the image restoration unit performs restoration and reconstruction processing on the sparsely sampled image, and uses the output of the last iterative module as the target image. The image restoration unit includes nine iterative modules with identical structures, and the output of any iterative module is determined by the following method:

[0041] r (k) =x (k-1) -ρΦ T (Φx (k-1) -y)

[0042] In the formula, r(k) is the intermediate quantity in the process of obtaining x(k) from x(k-1), x(k) is the output of the iteration module, x(k-1) is the input of the iteration module, k is the iteration index, the initial value is 1, k = k + 1 is executed after each iteration module, and the value range of k is 1 ≤ k ≤ 9; x(k-1) is the output result of the iteration module k-1; ρ is the step size of x(k-1) moving towards the negative gradient direction of the fidelity term in the (k-1)th iteration; Φ is the observation matrix, specifically the matrix Φ corresponding to the sparse array of focal plane pixels, which is a randomly generated (0,1) distribution of size 64×64 Gaussian random matrix, and y is the one-dimensional representation of the sparsely sampled image of the lidar.

[0043]

[0044] In the formula, x(k) is the output of the iterative module, λ is the non-zero parameter in the optimization problem, and Ψ is the sparse basis matrix in compressed sensing.

[0045] In conjunction with the second aspect, in one possible implementation of the second aspect, the multi-channel synchronous timing circuit is further used for:

[0046] The laser flight time is synchronized to the massive data acquisition unit.

[0047] In conjunction with the second aspect, in one possible implementation of the second aspect, the image restoration unit is trained using the following method:

[0048] 978 33×33 sub-images were extracted from the luminance component of each image in the training dataset to generate training data pairs. Used for training the network;

[0049] The loss function involved in training is determined as follows:

[0050]

[0051] In the formula, Nb is the number of images in the training dataset, which is 978; N is the size of each image in the training dataset, which is 1089; To obtain the output of the reconstructed image of the i-th image in the training dataset, i.e., the one-dimensional representation of the restored image, x i This is a one-dimensional representation of the i-th image sample;

[0052] The image restoration unit is trained based on the training data and the loss function until the training objective is met.

[0053] In conjunction with the second aspect, one possible implementation of the second aspect further includes a setting module, which is used to set the array arrangement of the sparsely arranged laser diode array on the focal plane; the array arrangement of the laser diode array is the same as that of the sparsely arranged avalanche photodiode array.

[0054] In conjunction with the second aspect, in one possible implementation of the second aspect, the setting module is specifically used to generate a Gaussian random matrix with a (0, 1) distribution and a size of 64×64, as the observation matrix; wherein, the proportion of 1 elements in the Gaussian random matrix is ​​set to 25%;

[0055] The positions of the sparsely arranged laser diode array and the observation matrix are matched one-to-one;

[0056] Laser diodes are placed at positions where the element is 1 in the observation matrix.

[0057] This application introduces the concept of sparse pixel arrays on the focal plane and combines it with compressed sensing reconstruction methods to achieve high-resolution 3D imaging with sparse pixel sampling of the focal plane signal. This overcomes the limitation of pixel count on the spatial resolution of 3D imaging, significantly reduces the pixel count requirement for focal plane devices in high-staring imaging resolution, and substantially reduces the requirements for data acquisition, transmission, and storage, thus saving hardware costs. This application innovatively uses a convolution-activation-convolution form as a nonlinear transformation to represent the sparse basis matrix, offering greater flexibility. Furthermore, this application innovatively utilizes a network block based on a residual channel attention mechanism (RCAB+) located after the activation function to assign different weights to network channels, enabling the neural network to focus more on channels with higher frequency information, resulting in clearer details in the reconstructed image. Attached Figure Description

[0058] Figure 1 A schematic flowchart of a lidar imaging method based on a sparse array of focal plane pixels provided in this application embodiment;

[0059] Figure 2 The array configurations of the focal plane sparse array laser diode array and the focal plane sparse array avalanche photodiode (APD) array provided in the embodiments of this application;

[0060] Figure 3 This is a schematic diagram illustrating the working principle of the image restoration method provided in the embodiments of this application.

[0061] Figure 4 A flowchart of the T(k) module in the iterative module k is provided for embodiments of this application;

[0062] Figure 5 A flowchart of the network block (RCAB+) based on the residual channel attention mechanism is provided for the embodiments of this application;

[0063] Figure 6 This is a schematic diagram of the structure of a lidar imaging device based on a sparse array of focal plane pixels, provided in an embodiment of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0065] Compressed sensing, also known as sparse sampling, is a novel sampling strategy that leverages the inherent sparsity of signals to perform sampling and compression simultaneously. This allows for high-probability reconstruction of the original signal with a significantly reduced sampling volume. This application of compressed sensing to LiDAR imaging effectively addresses the problem of surging LiDAR data volume, improves sampling efficiency, reduces the need for high transmission bandwidth and large storage space, and also reduces the number of sensors used, saving hardware costs.

[0066] The following is a combination of... Figure 1 The embodiments of this application will be described.

[0067] like Figure 1 The diagram shown is a schematic representation of the method flow provided in this application.

[0068] This application includes the following steps:

[0069] In step S101, the laser diode array sparsely arranged on the focal plane emits high-frequency laser light, which is directed at the target object in the field of view.

[0070] It should be noted that the following steps need to be performed before executing step S101:

[0071] The array arrangement of laser diode arrays with sparse focal planes is configured.

[0072] Specifically, such as Figure 2 As shown, a Gaussian random matrix with a (0, 1) distribution and a size of 64×64 is generated as the observation matrix. The percentage of 1 elements in the Gaussian random matrix is ​​set to 25%.

[0073] The positions of the sparsely arranged laser diode array on the focal plane and the observation matrix are matched one-to-one. That is, the size of the observation matrix is ​​64×64, and the size of the sparsely arranged laser diode array on the focal plane is also 64×64.

[0074] Laser diodes are placed at the positions where the element in the observation matrix is ​​1. Specifically, no laser diodes are placed at the positions where the element in the observation matrix is ​​0.

[0075] In step S102, the reflected echo after being reflected by the target object is received by a sparsely arranged avalanche photodiode array, processed to obtain the echo signal, and then sent to a parallel multi-channel synchronous timing circuit.

[0076] It should be noted that the laser diode array in this embodiment is arranged in the same way as the sparsely arranged avalanche photodiode array. Avalanche photodiodes (APDs) are arranged at the positions where the element in the observation matrix is ​​1, and no avalanche photodiodes (APDs) are arranged at the positions where the element in the observation matrix is ​​0. Other steps will not be described here.

[0077] In step S103, the parallel multi-channel synchronous timing circuit obtains the laser flight time based on the callback signal.

[0078] To execute step S103, the following steps are also required:

[0079] The multi-channel synchronous timing circuit synchronizes the laser flight time to the massive data acquisition unit.

[0080] In step S104, the massive data acquisition unit determines the distance information of the target object based on the laser flight time, converts the distance information into corresponding pixel values, obtains a sparse sampled image, and sends it to the image restoration unit.

[0081] It should be noted that, in the embodiments of this application, the image restoration unit is a deep network neural model.

[0082] The image restoration unit provided in this application is trained and optimized using the following methods:

[0083] Step S201: Extract 978 33×33 sub-images from the luminance component of each image in the training dataset to generate training data pairs. Used for training the network.

[0084] It should be noted that in this embodiment of the application, the Middlebury Stereo Data 2001, 2003, and 2005 are used as the training dataset.

[0085] Step S202, determine the loss function involved in training as follows:

[0086]

[0087] In the formula, Nb is the number of images in the training dataset, which is 978. N is the size of each image in the training dataset, which is 1089. To obtain the output of the reconstructed image of the i-th image in the training dataset, i.e., the one-dimensional representation of the restored image, x i Let be the one-dimensional representation of the i-th image sample.

[0088] Step S203: Train the image restoration unit according to the training data and loss function until the training objective is met.

[0089] In step S105, the image restoration unit restores and reconstructs the sparsely sampled image to obtain the target image.

[0090] Specifically, each iterative module in the image restoration unit performs restoration and reconstruction processing on the sparsely sampled image, and uses the output of the last iterative module as the target image. The image restoration unit includes 9 iterative modules with identical structures, and the output of any iterative module is determined by the following method:

[0091] r (k) =x (k-1) -ρΦ T (Φx (k-1) -y)

[0092] In the formula, r(k) is the intermediate quantity in the process of obtaining x(k) from x(k-1), x(k) is the output of the iteration module, x(k-1) is the input of the iteration module, k is the iteration index, initially 1, and k = k + 1 is executed after each iteration module. The value of k is 1 ≤ k ≤ 9. x(k-1) is the output result of the iteration module k-1. ρ is the step size by which x(k-1) moves towards the negative gradient direction of the fidelity term in the (k-1)th iteration. Φ is the observation matrix, specifically the matrix Φ corresponding to the sparse array of pixels on the focal plane, which is a randomly generated (0, 1) distribution of size 64×64 Gaussian random matrix, and y is the one-dimensional representation of the sparsely sampled image of the lidar.

[0093]

[0094] In the formula, x(k) is the output of the iterative module, λ is the non-zero parameter in the optimization problem, and Ψ is the sparse basis matrix in compressed sensing. That is, the coefficients of x in Ψ representation are sparse. To solve x(k), we first enter the T(k) module located after r(k), whose structure is as follows: Figure 4 As shown, the T(k) module serves two purposes. First, it innovatively uses a convolution-activation-convolution form as a nonlinear transformation to represent Ψ, whereas traditional methods require manually designing Ψ as a linear transformation matrix beforehand, lacking flexibility. Second, it innovatively utilizes a network block based on residual channel attention mechanism (RCAB+) located after the activation function to assign different weights to network channels, allowing the neural network to focus more on channels with higher frequency information, resulting in clearer details in the reconstructed image. The structure of RCAB+ is shown below. Figure 5 As shown.

[0095] After the T(k) module, the soft threshold function is used to solve for x(k), followed by an inverse transform module of the T(k) module. The output result x(k) of the iterative module k can then be obtained.

[0096] After nine iterative modules with identical structures, the final x(9) is the one-dimensional representation of the restored image. By restoring it to a two-dimensional image, the restored high-resolution image can be obtained.

[0097] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0098] Figure 6 An exemplary schematic diagram of a lidar imaging device based on a sparse array of focal plane pixels provided in an embodiment of this application is shown. Figure 6 As shown, the device has the function of implementing the above-mentioned lidar imaging method based on sparse array of focal plane pixels. The function can be implemented by hardware or by hardware executing corresponding software. The device may include: a laser diode array sparsely arranged on the focal plane, an avalanche photodiode array, a parallel multi-channel synchronous timing circuit, a massive data acquisition unit, and an image restoration unit.

[0099] A sparsely arranged array of laser diodes on the focal plane is used to emit high-frequency lasers to illuminate target objects in the field of view.

[0100] The avalanche photodiode array is used to receive the reflected echo after being reflected by the target object, process it to obtain the echo signal, and send the echo signal to the parallel multi-channel synchronous timing circuit.

[0101] The parallel multi-channel synchronous timing circuit obtains the laser flight time based on the callback signal.

[0102] The massive data acquisition unit determines the distance information of the target object based on the laser flight time, converts the distance information into corresponding pixel values, obtains a sparsely sampled image, and sends it to the image restoration unit.

[0103] The image restoration unit restores and reconstructs the sparsely sampled image to obtain the target image.

[0104] Optionally, the image restoration unit is specifically used for:

[0105] Each iterative module in the image restoration unit performs restoration and reconstruction processing on the sparsely sampled image, and uses the output of the last iterative module as the target image. The image restoration unit includes 9 iterative modules with identical structures, and the output of any iterative module is determined by the following method:

[0106] r (k) =x (k-1) -ρΦ T (Φx (k-1) -y)

[0107] In the formula, r(k) is the intermediate quantity in the process of obtaining x(k) from x(k-1), x(k) is the output of the iteration module, x(k-1) is the input of the iteration module, k is the iteration index, initially 1, and k = k + 1 is executed after each iteration module. The value of k is 1 ≤ k ≤ 9. x(k-1) is the output result of the iteration module k-1. ρ is the step size by which x(k-1) moves towards the negative gradient direction of the fidelity term in the (k-1)th iteration. Φ is the observation matrix, specifically the matrix Φ corresponding to the sparse array of pixels on the focal plane, which is a randomly generated (0, 1) distribution of size 64×64 Gaussian random matrix, and y is the one-dimensional representation of the sparsely sampled image of the lidar.

[0108]

[0109] In the formula, x(k) is the output of the iterative module, λ is the non-zero parameter in the optimization problem, and Ψ is the sparse basis matrix in compressed sensing.

[0110] Optionally, the multi-channel synchronous timing circuit is also used for:

[0111] The laser flight time is synchronized to the massive data acquisition unit.

[0112] Optionally, the image restoration unit is trained using the following method:

[0113] 978 33×33 sub-images were extracted from the luminance component of each image in the training dataset to generate training data pairs. Used for training the network.

[0114] The loss function involved in training is determined as follows:

[0115]

[0116] In the formula, Nb is the number of images in the training dataset, which is 978. N is the size of each image in the training dataset, which is 1089. To obtain the output of the reconstructed image of the i-th image in the training dataset, i.e., the one-dimensional representation of the restored image, x i Let be the one-dimensional representation of the i-th image sample.

[0117] The image restoration unit is trained based on the training data and the loss function until the training objective is met.

[0118] Optionally, a setting module is also included, which is used to set the array configuration of the sparsely arranged laser diode array on the focal plane. The array configuration of the laser diode array is the same as that of the sparsely arranged avalanche photodiode array.

[0119] Optionally, the configuration module is specifically used to generate a Gaussian random matrix with a (0, 1) distribution and a size of 64×64, as the observation matrix. The percentage of 1 elements in the Gaussian random matrix is ​​set to 25%.

[0120] The positions of the sparsely arranged laser diode array on the focal plane and the observation matrix are matched one-to-one.

[0121] Laser diodes are placed at positions where the element is 1 in the observation matrix.

[0122] This application introduces the concept of sparse pixel arrays on the focal plane and combines it with compressed sensing reconstruction methods to achieve high-resolution 3D imaging with sparse pixel sampling of the focal plane signal. This overcomes the limitation of pixel count on the spatial resolution of 3D imaging, significantly reduces the pixel count requirement for focal plane devices in high-staring imaging resolution, and substantially reduces the requirements for data acquisition, transmission, and storage, thus saving hardware costs. This application innovatively uses a convolution-activation-convolution form as a nonlinear transformation to represent the sparse basis matrix, offering greater flexibility. Furthermore, this application innovatively utilizes a network block based on a residual channel attention mechanism (RCAB+) located after the activation function to assign different weights to network channels, enabling the neural network to focus more on channels with higher frequency information, resulting in clearer details in the reconstructed image.

[0123] Those skilled in the art will clearly understand that the techniques in the embodiments of this application can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application or some parts of the embodiments.

[0124] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the service building apparatus and service loading apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0125] The embodiments described above do not constitute a limitation on the scope of protection of this application.

Claims

1. A lidar imaging method based on a sparse array of focal plane pixels, characterized in that, The method includes: A sparsely arranged array of laser diodes on the focal plane emits high-frequency laser light, which is directed at the target object in the field of view. The reflected echo after being reflected by the target object is received by a sparsely arranged avalanche photodiode array, processed to obtain an echo signal, and the echo signal is sent to a parallel multi-channel synchronous timing circuit. The parallel multi-channel synchronous timing circuit obtains the laser flight time based on the echo signal; The massive data acquisition unit determines the distance information of the target object based on the laser flight time, converts the distance information into corresponding pixel values, obtains a sparsely sampled image, and sends it to the image restoration unit. The image restoration unit restores and reconstructs the sparsely sampled image to obtain the target image.

2. The method according to claim 1, characterized in that, The image restoration unit restores and reconstructs the sparsely sampled image to obtain the target image, including: Each iterative module in the image restoration unit performs restoration and reconstruction processing on the sparsely sampled image, and uses the output of the last iterative module as the target image. The image restoration unit includes nine iterative modules with identical structures, and the output of any iterative module is determined by the following method: ； In the formula, For the reason get Intermediate quantities in the process For the output of the iterative module, The input to the iteration module is k, which is the iteration index. The initial value is 1. After each iteration module, k = k + 1 is executed. The value of k is 1 ≤ k ≤ 9. This represents the output of iteration module k-1; ρ represents the output of the (k-1)th iteration. The step size for moving towards the negative gradient direction of the fidelity term; Φ is the observation matrix, specifically the matrix Φ corresponding to the sparse array of pixels on the focal plane, which is a randomly generated (0,1) Gaussian random matrix of size 64×64, and y is the one-dimensional representation of the sparsely sampled image of the lidar. ； In the formula, Ψ is the output of the iterative module, λ is the non-zero parameter in the optimization problem, and Ψ is the sparse basis matrix in compressed sensing.

3. The method according to claim 1, characterized in that, The parallel multi-channel synchronous timing circuit obtains the laser flight time based on the echo signal, and then further includes: The multi-channel synchronous timing circuit synchronizes the laser flight time to the massive data acquisition unit.

4. The method according to claim 1, characterized in that, The image restoration unit is trained using the following method: 978 33×33 sub-images were extracted from the luminance component of each image in the training dataset to generate training data pairs. Used for training the network; The loss function involved in training is determined as follows: ； In the formula, Nb is the number of images in the training dataset, which is 978; N is the size of each image in the training dataset, which is 1089; To obtain the output of the reconstructed image of the i-th image in the training dataset, i.e., the one-dimensional representation of the restored image, This is a one-dimensional representation of the i-th image sample; The image restoration unit is trained based on the training data and the loss function until the training objective is met.

5. The method according to claim 1, characterized in that, A sparsely arranged array of laser diodes on the focal plane emits high-frequency laser light, illuminating the target object in the field of view. Prior to this, it also includes: The laser diode array is arranged in a sparsely arranged focal plane configuration; the laser diode array is arranged in the same way as the sparsely arranged avalanche photodiode array.

6. The method according to claim 5, characterized in that, The array arrangement of laser diode arrays with sparse focal planes includes: Generate a Gaussian random matrix with a (0, 1) distribution and a size of 64×64, as the observation matrix; wherein, the proportion of 1 elements in the Gaussian random matrix is ​​set to 25%; The positions of the sparsely arranged laser diode array and the observation matrix are matched one-to-one; Laser diodes are placed at positions where the element is 1 in the observation matrix.

7. A lidar imaging device based on a sparse array of focal plane pixels, characterized in that, The device includes: The system includes a sparsely arranged laser diode array, an avalanche photodiode array, a parallel multi-channel synchronous timing circuit, a massive data acquisition unit, and an image restoration unit. The sparsely arranged laser diode array on the focal plane is used to emit high-frequency lasers to illuminate the target object in the field of view; The avalanche photodiode array is used to receive the reflected echo after being reflected by the target object, process it to obtain the echo signal, and send the echo signal to the parallel multi-channel synchronous timing circuit. The parallel multi-channel synchronous timing circuit obtains the laser flight time based on the echo signal; The massive data acquisition unit determines the distance information of the target object based on the laser flight time, converts the distance information into corresponding pixel values, obtains a sparsely sampled image, and sends it to the image restoration unit. The image restoration unit restores and reconstructs the sparsely sampled image to obtain the target image.

8. The apparatus according to claim 7, characterized in that, The image restoration unit is specifically used for: Each iterative module in the image restoration unit performs restoration and reconstruction processing on the sparsely sampled image, and uses the output of the last iterative module as the target image. The image restoration unit includes nine iterative modules with identical structures, and the output of any iterative module is determined by the following method: ； In the formula, For the reason get Intermediate quantities in the process For the output of the iterative module, The input to the iteration module is k, which is the iteration index. The initial value is 1. After each iteration module, k = k + 1 is executed. The value of k is 1 ≤ k ≤ 9. This is the output of the iterative module k-1; ρ represents the value in the (k-1)th iteration. The step size for moving in the direction of the negative gradient of the fidelity term; Φ is the observation matrix, specifically the matrix Φ corresponding to the sparse array of pixels on the focal plane, which is a randomly generated (0,1) Gaussian random matrix of size 64×64, and y is the one-dimensional representation of the sparsely sampled image of the lidar. ； In the formula, Ψ is the output of the iterative module, λ is the non-zero parameter in the optimization problem, and Ψ is the sparse basis matrix in compressed sensing.

9. The apparatus according to claim 7, characterized in that, The multi-channel synchronous timing circuit is also used for: The laser flight time is synchronized to the massive data acquisition unit.

10. The apparatus according to claim 7, characterized in that, The image restoration unit is trained using the following method: 978 33×33 sub-images were extracted from the luminance component of each image in the training dataset to generate training data pairs. Used for training the network; The loss function involved in training is determined as follows: ； In the formula, Nb is the number of images in the training dataset, which is 978; N is the size of each image in the training dataset, which is 1089; To obtain the output of the reconstructed image of the i-th image in the training dataset, i.e., the one-dimensional representation of the restored image, This is a one-dimensional representation of the i-th image sample; The image restoration unit is trained based on the training data and the loss function until the training objective is met.

Images