A target recognition method, device and equipment of single-photon three-dimensional range image and medium

Abstract

This application provides a method, apparatus, device, and medium for target recognition using single-photon 3D range images. The method includes: obtaining a single-photon 3D range image by detecting echo signals using a coaxial array single-photon lidar system and performing frame accumulation and reconstruction processing; inputting the single-photon 3D range image into an improved YOLO V5 model for feature extraction and target recognition to determine the recognition result; wherein the improved YOLO V5 model is trained by integrating the spatial density function into a traditional convolutional kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network. This effectively solves the problems of low signal-to-noise ratio and slow recognition speed in traditional imaging and recognition technologies under extreme environments such as extremely low light, high dynamic range, or long distances, thus improving the accuracy of target recognition.

Description

Technical Field

[0001] This application relates to the fields of single-photon imaging and computer vision technology, and in particular to a method, apparatus, device and medium for target recognition of single-photon three-dimensional distance images. Background Technology

[0002] With the rapid development of optical detection technology and computer vision, imaging technology based on single-photon detection has shown great application potential in remote sensing, low-light environments, and other fields. Single-photon detection technology possesses extremely high sensitivity and temporal resolution, capable of recording the arrival time and location of individual photons, forming unique single-photon point cloud images. Unlike traditional optical images, single-photon point cloud images typically exhibit low signal-to-noise ratios and highly sparse data distributions, posing significant challenges to subsequent target recognition and information extraction.

[0003] Currently, target recognition methods for point cloud data mainly include traditional image processing methods and deep learning-based 3D point cloud processing methods. Traditional image processing methods first convert point cloud data into regularized 2D images or 3D voxel meshes through interpolation, voxelization, or projection, and then extract target features and perform recognition through edge detection, thresholding, and other operations. These methods heavily rely on manually designed features and complex parameter adjustments. For complex scenes and low signal-to-noise ratio (SNR) single-photon point cloud images, robustness and accuracy are often insufficient, and generalization ability is poor. Deep learning models, represented by PointNet and VoxelNet, process raw point cloud data directly or indirectly, achieving end-to-end learning and significantly improving target recognition performance. However, these methods typically have high computational complexity, requiring huge computing resources, making them difficult to meet the real-time requirements of application scenarios. More importantly, most of these models are designed for traditional LiDAR point clouds with high SNR. When directly applied to single-photon point cloud images with extremely low SNR and sparse photon counts, their performance drops sharply, making it difficult to extract effective features. Therefore, improving the accuracy and efficiency of target recognition using single-photon 3D distance images has become a significant technical challenge. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a target recognition method, device, equipment and medium for single-photon three-dimensional range images. By deeply coupling the transceiver coaxial array single-photon lidar system with the improved YOLO V5 model oriented towards the characteristics of single-photon three-dimensional range images, the problems of low target detection signal-to-noise ratio and slow recognition speed in extreme environments such as extremely weak light, high dynamics or long distances of traditional imaging and recognition technologies are effectively solved, thereby improving the accuracy of target recognition.

[0005] This application provides a target recognition method for single-photon three-dimensional range images, the target recognition method comprising: Acquire a single-photon three-dimensional range image; wherein, the single-photon three-dimensional range image is obtained by detecting the echo signal through a transceiver coaxial array single-photon lidar system and performing frame accumulation and reconstruction processing on the echo signal; The single-photon 3D range image is input into the improved YOLO V5 model to perform feature extraction and target recognition on the single-photon 3D range image, and the recognition result is determined; wherein, the recognition result includes the rectangular bounding box, confidence score and category label of each detected target; The improved YOLO V5 model is obtained by integrating the spatial density function into the traditional convolution kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network layer before training.

[0006] In one possible implementation, a single-photon three-dimensional range image is obtained by: The target is detected by a coaxial array single-photon lidar system, and the raw echo signal is obtained. The original echo signal is processed by time-correlated single-photon counting, and the frame count is accumulated for multiple photon arrival timestamps of each pixel unit to generate a photon count and time histogram. Based on the photon count and time histogram, the target's distance, intensity, and background information are estimated, and a gradient iterative denoising algorithm is used to reconstruct a single-photon three-dimensional distance image.

[0007] In one possible implementation, the step of estimating the target's distance information, intensity information, and background information based on the photon count and time histogram, and reconstructing a single-photon three-dimensional range image using a gradient iterative denoising algorithm, includes: Distance information is determined based on the peak position of the photon count and time histogram, intensity information is determined based on the peak height, and background information is determined based on the baseline level of the histogram. After performing near-end gradient update on the distance information, a point set surface reconstruction algorithm based on moving least squares is used to denoise the updated distance information; After performing near-end gradient update on the intensity information, low-pass filtering is performed on the updated intensity information of each pixel within its k nearest neighbors to remove noise; After performing a near-end gradient update on the background information, a fast Fourier transform frequency domain filter is used to denoise the updated background information. Repeat the above denoising process iteratively until the preset number of iterations is reached, and reconstruct the single-photon three-dimensional distance image.

[0008] In one possible implementation, the step of inputting the single-photon three-dimensional range image into an improved YOLOv5 model, performing feature extraction and target recognition on the single-photon three-dimensional range image, and determining the recognition result includes: The backbone network layer of the improved YOLO V5 model performs weighted convolutional feature extraction on the single-photon 3D distance image to determine a multi-scale feature map; The multi-scale feature map and the first feature map branch are bidirectionally fused based on the neck network layer of the improved YOLO V5 model to determine the fused feature map. The head network layer of the improved YOLO V5 model decouples the fused feature map and outputs the recognition result.

[0009] In one possible implementation, the backbone network layer based on the improved YOLO V5 model performs weighted convolutional feature extraction on the single-photon 3D distance image to determine a multi-scale feature map, including: The initial feature map is generated by extracting features from the single-photon three-dimensional distance image through an initial convolutional layer. The initial feature map is divided into a first feature map branch and a second feature map branch along the channel dimension; The first feature map branch is directly passed to the neck network layer as a direct feature path. The second feature map branch is input into a dense residual weighted convolution block for processing to determine the multi-scale feature map.

[0010] In one possible implementation, the dense residual weighted convolutional block includes a plurality of weighted convolutional layers and residual connections cascaded in sequence, wherein each weighted convolutional layer employs a weighted convolution operation with an integrated spatial density function.

[0011] In one possible implementation, the improved YOLO V5 model is trained in the following manner: Rectangular annotations are performed on the three-dimensional range images of single photons in the sample dataset to determine the model training dataset; The improved YOLO V5 model is trained end-to-end based on the model training dataset, and the trained improved YOLO V5 model and model weights are saved.

[0012] This application embodiment also provides a target recognition device for single-photon three-dimensional range images, the target recognition device comprising: The acquisition module is used to acquire a single-photon three-dimensional range image; wherein, the single-photon three-dimensional range image is obtained by a transceiver coaxial array single-photon lidar system detecting and performing frame accumulation and reconstruction processing on the echo signal; The processing module is used to input the single-photon 3D range image into the improved YOLO V5 model, perform feature extraction and target recognition on the single-photon 3D range image, and determine the recognition result; wherein, the recognition result includes the rectangular bounding box, confidence score, and category label of each detected target; wherein, the improved YOLO V5 model is trained by integrating the spatial density function into the traditional convolution kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network layer.

[0013] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the target recognition method for single-photon three-dimensional distance images described above are performed.

[0014] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the target recognition method for single-photon three-dimensional distance images as described above.

[0015] This application provides a target recognition method, apparatus, device, and medium based on a single-photon three-dimensional range image. The target recognition method includes: acquiring a single-photon three-dimensional range image; wherein the single-photon three-dimensional range image is obtained by detecting the echo signal using a transceiver coaxial array single-photon lidar system and performing frame accumulation and reconstruction processing; inputting the single-photon three-dimensional range image into an improved YOLO V5 model, performing feature extraction and target recognition on the single-photon three-dimensional range image, and determining the recognition result; wherein the recognition result includes the rectangular bounding box, confidence score, and category label of each detected target; wherein the improved YOLO V5 model is trained by integrating the spatial density function into a traditional convolutional kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network. By deeply coupling the transceiver coaxial array single-photon lidar system with the improved YOLO V5 model designed for the characteristics of single-photon three-dimensional range images, the problems of low target detection signal-to-noise ratio and slow recognition speed in extreme environments such as extremely low light, high dynamics, or long distances are effectively solved by traditional imaging and recognition technologies, thereby improving the accuracy of target recognition.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a target recognition method for a single-photon three-dimensional range image provided in this application embodiment; Figure 2 This is one of the structural schematic diagrams of a target recognition device for single-photon three-dimensional range imaging provided in an embodiment of this application; Figure 3 This is a second schematic diagram of the structure of a target recognition device for single-photon three-dimensional distance imaging provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0020] First, the applicable scenarios for this application will be introduced. This application can be applied to the fields of single-photon imaging and computer vision technology.

[0021] Research has revealed that existing target recognition methods for point cloud data mainly include traditional image processing methods and deep learning-based 3D point cloud processing methods. Traditional image processing methods first convert point cloud data into regularized 2D images or 3D voxel meshes through interpolation, voxelization, or projection, then extract target features and perform recognition through edge detection, thresholding, and other operations. These methods heavily rely on manually designed features and complex parameter adjustments, often exhibiting insufficient robustness and accuracy, and poor generalization ability, for complex scenes and low signal-to-noise ratio (SNR) single-photon point cloud images. Deep learning models, such as PointNet and VoxelNet, process raw point cloud data directly or indirectly, achieving end-to-end learning and significantly improving target recognition performance. However, these methods typically have high computational complexity, requiring substantial computing resources and making them unsuitable for applications with high real-time requirements. More importantly, most of these models are designed for traditional LiDAR point clouds with high SNR. When directly applied to single-photon point cloud images with extremely low SNR and sparse photon counts, their performance deteriorates sharply, making it difficult to extract effective features. Therefore, improving the accuracy and efficiency of target recognition using single-photon 3D distance images has become a significant technical challenge.

[0022] Based on this, the embodiments of this application provide a target recognition method for single-photon three-dimensional range images. By deeply coupling the transceiver coaxial array single-photon lidar system with the improved YOLO V5 model oriented towards the characteristics of single-photon three-dimensional range images, the method effectively solves the problems of low target detection signal-to-noise ratio and slow recognition speed in extreme environments such as extremely weak light, high dynamics or long distances, and improves the accuracy of target recognition.

[0023] Please see Figure 1 , Figure 1 This is a flowchart illustrating a target recognition method using a single-photon three-dimensional range image, as provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the target identification method includes: S101: Acquire a single-photon three-dimensional range image; wherein, the single-photon three-dimensional range image is obtained by a transceiver coaxial array single-photon lidar system detecting and performing frame accumulation and reconstruction processing on the echo signal.

[0024] In this step, a single-photon three-dimensional range image is obtained by using a transceiver coaxial array single-photon lidar system to detect and perform frame accumulation and reconstruction processing on the echo signal.

[0025] In one possible implementation, a single-photon three-dimensional range image is obtained by: A: The target is detected by a coaxial array single-photon lidar system to obtain the raw echo signal.

[0026] Here, the pulse width and pulse repetition frequency of the pulsed laser in the transceiver coaxial array single-photon lidar system are determined based on the required detection distance. When the laser pulse width is... The pulse repetition frequency is At that time, the maximum unambiguous range of a single-photon lidar is: The depth distance is Multiple probes are performed on different targets at different distances to obtain the original echo signals.

[0027] The transceiver coaxial array single-photon lidar system consists of a 1550nm pulsed laser source, a 6-inch Cassegrain telescope with a 1500mm focal length, and a 64*64 array of single-photon detectors.

[0028] B: Perform time-correlated single-photon counting processing on the original echo signal, accumulate the frame count for multiple photon arrival timestamps of each pixel unit, and generate a photon count and time histogram.

[0029] Here, time-correlated single-photon counting is performed on the arrival timestamps of photons detected multiple times by each pixel unit in the array single-photon detector to generate a photon count-time histogram.

[0030] It should be noted that the photon count vs. time histogram is a histogram of the relationship between photon count and time.

[0031] C: Based on the photon count and time histogram, the target's distance information, intensity information, and background information are estimated, and a gradient iterative denoising algorithm is used to reconstruct a single-photon three-dimensional distance image.

[0032] Here, distance, intensity, and background information for different distances and targets are estimated based on the photon count-time histogram, and a gradient iterative denoising algorithm is used to reconstruct a high-quality single-photon three-dimensional distance image.

[0033] In one possible implementation, the step of estimating the target's distance information, intensity information, and background information based on the photon count and time histogram, and reconstructing a single-photon three-dimensional range image using a gradient iterative denoising algorithm, includes: (1): Distance information is determined based on the peak position of the photon count and time histogram, intensity information is determined based on the peak height, and background information is determined based on the baseline level of the histogram.

[0034] Here, distance information is determined based on the peak position of the photon count and time histogram, intensity information is determined based on the peak height, and background information is determined based on the histogram baseline level.

[0035] (2): After performing near-end gradient update on the distance information, the updated distance information is denoised using a point set surface reconstruction algorithm based on moving least squares.

[0036] Here, after performing near-end gradient update on the distance information, a point set surface reconstruction algorithm based on moving least squares is used to denoise the updated distance information.

[0037] (3): After performing near-end gradient update on the intensity information, perform low-pass filtering on the updated intensity information for each pixel within its k nearest neighbors to denoise it.

[0038] Here, neighborhood adaptive low-pass filtering is applied to the intensity information for noise reduction. The intensity information has strong spatial correlation but is easily affected by outlier photons, so local low-pass filtering guided by k nearest neighbors is adopted.

[0039] (4): After performing near-end gradient update on the background information, the updated background information is denoised by using fast Fourier transform frequency domain filtering.

[0040] Here, frequency domain sparsity-driven denoising is implemented on the background information. The background component exhibits a slowly varying characteristic in the time domain and has significant low-frequency energy concentration in the frequency domain. Therefore, FFT frequency domain hard threshold filtering is adopted.

[0041] (5): Repeat the above denoising process until the preset number of iterations is reached, and reconstruct the single-photon three-dimensional distance image.

[0042] Here, the above steps are repeated multiple times until the preset number of iterations is reached. Then, the denoised distance information, intensity information, and background information are reconstructed to obtain a single-photon three-dimensional distance image.

[0043] This application proposes a three-information coupled gradient iterative denoising framework: for distance information, a "near-end gradient update + moving least squares (MLS) surface reconstruction" is used, leveraging the spatial geometric continuity constraint of the point cloud to suppress distance jump noise while preserving edge sharpness; for intensity information, a "near-end gradient update + nearest neighbor low-pass filtering" is implemented to adaptively smooth photon count fluctuations within the pixel neighborhood, avoiding detail blurring caused by global mean filtering; for background information, a "near-end gradient update + FFT frequency domain hard thresholding denoising" is applied to accurately separate broadband background noise from narrowband target echoes. After joint iterative reconstruction, the three elements provide YOLOv5 recognition with a high signal-to-noise ratio, clear structure, and explicit physical meaning in a two-dimensional input, fundamentally avoiding information loss caused by point cloud voxelization or projection.

[0044] S102: Input the single-photon three-dimensional range image into the improved YOLO V5 model, perform feature extraction and target recognition on the single-photon three-dimensional range image, and determine the recognition result; wherein, the recognition result includes the rectangular bounding box, confidence score and category label of each detected target.

[0045] In this step, the single-photon three-dimensional range image is input into a pre-trained improved YOLO V5 model to extract features and identify targets from the single-photon three-dimensional range image, and the identification result is determined.

[0046] The improved YOLO V5 model is obtained by integrating the spatial density function into the traditional convolution kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network layer before training.

[0047] Here, the weighted convolution used in the improved YOLO V5 is achieved by introducing a spatial density function Φ∈ Ka×Kb Element-wise modulation of the convolution kernel in traditional convolution allows it to adaptively allocate different weights based on the spatial location of pixels, thereby highlighting the feature responses of important regions. This effectively enhances the model's ability to perceive image details and structure, significantly improving the network's feature extraction performance. Specifically, for a given input feature map and convolution kernel, the formula for weighted convolution is:

[0048] in, For the input feature map, The first one after being modulated by the spatial density function Φ f There are 1 convolutional kernel, where x and y are the coordinates of the feature map. , These represent the height and width of the convolution kernel, respectively. a and b are the local coordinate indices within the convolution kernel. The spatial density function Φ at the convolution kernel position ( a , b The value at ) For the original number f The value of the convolution kernel at position (a, b).

[0049] As shown in the above formula, the introduction of weighted convolution can achieve perfect compatibility with existing convolutional neural network architectures without increasing the number of trainable parameters, with minimal increase in computational overhead. To efficiently compute the density function, it is designed as a symmetric, positive semi-definite matrix with rank 1. ,in satisfy And the center value is fixed at 1. For example, for a commonly used 3×3 convolution kernel, only one hyperparameter needs to be adjusted. Its density matrix is ​​expressed as:

[0050] In one possible implementation, the step of inputting the single-photon three-dimensional range image into an improved YOLOv5 model, performing feature extraction and target recognition on the single-photon three-dimensional range image, and determining the recognition result includes: a: Based on the backbone network layer of the improved YOLO V5 model, weighted convolutional feature extraction is performed on the single-photon three-dimensional distance image to determine the multi-scale feature map.

[0051] Here, weighted convolutional feature extraction is performed on the single-photon 3D distance image using the backbone network layer to determine the multi-scale feature map.

[0052] In one possible implementation, the backbone network layer based on the improved YOLO V5 model performs weighted convolutional feature extraction on the single-photon 3D distance image to determine a multi-scale feature map, including: The initial convolutional layer extracts features from the single-photon 3D distance image to generate an initial feature map. The initial feature map is then divided into a first feature map branch and a second feature map branch along the channel dimension. The first feature map branch is directly passed to the neck network layer as a direct feature path. The second feature map branch is input into a dense residual weighted convolutional block for processing to determine a multi-scale feature map.

[0053] Here, the backbone network is based on the CSPDarknet53 structure. It extracts features from the input single-photon 3D range image and divides the feature map into two parts. One part is directly processed, and the other part is merged with the former after passing through a dense residual weighted convolution block. This reduces the amount of computation and maintains the diversity of features.

[0054] The dense residual weighted convolutional block includes multiple weighted convolutional layers and residual connections cascaded in sequence, wherein each weighted convolutional layer uses a weighted convolutional operation with an integrated spatial density function instead of a traditional convolutional operation.

[0055] b: Based on the neck network layer of the improved YOLO V5 model, the multi-scale feature map and the first feature map branch are bidirectionally fused to determine the fused feature map.

[0056] Here, the neck network receives the multi-scale feature maps (mathbf{F}_2, mathbf{F}_3, mathbf{F}_4) output by the backbone network and the feature maps of the aforementioned first feature map branch (as a high-resolution supplement), and performs two-level fusion: Starting from the highest-level semantic feature mathbf{F}_4 in the top-down path, it undergoes 1×1 convolution dimensionality reduction + upsampling (×2), and then element-wise addition and fusion with mathbf{F}_3; the fusion result is then subjected to another 1×1 convolution + upsampling (×2), and added to mathbf{F}_2 to generate enhanced mid-level features. In the bottom-up path, the feature map of the first feature map branch is subjected to 3×3 convolution + downsampling (×2), and then concatenated and fused with mathbf{F}_3; this fused feature is then subjected to another 3×3 convolution + downsampling (×2), and concatenated and fused with mathbf{F}_4 to generate enhanced deep features. Ultimately, the neck network outputs multiple sets of fused feature maps.

[0057] c: The recognition result is output by decoupling the fused feature map based on the head network layer of the improved YOLO V5 model.

[0058] Here, the head network feeds the fused feature maps into a decoupled prediction head: each prediction head contains two parallel branches: (1) Regression branch: predicts the target center point offset (t_x, t_y), width and height scaling factors (t_w, t_h), and depth offset t_d, for a total of 5 dimensions; (2) Classification branch: predicts the probability distribution of the target's category. The regression branch is optimized using IoU-awareLoss, and the classification branch is optimized using FocalLoss to suppress background class samples. Finally, for each spatial location (h, w) on each feature map, a structured prediction tuple is output: ( xc , yc , w , h , d ,conf,pcls). Among them ( xc , yc It is obtained by decoding the grid coordinates and offsets. w , h Mapped to pixel size d The bounding box confidence score is determined by a lookup table from the distance image depth map. After non-maximum suppression (NMS, IoU threshold of 0.45), the final recognition result is output, including the target category label, two-dimensional bounding box coordinates (mapped to the original distance image plane), corresponding three-dimensional spatial coordinates (X,Y,Z), and confidence score.

[0059] In one possible implementation, the improved YOLO V5 model is trained in the following manner: Rectangular annotations are performed on the single-photon 3D distance images of the samples in the sample dataset to determine the model training dataset; the improved YOLO V5 model is trained end-to-end based on the model training dataset, and the trained improved YOLO V5 model and model weights are saved.

[0060] Here, the Labelme tool is used to label the single-photon 3D distance images of the samples in the sample dataset with rectangles to determine the model training dataset. The improved YOLO V5 model is trained using the preprocessed training dataset, and the trained model and training weights are saved.

[0061] It's worth noting that YOLO V5's architecture integrates a high-efficiency backbone network, a powerful neck network, and a fast, decoupled head network, forming an end-to-end object detection model. The backbone network uses a CSP-based Darknet53 structure, with the C3 module as its core. By splitting and merging feature maps along the channel dimension, it achieves effective reuse of gradient information and a significant reduction in computational cost, ensuring improved computational efficiency and inference speed while extracting depth. The neck network innovatively combines bottom-up and top-down structures. This bidirectional feature fusion mechanism enables the model to effectively handle drastic changes in target scale in single-photon 3D range images, achieving simultaneous detection of targets of varying distances and sizes. The head network employs a decoupled design, separating the target classification task from the bounding box regression task, allowing the model to focus more on its respective targets, improving prediction accuracy and convergence speed.

[0062] In this application, the existing YOLOv5 directly processes single-photon distance images, but due to image sparsity (effective pixel ratio is often <5%), the convolution kernel response is distorted, the signal-to-noise ratio of the early layers of the backbone network drops sharply, and the classification confidence is generally below 0.3. This invention achieves a fundamental improvement by embedding the spatial density function Φ into the convolution kernel weights, so that the convolution operation automatically assigns higher response weights to high photon count regions and weights to low / zero count regions approaching 0. This design does not increase the number of trainable parameters (it is still the standard number of convolution parameters), but improves the effective activation pixel rate of the first layer feature map of the backbone network and reduces the attenuation of gradient flow in sparse regions; combined with the "feature splitting-weighted fusion" mechanism in the CSPDarknet53 backbone, it maintains the integrity of deep semantic features and strengthens shallow spatial localization features through residual weighted blocks, thereby improving the detection of small targets.

[0063] In a specific implementation, the first step involves selecting a suitable pulse width laser and setting an appropriate repetition frequency, then using a transceiver coaxial array single-photon lidar system to acquire data multiple times from different targets at different distances, storing the original dataset. The second step involves preprocessing the original data acquired by the transceiver coaxial array single-photon lidar based on the principle of time-correlated single-photon counting, obtaining a photon count-time histogram dataset after frame accumulation. The third step involves preliminarily estimating the target's distance, intensity, and background information based on the photon count-time histogram, and then using a gradient iterative denoising algorithm to iterate through these parameters multiple times, ultimately reconstructing a sample single-photon 3D range image dataset. The fourth step involves using Labelme software to perform rectangular annotations on the sample single-photon 3D range image dataset, saving the annotation results, and exporting them as a training dataset in a specific format. An improved YOLO V5 model is then constructed by introducing a density function into the convolution operation to build a weighted convolution and replacing all traditional convolution operations in the YOLO V5 model. Fifth, use the improved YOLO V5 model constructed in step four and the preprocessed sample dataset to train target recognition, and save the trained model and training weights. Sixth, use the saved trained model and training weights to construct a 3D range image of the target for unknown distances and unknown targets in the first three steps, input it into the trained YOLO V5 model to perform single-photon 3D range image target recognition, and output and save the target recognition results.

[0064] This application provides a target recognition method based on a single-photon three-dimensional range image. The target recognition method includes: acquiring a single-photon three-dimensional range image; wherein the single-photon three-dimensional range image is obtained by detecting the echo signal through a transceiver coaxial array single-photon lidar system and performing frame accumulation and reconstruction processing; inputting the single-photon three-dimensional range image into an improved YOLO V5 model, performing feature extraction and target recognition on the single-photon three-dimensional range image, and determining the recognition result; wherein the recognition result includes the rectangular bounding box, confidence score, and category label of each detected target; wherein the improved YOLO V5 model is trained by integrating the spatial density function into the traditional convolution kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network. By deeply coupling the transceiver coaxial array single-photon lidar system with the improved YOLO V5 model oriented towards the characteristics of single-photon three-dimensional range imaging, a "detection-reconstruction-recognition" closed-loop technology is formed, which effectively solves the problems of low target detection signal-to-noise ratio and slow recognition speed in extreme environments such as extremely low light, high dynamics or long distances, and improves the accuracy of target recognition.

[0065] Please see Figure 2 , Figure 3 , Figure 2This is one of the structural schematic diagrams of a target recognition device for single-photon three-dimensional range imaging provided in an embodiment of this application; Figure 3 This is a second schematic diagram of the structure of a target recognition device for single-photon three-dimensional distance imaging provided in an embodiment of this application. Figure 2 As shown, the target recognition device 300 for single-photon three-dimensional range imaging includes: The acquisition module 210 is used to acquire a single-photon three-dimensional range image; wherein, the single-photon three-dimensional range image is obtained by a transceiver coaxial array single-photon lidar system detecting and performing frame accumulation and reconstruction processing on the echo signal; The processing module 220 is used to input the single-photon 3D range image into the improved YOLO V5 model, perform feature extraction and target recognition on the single-photon 3D range image, and determine the recognition result; wherein, the recognition result includes the rectangular bounding box, confidence score, and category label of each detected target; wherein, the improved YOLO V5 model is trained by integrating the spatial density function into the traditional convolution kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network layer.

[0066] Furthermore, the acquisition module 210 acquires the single-photon three-dimensional range image in the following manner: The target is detected by a coaxial array single-photon lidar system, and the raw echo signal is obtained. The original echo signal is processed by time-correlated single-photon counting, and the frame count is accumulated for multiple photon arrival timestamps of each pixel unit to generate a photon count and time histogram. Based on the photon count and time histogram, the target's distance, intensity, and background information are estimated, and a gradient iterative denoising algorithm is used to reconstruct a single-photon three-dimensional distance image.

[0067] Furthermore, the acquisition module 210 is used to estimate the target's distance information, intensity information, and background information based on the photon count and time histogram, and reconstructs a single-photon three-dimensional distance image using a gradient iterative denoising algorithm. Distance information is determined based on the peak position of the photon count and time histogram, intensity information is determined based on the peak height, and background information is determined based on the baseline level of the histogram. After performing near-end gradient update on the distance information, a point set surface reconstruction algorithm based on moving least squares is used to denoise the updated distance information; After performing near-end gradient update on the intensity information, low-pass filtering is performed on the updated intensity information of each pixel within its k nearest neighbors to remove noise; After performing a near-end gradient update on the background information, a fast Fourier transform frequency domain filter is used to denoise the updated background information. Repeat the above denoising process iteratively until the preset number of iterations is reached, and reconstruct the single-photon three-dimensional distance image.

[0068] Furthermore, the processing module 220 is used to input the single-photon three-dimensional range image into the improved YOLO V5 model, perform feature extraction and target recognition on the single-photon three-dimensional range image, and determine the recognition result: The backbone network layer of the improved YOLO V5 model performs weighted convolutional feature extraction on the single-photon 3D distance image to determine a multi-scale feature map; The multi-scale feature map and the first feature map branch are bidirectionally fused based on the neck network layer of the improved YOLO V5 model to determine the fused feature map. The head network layer of the improved YOLO V5 model decouples the fused feature map and outputs the recognition result.

[0069] Furthermore, the processing module 220 performs weighted convolutional feature extraction on the single-photon 3D distance image using the backbone network layer based on the improved YOLO V5 model to determine a multi-scale feature map: The initial feature map is generated by extracting features from the single-photon three-dimensional distance image through an initial convolutional layer. The initial feature map is divided into a first feature map branch and a second feature map branch along the channel dimension; The first feature map branch is directly passed to the neck network layer as a direct feature path. The second feature map branch is input into a dense residual weighted convolution block for processing to determine the multi-scale feature map.

[0070] Furthermore, such as Figure 3 As shown, the target recognition device 200 for single-photon three-dimensional distance images also includes a training module 230, which trains the improved YOLO V5 model in the following manner: Rectangular annotations are performed on the three-dimensional range images of single photons in the sample dataset to determine the model training dataset; The improved YOLO V5 model is trained end-to-end based on the model training dataset, and the trained improved YOLO V5 model and model weights are saved.

[0071] This application provides a target recognition device for single-photon three-dimensional range images. The target recognition device includes: an acquisition module for acquiring single-photon three-dimensional range images; wherein the single-photon three-dimensional range images are obtained by a transceiver coaxial array single-photon lidar system detecting and performing frame accumulation and reconstruction processing on the echo signals; and a processing module for inputting the single-photon three-dimensional range images into an improved YOLO V5 model to perform feature extraction and target recognition on the single-photon three-dimensional range images, and determining the recognition result; wherein the recognition result includes the rectangular bounding box, confidence score, and category label of each detected target; wherein the improved YOLO V5 model is trained by integrating the spatial density function into the traditional convolution kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network. By deeply coupling the transceiver coaxial array single-photon lidar system with the improved YOLO V5 model oriented towards the characteristics of single-photon three-dimensional range imaging, a "detection-reconstruction-recognition" closed-loop technology is formed, which effectively solves the problems of low target detection signal-to-noise ratio and slow recognition speed in extreme environments such as extremely low light, high dynamics or long distances, and improves the accuracy of target recognition.

[0072] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0073] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps of the target recognition method for single-photon three-dimensional distance images in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0074] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the target recognition method for single-photon three-dimensional distance images in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0075] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0077] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network layer units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0078] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0079] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network layer device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A target recognition method based on a single-photon three-dimensional range image, characterized in that, The target recognition method includes: Acquire a single-photon three-dimensional range image; wherein, the single-photon three-dimensional range image is obtained by detecting the echo signal through a transceiver coaxial array single-photon lidar system and performing frame accumulation and reconstruction processing on the echo signal; The single-photon 3D range image is input into the improved YOLO V5 model to perform feature extraction and target recognition on the single-photon 3D range image, and the recognition result is determined; wherein, the recognition result includes the rectangular bounding box, confidence score and category label of each detected target; The improved YOLO V5 model is obtained by integrating the spatial density function into the traditional convolution kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network layer before training.

2. The target recognition method according to claim 1, characterized in that, The three-dimensional range image of a single photon is obtained through the following methods: The target is detected by a coaxial array single-photon lidar system, and the raw echo signal is obtained. The original echo signal is processed by time-correlated single-photon counting, and the frame count is accumulated for multiple photon arrival timestamps of each pixel unit to generate a photon count and time histogram. Based on the photon count and time histogram, the target's distance, intensity, and background information are estimated, and a gradient iterative denoising algorithm is used to reconstruct a single-photon three-dimensional distance image.

3. The target recognition method according to claim 2, characterized in that, The step of estimating the target's distance, intensity, and background information based on the photon count and time histogram, and reconstructing a single-photon three-dimensional range image using a gradient iterative denoising algorithm, includes: Distance information is determined based on the peak position of the photon count and time histogram, intensity information is determined based on the peak height, and background information is determined based on the baseline level of the histogram. After performing near-end gradient update on the distance information, a point set surface reconstruction algorithm based on moving least squares is used to denoise the updated distance information; After performing near-end gradient update on the intensity information, low-pass filtering is performed on the updated intensity information of each pixel within its k nearest neighbors to denoise it. After performing a near-end gradient update on the background information, a fast Fourier transform frequency domain filter is used to denoise the updated background information. Repeat the above denoising process iteratively until the preset number of iterations is reached, and reconstruct the single-photon three-dimensional distance image.

4. The target recognition method according to claim 1, characterized in that, The process of inputting the single-photon three-dimensional range image into the improved YOLO V5 model, performing feature extraction and target recognition on the single-photon three-dimensional range image, and determining the recognition result includes: The backbone network layer of the improved YOLO V5 model performs weighted convolutional feature extraction on the single-photon 3D distance image to determine a multi-scale feature map; The multi-scale feature map and the first feature map branch are bidirectionally fused based on the neck network layer of the improved YOLO V5 model to determine the fused feature map; The head network layer of the improved YOLO V5 model decouples the fused feature map and outputs the recognition result.

5. The target recognition method according to claim 4, characterized in that, The backbone network layer based on the improved YOLO V5 model performs weighted convolutional feature extraction on the single-photon 3D distance image to determine a multi-scale feature map, including: The initial feature map is generated by extracting features from the single-photon three-dimensional distance image through an initial convolutional layer. The initial feature map is divided into a first feature map branch and a second feature map branch along the channel dimension; The first feature map branch is directly passed to the neck network layer as a direct feature path. The second feature map branch is input into a dense residual weighted convolution block for processing to determine the multi-scale feature map.

6. The target recognition method according to claim 5, characterized in that, The dense residual weighted convolutional block includes multiple weighted convolutional layers and residual connections cascaded in sequence, wherein each weighted convolutional layer employs a weighted convolution operation with an integrated spatial density function.

7. The target recognition method according to claim 1, characterized in that, The improved YOLOv5 model was trained using the following method: Rectangular annotations are performed on the three-dimensional range images of single photons in the sample dataset to determine the model training dataset; The improved YOLO V5 model is trained end-to-end based on the model training dataset, and the trained improved YOLO V5 model and model weights are saved.

8. A target recognition device for single-photon three-dimensional distance imaging, characterized in that, The target identification device includes: The acquisition module is used to acquire a single-photon three-dimensional range image; wherein, the single-photon three-dimensional range image is obtained by a transceiver coaxial array single-photon lidar system detecting and performing frame accumulation and reconstruction processing on the echo signal; The processing module is used to input the single-photon 3D range image into the improved YOLO V5 model, perform feature extraction and target recognition on the single-photon 3D range image, and determine the recognition result; wherein, the recognition result includes the rectangular bounding box, confidence score, and category label of each detected target; wherein, the improved YOLO V5 model is trained by integrating the spatial density function into the traditional convolution kernel to form a weighted convolution operation, and replacing all traditional convolution operations in the backbone, neck, and head layers of the YOLO V5 network layer.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the target recognition method for single-photon three-dimensional range images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the target recognition method for single-photon three-dimensional distance images as described in any one of claims 1 to 7.

Images