A reconstruction detection framework system based on fourier single-pixel imaging system

Abstract

The application discloses a reconstruction detection framework system based on a Fourier single-pixel imaging system, extracts features from an undersampling image through a reconstruction network, and obtains a Fourier single-pixel imaging reconstruction result; the reconstruction network can reduce model complexity while guaranteeing excellent reconstruction performance; the Fourier single-pixel imaging reconstruction result is sent into a target detection module for target detection, a target detection network is selected as a benchmark model of the detection module, a fast spatial pyramid pooling module in the target detection network is optimized into a hybrid fast spatial pyramid pooling module, and a deformable convolution network is introduced into the target detection module; a joint training strategy module uses a detection loss to guide training of the Fourier single-pixel image reconstruction network through back propagation, forces the reconstruction network to generate an image more suitable for detection, and improves the final performance of target detection, and has the technical features of being capable of enhancing the quality of the undersampling image, improving target detection performance and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to a reconstruction detection framework system, more particularly, to a reconstruction detection framework system based on a Fourier ptychographic system, and belongs to the field of optical imaging. BACKGROUND

[0002] Target detection is widely used in robot navigation, intelligent video monitoring, industrial detection, underwater detection and other fields, and is a popular direction in the field of computer vision. Since the image source of traditional target detection comes from array imaging devices, there are many challenges in extreme environmental conditions, such as light attenuation, light scattering effect, and limitation of observation angle and depth in long-distance environment. Therefore, it is urgent to explore new imaging technology to overcome these challenges.

[0003] Fourier ptychographic imaging is a high-robustness imaging technology that uses a single-pixel detector without spatial resolution capability to reconstruct target scene information. Compared with traditional optical imaging, Fourier ptychographic imaging has the advantages of strong anti-noise capability, wide spectral band, non-local imaging, and long imaging range. Therefore, combining target detection technology with Fourier ptychographic imaging has the potential to detect targets in optical-unfriendly environments, especially in scattering media and remote sensing of large-scale dynamic scenes. However, in order to improve the speed of Fourier ptychographic imaging, the target scene needs to be undersampled, which inevitably leads to ringing effect in the imaging result. In addition, with the decrease of sampling rate, the quality of the reconstruction result will also decrease. All these shortcomings limit the subsequent target detection task of Fourier ptychographic imaging.

[0004] In recent years, the reconstruction algorithm of Fourier ptychographic imaging based on deep learning has to some extent alleviated the contradiction between imaging quality and sampling rate, making it possible to use Fourier ptychographic imaging reconstruction results for target detection. However, there are still some unsolved problems in the task of Fourier ptychographic imaging target detection. First, there are a large number of parameters in the current high-quality Fourier ptychographic imaging reconstruction network; after combining the large-parameter reconstruction network with the target detection network, the model is difficult to train and deploy. In addition, most reconstruction methods prefer to generate visually satisfactory images, ignoring the purpose of the target detection task. The independent optimization of the Fourier ptychographic imaging reconstruction task and the target detection task leads to a gap between the reconstruction task and the detection task, hindering the detector to achieve optimal detection performance. Therefore, it is necessary to design a joint framework that can perform image reconstruction and real-time detection. SUMMARY

[0005] In order to solve the above prior art problems, the present application provides a reconstruction detection framework system based on a Fourier single-pixel imaging system, which can enhance the quality of the undersampled image and improve the target detection performance.

[0006] In order to achieve the above purpose, the present application is realized by the following technical scheme:

[0007] A reconstruction detection framework system based on a Fourier single-pixel imaging system, comprising an image reconstruction module, a target detection module and a joint training strategy module connected in communication;

[0008] The image reconstruction module comprises a spatially adaptive reconstruction network, which extracts features from the undersampled image for Fourier single-pixel imaging reconstruction through the reconstruction network, and obtains the Fourier single-pixel imaging reconstruction result; the image reconstruction module comprises a spatial feature attention module and an adaptive feature aggregation module, and the reconstruction network can significantly reduce the model complexity while ensuring excellent reconstruction performance, thereby reducing the additional computational burden imposed in the reconstruction-detection framework;

[0009] The Fourier single-pixel imaging reconstruction result is sent to the target detection module for target detection, in which the target detection network YOLOv5 is selected as the benchmark model of the detection module, and the fast spatial pyramid pooling module in the target detection network YOLOv5 is optimized into a hybrid fast spatial pyramid pooling module to overcome the problem of more information loss at low sampling by enhancing the extraction of global information; and a deformable convolution network is introduced in the target detection module to effectively alleviate the problem that the fixed convolution layer cannot fully capture irregular-shaped objects.

[0010] The joint training strategy module uses the detection loss to guide the training of the Fourier single-pixel image reconstruction network through backpropagation, forcing the reconstruction network to generate images more suitable for detection, to further improve the final performance of target detection at low sampling rate.

[0011] Preferably, the reconstruction network is a 7-level encoder-decoder network with a structure of 4 layers of encoders and 3 layers of decoders, and each layer of the reconstruction network contains a plurality of spatial feature attentions, so that the network can comprehensively understand and effectively utilize spatial information for Fourier single-pixel image reconstruction.

[0012] Preferably, the reconstruction stage of the image reconstruction module is as follows:

[0013] Step 1) In the encoder, the image acquired in the Fourier ptychographic system is first subjected to a 3x3 convolution operation with a step size of 1 to adjust the channel, and then subjected to a spatial feature attention module and a 3x3 convolution operation with a step size of 2 for downsampling and feature extraction. The extracted features are then used as the input of the decoder;

[0014] Step 2) In the decoder, the features are upsampled and reconstructed by a pixel shuffle 1x1 convolution operation and a spatial feature attention module. To avoid information loss, the features at each downsampling step in the encoder are connected to the corresponding decoder layer through a skip connection;

[0015] Step 3) At the end of the decoder, a 3x3 convolution operation with a step size of 1 is used to adjust the features to a single channel;

[0016] Finally, the pixel-wise addition is added to the original image to obtain the reconstruction result of the image, i.e., the reconstructed image.

[0017] Preferably, in step 2), since the features in the encoder and the decoder are different in semantics, an adaptive feature aggregation module is used at the skip connection point of the decoder. The adaptive feature aggregation module dynamically aggregates information between different levels of features and filters redundant information.

[0018] Preferably, the detection stage of the target detection module:

[0019] Step 1) The reconstructed image is input into the detection network. The target detection network YOLOv5 includes a backbone network, a neck, and a plurality of detection heads;

[0020] Step 2) In the backbone network, the mixed fast pyramid pooling module can more comprehensively capture global and local information of the features. The mixed fast pyramid pooling module combines average pooling and maximum pooling to fuse global and local information during feature extraction. The mixed fast pyramid pooling module introduces an additional average pooling branch based on the original maximum pooling branch, and similarly connects three average pooling layers.

[0021] Step 3) In the neck, deformable convolution is introduced to improve the processing capability of deformed targets in low sampling images. Through deformable convolution, the kernel shape is dynamically adjusted to adapt to the changes in the position, size, and direction of the target in the input feature map.

[0022] Preferably, the joint training strategy of the joint training strategy module:

[0023] Step 1) In the reconstruction stage, the network for reconstruction is trained for different sampling rates, the training process uses the ADAM optimizer to iterate, 100 iterations are carried out, the batch size is set to 16, the initial learning rate is 1x10-4, and the cosine annealing is used to adjust the learning rate, and the best model is saved during the training process;

[0024] Step 2) In the detection stage, the FPSI reconstruction result is input into the detector for training; the training is carried out using the ADAM optimizer, 200 iterations are carried out, the batch size is 16, the learning rate is 1x10-2, and the best model is saved during the training process;

[0025] Step 3) In the joint training stage, the weights of the reconstruction stage and the detection stage are loaded, and the reconstruction network and the detection network are trained at the same time, the respective parameters are adjusted through the detection loss and the reconstruction loss, and whether the training is ended is determined through the respective loss changes and the saved result pictures in each round of process.

[0026] Preferably, the total loss of the reconstruction and detection framework system based on the Fourier single-pixel imaging system is as follows, wherein α and β are both set to 1:

[0027] L = αL rec + βL det (1)

[0028] In the reconstruction module, the loss function is mainly composed of pixel loss and perception loss, and the reconstruction module loss is as follows, and λ is set to 0.01:

[0029] L rec = L1+ λL per (2)

[0030] The pixel loss L1is expressed as:

[0031]

[0032] Wherein n is the number of pixels in the image, y a is the FPSI reconstruction result, and y is the original image;

[0033] The perception loss is denoted as L per , and is expressed as:

[0034]

[0035] Wherein, m represents the number of layers of VGG-19 network, and φ i represents each layer in VGG-19 used to calculate the loss;

[0036] In the detection module, the loss of the detection network is simply expressed as:

[0037] Ldet = alpha box L Clou + alpha obj L obj + alpha cls L cls (5)

[0038] wherein alpha * represents the weight of the corresponding item, L CIou represents the regression bounding box loss, L obj represents the object confidence loss, L cls represents the class loss;

[0039] L CIou is a regression bounding box loss function that comprehensively considers the overlapping area, center point distance and width-length ratio; L CIou can be expressed as:

[0040]

[0041]

[0042]

[0043] wherein IOU is the intersection over union between the detection frame and the real frame, rho(b,b gt ) is the distance between the center of the detection frame and the real frame, w and w gt are the width values of the detection frame and the real frame respectively, h and h gt are the length values of the detection frame and the real frame respectively, L obj and L cls in formula (5) are binary cross entropy functions, and the formula is as follows:

[0044]

[0045] wherein y is a binary label 0 or 1, and p(y) is the probability that the output belongs to the label y.

[0046] Beneficial effects: when detecting the target of the low sampling rate image, the quality of the image can be effectively improved, and a closer upstream and downstream relationship is established to further improve the accuracy of the target detection task; the quality of the undersampled image can be enhanced, and the target detection performance can be improved. In addition, compared with other reconstruction and detection combined methods, the method is more excellent in detection accuracy and model efficiency, and provides a new idea and research direction for the combination of single-pixel imaging reconstruction and detection tasks. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is the reconstruction detection framework diagram of the application.

[0048] Figure 2is one of the real experimental schematic diagram of the present application.

[0049] Figure 3 is the second real experimental schematic diagram of the present application.

[0050] Figure 4 is the third real experimental schematic diagram of the present application.

[0051] Figure 5 is the fourth real experimental schematic diagram of the present application. DETAILED DESCRIPTION

[0052] The application is further described below in conjunction with the accompanying drawings of the specification, but the application is not limited to the following examples.

[0053] The reconstruction-detection framework system based on the Fourier single-pixel imaging system includes an image reconstruction module and a target detection module. In the Fourier single-pixel imaging reconstruction module, a high-performance spatial adaptive reconstruction network is designed, which aims to extract features from undersampled images for Fourier single-pixel imaging reconstruction. The network mainly consists of a spatial feature attention module and an adaptive feature aggregation module, which significantly reduces the model complexity while ensuring excellent reconstruction performance, thereby reducing the additional computational burden imposed in the reconstruction-detection framework. Then, the Fourier single-pixel imaging reconstruction result is sent to the target detection module for target detection. In the target detection module, the YOLOv5 network is improved. Among them, a hybrid fast spatial pyramid pooling structure is proposed, which enhances the extraction of global information compared with the original fast spatial pyramid pooling structure. In addition, a deformable convolution network is introduced to effectively alleviate the problem that fixed convolution layers cannot fully capture irregularly shaped objects. Finally, in order to better assist the subsequent target detection task, a joint training strategy module of the reconstruction network and the target detection network is designed. The strategy aims to further improve the target detection performance under low sampling rate. Numerical simulation and real experiment results show that the proposed Fourier single-pixel imaging reconstruction-detection framework can enhance the quality of undersampled images and improve the target detection performance. In addition, compared with other reconstruction and detection combined methods, it is more superior in detection accuracy and model efficiency. It provides a new idea and research direction for the combination of single-pixel imaging reconstruction and detection tasks:

[0054] 1) In the FSPI reconstruction phase, a spatial information attention module and an adaptive feature aggregation module are designed, which significantly enhance the network's ability to extract multi-scale features and suppress noise. Effectively reduces the number of parameters of the reconstruction network.

[0055] 2) In the target detection phase, the fast spatial pyramid pooling module is improved to a hybrid fast spatial pyramid pooling module, enhancing its ability to extract global information. In addition, to alleviate the influence of ringing effect on target deformation, deformable convolution is introduced.

[0056] 3) A joint training strategy is designed to guide the training of the Fourier ptychographic image reconstruction network through backpropagation using detection loss. The reconstruction network is forced to generate images more suitable for detection, further improving the final performance of target detection at low sampling rates.

[0057] As shown in Figures 1-5 A specific embodiment of a reconstruction and detection framework system based on a Fourier ptychographic imaging system is shown, which includes an image reconstruction module, a target detection module and a joint training strategy module connected in communication;

[0058] The image reconstruction module includes a spatially adaptive reconstruction network that extracts features from the undersampled image for Fourier ptychographic reconstruction through the reconstruction network to obtain the Fourier ptychographic reconstruction result. The image reconstruction module includes a spatial feature attention module and an adaptive feature aggregation module. The reconstruction network can significantly reduce the model complexity while ensuring excellent reconstruction performance, thereby reducing the additional computational burden imposed in the reconstruction-detection framework;

[0059] The Fourier ptychographic reconstruction result is sent to the target detection module for target detection. In the detection module, the target detection network YOLOv5 is selected as the benchmark model of the detection module. The fast spatial pyramid pooling module in the target detection network YOLOv5 is optimized into a hybrid fast spatial pyramid pooling module to enhance the extraction of global information and overcome the problem of more information loss at low sampling. A deformable convolution network is introduced in the target detection module to effectively alleviate the problem that fixed convolution layers cannot fully capture irregularly shaped objects.

[0060] The joint training strategy module uses detection loss to guide the training of the Fourier ptychographic image reconstruction network through backpropagation. The reconstruction network is forced to generate images more suitable for detection to further improve the final performance of target detection at low sampling rates.

[0061] 1、The reconstruction network (reconstruction network) structure is a 7-level encoder-decoder network, which has the characteristics of 4 layers of encoder and 3 layers of decoder. Each layer of the spatial adaptive reconstruction network contains a number of spatial feature attentions, so that the network can comprehensively understand and effectively use spatial information for Fourier single-pixel image reconstruction. In the encoder, the image first undergoes a 3x3 convolution operation with a step size of 1 to adjust the channels. Then, it passes through the spatial information attention module and the 3x3 convolution operation with a step size of 2 for downsampling and feature extraction. The extracted features are then used as input to the decoder. In the decoder, these features are upsampled and reconstructed through pixel shuffle and 1x1 convolution operations and the spatial information attention module. To avoid information loss, the features at each downsampling step in the encoder are connected to the corresponding decoder layer through a skip connection. Since the features in the encoder and decoder are semantically different, an adaptive feature aggregation module is used at the skip connection point of the decoder. The adaptive feature aggregation module dynamically aggregates information between different levels of features and filters redundant information. At the end of the decoder, a 3x3 convolution operation with a step size of 1 is used to adjust the features to a single channel. Finally, they are added to the original image pixel by pixel to obtain the final reconstruction result.

[0062] 2、YOLOv5 is selected as the baseline model of the detection module. In view of the problem that the fast spatial pyramid pooling module in YOLOv5 only uses maximum pooling, which causes more information loss at low sampling, a hybrid fast spatial pyramid pooling module is designed. In addition, in order to solve the object distortion problem caused by low sampling, deformable convolution is introduced into the neck.

[0063] 3、There are fundamental differences between Fourier single-pixel image reconstruction and object detection tasks because their goals are not completely consistent. The main goal of Fourier single-pixel image reconstruction is to enhance image quality, which is usually measured by indicators such as peak signal-to-noise ratio and structural similarity index. On the contrary, the goal of object detection is to improve the accuracy of machine vision, which is usually evaluated using indicators such as overall class average accuracy. In order to solve these different goals, the tasks trained separately are combined, and the detection loss of backpropagation is used to guide the training of the Fourier single-pixel image reconstruction network, forcing it to generate images more suitable for detection. The specific operation of the reconstruction and detection method based on Fourier single-pixel imaging is shown in Figure 2

[0064] Step 1: In the reconstruction phase, the network for reconstruction is trained for different sampling rates. The training process uses the ADAM optimizer for iteration, which requires about 100 iterations. The batch size is set to 16, the initial learning rate is 1x10-4, and cosine annealing is used to adjust the learning rate.

[0065] ​Step 2: In the detection stage, the FPSI reconstruction result is input into the detector for training. Similarly, training is performed using the ADAM optimizer, which takes about 200 iterations, a batch size of 16, and a learning rate of 1x10-2. The best model is saved during training.

[0066] Step 3: In the joint training stage, the pre-trained weights are loaded and the reconstruction network and the detection network are trained to fine-tune the respective parameters. The training is ended by observing the changes in the respective loss and the saved results in each round.

[0067] In a preferred embodiment, the reconstruction network is a 7-level encoder-decoder network with a structure of 4 layers of encoder and 3 layers of decoder. Each layer of the reconstruction network includes several spatial feature attentions to enable the network to comprehensively understand and effectively utilize spatial information for Fourier ptychographic image reconstruction.

[0068] In a preferred embodiment, the reconstruction stage of the image reconstruction module is as follows:

[0069] Step 1) In the encoder, the image obtained from the Fourier ptychographic imaging system is first subjected to a 3x3 convolution operation with a step size of 1 to adjust the channels. Then, through the spatial feature attention module and the 3x3 convolution operation with a step size of 2, the image is downsampled and features are extracted. The extracted features are then used as input for the decoder.

[0070] Step 2) In the decoder, the features are upsampled and reconstructed through the pixel shuffle 1x1 convolution operation and the spatial feature attention module. To avoid information loss, the features from each downsample step in the encoder are connected to the corresponding decoder layer through a skip connection.

[0071] Step 3) At the end of the decoder, a 3x3 convolution operation with a step size of 1 is used to adjust the features to a single channel.

[0072] Finally, the pixel-wise addition is added to the original image to obtain the reconstruction result of the image, i.e., the reconstructed image.

[0073] In a preferred embodiment, in step 2), since the features in the encoder and the decoder are different in semantics, an adaptive feature aggregation module is used at the skip connection point of the decoder. The adaptive feature aggregation module dynamically aggregates information between different levels of features and filters redundant information.

[0074] In a preferred embodiment, the detection stage of the target detection module is as follows:

[0075] Step 1) The reconstructed image is input into the detection network. The target detection network YOLOv5 includes a backbone network, a neck, and several detection heads.

[0076] Step 2) In the backbone network, the global and local information of the features can be more comprehensively captured by mixing the fast pyramid pooling module, which combines average pooling and maximum pooling to fuse global and local information during feature extraction. The mixed fast pyramid pooling module introduces an additional average pooling branch based on the original maximum pooling branch, similarly connecting three average pooling layers.

[0077] Step 3) In the neck, deformable convolution is introduced to improve the processing capability of the deformation target in the low sampling image; through deformable convolution, the kernel shape is dynamically adjusted to adapt to the convolution operation of the changes in the target position, size, and direction in the input feature map.

[0078] In a preferred embodiment, the joint training strategy of the joint training strategy module is:

[0079] Step 1) In the reconstruction phase, the network for reconstruction is trained for different sampling rates. The training process uses the ADAM optimizer for iteration, with 100 iterations, a batch size of 16, an initial learning rate of 1x10-4, and cosine annealing to adjust the learning rate. The best model is saved during the training process;

[0080] Step 2) In the detection phase, the FPSI reconstruction result is input into the detector for training; the ADAM optimizer is used for training, with 200 iterations, a batch size of 16, and a learning rate of 1x10-2. The best model is saved during the training process;

[0081] Step 3) In the joint training phase, the weights of the reconstruction phase and the detection phase are loaded, and the reconstruction network and the detection network are trained simultaneously. The parameters of each are adjusted through the detection loss and the reconstruction loss. The training is ended by determining whether to end the training through the statistical loss change of each and the saved result image in each round of process.

[0082] Preferably, the total loss of the reconstruction and detection framework system based on the Fourier single-pixel imaging system is as follows, where α and β are both set to 1:

[0083] L = αL rec + βL det (10)

[0084] In the reconstruction module, the loss function is mainly composed of pixel loss and perception loss. The reconstruction module loss is as follows, where λ is set to 0.01:

[0085] L rec = L1+ λL per (11)

[0086] The pixel loss L1 is expressed as:

[0087]

[0088] where n is the number of pixels in the image, y a is the FPSI reconstruction result, and y is the original image;

[0089] The perceptual loss is denoted as L per , which is expressed as:

[0090]

[0091] where m represents the number of layers of the VGG-19 network, and φ i represents each layer in the VGG-19 used to calculate the loss;

[0092] In the detection module, the loss of the detection network is simply denoted as:

[0093] L det = a box L Clou + a obj L obj + a cls L cls (14)

[0094] where a * represents the weight of the corresponding item, L CIou represents the regression box loss, L obj represents the object confidence loss, and L cls represents the class loss;

[0095] L CIou is a regression box loss function that comprehensively considers the overlapping area, center point distance, and width-length ratio; L CIou can be expressed as:

[0096]

[0097]

[0098]

[0099] where IOU is the intersection over union between the detection box and the real box, p(b,b gt ) is the distance between the center of the detection box and the real box, w and w gt are the width values of the detection box and the real box, respectively, and h and h gt are the length values of the detection box and the real box, respectively, L obj and L cls in formula (5) are both binary cross-entropy functions, and the formula is as follows:

[0100]

[0101] where y is a binary label 0 or 1, and p(y) is the probability that the output belongs to label y.

[0102] The present application is based on joint training for image reconstruction and target detection, which is used to improve the accuracy of the current low sampling rate Fourier single-pixel image target detection. In order to process the low sampling rate Fourier single-pixel image quality low reconstruction task, a spatial adaptive reconstruction network is designed to quickly reconstruct the low sampling rate image to improve the image quality. The mixed fast spatial pooling pyramid fusion and deformable convolution are fused into the target detection network to improve the detection performance. A joint training strategy is designed to strengthen the synergy between high-level and low-level visual tasks, thereby further improving the detection accuracy.

[0103] Finally, it should be noted that the present application is not limited to the above embodiments, but can have many variations. All variations that can be directly derived or inferred from the disclosed content by those of ordinary skill in the art should be considered within the scope of the present application.

Claims

1. A reconstruction detection framework system based on a Fourier single-pixel imaging system, characterized by: It includes a communication-connected image reconstruction module, a target detection module, and a joint training strategy module; The image reconstruction module includes a spatially adaptive reconstruction network that extracts features from undersampled images for Fourier single-pixel imaging (FSPI) reconstruction, resulting in Fourier single-pixel imaging reconstruction results. The image reconstruction module includes a spatial feature attention module and an adaptive feature aggregation module. The reconstruction network significantly reduces model complexity while ensuring excellent reconstruction performance, thereby reducing the additional computational burden imposed by the reconstruction detection framework. The Fourier single-pixel imaging reconstruction results are fed into the target detection module for target detection. In this module, the target detection network YOLOv5 is selected as the benchmark model. The fast spatial pyramid pooling module in the target detection network YOLOv5 is optimized into a hybrid fast spatial pyramid pooling module to enhance the extraction of global information and overcome the problem of high information loss during low sampling. A deformable convolutional network is introduced in the target detection module to effectively alleviate the problem that fixed convolutional layers cannot fully capture irregularly shaped objects. The joint training strategy module uses detection loss to guide the training of the reconstruction detection framework system of the Fourier single-pixel imaging system through backpropagation, forcing the reconstruction network to generate images that are more suitable for detection, so as to further improve the final performance of target detection at low sampling rates.

2. The reconstruction detection framework system based on the Fourier single-pixel imaging system according to claim 1, characterized in that: The reconstruction network is a 7-level encoder-decoder network with a 4-layer encoder and 3-layer decoder structure. Each layer of the reconstruction network contains several spatial feature attentions, so that the network can fully understand and effectively utilize spatial information for Fourier single-pixel image reconstruction.

3. The reconstruction detection framework system based on the Fourier single-pixel imaging system according to claim 1, characterized in that: Reconstruction phase using the image reconstruction module: Step 1) In the encoder, the image first undergoes a 3×3 convolution operation with a stride of 1 to adjust the channels. Then, it passes through the spatial feature attention module and a 3×3 convolution operation with a stride of 2 for downsampling and feature extraction. The extracted features are then used as input to the decoder. Step 2) In the decoder, the features are upsampled and reconstructed through pixel-shuffled 1×1 convolution operations and spatial feature attention modules. To avoid information loss, the features of each downsampling step in the encoder are connected to the corresponding decoder layer through skip connections; Step 3) At the end of the decoder, a 3×3 convolution operation with a stride of 1 is used to resize the features into a single channel; Finally, the pixels are added to the original image to obtain the reconstruction result of the image, that is, the reconstructed image.

4. The reconstruction detection framework system based on the Fourier single pixel imaging system according to claim 3, characterized in that: In step 2), since the features in the encoder and decoder are semantically different, an adaptive feature aggregation module is used at the jump connection point of the decoder to dynamically aggregate information between features at different levels and filter out redundant information.

5. The reconstruction detection framework system based on Fourier single pixel imaging system according to claim 1, characterized in that: Detection stage of the target detection module: Step 1) Input the reconstructed image into the detection network. The target detection network YOLOv5 includes a backbone network, a neck, and several detection heads; Step 2) In the backbone network, the global and local information of features can be more comprehensively captured by the hybrid fast pyramid pooling module. The hybrid fast pyramid pooling module combines average pooling and maximum pooling to fuse global and local information during feature extraction. The hybrid fast pyramid pooling module introduces an additional average pooling branch based on the original maximum pooling branch, similarly connecting three average pooling layers; Step 3) In the neck, deformable convolution is introduced to improve the processing capability of deformable targets in low-sampled images; through deformable convolution, the convolution operation whose kernel shape is dynamically adjusted to adapt to the changes in the target position, size and orientation in the input feature map.

6. The reconstruction detection framework system based on Fourier single pixel imaging system according to claim 4, characterized in that: Joint training strategy module joint training strategy: Step 1) In the reconstruction phase, the network used for reconstruction is trained for different sampling rates. The training process is iterated using the ADAM optimizer for 100 iterations, with a batch size of 16 and an initial learning rate of 1×10-4. Cosine annealing is used to adjust the learning rate, and the best model is saved during the training process. Step 2) In the detection phase, the FSPI reconstruction results are input into the detector for training; The training was performed using the ADAM optimizer for 200 iterations, a batch size of 16, a learning rate of 1×10-2, and the best model was saved during training. Step 3) In the joint training phase, the weights of the reconstruction phase and the detection phase are loaded, and the reconstruction network and the detection network are trained simultaneously. The parameters of each network are adjusted together through the detection loss and the reconstruction loss. The decision on whether to end the training is made by statistically analyzing the changes in each loss and the result graph saved during each round.

7. The reconstruction detection framework system based on the Fourier single pixel imaging system according to any one of claims 1 to 6, characterized in that: The total loss of the reconstruction detection framework system based on the Fourier single-pixel imaging system is calculated as follows, where α and β are both set to 1: L=αL rec +βL det (1) In the reconstruction module, the loss function mainly consists of pixel loss and perceptual loss. The reconstruction module loss is shown below, and λ is set to 0.01: L rec =L1+λL per (2) The pixel loss L1 is expressed as: Where n is the number of pixels in the image, y a is the FSPI reconstruction result, y is the original image; The perceptual loss is denoted as L per , expressed as: Among them, m represents the number of layers of the VGG-19 network, φ i Represents each layer used to calculate the loss in VGG-19; In the target detection module, the loss of the detection network is simply expressed as: L det =a box L Clou +a obj L obj +a cls L cls (5) where α box , α obj , α cls Indicates the weight of the corresponding item, L CIou represents the regression box loss, L obj represents the object confidence loss, L cls represents the category loss; L CIou It is a regression box loss function that comprehensively considers the overlapping area, center point distance and width-to-length ratio; L CIou It can be expressed as: Among them, IOU is the intersection-union ratio between the detection box and the real box, ρ(b,b gt ) is the distance between the center of the detection frame and the center of the real frame, w and w gt The widths of the detection frame and the real frame, h and h respectively gt are the lengths of the detection frame and the true frame respectively. L in formula (5) obj and L cls They are all binary cross entropy functions, and the formula is as follows: where y i is a binary label 0 or 1, p(y i ) is the output belonging to label y i probability.

Images