Composite biomimetic ghost imaging method and system

Abstract

The application discloses a composite biomimetic ghost imaging method and system, and the method comprises the following steps: based on using a feedback type axial vibration retina technology, two-dimensional motion detection of a target in a complex background is realized, and azimuth information and an angular velocity of the target are acquired; according to the azimuth information and the angular velocity of the target, target recognition under an imaging-free condition is carried out on a signal in a specific direction; according to a target recognition result, a sub visual field where the target is located is determined, and variable-resolution ghost imaging is carried out on the visual field. The ghost imaging technology with a large visual field and high efficient sensing performance is realized by using multiple detectors, and the ghost imaging technology has the functions of integration of detection, recognition and tracking.

Description

Technical Field

[0001] This invention belongs to the field of photoelectric sensing technology, and more specifically, relates to a composite bionic ghost imaging method and system. Background Technology

[0002] Optoelectronic sensing technology has always been an indispensable technology in many application fields, such as security monitoring, autonomous driving, intelligent manufacturing, IFF (Identification Friend or Foe), and intelligent reconnaissance, among other defense and civilian applications. The most important aspects of optoelectronic sensing technology include three parts, which, in the order of the optical system and the target, encompass detection, identification, and tracking. These three parts are usually applied as a whole in various fields. However, due to the different functional attributes of each part, the functional requirements of the optoelectronic sensing system also differ. For example, detection focuses on searching for targets with a large field of view; after the target is found, high-resolution imaging is used for target identification, thus requiring higher resolution in the identification stage. Furthermore, after target identification, the focus is on keeping up with the target at a faster rate, meaning the tracking stage needs superior real-time performance. Therefore, the integrated function of detection, identification, and tracking has significant advantages and is more suitable for various scenarios. Current optoelectronic imaging systems can meet these functional requirements to a certain extent, but their coverage is not good. This is mainly due to two reasons: firstly, the functional requirements of each stage are different, and the single function of the imaging sensor is insufficient; secondly, the functional overlap between different stages leads to a large amount of information redundancy and makes real-time performance more difficult. Therefore, how to develop more effective photoelectric sensors to meet the needs of integrated detection, identification, and tracking applications has become an urgent problem to be solved.

[0003] In recent years, ghost imaging technology has achieved spatial resolution imaging using single-pixel detectors in conjunction with light intensity fluctuations. This method boasts advantages such as simple structure and strong anti-interference capabilities, making it applicable to various scenarios. However, since ghost imaging requires multiple correlation measurements, its imaging efficiency still needs improvement. Researchers have explored ways to enhance ghost imaging efficiency, particularly using multiple detectors for parallel ghost imaging, which has become a crucial approach to improving ghost imaging quality. For example, using a four-quadrant detector to reconstruct the target image achieves a fourfold increase in reconstruction efficiency compared to a single detector. Furthermore, the four-quadrant detector itself possesses excellent tracking performance; therefore, ghost imaging has become a potential method for achieving both high-resolution imaging and efficient tracking. However, for efficient imaging with a large field of view, ghost imaging still needs to overcome some technical bottlenecks. For large field-of-view imaging, biomimetic imaging inspired by insect eyes provides a good approach; for example, dragonflies can perceive a 360-degree scene. For efficient perception, the variable resolution characteristics of the human eye allow for dynamic adaptive resource allocation, ensuring that the human eye always possesses the ability to achieve high-resolution imaging of targets of interest. With the rapid development of computing power and the semiconductor industry, computational imaging, represented by ghost imaging, has more flexible configuration characteristics and provides new solutions for more intelligent integrated applications of detection, identification and tracking.

[0004] In view of this, the present invention combines the compound eye of insects with the variable resolution imaging perception of the human eye to form a composite bionic ghost imaging, which provides different working modes at different stages, thereby achieving a more matched perception capability. Summary of the Invention

[0005] This invention addresses the aforementioned problems in the prior art. Therefore, a composite bionic ghost imaging method and system are needed, which utilizes multiple detectors to achieve ghost imaging technology with a large field of view and high-efficiency sensing performance, integrating detection, identification, and tracking functions.

[0006] According to a first aspect of the present invention, a composite bionic ghost imaging method is provided, the method comprising: Based on the use of feedback axial vibration retinal technology to achieve two-dimensional motion detection of targets in complex backgrounds, the target's orientation information and angular velocity are obtained. Based on the target's orientation information and angular velocity, target identification is performed on the signal in a specific direction under imaging-free conditions. Based on the target recognition results, the sub-field of view where the target is located is determined, and variable resolution ghost imaging is performed on the field of view.

[0007] Furthermore, the method for achieving two-dimensional motion detection of targets in complex backgrounds using feedback-based axial vibration retinal technology, and acquiring the target's orientation information and angular velocity, includes: When detecting the target's location, data within the corresponding field of view is first collected, and the output voltage values ​​of the detector array are recorded when the target is in each direction. The target information is received using a photoelectric detection system. For targets in different directions, a voltage value sequence is obtained based on the different voltage output values ​​generated by the detectors at different positions. The target's location information is calculated based on the voltage value sequence.

[0008] Further, calculating the target's azimuth information based on the voltage value sequence includes: The voltage value sequence is used as the input to the neural network. I in According to the ideal location of the target I sta By training a neural network, we can find values ​​that match the input values. I in To output value I sta The process involves assigning corresponding neural network weights and activation values, and then configuring the learned neural network weights and activation values ​​into the neural network. Based on the trained neural network, for targets with unknown locations, the target's orientation information can be calculated according to the voltage value sequence corresponding to the target and the weights and excitation values ​​of the neural network.

[0009] Furthermore, the method for achieving two-dimensional motion detection of targets in complex backgrounds using feedback-based axial vibration retinal technology to obtain the target's orientation information and angular velocity also includes: The initial velocity is obtained by roughly calculating the target's angular velocity based on the time interval between the rising edges generated by two adjacent single-pixel detectors as the target flies past. Based on the initial angular velocity, the vibration velocity of the retina is gradually controlled by feedback to match the vibration velocity with the target angular velocity.

[0010] Further, the step of progressively controlling the vibration velocity of the retina based on the initial measured angular velocity, so that the vibration velocity matches the target angular velocity, includes: Assume that the initial retinal vibration frequency corresponds to a certain cross-correlation asymmetry of the single-pixel detector signal. a 0, taking the minimum cross-correlation asymmetry. a min = | a 0|, and simultaneously set the minimum adjustable value of the retinal vibration frequency to Δ. oh min The sign is positive, used to adjust the retinal vibration frequency, and the cross-correlation trend indicator flag is set to an initial value of 0; Fine-tuning the retinal vibration frequency Δ oh min Calculate the cross-correlation asymmetry at this time.a 1, and will a The absolute value of 1 and the set minimum value a min Compare: like |a 1|< a min If the cross-correlation trend indicator flag is set to 1, it indicates that the cross-correlation asymmetry decreases and the frequency further increases along Δ. oh min Changes in direction, take the minimum cross-correlation value a min For the updated |a 1|, and repeat the loop starting from the fine-tuning step until the frequency adjustment exceeds the matching frequency, resulting in cross-correlation asymmetry. |a 1 | Higher than the version updated at that time a min Finally, the angular velocity of the target is calculated.

[0011] Furthermore, the step of performing target recognition without imaging based on the target's azimuth information and angular velocity in a specific direction includes: Based on the detector voltage signal collected from a specific direction, the signal characteristics in that direction are determined and compared with the signal characteristics of the ideal signal. When the similarity between the collected signal and the ideal signal exceeds a certain threshold, the target in that direction is identified as the predetermined target.

[0012] Further, the step of determining the sub-field of view where the detected target is located based on the target recognition result, and performing variable resolution ghost imaging on the field of view, includes: Obtain the central concave region of interest of the panoramic image and set the variable resolution annular speckle parameters according to the logarithmic polar coordinate transformation; A human-eye variable resolution ring speckle model was constructed to generate a human-eye variable resolution ring speckle sequence. Ghost imaging reconstruction is performed based on the aforementioned human eye variable resolution ring speckle sequence and the acquired sub-detector signals. Based on the results of reconstruction from multiple adjacent projection units, three-dimensional imaging of the target is achieved.

[0013] Furthermore, the step of acquiring the central concave region of interest of the panoramic image and setting the variable-resolution annular speckle parameters according to the logarithmic polar coordinate transformation includes: Following the variable resolution arrangement of the receptors in the human eye's retina, the projected pattern is divided into a central concave region and an edge region. The central concave region uses uniform high-resolution Cartesian sampling, while the edge region uses logarithmic polar coordinate variable resolution sampling. The edge region image is compressed to different degrees using the characteristics of the logarithmic polar coordinate model. Based on the imaging area of ​​the target, determine the center position of the human eye speckle and the radius of the central high-resolution region.

[0014] Furthermore, the construction of the human eye-like variable-resolution circular speckle model and the generation of the human eye-like variable-resolution circular speckle sequence include: use r The radius of the outer ring of the central concave region represents the distance from the pixel to the center position. r The area within 0 is the central region. r The area outside 0 is the edge region; Based on the differences in polar radius and polar angle, the marginal region is divided into: P Rings, each ring Q 1 pixel; In the human-eye-inspired logarithmic polar coordinate model, it is assumed that... p and q They represent the first p Ring and the first q If there are 10 pixels, the variable resolution structure of the edge region can be calculated using the following system of equations: ; in, r p Indicates the first p The radius of the ring containing the ring pixel. r 1 represents the radius of the ring containing the first ring pixel. i q Indicates the first q The angle corresponding to each pixel. e Indicates the inter-ring growth coefficient; Based on the human eye logarithmic polar coordinate model and uniform projection pattern generation method, a human eye spatial variable resolution projection pattern is generated.

[0015] Furthermore, based on the aforementioned human-eye variable-resolution circular speckle sequence and combined with the acquired sub-detector signals, ghost imaging reconstruction is performed using the following formula: ; in, To mimic the speckle sequence of the human eye, The image reconstructed from the corresponding ghost image. This represents the total light intensity received by the sub-detector.

[0016] Furthermore, the process of achieving three-dimensional imaging of the target based on the reconstructed results from multiple adjacent projection units includes: By using a computational ghost imaging algorithm, the detector signals corresponding to adjacent projection units are reconstructed to obtain a reconstructed multi-view image sequence. Based on the reconstructed multi-view image sequence, the distance values ​​of different targets are calculated using a stereo matching method to achieve three-dimensional imaging of the scene; The stereo matching method includes: Images from the pair detectors I k and images from adjacent sub-detectors I k+1 Perform homography transformation and search for the best matching point on the corresponding epipolar line; With the first k Using the pixels in the view corresponding to each sub-detector as a reference, calculate the adjacent pixels of the first... k The similarity of pixel values ​​within the search window corresponding to +1 sub-detectors in the view is used. If the similarity meets a set threshold, the pixel is determined to be the corresponding matching point, based on the disparity value between the pixels. d By utilizing the relationship between disparity values ​​and depth values, three-dimensional information can be acquired.

[0017] According to a second aspect of the present invention, a composite bionic ghost imaging system is provided, the system comprising: A single-pixel detector array is used to acquire light signals from the imaging plane at different angles; Digital micromirror device for modulating spatial light information; A light source for projecting white light onto the digital micromirror device; A data acquisition device is used to convert the light intensity value of a real single-pixel detector into an electrical signal. Optical fibers and lenses are used to project the speckle of the projection unit and to receive the optical signals from the detector. The host computer is configured as follows: Based on the use of feedback axial vibration retinal technology to achieve two-dimensional motion detection of targets in complex backgrounds, the target's orientation information and angular velocity are obtained. Based on the target's orientation information and angular velocity, target identification is performed on the signal in a specific direction under imaging-free conditions. Based on the target recognition results, the sub-field of view where the target is located is determined, and variable resolution ghost imaging is performed on the field of view.

[0018] The present invention has at least the following beneficial effects: (1) Flexible and diverse methods. Multiple detectors can be used for large-scale detection and high-resolution imaging after the target is detected, which is beneficial for target identification.

[0019] (2) Strong wide field of view detection capability. The compound eye structure is adopted to form a periodic arrangement structure, which is laid out on the curved surface, which is more conducive to wide field of view detection.

[0020] (3) Strong redundancy compression capability. By using a variable resolution speckle layout similar to the human retina, high-resolution speckle is formed for the target, while low-resolution speckle modulation is performed for non-interesting targets, which is beneficial for compressing redundant data. Attached Figure Description

[0021] In drawings that are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with or without letter suffixes may indicate different instances of similar parts. The drawings generally illustrate various embodiments by way of example rather than limitation and, together with the description and claims, serve to explain embodiments of the invention. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and not intended to be exhaustive or exclusive embodiments of the apparatus or method.

[0022] Figure 1 This is a flowchart of a composite bionic ghost imaging method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating target detection in a composite bionic ghost imaging method according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating how a composite bionic ghost imaging method, based on target recognition results, determines the sub-field of view where the target is located and performs variable resolution ghost imaging on that field of view. Figure 4 This is a two-dimensional schematic diagram of a composite bionic ghost imaging system according to an embodiment of the present invention, wherein the optical fiber is 100, the light source is 101, the digital micromirror device is 102, the four-quadrant detector is 103, the data acquisition card is 104, and the host computer is 105. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples, but this is not intended to limit the present invention. If there is no necessary sequential relationship between the various steps described herein, the order in which they are described as examples should not be considered a limitation. Those skilled in the art should understand that the order can be adjusted, as long as it does not disrupt the logical consistency between them and render the entire process impossible.

[0024] This invention provides a composite bionic ghost imaging method; please refer to [link / reference]. Figure 1 , Figure 1 A flowchart of a composite bionic ghost imaging method according to an embodiment of the present invention is shown. The method includes the following steps: Step S100: Based on the use of feedback axial vibration retinal technology to realize two-dimensional motion detection of the target in a complex background, the orientation information and angular velocity of the target are obtained.

[0025] It should be noted that in the composite bionic ghost imaging system, the four fiber optic arrays connected to the projection unit and detector are arranged in an array structure (including planar, curved, etc.; here, the system is configured to correspond to four sub-fields of view). Two-dimensional motion detection of targets against complex backgrounds is achieved using feedback-based axial vibration retina technology, including measuring target orientation and angular velocity. When detecting target orientation, experimental data is first collected, recording the light intensity values ​​received by the fiber optics at four orientations. The photoelectric detection system receives target information; for targets at different orientations, the light signals collected by the fiber optics at different positions will generate different voltage output values. The measured voltage value sequence is input into the algorithm to calculate and obtain the target's position information.

[0026] In some embodiments, for targets at different locations, the optical signals collected by optical fibers at different locations will generate different voltage output values, and these voltage output values ​​are used as inputs to the neural network. I in Because we knew the ideal location of the target beforehand in the experiment. I sta So, by training a neural network, we can find the value that matches the input value. I in To output value I sta The process involves assigning corresponding neural network weights and activation values, and then using the learned values ​​as the inherent parameters of the neural network to input into the system. After such training, the location of an unknown target can be calculated using the voltage output of the detector and the weights and activation values ​​of the neural network.

[0027] Furthermore, when a target enters a sub-field of view, the output signal of the small eye will be superimposed with a rising edge on the gradually varying background light. The width of the rising edge is related to the target's angular velocity and also to the detection performance of the photodetector. If the target flies past two small eyes A and B consecutively, the angular velocity of the target can be calculated by using the time difference and the corresponding angle difference between the rising edges of the two sub-detectors.

[0028] In some embodiments, such as Figure 2 As shown, the flowchart of a composite bionic ghost imaging method disclosed in this embodiment includes the following steps: The first part involves roughly calculating the target's angular velocity based on the time interval between the rising edges generated when the target flies through the quadrants of two adjacent detectors. This is done by recording the moments when the output signals of the two sub-detectors exceed a certain threshold, subtracting them to obtain the time interval, and then dividing the angular interval between the sub-detectors by this time interval to obtain the initial velocity measurement.

[0029] The second part involves gradually controlling the vibration velocity of the retina, starting from the initial angular velocity measurement, to match it with the target angular velocity.

[0030] Assume that the initial retinal vibration frequency corresponds to a certain cross-correlation asymmetry of the single-pixel detector signal. a 0, taking the minimum cross-correlation asymmetry. a min = | a 0|. Simultaneously, let the minimum adjustable value of the retinal vibration frequency be Δ. oh min The sign is positive, and it is used to adjust the retinal vibration frequency. The cross-correlation trend indicator flag is initialized to 0. Then, fine-tuning is attempted to change the retinal vibration frequency Δ. oh min Calculate the cross-correlation asymmetry at this time. a 1, and its absolute value is compared with the set minimum value. a min Comparison. If |a 1|< a min This indicates that frequency changes can make the vibration frequency more closely match the target angular velocity. Therefore, setting the cross-correlation trend flag to 1 indicates that the cross-correlation asymmetry is reduced, and the frequency needs to be further adjusted along the Δ... oh min A change in direction is required to complete the matching. Then, the minimum cross-correlation value is taken. a min For the updated |a 1|, and repeat the loop starting from the fine-tuning step until the frequency adjustment exceeds the matching frequency, resulting in cross-correlation asymmetry. |a 1 | Higher than the version updated at that time a min Finally, the angular velocity of the target motion can be calculated.

[0031] Step S200: Based on the target's orientation information and running angular velocity, target identification is performed on the signal in a specific direction under imaging-free conditions.

[0032] In some embodiments, target identification without imaging is performed on signals in a specific direction based on the results of target detection. The signal characteristics in that direction are determined based on the detector voltage signal acquired in that direction and compared with the signal under ideal conditions. When the similarity between the acquired signal and the ideal signal exceeds a certain threshold, the target in that direction can be identified as the predetermined desired target.

[0033] Finally, in step S300, based on the target recognition result, the sub-field of view where the target is located is determined, and variable resolution ghost imaging is performed on the field of view.

[0034] In some embodiments, such as Figure 3 As shown, the step of determining the sub-field of view where the detected target is located based on the target recognition result, and performing variable resolution ghost imaging on the field of view, specifically includes: Step S301: Obtain the central concave region of interest of the panoramic image, and set the variable resolution annular speckle parameters according to the logarithmic polar coordinate transformation.

[0035] For example, in this step, the foveal region of interest of the panoramic image is selected according to application requirements, and the variable-resolution annular speckle parameters are set according to logarithmic polar coordinate transformation. Following the variable-resolution arrangement of the receptors in the human eye's retina, the projected pattern is divided into a foveal region and an edge region. The foveal region uses traditional uniform high-resolution Cartesian sampling to avoid the redundancy of logarithmic polar coordinates at the center; while the edge region uses variable-resolution logarithmic polar coordinate sampling, utilizing the characteristics of the logarithmic polar coordinate model to compress the edge region image to varying degrees. Both together achieve the human eye-like imaging characteristic of "high resolution at the center and low resolution at the edges." Based on the target's imaging area range obtained in step two, a suitable human eye-like speckle center position and the radius of the high-resolution central region are selected. Here, the center position and the radius of the high-resolution central region are dynamic variables and are correlated with the detection and recognition results in steps S100 and S200.

[0036] Step S302: Construct a human eye variable resolution ring speckle model and generate a human eye variable resolution ring speckle sequence.

[0037] For example, in a human-eye-inspired variable-resolution annular speckle pattern, the central high-resolution sampling region is generated using a traditional Cartesian coordinate system, which avoids the oversampling problem of logarithmic polar coordinates in the central region; the edge regions are generated using a human-eye-inspired logarithmic polar coordinate model. If using... r The radius of the outer ring of the central concave region represents the distance from the pixel to the center position. r The area within 0 is the central region. r The area outside 0 is the edge region.

[0038] Based on the differences in polar radius and polar angle, the marginal region can be divided into...P Rings, each ring Q 1 pixel.

[0039] In the human-eye-inspired logarithmic polar coordinate model, it is assumed that... p and q They represent the first p Ring and the first q If there are 10 pixels, the variable resolution structure of the edge region can be calculated by the following set of equations: ; in, r p Indicates the first p The radius of the ring containing the ring pixel. r 1 represents the radius of the ring containing the first ring pixel. i q Indicates the first q The angle corresponding to each pixel. e This represents the inter-ring growth coefficient.

[0040] Based on the human eye logarithmic polar coordinate model and the uniform projection pattern generation method, a human eye spatial variable resolution projection pattern can be generated on the DMD.

[0041] Step S303: Based on the human-eye variable resolution ring speckle sequence, ghost imaging reconstruction is performed in combination with the acquired sub-detector signals.

[0042] For example, to reconstruct the ghost imaging result better and faster, a compressed sensing algorithm is used to reconstruct the acquired signal. Since a single-pixel detector is used in calculating ghost imaging, the image reconstruction can also be achieved using a single-pixel imaging algorithm, which can be expressed as: ; in, To mimic the speckle sequence of the human eye, The image reconstructed from the corresponding ghost image. This represents the total light intensity received by the sub-detector. To acquire target images more efficiently, a parallel algorithm is used to optimize the speckle pattern. The entire projection area is divided into corresponding parts, and speckle patterns with the same settings are projected simultaneously, thereby enabling faster target reconstruction.

[0043] Step S304: Based on the results of reconstruction from multiple adjacent projection units, achieve three-dimensional imaging of the target.

[0044] For example, based on the reconstructed result of multiple adjacent projection units, the three-dimensional imaging of the target can be achieved through the following method: By employing a computational ghost imaging algorithm, images are reconstructed from the detector signals corresponding to adjacent projection units to obtain the corresponding results. Based on the reconstructed multi-view image sequence, a stereo matching method is used to calculate the distance values ​​of different targets, thereby achieving three-dimensional imaging of the scene.

[0045] The stereo matching method includes: Images from the pair detectors I k and images from adjacent sub-detectors I k+1 Perform homography transformation and search for the best matching point on the corresponding epipolar line; With the first k Using the pixels in the view corresponding to each sub-detector as a reference, calculate the adjacent pixels of the first... k The similarity of pixel values ​​within the search window of the view corresponding to +1 sub-detectors is used. If the similarity meets the set threshold, the pixel is determined to be the corresponding matching point. Based on the disparity value d between the pixels, the relationship between the disparity value and the depth value is used to obtain three-dimensional information.

[0046] like Figure 4 As shown, an embodiment of the present invention provides a composite bionic ghost imaging system, which includes an optical fiber 100, a light source 101, a digital micromirror device 102, a four-quadrant detector 103, a data acquisition card 104, and a host computer 105.

[0047] The optical fiber 100 is used to transmit optical signals, including for projecting speckle patterns generated by the digital micromirror device and for receiving optical signals to the four-quadrant detector 103.

[0048] Each detector target surface in the four-quadrant detector 103 is used to receive light signals from different directions, and the overlap ratio of the fields of view is small, thereby obtaining a larger field of view.

[0049] The digital micromirror device 102 is used to modulate spatial light information.

[0050] The light source 101 is used to project white light onto the digital micromirror device.

[0051] The data acquisition device 104 is used to convert the light intensity value of the actual single-pixel detector into an electrical signal.

[0052] The host computer 105 is used to output control commands and is specifically configured as follows: Based on the use of feedback axial vibration retinal technology to achieve two-dimensional motion detection of targets in complex backgrounds, the target's orientation information and angular velocity are obtained. Based on the target's orientation information and angular velocity, target identification is performed on the signal in a specific direction under imaging-free conditions. Based on the target recognition results, the sub-field of view where the target is located is determined, and variable resolution ghost imaging is performed on the field of view.

[0053] In some embodiments, the host computer 105 is further configured to: When detecting the location of a target, data within the field of view is first collected, and the output voltage values ​​of the detector array are recorded when the target is in various directions. The target information is received using a photoelectric detection system. For targets in different directions, a voltage value sequence is obtained based on the different voltage output values ​​generated by the detectors at different positions. The location information of the target is calculated based on the voltage value sequence.

[0054] In some embodiments, the host computer 105 is further configured to: The voltage value sequence is used as the input to the neural network. I in According to the ideal location of the target I sta By training a neural network, we can find values ​​that match the input values. I in To output value I sta The process involves assigning corresponding neural network weights and activation values, and then configuring the learned neural network weights and activation values ​​into the neural network. Based on the trained neural network, for targets with unknown locations, the target's orientation information can be calculated according to the voltage value sequence corresponding to the target and the weights and excitation values ​​of the neural network.

[0055] In some embodiments, the host computer 105 is further configured to: The initial velocity is obtained by roughly calculating the target's angular velocity based on the time interval between the rising edges generated by two adjacent single-pixel detectors as the target flies past. Based on the initial angular velocity, the vibration velocity of the retina is gradually controlled by feedback to match the vibration velocity with the target angular velocity.

[0056] In some embodiments, the host computer 105 is further configured to: Assume that the initial retinal vibration frequency corresponds to a certain cross-correlation asymmetry of the single-pixel detector signal. a 0, taking the minimum cross-correlation asymmetry. a min = | a 0|, and simultaneously set the minimum adjustable value of the retinal vibration frequency to Δ. oh min The sign is positive, used to adjust the retinal vibration frequency, and the cross-correlation trend indicator flag is set to an initial value of 0; Fine-tuning the retinal vibration frequency Δ ohmin Calculate the cross-correlation asymmetry at this time. a 1, and will a The absolute value of 1 and the set minimum value a min Compare: like |a 1|< a min If the cross-correlation trend indicator flag is set to 1, it indicates that the cross-correlation asymmetry decreases and the frequency further increases along Δ. oh min Changes in direction, take the minimum cross-correlation value a min For the updated |a 1|, and repeat the loop starting from the fine-tuning step until the frequency adjustment exceeds the matching frequency, resulting in cross-correlation asymmetry. |a 1 | Higher than the version updated at that time a min Finally, the angular velocity of the target is calculated.

[0057] In some embodiments, the host computer 105 is further configured to: Based on the detector voltage signal collected from a specific direction, the signal characteristics in that direction are determined and compared with the signal characteristics of the ideal signal. When the similarity between the collected signal and the ideal signal exceeds a certain threshold, the target in that direction is identified as the predetermined target.

[0058] In some embodiments, the host computer 105 is further configured to: Obtain the central concave region of interest of the panoramic image and set the variable resolution annular speckle parameters according to the logarithmic polar coordinate transformation; A human-eye variable resolution ring speckle model was constructed to generate a human-eye variable resolution ring speckle sequence. Ghost imaging reconstruction is performed based on the aforementioned human eye variable resolution ring speckle sequence and the acquired sub-detector signals. Based on the results of reconstruction from multiple adjacent projection units, three-dimensional imaging of the target is achieved.

[0059] In some embodiments, the host computer 105 is further configured to: Following the variable resolution arrangement of the receptors in the human eye's retina, the projected pattern is divided into a central concave region and an edge region. The central concave region uses uniform high-resolution Cartesian sampling, while the edge region uses logarithmic polar coordinate variable resolution sampling. The edge region image is compressed to different degrees using the characteristics of the logarithmic polar coordinate model. Based on the imaging area of ​​the target, determine the center position of the human eye speckle and the radius of the central high-resolution region.

[0060] In some embodiments, the host computer 105 is further configured to: use r The radius of the outer ring of the central concave region represents the distance from the pixel to the center position. r The area within 0 is the central region. r The area outside 0 is the edge region; Based on the differences in polar radius and polar angle, the marginal region is divided into: P Rings, each ring Q 1 pixel; In the human-eye-inspired logarithmic polar coordinate model, it is assumed that... p and q They represent the first p Ring and the first q If there are 10 pixels, the variable resolution structure of the edge region can be calculated using the following system of equations: ; in, r p Indicates the first p The radius of the ring containing the ring pixel. r 1 represents the radius of the ring containing the first ring pixel. i q Indicates the first q The angle corresponding to each pixel. e Indicates the inter-ring growth coefficient; Based on the human eye logarithmic polar coordinate model and uniform projection pattern generation method, a human eye spatial variable resolution projection pattern is generated.

[0061] In some embodiments, the host computer 105 is further configured to perform ghost imaging reconstruction based on the human-eye variable-resolution annular speckle sequence and the acquired sub-detector signals using the following formula: ; in, To mimic the speckle sequence of the human eye, The image reconstructed from the corresponding ghost image. This represents the total light intensity received by the sub-detector.

[0062] In some embodiments, the host computer 105 is further configured to: By using a computational ghost imaging algorithm, the detector signals corresponding to adjacent projection units are reconstructed to obtain a reconstructed multi-view image sequence. Based on the reconstructed multi-view image sequence, the distance values ​​of different targets are calculated using a stereo matching method to achieve three-dimensional imaging of the scene; The stereo matching method includes: Images from the pair detectors Ik and images from adjacent sub-detectors I k+1 Perform homography transformation and search for the best matching point on the corresponding epipolar line; With the first k Using the pixels in the view corresponding to each sub-detector as a reference, calculate the adjacent pixels of the first... k The similarity of pixel values ​​within the search window corresponding to +1 sub-detectors in the view is used. If the similarity meets a set threshold, the pixel is determined to be the corresponding matching point, based on the disparity value between the pixels. d By utilizing the relationship between disparity values ​​and depth values, three-dimensional information can be acquired.

[0063] It should be noted that the composite bionic ghost imaging system described in this embodiment and the previously described composite bionic ghost imaging method belong to the same technical concept and can produce the same technical effect, which will not be elaborated here.

[0064] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on the invention that have equivalent elements, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or alterations. Elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, and such examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered illustrative only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0065] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the invention. This should not be construed as an intention that a feature of an unclaimed invention is necessary for any claim. Rather, the subject matter of the invention may be less than all the features of a particular embodiment of the invention. Thus, the following claims are incorporated herein by reference as examples or embodiments, wherein each claim is independently considered as a separate embodiment, and these embodiments are contemplated as being able to be combined with each other in various combinations or arrangements. The scope of the invention should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. A composite bionic ghost imaging method, characterized in that, The method includes: Based on the use of feedback axial vibration retinal technology to achieve two-dimensional motion detection of targets in complex backgrounds, the target's orientation information and angular velocity are obtained. Based on the target's orientation information and angular velocity, target identification is performed on the signal in a specific direction under imaging-free conditions. Based on the target recognition results, the sub-field of view where the detected target is located is determined, and variable resolution ghost imaging is performed on the field of view; The step of determining the sub-field of view where the detected target is located based on the target recognition result, and performing variable resolution ghost imaging on the field of view, includes: Obtain the central concave region of interest of the panoramic image and set the variable resolution annular speckle parameters according to the logarithmic polar coordinate transformation; A human-eye variable resolution ring speckle model was constructed to generate a human-eye variable resolution ring speckle sequence. Ghost imaging reconstruction is performed based on the aforementioned human eye variable resolution ring speckle sequence and the acquired sub-detector signals. Based on the results reconstructed from multiple adjacent projection units, three-dimensional imaging of the target is achieved. The process of acquiring the central concave region of interest of the panoramic image and setting the variable-resolution annular speckle parameters according to the logarithmic polar coordinate transformation includes: Following the variable resolution arrangement of the receptors in the human eye's retina, the projected pattern is divided into a central concave region and an edge region. The central concave region uses uniform high-resolution Cartesian sampling, while the edge region uses logarithmic polar coordinate variable resolution sampling. The edge region image is compressed to different degrees using the characteristics of the logarithmic polar coordinate model. Based on the imaging area of ​​the target, determine the center position of the human eye speckle and the radius of the central high-resolution region; The construction of a human-eye-like variable-resolution circular speckle model and the generation of a human-eye-like variable-resolution circular speckle sequence include: use r The radius of the outer ring of the central concave region represents the distance from the pixel to the center position. r The area within 0 is the central region. r The area outside 0 is the edge region; Based on the differences in polar radius and polar angle, the marginal region is divided into: P Rings, each ring Q 1 pixel; In the human-eye-inspired logarithmic polar coordinate model, it is assumed that... p and q They represent the first p Ring and the first q If there are 10 pixels, the variable resolution structure of the edge region can be calculated using the following system of equations: in, r p Indicates the first p The radius of the ring containing the ring pixel. r 1 represents the radius of the ring containing the first ring pixel. θ q Indicates the first q The angle corresponding to each pixel. ε Indicates the inter-ring growth coefficient; Based on the human eye logarithmic polar coordinate model and uniform projection pattern generation method, a human eye spatial variable resolution projection pattern is generated. Based on the aforementioned human-eye variable-resolution annular speckle sequence, and combined with the acquired sub-detector signals, ghost imaging reconstruction is performed using the following formula: in, To mimic the speckle sequence of the human eye, The image reconstructed from the corresponding ghost image. This represents the total light intensity received by the sub-detector.

2. The method according to claim 1, characterized in that, The method for detecting two-dimensional motion of targets against complex backgrounds using feedback-based axial vibration retinal technology, and acquiring the target's orientation information and angular velocity, includes: When detecting the location of a target, data within the field of view is first collected, and the output voltage values ​​of the detector array are recorded when the target is in various directions. The target information is received using a photoelectric detection system. For targets in different directions, a voltage value sequence is obtained based on the different voltage output values ​​generated by the detectors at different positions. The location information of the target is calculated based on the voltage value sequence.

3. The method according to claim 2, characterized in that, The step of calculating the target's azimuth information based on the voltage value sequence includes: The voltage value sequence is used as the input to the neural network. I in According to the ideal location of the target I sta By training a neural network, we can find values ​​that match the input values. I in To output value I sta The process involves assigning corresponding neural network weights and activation values, and then configuring the learned neural network weights and activation values ​​into the neural network. Based on the trained neural network, for targets with unknown locations, the target's orientation information can be calculated according to the voltage value sequence corresponding to the target and the weights and excitation values ​​of the neural network.

4. The method according to claim 2 or 3, characterized in that, The method for achieving two-dimensional motion detection of targets in complex backgrounds using feedback axial vibration retinal technology, and obtaining the target's orientation information and angular velocity, also includes: The initial velocity is obtained by roughly calculating the target's angular velocity based on the time interval between the rising edges generated by two adjacent single-pixel detectors as the target flies past. Based on the initial angular velocity, the vibration velocity of the retina is gradually controlled by feedback to match the vibration velocity with the target angular velocity.

5. The method according to claim 4, characterized in that, The step of progressively controlling the vibration velocity of the retina based on the initial measured angular velocity, so that the vibration velocity matches the target angular velocity, includes: Assume that the initial retinal vibration frequency corresponds to a certain cross-correlation asymmetry of the single-pixel detector signal. a 0, taking the minimum cross-correlation asymmetry. a min = | a 0|, and simultaneously set the minimum adjustable value of the retinal vibration frequency to Δ. ω min The sign is positive, used to adjust the retinal vibration frequency, and the cross-correlation trend indicator flag is set to an initial value of 0; Fine-tuning the retinal vibration frequency Δ ω min Calculate the cross-correlation asymmetry at this time. a 1, and will a The absolute value of 1 and the set minimum value a min Compare: like |a 1| < a min If the cross-correlation trend indicator flag is set to 1, it indicates that the cross-correlation asymmetry decreases and the frequency further increases along Δ. ω min Changes in direction, take the minimum cross-correlation value a min For the updated |a 1|, and repeat the loop starting from the fine-tuning step until the frequency adjustment exceeds the matching frequency, resulting in cross-correlation asymmetry. |a 1 | Higher than the version updated at that time a min Finally, the angular velocity of the target is calculated.

6. The method according to claim 1, characterized in that, The step of performing target recognition without imaging based on the target's azimuth information and angular velocity in a specific direction includes: Based on the detector voltage signal collected from a specific direction, the signal characteristics in that direction are determined and compared with the signal characteristics of the ideal signal. When the similarity between the collected signal and the ideal signal exceeds a certain threshold, the target in that direction is identified as the predetermined target.

7. The method according to claim 1, characterized in that, The method of achieving three-dimensional imaging of the target based on the reconstruction results of multiple adjacent projection units includes: By using a computational ghost imaging algorithm, the detector signals corresponding to adjacent projection units are reconstructed to obtain a reconstructed multi-view image sequence. Based on the reconstructed multi-view image sequence, the distance values ​​of different targets are calculated using a stereo matching method to achieve three-dimensional imaging of the scene; The stereo matching method includes: Images of the pair detectors I k and images from adjacent sub-detectors I k+1 Perform homography transformation and search for the best matching point on the corresponding epipolar line; With the first k Using the pixels in the view corresponding to each sub-detector as a reference, calculate the adjacent pixels of the first... k The similarity of pixel values ​​within the search window corresponding to +1 sub-detectors in the view is used. If the similarity meets a set threshold, the pixel is determined to be the corresponding matching point, based on the disparity value between the pixels. d By utilizing the relationship between disparity values ​​and depth values, three-dimensional information can be acquired.

8. A composite bionic ghost imaging system for implementing the method as described in any one of claims 1 to 7, characterized in that, The system includes: A single-pixel detector array is used to acquire light signals from the imaging plane at different angles; Digital micromirror device for modulating spatial light information; A light source for projecting white light onto the digital micromirror device; A data acquisition device is used to convert the light intensity value of a real single-pixel detector into an electrical signal. Optical fibers and lenses are used to project the speckle of the projection unit and to receive the optical signals from the detector. The host computer is configured as follows: Based on the use of feedback axial vibration retinal technology to achieve two-dimensional motion detection of targets in complex backgrounds, the target's orientation information and angular velocity are obtained. Based on the target's orientation information and angular velocity, target identification is performed on the signal in a specific direction under imaging-free conditions. Based on the target recognition results, the sub-field of view where the target is located is determined, and variable resolution ghost imaging is performed on the field of view.

Images