A dynamic gain regulation photoelectric hybrid neural network computing system for low-altitude unmanned aerial vehicle debris search and rescue

Abstract

This invention discloses a dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue, comprising a UAV camera, a convolutional neural network preprocessing module, a dynamic range adaptive matching module, a matrix calculation mapping control module, a weighted DAC module, an optical matrix calculation chip, a scaling restoration module, and a convolutional neural network post-processing module. The system identifies weak feature regions and performs adaptive gain amplification through the dynamic range adaptive matching module. The matrix calculation mapping control module adapts the optical calculation format, the optical matrix calculation chip performs high-speed, low-power matrix operations, and the scaling restoration module restores the true signal amplitude, ultimately outputting the target recognition result. This invention improves the accuracy of weak vital sign detection, reduces end-to-end latency and power consumption, adapts to UAV SWaP constraints, and is suitable for low-altitude UAV rubble search and rescue scenarios.

Description

Technical Field

[0001] This invention relates to the field of drone search and rescue technology, specifically to a dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude drone rubble search and rescue. Background Technology

[0002] Artificial intelligence, especially deep learning, has become a core driving force for technological innovation across various industries. As a key component of deep learning, artificial neural networks, by mimicking the structure and function of biological nervous systems, construct mathematical models capable of handling complex tasks, achieving remarkable success in fields such as computer vision, autonomous navigation, and intelligent monitoring. With the widespread adoption of drone technology, deploying neural networks on low-altitude drone platforms to achieve functions such as environmental perception, target recognition, and autonomous decision-making has become an important research direction. Especially in extreme scenarios such as post-disaster rubble search and rescue, drones can replace personnel in entering dangerous areas to complete high-risk tasks such as searching for signs of life, exploring minute gaps, and locating trapped individuals, demonstrating irreplaceable application value.

[0003] Convolutional Neural Networks (CNNs) are a class of specially designed deep learning models that excel at processing image data. By simulating the processing mechanisms of the human visual cortex, CNNs utilize structures such as convolutional layers, pooling layers, and fully connected layers to extract features from the input data layer by layer, from low-level edges and textures to high-level object parts and even the entire object. Currently, in high-precision image recognition, object detection, and other tasks, researchers typically use full-precision network models, where the weights and activation values ​​in the network are represented using 32-bit floating-point numbers. This high-precision representation preserves rich feature information and is the foundation for ensuring high accuracy in network models.

[0004] To enable real-time neural network inference on drones, low-altitude drone platforms typically utilize embedded computing platforms centered around ARM-based CPUs or integrated GPUs (System-on-a-Chip). Compared to dedicated hardware such as FPGAs and ASICs, general-purpose processors offer advantages such as shorter development cycles, more flexible programming, and a mature ecosystem, allowing for convenient model porting using mainstream deep learning frameworks. The processor's parallel computing capabilities directly determine the efficiency and real-time performance of drones in tasks such as rubble search and rescue, real-time obstacle avoidance, and detection of weak vital signs.

[0005] While full-precision convolutional neural networks hold great promise for applications in low-altitude drones, their deployment faces far greater challenges in extreme scenarios such as post-disaster rubble search and rescue than in conventional scenarios.

[0006] First, full-precision neural network models have a huge number of parameters and high computational complexity. However, UAV platforms are strictly limited by size, weight, and power consumption (SWaP), and their processor computing power is far lower than that of ground equipment. Directly running complex full-precision CNN models would result in high inference latency and high power consumption, making it difficult to meet the real-time and long-endurance requirements of rubble search and rescue missions. Second, the post-disaster rubble environment is extremely complex, with problems such as extremely low lighting, dust scattering, chaotic structures, and extremely low local contrast. UAVs flying over rubble are also affected by airflow disturbances and high-frequency vibrations of the fuselage, resulting in a large amount of noise and motion blur superimposed on the acquired images. Full-precision networks are extremely sensitive to noise, which can easily lead to false positives and false negatives of weak life features. In search and rescue scenarios, the fault tolerance is close to zero, and any missed detection may lead to search and rescue failure. Finally, UAVs have limited endurance, and high-load computing will consume power rapidly, significantly shortening the effective search and rescue time. Therefore, achieving high-speed, low-power, and highly robust neural network inference without sacrificing the accuracy of weak feature detection is crucial for UAVs in post-disaster rubble search and rescue.

[0007] Optical matrix computing chips can efficiently perform matrix multiplication operations and have significant advantages in low power consumption and high bandwidth. However, their direct application in post-disaster rubble search and rescue scenarios faces an insurmountable signal-to-noise ratio bottleneck. Optical computing is essentially analog computing, and is affected by laser relative intensity noise, thermal noise, and shot noise from photodetectors, resulting in an unavoidable analog noise floor. In rubble search and rescue missions, the targets to be detected are mostly extremely low-amplitude, low-contrast features such as tiny gaps, buried limbs, faint life outlines, and fine structures in dark areas. Moreover, images are already superimposed with dust noise and vibration interference. When such weak signals enter the optical matrix chip, they are easily submerged or confused by the chip's inherent analog noise floor. Although all-digital circuits have higher power consumption, they do not introduce analog noise. Therefore, when existing general-purpose optical matrix computing chips are directly used in UAV post-disaster rubble search and rescue, they are prone to missing trapped individuals due to the loss of weak features, failing to meet the safety and reliability requirements of search and rescue missions.

[0008] To address the issues of weak signals being submerged by analog noise and the confusion between noise and weak features in convolutional neural networks using optical matrix operation chips, existing research has proposed noise suppression and weak signal enhancement strategies based on dynamic thresholds.

[0009] For example, some researchers have proposed a noise-perceived dynamic thresholding mechanism. This mechanism constructs a decision threshold linked to the noise floor by online measurement of the standard deviation of optical domain noise, ensuring that the threshold is significantly higher than the random noise level. The convolution output is then subjected to threshold-based decision-making: portions below the threshold are classified as simulated noise and set to zero, while portions above the threshold are classified as valid signals and undergo gain compensation. This approach preserves weak, effective features while suppressing floor noise interference. This method achieved stable visual inference performance on a high-noise optical matrix chip. Further research has verified the effectiveness of the dynamic thresholding mechanism in low-light and low signal-to-noise ratio scenarios. Through real-time noise floor estimation and a dynamic threshold filtering structure, the decision threshold is adaptively adjusted in the electrical domain post-processing module based on the noise level of the optical computation output, effectively separating noise from weak target features and significantly improving the robustness of target detection under low-light conditions. Related research also shows that in optical convolutional neural networks based on Mach-Zehnder interferometer arrays, dynamic thresholding can be used in conjunction with pooling and normalization operations to form a hierarchical adaptive noise-resistant structure: shallow layers have small output amplitudes and dense weak features, so a stricter dynamic threshold is used to finely distinguish noise; deep layers have higher semantic feature amplitudes, so a relatively loose threshold is used to reduce information loss. This hierarchical dynamic thresholding strategy can effectively alleviate the inundation and confusion of weak signals by optical matrix simulation noise. Building on this, some studies have introduced adaptive soft thresholding into optoelectronic hybrid convolutional neural networks. The threshold is updated in real time by the optical computation noise basis. Compared with hard thresholding, it can smooth the output and avoid artifacts while suppressing noise, making it more suitable for continuous simulation optical computing systems and further improving the accuracy of weak signal recovery.

[0010] However, these methods still have significant shortcomings. They are not designed for the extreme environment and extremely low fault tolerance requirements of post-disaster rubble search and rescue, and are difficult to directly adapt to UAV optoelectronic hybrid computing systems. Existing dynamic thresholding and noise suppression strategies only perform post-processing noise reduction at the optical computing output end, failing to address the core issue that weak life features in rubble are already below the simulated noise floor before entering the optical matrix chip, and cannot fundamentally prevent weak signals from being submerged by noise. At the same time, existing methods do not take into account the unique characteristics of rubble scenes such as dust scattering, extremely low illumination, messy textures, and high vibration interference, and have not established a dynamic thresholding mechanism that combines multiple factors such as image local variance, ambient brightness, aircraft vibration, and dust noise, making it difficult to accurately distinguish between real weak features and rubble noise. Furthermore, the existing solutions are not optimized for the SWaP constraints of UAVs and the ultra-low fault tolerance requirements of search and rescue missions. When increasing the amplitude of weak signals, they are prone to problems such as optical modulator saturation and signal distortion. They cannot achieve a balance between low power consumption, high real-time performance, and high detection robustness, and cannot meet the core requirements of zero missed detection of weak life signs, high reliability, and low latency in post-disaster rubble search and rescue. Summary of the Invention

[0011] To address the shortcomings of existing optical matrix computing chips in low-altitude UAV rubble search and rescue scenarios, such as the submersion of weak life features by simulated noise, inability to adapt to UAV SWaP constraints, easy signal amplification saturation, and easy missed detection of weak targets, this invention provides a dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue. By introducing a digital domain dynamic range adaptive matching mechanism at the optical computing front end, combined with local variance analysis and multi-factor coupled dynamic threshold decision, it accurately identifies weak feature areas such as tiny gaps, weak life outlines, buried limbs, and fine structures in dark areas in rubble scenarios. It then performs block-independent adaptive dynamic gain amplification on these areas, artificially raising the weak signals to the linear high signal-to-noise ratio operating range of the optical matrix computing chip, making them significantly higher than the chip's inherent simulated noise floor. After the optical computing is completed, it performs inverse proportional restoration based on the gain coefficient to restore the true signal amplitude. Thus, without introducing additional noise or causing signal saturation, it retains the advantages of high speed, low power consumption, and high parallelism of optical computing, while effectively solving the problems of weak features being submerged by simulated noise and insufficient detection reliability.

[0012] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0013] A dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude drone rubble search and rescue, the system includes a drone camera, a convolutional neural network preprocessing module, a dynamic range adaptive matching module, a matrix calculation mapping control module, a weight DAC module, an optical matrix calculation chip, a scaling module, and a convolutional neural network postprocessing module;

[0014] The convolutional neural network preprocessing module is used to convert the ruin scene images captured by the UAV camera into tensor format and perform block processing on the images based on the UAV SWaP constraints.

[0015] The dynamic range adaptive matching module performs local variance analysis on each image feature map block, and calculates a dynamic threshold by combining the image ambient brightness, the real-time vibration intensity of the UAV IMU, the scattering noise of the rubble dust and the priority of the search and rescue mission. Blocks with local variance lower than the dynamic threshold are identified as weak feature regions including tiny gaps, weak life outlines, buried limb features, and dark fine structures. For such blocks, adaptive dynamic gain amplification is performed in the digital domain to raise the signal amplitude to the linear high signal-to-noise ratio working range of the optical matrix computing chip. Blocks with local variance greater than or equal to the dynamic threshold are identified as significant rubble textures or background regions and are directly output without gain processing.

[0016] The weight DAC module is used to convert the weight matrices of the convolutional layer and the fully connected layer into analog driving signals and configure them to the optical matrix computing chip.

[0017] The matrix calculation mapping control module converts the convolution operation into matrix multiplication form through the im2col algorithm. It receives and integrates the gain amplified block data and the unamplified pass-through block data output by the dynamic range adaptive matching module. After format normalization, it is converted into the corresponding analog electrical signal and modulated into the input optical signal. The optical matrix calculation chip combines the configured weight matrix to complete the matrix multiplication and accumulation operation.

[0018] The output optical signal of the optical matrix computing chip is converted into a digital signal by photoelectric conversion and ADC sampling, and then sent back to the matrix computing mapping control module. The matrix computing mapping control module completes the reconstruction of the calculation results, timing synchronization and format regularization, and sends the complete feature map data to the scaling restoration module.

[0019] The proportional restoration module performs inverse attenuation restoration on the complete feature map data according to the dynamic gain coefficient to restore the true amplitude of the signal; the convolutional neural network post-processing module is used to perform activation, pooling and normalization operations on the restored data and output the final recognition result.

[0020] Furthermore, the convolutional neural network preprocessing module is used to select a suitable optical matrix computing chip based on the drone payload size constraints for low-altitude drone disaster relief and rescue scenarios; and to determine the size and number of image blocks based on the power consumption constraints of the airborne equipment and the bit width and computing power of the optical matrix computing chip, wherein the block size matches the bit width of the computing unit, and the number of blocks is positively correlated with the chip computing power and negatively correlated with the power consumption constraints of the airborne equipment; and then, based on the determined block size and number of blocks, the images captured by the drone camera are processed into blocks.

[0021] Furthermore, the dynamic range adaptive matching module includes a dynamic range calibration unit, a dynamic threshold calculation unit, a block feature decision unit, a gain coefficient derivation unit, a signal amplitude amplification unit, and a signal output unit;

[0022] The dynamic range calibration unit is used to pre-calibrate the linear safe operating range of the optical matrix calculation chip and electro-optic modulator according to the chip range, and constrain the signal amplitude within the preset range of the chip's full range;

[0023] The dynamic threshold calculation unit is used to construct a multi-factor coupled dynamic threshold for each image feature map by combining image ambient brightness, real-time vibration amplitude of the UAV IMU, and noise from dust scattering in the debris. :

[0024] ;

[0025] In the formula, For image ambient brightness; It is a noise basis mapping function based on ambient brightness, which is a monotonically decreasing function, and the maximum noise basis of the dark area is not less than 1.2 times the minimum noise basis of the bright area; Real-time vibration amplitude of the UAV's IMU; This is a vibration correction factor, which is positively correlated with the UAV rotor speed, flight attitude, and airflow disturbance intensity, and satisfies... It is used to correct the effect of vibration on the lifting of the noise substrate of optical chips; The noise intensity caused by dust scattering from the ruins; The dust noise weighting coefficient is positively correlated with the proportion of high-frequency components and the local entropy value of the image, and satisfies the following conditions: This is used to correct the interference of dust scattering from ruins on the signal-to-noise ratio of images;

[0026] The block feature decision unit performs local variance analysis on each image feature map block, and combines it with the corresponding dynamic threshold to determine the blocks with local variance lower than the dynamic threshold as weak feature regions including tiny gaps, weak life outlines, buried limb features, and dark fine structures.

[0027] The gain coefficient derivation unit is used to independently calculate the adaptive gain coefficient with a safety upper limit constraint for each weak feature block:

[0028] ;

[0029] in, The adaptive gain coefficient for the i-th weak feature block is used to perform digital domain amplitude amplification on the block. This is a safety margin coefficient, whose value is positively correlated with the linearity redundancy requirement of the optical modulator and with the dynamic threshold. The value of is negatively correlated with the search and rescue miss detection offset coefficient. Positive correlation, and satisfying ; The offset coefficient for preventing missed detections in search and rescue operations is positively correlated with the priority of search and rescue missions. The upper limit amplitude of the linear operating range of the optical matrix calculation chip and optical modulator; The equivalent amplitude of the inherent analog noise floor of the optical matrix calculation chip; This represents the maximum value of the original feature amplitude within the current weak feature block;

[0030] The signal amplitude amplification unit is used to perform adaptive nonlinear dynamic gain amplification in the digital domain for weak feature regions using an adaptive gain coefficient, so that the signal amplitude is uniformly raised to the linear high signal-to-noise ratio operating range of the optical matrix computing chip.

[0031] The signal output unit is used to output the weak feature area after amplifying the signal amplitude, and to directly output the significant texture or background area of ​​the ruins.

[0032] Furthermore, the safety margin coefficient Obtain adaptively through the following steps:

[0033] Based on the linear operating range of the optical matrix computing chip and the electro-optic modulator, a benchmark value for calibrating the safety margin coefficient is determined. ,satisfy ;

[0034] Based on the current dynamic threshold Search and rescue mission priorities and UAV SWaP constraints are configured and adjusted according to preset hierarchical rules. The step size adjustment is negatively correlated with the dynamic threshold and the airborne power consumption constraint strength, but positively correlated with the priority of search and rescue missions.

[0035] Based on preset step size The benchmark value for the safety margin factor Iterative adjustments are made to refine the safety margin coefficient. With dynamic threshold The correlation is negative; the correction is verified after each round of adjustment. Does it meet the requirements? Furthermore, if the amplified signal amplitude falls within the linear operating range of the optical modulator, and if not, the safety margin coefficient is adjusted in the opposite direction until the corresponding constraint is met.

[0036] Taking into account the power consumption constraints of the UAV's onboard payload and the priority of the search and rescue mission, A second fine-tuning is performed, and the corresponding constraints are checked again after the fine-tuning. If they are not met, the coefficients before the fine-tuning are used as the benchmark for amplitude correction to ensure that the final coefficients meet all constraint requirements.

[0037] Furthermore, after the block feature decision unit completes the type decision for each image feature map block, it adaptively adjusts the basic overlap rate between weak feature blocks and their adjacent blocks based on the decision results. For blocks determined to be weak features, the adjacent overlap rate is corrected upward by a preset ratio based on the basic overlap rate. For blocks determined to be non-weak features, the basic overlap rate remains unchanged. The basic overlap rate is jointly determined by the bit width of the computing unit of the optical matrix computing chip and the onboard computing power constraint.

[0038] Furthermore, the scaling restoration module performs inverse restoration on each image block based on the gain coefficient matrix to recover the true signal amplitude:

[0039] ;

[0040] In the formula, This represents the output result of the i-th image block after processing by the optical matrix computing chip. This represents the true amplitude of the signal after the i-th image block is recovered. is the adaptive gain coefficient corresponding to the i-th image block.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] First, the dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue of the present invention adaptively boosts the gain of weak features at the optical computing front end, raising unusable signals below the noise floor to a linear high signal-to-noise ratio range, thus preventing weak life features from being submerged by the noise simulated by the optical chip from the source, and greatly improving the detection robustness of targets with extremely low amplitude and low contrast.

[0043] Secondly, the dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue of the present invention adopts a block-independent gain and dynamic threshold coupling mechanism. It combines multiple factors such as ambient brightness, UAV vibration, dust scattering, and search and rescue priority to adaptively adjust the decision threshold and amplification intensity, which can accurately adapt to complex and extreme rubble scenarios and significantly reduce the false detection rate and false detection rate.

[0044] Third, the dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue of the present invention adaptively determines the block strategy based on the constraints of optical chip computing unit bit width, computing power and airborne power consumption. While ensuring the integrity of weak features, it optimizes the computing load, perfectly adapts to the SWaP constraints of UAVs, and achieves low power consumption, low latency and high real-time inference.

[0045] Fourth, the dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue of the present invention adopts a reversible control mechanism of "amplification first - optical computing then restoration", which scales the signal and noise proportionally, prevents noise accumulation and deterioration of the signal-to-noise ratio, and can enjoy the performance advantages of optical computing while achieving the detection accuracy and reliability of digital circuits. Attached Figure Description

[0046] Figure 1 This is a diagram of the architecture of the dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude drone rubble search and rescue according to the present invention. Detailed Implementation

[0047] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0048] This invention discloses a dynamic gain-modulated optoelectronic hybrid neural network computing system for low-altitude drone rubble search and rescue. The system includes a drone camera, a convolutional neural network preprocessing module, a dynamic range adaptive matching module, a matrix calculation mapping control module, a weight DAC module, an optical matrix calculation chip, a scaling module, and a convolutional neural network postprocessing module.

[0049] The convolutional neural network preprocessing module is used to convert the ruin scene images captured by the UAV camera into tensor format and perform block processing on the images based on the UAV SWaP constraints.

[0050] The dynamic range adaptive matching module performs local variance analysis on each image feature map block, and calculates a dynamic threshold by combining the image ambient brightness, the real-time vibration intensity of the UAV IMU, the scattering noise of the rubble dust and the priority of the search and rescue mission. Blocks with local variance lower than the dynamic threshold are identified as weak feature regions including tiny gaps, weak life outlines, buried limb features, and dark fine structures. For such blocks, adaptive dynamic gain amplification is performed in the digital domain to raise the signal amplitude to the linear high signal-to-noise ratio working range of the optical matrix computing chip. Blocks with local variance greater than or equal to the dynamic threshold are identified as significant rubble textures or background regions and are directly output without gain processing.

[0051] The weight DAC module is used to convert the weight matrices of the convolutional layer and the fully connected layer into analog driving signals and configure them to the optical matrix computing chip.

[0052] The matrix calculation mapping control module converts the convolution operation into matrix multiplication form through the im2col algorithm. It receives and integrates the gain amplified block data and the unamplified pass-through block data output by the dynamic range adaptive matching module. After format normalization, it is converted into the corresponding analog electrical signal and modulated into the input optical signal. The optical matrix calculation chip combines the configured weight matrix to complete the matrix multiplication and accumulation operation.

[0053] The output optical signal of the optical matrix computing chip is converted into a digital signal by photoelectric conversion and ADC sampling, and then sent back to the matrix computing mapping control module. The matrix computing mapping control module completes the reconstruction of the calculation results, timing synchronization and format regularization, and sends the complete feature map data to the scaling restoration module.

[0054] The proportional restoration module performs inverse attenuation restoration on the complete feature map data according to the dynamic gain coefficient to restore the true amplitude of the signal; the convolutional neural network post-processing module is used to perform activation, pooling and normalization operations on the restored data and output the final recognition result.

[0055] After the system powers on and starts up, it completes overall initialization and hardware self-test. The UAV's onboard general-purpose processor then enters the working state and begins loading the pre-trained convolutional neural network model weight file. The processor sequentially reads the weight parameters of the convolutional layers and fully connected layers, and converts the weight matrix into a configurable parameter format according to the array structure and port mapping rules of the optical matrix computing chip. Through a high-speed configuration interface, the weight matrix is ​​configured online into the programmable optical array inside the optical matrix computing chip, completing the hardware mapping of the neural network computing core and ensuring that the optical matrix computing chip can correctly perform vector-matrix multiplication operations.

[0056] The convolutional neural network preprocessing module addresses the complex scenarios and payload constraints of low-altitude UAV disaster relief and rubble search and rescue. Based on the UAV platform's size, weight, and installation space limitations, a light matrix computing chip with matching computing power, size, and power consumption is selected. The dynamic range calibration unit synchronously performs initialization calibration during system startup. Based on the light matrix computing chip's rated input range, saturation threshold, and the linear operating range of the electro-optic modulator, the safe input range of the chip is pre-calibrated. During calibration, the dynamic range calibration unit obtains the critical amplitude point from the linear operating region to the saturation region of the chip by inputting a standard amplitude calibration signal. This is used to determine the signal safety upper limit, constraining the input signal amplitude within a preset reasonable range of the chip's full-scale range. By pre-calibrating the linear safe operating range, weak features can be effectively prevented from entering the saturation region due to excessive amplitude during subsequent adaptive gain amplification, preventing signal distortion, modulator overdrive, and abnormal output of the light chip, ensuring the entire optoelectronic hybrid computing path operates in a stable, linear, and low-distortion state.

[0057] During the search and rescue operation in the ruins, the drone's camera captures low-altitude scene images in real time, outputting continuous video frames one by one to the convolutional neural network preprocessing module. The convolutional neural network preprocessing module sequentially performs image scaling, pixel value normalization, channel order conversion, and data format regularization on the input image, reducing the impact of lighting changes and resolution differences on subsequent feature analysis. It converts the original two-dimensional image into a four-dimensional tensor format that meets the requirements of neural network input, forming standard feature map data that can be used for subsequent matrix operations.

[0058] After receiving feature map data in tensor format, the matrix computation mapping control module reconstructs the convolution computation structure using the im2col algorithm. This unfolds the traditional convolution operation and converts it into a general vector-matrix multiplication operation, enabling the convolution computation to efficiently adapt to the parallel computing architecture of the optical matrix computing chip and fully leverage the advantages of high speed, low power consumption, and high throughput in optical computing. The convolutional neural network preprocessing module includes an image segmentation unit. This unit adaptively determines the segment size and number of image feature maps by considering the power consumption constraints of the onboard equipment, the bit width of the computing unit in the optical matrix computing chip, the array size, and the maximum computing power parameters. This ensures that the segment size matches the bit width of the chip's computing unit, and that the number of segments is positively correlated with the chip's computing power and negatively correlated with the power consumption constraints of the onboard equipment, thus reducing system power consumption while ensuring real-time computation. After determining the segmentation rules according to the above strategy, the image segmentation unit performs regularization segmentation processing on the tensor format image feature maps, providing a data foundation for subsequent weak feature detection and dynamic gain amplification.

[0059] The dynamic range adaptive matching module is the core unit of this invention for achieving reliable detection of weak features. It integrates a dynamic range calibration unit, a dynamic threshold calculation unit, a block feature decision unit, a gain coefficient derivation unit, a signal amplitude amplification unit, and a signal output unit. These units work together to achieve accurate identification and adaptive gain control of weak features.

[0060] The dynamic threshold calculation unit is responsible for constructing a multi-factor coupled dynamic threshold. Its calculation process fully incorporates the complex environmental factors and mission requirements of rubble search and rescue scenarios. Specifically, this dynamic threshold calculation unit combines key influencing factors such as image ambient brightness, real-time vibration amplitude of the UAV IMU, and rubble dust scattering noise to construct a multi-factor coupled dynamic threshold for each image feature map. :

[0061] ;

[0062] In the formula, For image ambient brightness; The noise floor mapping function is based on ambient brightness. This function exhibits a monotonically decreasing characteristic, ensuring that the noise floor estimate in dark scenes is not less than 1.2 times the minimum noise floor in bright scenes, in order to adapt to changes in noise levels under different lighting conditions. Real-time vibration amplitude of the UAV's IMU; This is a vibration correction factor, which is positively correlated with the UAV rotor speed, flight attitude, and airflow disturbance intensity, and satisfies... It is used to correct the effect of vibration on the lifting of the noise substrate of optical chips; The noise intensity caused by dust scattering from the ruins; The dust noise weighting coefficient is positively correlated with the proportion of high-frequency components and the local entropy value of the image, and satisfies the following conditions: This is used to correct the interference of scattering from rubble dust on the image signal-to-noise ratio. Through the above-mentioned multi-factor coupled dynamic threshold calculation model, the system can accurately and adaptively adjust the decision threshold of weak features to match the complex and ever-changing rubble environment and the dynamic search and rescue mission requirements, providing a reliable basis for subsequent block feature decision-making.

[0063] The block feature decision unit receives image feature map blocks after normalization by the image block segmentation unit. For each block, it performs refined local variance analysis to quantify the texture complexity and feature saliency within the block. This unit traverses all pixels within the block using a sliding window, calculates the local variance of pixel values ​​within the window, and generates a global local variance feature value for that block based on the variance statistics of all sliding windows within the block. This feature value characterizes the dispersion and texture richness of pixel values ​​within the block. Subsequently, the block feature decision unit couples the local variance feature value of this block with the corresponding block multi-factor dynamic threshold output by the dynamic threshold calculation unit. A comparison is performed to accurately determine the block type: if the local variance eigenvalue of the block is lower than the dynamic threshold... If the local variance eigenvalue of the block is greater than or equal to the dynamic threshold, then the block is determined to be a region with weak features. Typical features include small gaps in a ruin scene, faint life outlines of buried persons, features of buried limbs, and low-contrast, low-amplitude targets such as fine structures in dark areas. If the block is determined to be a significant textured area of ​​ruins or a background area, no gain amplification processing is required, and it is directly output to the subsequent process.

[0064] Through the aforementioned decision mechanism based on the coupling of local variance and dynamic threshold, the system can accurately identify weak life features in ruins that are easily submerged by noise, while avoiding ineffective gain on high-contrast and significant textures. Under the premise of ensuring an extremely low false negative rate, the system optimizes computational efficiency and power consumption, perfectly adapting to the extremely complex scenario requirements of low-altitude UAV search and rescue in ruins.

[0065] The gain coefficient derivation unit is the core computational unit for realizing block-based adaptive gain control. For each weak feature block output by the block feature decision unit, it independently calculates the adaptive gain coefficient with a safety upper limit constraint, ensuring that the gain amplification process both meets the signal boosting requirements and strictly avoids the risk of optical chip saturation. This unit uses the following gain coefficient calculation model:

[0066] ;

[0067] in, The adaptive gain coefficient for the i-th weak feature block is used to perform digital domain amplitude amplification on the block. This is a safety margin coefficient, whose value is positively correlated with the linearity redundancy requirement of the optical modulator and with the dynamic threshold. The value of is negatively correlated with the search and rescue miss detection offset coefficient. Positive correlation, and satisfying This is used to reserve a safety margin for signal amplification, so as to avoid the amplified signal amplitude reaching the upper limit of the linear range of the optical chip. The upper limit amplitude of the linear operating range of the optical matrix calculation chip and optical modulator is pre-calibrated by the dynamic range calibration unit during the system initialization phase. The equivalent amplitude of the inherent analog noise floor of the optical matrix calculation chip is a fixed parameter determined by the chip's hardware characteristics. This represents the maximum original feature amplitude within the current weak feature block, used to characterize the original signal strength within the block. Search and rescue anti-missed detection offset coefficient. It is positively correlated with the search and rescue mission level; the higher the mission level, the greater the shift coefficient in preventing missed detections during search and rescue operations. The larger the value, the greater the safety margin coefficient. Without triggering the saturation of the optical chip, the gain coefficient of the weak feature block is increased as much as possible, raising the amplitude of weak life features to a high signal-to-noise ratio range far above the noise floor, so as to avoid weak life features being misjudged as noise to the greatest extent, thereby meeting the core requirement of zero missed detection during the golden rescue period.

[0068] As a preferred example, the safety margin coefficient can be implemented using the following linear mapping method:

[0069] ;

[0070] In the formula, Based on the safety margin, and These are the dynamic threshold adjustment coefficient and the search and rescue priority adjustment coefficient, respectively. The parameters must satisfy: when Take the maximum value. When taking the minimum value, ;when Take the minimum value. When taking the maximum value, . , and The safety margin coefficient can be determined by fitting calibration data during system initialization or signal sample data from historical scenarios, ensuring that it meets requirements under different dynamic thresholds and search and rescue mission levels. The constraints are as follows. Preferably, the fitted coefficients are then calibrated a second time based on the actual optical chip model and the noise level of the ruin environment to adapt to the needs of different application scenarios.

[0071] As another preferred example, the safety margin factor The results are obtained through an adaptive iterative process involving multi-factor coupling, with the following specific steps:

[0072] Step 1: Based on the rated input range and saturation threshold of the optical matrix calculation chip, as well as the linear response range of the electro-optic modulator, pre-calibrate the baseline value of the safety margin coefficient. ,satisfy The constraints are defined. This benchmark value provides an initial reference for subsequent adaptive adjustments, ensuring that the initial safety margin of gain amplification conforms to the hardware characteristics of the optical chip, thus mitigating the risk of signal saturation from the source.

[0073] Step 2, based on the current dynamic threshold Search and rescue mission priorities and UAV SWaP (size, weight, power consumption) constraints are configured and adjusted according to preset hierarchical rules. Among them, the adjustment step size is negatively correlated with the dynamic threshold and the airborne power consumption constraint strength, and positively correlated with the search and rescue mission priority. When the dynamic threshold is low and the search and rescue priority is high, a larger adjustment step size is configured to improve the gain strength. When the power consumption constraint is strict, a smaller step size is configured to control the system power consumption, so as to achieve a dynamic balance between detection performance and power consumption.

[0074] Step 3, based on the preset step size The benchmark value for the safety margin factor Iterative adjustments are made to refine the safety margin coefficient. With dynamic threshold There is a negative correlation, meaning that the lower the dynamic threshold, the larger the safety margin coefficient, and the higher the corresponding gain strength; the correction is verified after each round of adjustment. Does it meet the requirements? Furthermore, it checks whether the amplified signal amplitude falls within the linear operating range of the optical modulator. If not, it adjusts the safety margin coefficient in the opposite direction and iterates again until all constraints are met.

[0075] Step 4: Based on iterative adjustments, combine the power consumption constraints of the UAV's onboard payload with the search and rescue mission priority-corresponding search and rescue anti-missed detection offset coefficient. Using table lookup methods, etc. Perform a second amplitude limiting fine-tuning to obtain the fine-tuned result. ; for the fine-tuning The corresponding constraints are checked again. If they are not met, the coefficient before fine-tuning is used as the benchmark for amplitude limiting correction to ensure that the final output safety margin coefficient fully meets all the constraints such as the linear range, value range, and airborne power consumption of the optical chip.

[0076] This gain coefficient calculation model possesses inherent adaptive properties: for weak feature blocks with lower original amplitudes, Smaller, corresponding to the gain coefficient It automatically increases, achieving precise control by amplifying the weakest signal; simultaneously, it utilizes a safety margin coefficient. The constraints ensure that the amplitude of all amplified signals falls strictly within the linear safe operating range of the optical chip. This artificially elevates weak signals to a high signal-to-noise ratio range far above the noise floor while completely avoiding signal saturation distortion, thus meeting the needs of weak life sign detection in low-altitude UAV rubble search and rescue scenarios.

[0077] The signal amplitude amplification unit receives the adaptive gain coefficients for each weak feature block output by the gain coefficient derivation unit. For each block identified as a weak feature region, it performs precise adaptive dynamic gain amplification in the digital domain. This unit multiplies the original feature amplitude of all pixels within a block by the corresponding block's gain coefficient, working on a block-by-block basis. This invention achieves a linear amplitude boost, uniformly raising weak signals that were originally below the analog noise floor of the optical chip to the linear high signal-to-noise ratio operating range of the optical matrix computing chip. This ensures that the signal strength is significantly higher than the chip's inherent noise floor, fundamentally preventing weak life features from being submerged by noise and guaranteeing the reliability of the optical computing process. The invention employs a block-based independent gain mechanism, performing amplification only on weak feature regions, while leaving significant texture areas of ruins and background regions unaffected. This achieves targeted enhancement of low-amplitude targets while avoiding excessive amplification of high-contrast effective features. While improving detection accuracy, it effectively reduces the system's computational and power consumption overhead, perfectly adapting to the SWaP constraints of UAVs.

[0078] The signal output unit, serving as the output interface of the dynamic range adaptive matching module, is responsible for integrating and outputting the segmented data. For segments with weak feature regions that have undergone gain amplification, the data is normalized according to the segment position and gain coefficient before being output. For segments identified as significant textures or background areas of ruins, the original data is preserved and directly output to subsequent processes. This unit ensures that all segmented data is output completely in the original image spatial order, providing a standardized input with a unified format and continuous timing for subsequent optical matrix calculations, thus guaranteeing the timing synchronization and data integrity of the entire optoelectronic hybrid computing link.

[0079] In a preferred embodiment of the present invention, after completing the type determination of each image feature map block, the block feature determination unit adaptively adjusts the overlap rate between blocks based on the determination results to solve the problem of missed detection of weak features caused by block boundary truncation, while taking into account system computational efficiency and UAV SWaP constraints. Specifically, the system pre-determines the basic overlap rate between blocks based on the bit width and array size of the optical matrix calculation chip, combined with airborne computing power and power consumption constraints. This basic overlap rate is the default overlap parameter for the entire image blocks, controlling computational redundancy while ensuring feature continuity. After completing the type determination, the block feature decision unit performs differentiated overlap rate adjustments for different types of blocks: For blocks judged as having weak features (including low-amplitude targets such as tiny gaps, weak life outlines, and buried limb features), the overlap rate between the block and its adjacent blocks is adjusted upwards by a preset ratio based on the basic overlap rate, increasing the overlapping area between blocks. This avoids truncating or missing weak features due to their location at block boundaries, thus improving the detection completeness of low-contrast targets. For blocks judged as having non-weak features (i.e., areas with significant texture in ruins or background areas), the basic overlap rate is kept unchanged to avoid meaningless computational redundancy. This optimizes system computing power and power consumption while ensuring detection accuracy, perfectly adapting to the resource constraints of the UAV platform.

[0080] The scaling module receives the digital results after photoelectric conversion and ADC sampling, and combines them with the gain coefficient matrix generated by the dynamic range adaptive matching module to perform a precise inverse scaling operation on each image block to recover the true amplitude of the signal. Its calculation model is as follows:

[0081] ;

[0082] In the formula, This represents the output result of the i-th image block after processing by the optical matrix computing chip. This represents the true amplitude of the signal after the i-th image block is recovered. For the i-th image block, the adaptive gain coefficient is given. For significant texture / background blocks that have not undergone gain amplification, After restoration, the signal retains its original amplitude. The proportional restoration module attenuates the optical computing output in blocks proportionally according to the corresponding gain coefficient, accurately restoring the signal amplitude of the weak feature blocks from the linear high signal-to-noise ratio range of the optical chip to the original input level. This mechanism ensures that the signal is scaled proportionally to the inherent analog noise floor of the optical matrix computing chip, preventing noise accumulation and amplification during optical computing. The final output signal's signal-to-noise ratio is completely consistent with the original input signal. This retains the core advantages of optical computing—high speed, low power consumption, and high parallelism—while completely solving the industry pain point of weak features being overwhelmed by analog noise, achieving a perfect fusion of optical computing performance and digital circuit detection accuracy.

[0083] The optical matrix computing chip utilizes its physical characteristics to complete large-scale matrix multiplication operations in an extremely short time. The output optical signal is converted into an electrical signal via a balanced photodetector, sampled into a digital signal by the output ADC module, and then reassembled by the matrix computing mapping control module. After timing synchronization and format normalization, the result is sent back to the scaling restoration module. Upon receiving the result, the scaling restoration module performs inverse attenuation restoration based on the previously applied gain coefficient, thereby eliminating the background noise generated during the optical chip's analog calculations. Finally, the convolutional neural network post-processing module continues to perform subsequent calculation steps such as ReLU activation and pooling on the restored complete feature map until the final target detection result is output.

[0084] In a typical UAV dynamic tracking test, this system accurately identified and locked onto the target person even when the UAV flew at a speed of 8 m / s, and reliably controlled the UAV to perform the target tracking task. The end-to-end latency from image acquisition to target recognition output is less than 50 milliseconds, and the real-time processing performance is significantly better than traditional airborne computing solutions. Since the core matrix multiplication and accumulation operations are performed by a low-power optical matrix computing chip, the peak power consumption of this system is reduced by about 50% compared to a pure electronic GPU solution with equivalent computing power. This effectively extends the single-flight endurance of the UAV, significantly improves the efficiency and coverage of rubble inspection operations, and fully demonstrates the practical value and performance advantages of this invention in low-altitude UAV rubble search and rescue scenarios.

[0085] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0086] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue, characterized in that, The system includes a drone camera, a convolutional neural network preprocessing module, a dynamic range adaptive matching module, a matrix calculation mapping control module, a weight DAC module, an optical matrix calculation chip, a scaling module, and a convolutional neural network postprocessing module. The convolutional neural network preprocessing module is used to convert the ruin scene images captured by the UAV camera into tensor format and perform block processing on the images based on the UAV SWaP constraints. The dynamic range adaptive matching module performs local variance analysis on each image feature map block, and calculates a dynamic threshold by combining the image ambient brightness, the real-time vibration intensity of the UAV IMU, the scattering noise of the rubble dust and the priority of the search and rescue mission. Blocks with local variance lower than the dynamic threshold are identified as weak feature regions including tiny gaps, weak life outlines, buried limb features, and dark fine structures. For such blocks, adaptive dynamic gain amplification is performed in the digital domain to raise the signal amplitude to the linear high signal-to-noise ratio working range of the optical matrix computing chip. Blocks with local variance greater than or equal to the dynamic threshold are identified as significant rubble textures or background regions and are directly output without gain processing. The weight DAC module is used to convert the weight matrices of the convolutional layer and the fully connected layer into analog driving signals and configure them to the optical matrix computing chip. The matrix calculation mapping control module converts the convolution operation into matrix multiplication form through the im2col algorithm. It receives and integrates the gain amplified block data and the unamplified pass-through block data output by the dynamic range adaptive matching module. After format normalization, it is converted into the corresponding analog electrical signal and modulated into the input optical signal. The optical matrix calculation chip combines the configured weight matrix to complete the matrix multiplication and accumulation operation. The output optical signal of the optical matrix computing chip is converted into a digital signal by photoelectric conversion and ADC sampling, and then sent back to the matrix computing mapping control module. The matrix computing mapping control module completes the reconstruction of the calculation results, timing synchronization and format regularization, and sends the complete feature map data to the scaling restoration module. The proportional restoration module performs inverse attenuation restoration on the complete feature map data according to the dynamic gain coefficient to restore the true amplitude of the signal; The convolutional neural network post-processing module is used to perform activation, pooling, and normalization operations on the restored data and output the final recognition result.

2. The dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue according to claim 1, characterized in that, The convolutional neural network preprocessing module, designed for low-altitude UAV disaster relief and rubble search scenarios, selects a suitable optical matrix computing chip based on the UAV payload size constraints. It then determines the size and number of image blocks based on the power consumption constraints of the onboard equipment, as well as the bit width and computing power of the optical matrix computing chip's computing units. The block size matches the bit width of the computing units, and the number of blocks is positively correlated with the chip's computing power and negatively correlated with the onboard equipment's power consumption constraints. Finally, based on the determined block size and number, the images captured by the UAV camera are processed into blocks.

3. The dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue according to claim 1, characterized in that, The dynamic range adaptive matching module includes a dynamic range calibration unit, a dynamic threshold calculation unit, a block feature decision unit, a gain coefficient derivation unit, a signal amplitude amplification unit, and a signal output unit. The dynamic range calibration unit is used to pre-calibrate the linear safe operating range of the optical matrix calculation chip and electro-optic modulator according to the chip range, and constrain the signal amplitude within the preset range of the chip's full range; The dynamic threshold calculation unit is used to construct a multi-factor coupled dynamic threshold for each image feature map by combining image ambient brightness, real-time vibration amplitude of the UAV IMU, and noise from dust scattering in the debris. : ； In the formula, For image ambient brightness; It is a noise basis mapping function based on ambient brightness, which is a monotonically decreasing function, and the maximum noise basis of the dark area is not less than 1.2 times the minimum noise basis of the bright area; Real-time vibration amplitude of the UAV's IMU; This is a vibration correction factor, which is positively correlated with the UAV rotor speed, flight attitude, and airflow disturbance intensity, and satisfies... It is used to correct the effect of vibration on the lifting of the noise substrate of optical chips; The noise intensity caused by dust scattering from the ruins; The dust noise weighting coefficient is positively correlated with the proportion of high-frequency components and the local entropy value of the image, and satisfies the following conditions: This is used to correct the interference of dust scattering from ruins on the signal-to-noise ratio of images; The block feature decision unit performs local variance analysis on each image feature map block, and combines it with the corresponding dynamic threshold to determine the blocks with local variance lower than the dynamic threshold as weak feature regions including tiny gaps, weak life outlines, buried limb features, and dark fine structures. The gain coefficient derivation unit is used to independently calculate the adaptive gain coefficient with a safety upper limit constraint for each weak feature block: ； in, The adaptive gain coefficient for the i-th weak feature block is used to perform digital domain amplitude amplification on the block. This is a safety margin coefficient, whose value is positively correlated with the linearity redundancy requirement of the optical modulator and with the dynamic threshold. The value of is negatively correlated with the search and rescue miss detection offset coefficient. Positive correlation, and satisfying ; The offset coefficient for preventing missed detections in search and rescue operations is positively correlated with the priority of search and rescue missions. The equivalent amplitude of the inherent analog noise floor of the optical matrix computing chip. The upper limit amplitude of the linear range of the optical matrix calculation chip and optical modulator; This represents the maximum value of the original feature amplitude within the current weak feature block; The signal amplitude amplification unit is used to perform adaptive nonlinear dynamic gain amplification in the digital domain for weak feature regions using an adaptive gain coefficient, so that the signal amplitude is uniformly raised to the linear high signal-to-noise ratio operating range of the optical matrix computing chip. The signal output unit is used to output the weak feature area after amplifying the signal amplitude, and to directly output the significant texture or background area of ​​the ruins.

4. The dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue according to claim 3, characterized in that, The safety margin coefficient Obtain adaptively through the following steps: Based on the linear operating range of the optical matrix computing chip and the optical modulator, a benchmark value for calibrating the safety margin coefficient is determined. ,satisfy ; Based on the current dynamic threshold Search and rescue mission priorities and UAV SWaP constraints are configured and adjusted according to preset hierarchical rules. ; The adjustment step size is negatively correlated with the dynamic threshold and the airborne power consumption constraint strength, and positively correlated with the priority of search and rescue missions; Based on preset step size The benchmark value for the safety margin factor Iterative adjustments are made to refine the safety margin coefficient. With dynamic threshold The correlation is negative; the correction is verified after each round of adjustment. Does it meet the requirements? Furthermore, if the amplified signal amplitude falls within the linear operating range of the optical modulator, and if not, the safety margin coefficient is adjusted in the opposite direction until the corresponding constraint is met. Taking into account the power consumption constraints of the UAV's onboard payload and the priority of the search and rescue mission, A second fine-tuning is performed, and the corresponding constraints are checked again after the fine-tuning. If they are not met, the coefficients before the fine-tuning are used as the benchmark for amplitude correction to ensure that the final coefficients meet all constraint requirements.

5. The dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue according to claim 3, characterized in that, After the block feature decision unit completes the type decision of each image feature map block, it adaptively adjusts the basic overlap rate between weak feature blocks and their adjacent blocks according to the decision result. For blocks that are judged as weak features, the adjacent overlap rate is corrected upward by a preset ratio based on the basic overlap rate. For blocks that are judged as non-weak features, the basic overlap rate remains unchanged. The basic overlap rate is determined by the bit width of the computing unit of the optical matrix computing chip and the onboard computing power constraint.

6. The dynamic gain-controlled optoelectronic hybrid neural network computing system for low-altitude UAV rubble search and rescue according to claim 1, characterized in that, The scaling restoration module performs inverse restoration on each image block according to the gain coefficient matrix to recover the true signal amplitude: ； In the formula, This represents the output result of the i-th image block after processing by the optical matrix computing chip. This represents the true amplitude of the signal after the i-th image block is recovered. is the adaptive gain coefficient corresponding to the i-th image block.

Images