Dynamic variable array plane bionic compound eye optical detection system

Abstract

The invention discloses a bionic compound eye optical detection system with a dynamic variable array plane, and belongs to the technical field of advanced optical detection and bionic intelligent sensing. The system comprises a dynamic variable array plane subsystem, an intelligent control and processing unit and a communication peripheral unit. The core is that the dynamic variable array plane subsystem is integrated with a substrate capable of being electrically controlled to deform and a sub-eye unit array distributed on the substrate, and each sub-eye comprises a variable focal length lens, a switchable light splitting module and a multi-band image sensor. The intelligent control and processing unit operates optimization algorithms such as view field-resolution tradeoff, field depth expansion and intelligent wave band fusion, and integrated intelligent cooperative control over the macroscopic array surface form, the microscopic sub-eye optical parameters and the image processing flow is achieved. The defects that a traditional mechanical adjustment compound eye system is slow in response and poor in flexibility are overcome, autonomous dynamic reconstruction can be achieved according to detection scenes and task requirements, and the method is particularly suitable for wide-area searching and accurate positioning of high-speed small targets.

Description

Technical Field

[0001] This invention belongs to the field of advanced optical detection and biomimetic intelligent sensing technology, specifically relating to a biomimetic compound eye optical detection system. More specifically, this invention relates to an adaptive biomimetic compound eye optical detection system whose array morphology, optical parameters, and operating modes can be electrically controlled, autonomously, and dynamically reconfigured according to the detection scenario and mission requirements. It is particularly suitable for wide-area search, precise positioning, identification, tracking, and early warning of high-speed, small targets such as missiles and drones. Background Technology

[0002] Bionic compound eye systems mimic the multi-pore structure and function of insect compound eyes in nature. Due to their potential to achieve a large field of view, high sensitivity, and rapid perception of moving targets, they have attracted widespread attention in fields such as wide-area surveillance, drone obstacle avoidance, and advanced security.

[0003] Existing technical solutions can be mainly divided into two categories: The first type is a static compound eye system based on a rigid fixed substrate. This type of system arranges multiple small imaging units ("sub-eyes") in a matrix on a rigid planar or curved substrate. Its working principle is that each sub-eye images independently, and the images are ultimately stitched together using an image stitching algorithm to create a large field-of-view panoramic image. Although this type of system has a relatively simple structure, its field of view, resolution, and focal length are fixed after the system is manufactured, making dynamic adjustments impossible according to mission requirements. This results in extremely poor flexibility and an inability to cope with complex and ever-changing detection scenarios.

[0004] The second type is the compound eye system, which introduces mechanical adjustment mechanisms to improve flexibility. This is the closest prior art to the present invention. Such systems attempt to adjust the parameters of the sub-eyes mechanically. Typical solutions include: equipping each sub-eye with a miniature stepper motor and lead screw mechanism to mechanically adjust the pitch and tilt angles of its optical lens; using a precision linear guide and servo motor system to change the physical spacing between sub-eye modules; and using mechanisms such as voice coil motors inside the sub-eyes to fine-tune the lens focal length.

[0005] However, this mechanical transmission-based solution has a series of inherent and insurmountable technical bottlenecks: 1. Slow response speed, making it difficult to track high-speed targets: The significant inertia of mechanical moving parts results in response delays typically on the order of seconds, making it impossible to achieve rapid reconstruction at the millisecond level. This makes it unable to effectively and continuously track and identify high-speed maneuvering targets such as missiles and drones.

[0006] 2. The system is bulky and complex, limiting the array size and flexibility: When attempting to expand the sub-eye array size to improve performance, the accompanying mechanical adjustment mechanisms increase exponentially in size, weight, power consumption, and complexity. In addition, all sub-eyes are fixed on the same rigid base, making it impossible to achieve dynamic changes in the overall macroscopic shape of the detection array (such as from a plane to a sphere), fundamentally limiting the flexibility of the system configuration.

[0007] 3. Significant challenges in control precision and reliability: To achieve seamless wide-area imaging, high-precision coordinated control of dozens or even hundreds of independent mechanical actuators is required, resulting in extremely complex control algorithms. During long-term operation, wear and aging of mechanical components inevitably introduce errors, leading to system performance degradation, reduced reliability, and high maintenance costs.

[0008] At the optical component level, electrowetting liquid lenses, microelectromechanical systems (MEMS) actuators, and liquid crystal tunable filters are known electro-optical technologies, each exhibiting advantages in response speed and integration. However, existing technologies lack a systematic architecture and collaborative control method capable of organically integrating these advanced electro-optical components into a unified whole, particularly addressing the core challenge of real-time, high-precision, and integrated collaborative control of macroscopic array morphology and microscopic sub-eye optical parameters.

[0009] Therefore, there is an urgent need in this field for a completely new technological approach to fundamentally break through the constraints of mechanical adjustment and achieve a leap in the perception dimension and intelligence level of the bionic compound eye system. Summary of the Invention

[0010] I. Purpose of the Invention The purpose of this invention is to overcome the shortcomings of existing bionic compound eye systems, especially those based on mechanical adjustment, such as slow response speed, poor system flexibility, complex collaborative control, and low reliability. It aims to provide a novel adaptive bionic compound eye optical detection system based on dynamically variable arrays and intelligent collaborative control. This invention focuses on fundamental changes in system architecture and the use of hardware and software collaborative control algorithms to solve new technical problems.

[0011] Based on the above overall objectives, the specific objectives of the present invention include: 1. A system capable of dynamic electronically reconfigurable macroscopic array morphology is provided: the aim is to replace the traditional rigid or mechanically movable base with an electrically deformable base, enabling the entire detection array to switch rapidly and continuously between various morphologies such as planar, convex, concave, and even free-form surfaces, thereby fundamentally eliminating the limitations of mechanical inertia on the system's response speed and flexibility.

[0012] 2. A system is provided that enables independent electronically controlled adjustment of the optical parameters of microscopic sub-eyes: the system aims to enable each sub-eye to independently and rapidly adjust its focal length and working band without mechanical movement by integrating electronically controlled optical components such as variable focal length lenses and switchable beam splitting modules, thus providing the underlying hardware foundation for the system's refined and adaptive detection.

[0013] 3. A system capable of intelligent collaborative control of macroscopic morphology and microscopic parameters is provided: The aim is to construct an intelligent control core that integrates multiple optimization strategy algorithms to achieve closed-loop collaborative control of deformable substrate, variable focal length sub-eye, beam splitting module and image processing flow, so that the system can autonomously and dynamically optimize its global detection performance according to high-level task instructions and real-time environmental perception data.

[0014] 4. Provide a system capable of outputting multi-dimensional fused information: The aim is to enable the system to output not only wide-area images, but also fused intelligence including target semantic information (such as attributes and threat levels), high-precision three-dimensional spatial coordinates, motion trajectory and future trend predictions, through the synergy of the above hardware and software, so as to provide direct support for subsequent decision-making and interception operations.

[0015] II. Technical Solution To achieve the aforementioned objectives, this invention provides an adaptive bionic compound eye optical detection system based on a dynamically variable array. This system, through the construction of two core technological pillars—a "dynamically variable array" and "intelligent collaborative control"—transforms a traditional, static, and functionally fixed optical system into a dynamic, functionally reconfigurable intelligent sensing system.

[0016] Please see Figure 1 The system, centered on closed-loop intelligent control of "perception-decision-reconstruction," comprises the following three parts: The dynamic variable array subsystem (1) serves as the physical sensing front end of the system. The intelligent control and processing unit (2) is electrically connected to the dynamic variable array subsystem (1) and serves as the decision-making and control core of the system. The communication and peripheral unit (3) is connected to the intelligent control and processing unit (2) and serves as the interface for the system to interact with the outside world.

[0017] 1. Dynamically Variable Surface Subsystem (1) The dynamic variable array subsystem (1) is an innovation of the traditional multi-aperture imaging system. Its fundamental innovation lies in upgrading the fixed, mechanically adjustable imaging module into an intelligent sub-eye unit array integrated on a unified, electrically controllable deformable substrate, with independently adjustable optical parameters. The system includes: Deformable substrate (11): serving as a key actuator for realizing the electronically controlled reconstruction of macroscopic array morphology. The deformable substrate (11) is configured to dynamically, continuously, and rapidly switch between various morphologies such as planar, convex spherical, concave spherical, and even freeform surfaces in response to electronic control signals. The deformable substrate (11) includes: a silicon substrate (111), a MEMS actuator array (112), and a flexible PCB (113).

[0018] In a preferred embodiment, the deformable substrate (11) is a microelectromechanical system (MEMS) actuator array, which drives the substrate to produce local or overall deformation by precisely controlling the independent displacement of each micro actuator.

[0019] As another preferred embodiment, the deformable substrate (11) is a microfluidic pressure chamber array. By injecting or extracting fluid into the sealed independent chamber, the volume change of the chamber is controlled, thereby driving the flexible optical film layer covering it to deform.

[0020] This integrated electronically controlled deformation solution replaces the discrete mechanical structure for "lens relative position and orientation adjustment" in traditional systems, improving the response speed of macroscopic reconstruction from the second level to the millisecond level.

[0021] Sub-eye unit array (12): Fixedly integrated on the deformable substrate (11) in the form of a high-density distributed array. Each sub-eye unit acts as an independent intelligent sensing node, and its optical parameters can be independently and electrically reconstructed, including: Variable focal length lens (121): configured to continuously change its focal length without mechanical movement in response to an electronically controlled signal. This can be achieved, but is not limited to, an electrowetting liquid lens or an Alvarez lens pair. This component implements the traditional "lens focal length adjustment" function and offers advantages such as millisecond-level response, small size, and low power consumption.

[0022] Switchable beam splitter module (122): Located behind the optical path of the variable focal length lens (121), it is configured to dynamically select the optical band it transmits in response to an electronic control signal. For example, a switchable filter based on liquid crystal or electrochromic material is used, so that its working band can be switched between visible light, near infrared, mid-to-long-wave infrared, etc., thereby realizing dynamic multi-band fusion sensing.

[0023] Multi-band image sensor (123): used to receive light transmitted through the switchable beam splitter (122) and convert it into an electrical signal. The multi-band image sensor (123) includes at least a high-resolution visible light image sensor and a high-sensitivity thermal infrared image sensor to ensure the system's detection capability under all-day and complex weather conditions.

[0024] 2. Intelligent control and processing unit (2) The intelligent control and processing unit (2) is electrically connected to the dynamic variable array subsystem (1), forming the central hub of closed-loop control. It not only inherits the function of real-time multi-channel image stitching and correction, but also achieves unified, coordinated, and intelligent control of macroscopic array morphology, microscopic sub-eye parameters, and image processing flow through embedded multiple optimization strategy algorithms. It further includes: Environmental perception module (21): Provides real-time data input for control decisions, including inertial measurement unit (for sensing its own attitude), ambient light sensor (for sensing ambient light intensity), laser ranging module (for obtaining initial target distance), etc.

[0025] Image processing module (22): performs real-time stitching and geometric correction on the multi-channel image data collected by the multi-band image sensor (123), and runs a deep learning target detection and recognition algorithm to automatically identify specific targets such as missiles and drones and their attributes.

[0026] Core processor (23): As the control center of the system, it is configured to perform the following intelligent collaborative control operations: It receives task instructions from the outside and real-time feedback data from the environment perception module (21).

[0027] The system analyzes and makes decisions based on multiple pre-defined optimization strategies and algorithms, generating coordinated control commands. These algorithms are the core difference from the simple control logic in existing technologies, and specifically include: Field of view-resolution trade-off algorithm: The core processor (23) is configured to: in wide-field search mode, control the deformable substrate (11) to form a large curvature convex surface, and set all or most of the variable focal length lenses (121) to a short focal length state to maximize the field of view coverage; once a specific target is identified by the image processing module (22), control the deformable substrate (11) of the target projection area to be locally flattened, and simultaneously set the variable focal length lenses (121) of the sub-eye units in the area to a long focal length state, thereby forming a high-resolution "local telescope array" in the area, realizing a seamless switch from "wide-field search" to "local staring".

[0028] Depth of field extension algorithm: The core processor (23) is configured to actively set different sub-eye units at different focusing distances, making them "dedicated sensors" for different depth of field ranges; then, by fusing the multi-focal plane images collected by these sub-eye units, a fused image that is clear from near to far is generated, thereby extending the overall depth of field of the system.

[0029] Intelligent band fusion strategy: The core processor (23) is configured to dynamically adjust the working mode of the switchable beam splitter (122) based on environmental perception data (such as light intensity and haze index), and adaptively adjust the fusion weights of different band images, such as visible light and infrared, at the data level, feature level, or decision level. For example, in foggy weather, the weight of infrared band data is automatically enhanced to improve detection distance and recognition rate.

[0030] The coordinated control command is sent to the dynamic variable array subsystem (1) to precisely drive the shape of the deformable substrate (11), the focal length of each of the variable focal length lenses (121), and the working state of each of the switchable beam splitting modules (122).

[0031] In conjunction with the image processing module (22), based on the processed image data and real-time system control parameters, the high-precision three-dimensional spatial coordinates, motion vectors, trajectory and future trend of the identified target are calculated.

[0032] 3. Communication and peripheral unit (3) The communication and peripheral unit (3) is responsible for the interaction between the system and the outside world, including a display device (31) for monitoring the system status, an alarm device (32) for issuing alarms, and a data interface (33) for outputting processed target data (such as coordinates and trajectory) to external actuators (such as fire control systems).

[0033] Compared with existing technologies, this invention achieves a significant performance improvement and a qualitative breakthrough in functionality by constructing two pillars: "dynamic variable array" and "intelligent cooperative control." See the appendix for a detailed comparison. Figure 4 . Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall system architecture of this invention, namely "perception-decision-reconstruction".

[0036] in: 1-Dynamic variable array subsystem, 11-Deformable substrate, 111-Silicon substrate, 112-MEMS actuator array, 113-Flexible PCB, 12-Sub-eye unit array, 121-Variable focal length lens, 122-Switchable beam splitter module, 123-Multi-band image sensor; Intelligent control and processing unit, 21-Environmental perception module, 22-Image processing module, 23-Core processor; Communication and peripheral unit, 31-display device, 32-alarm device, 33-external actuator interface.

[0037] Figure 4 It is a comparison with existing technologies.

[0038] Figure 2 This is a schematic diagram of the main control flow.

[0039] Figure 3 This is a detailed configuration process diagram. Detailed Implementation

[0041] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that these embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0042] 1. System Physical Implementation and Key Component Details 1.1 Specific implementation plan of deformable substrate (11) In a preferred embodiment of the present invention, the deformable substrate (11) adopts an array-type scheme based on MEMS micro actuators, and its specific structure, connection and control method are as follows: Structural Composition: The deformable substrate (11) includes a silicon pedestal (111) on which a matrix (e.g., 16x16) array of piezoelectric MEMS actuators (112) is integrated using microfabrication technology. The size of each actuator unit is determined according to the actual physical layout (e.g., 1mm x 1mm), and it can generate a Z-axis displacement of 0-100μm under a driving voltage of 0-60V. A flexible PCB board (113) is fixedly bonded to the upper surface of the MEMS actuator array (112) using anisotropic conductive adhesive.

[0043] Connection relationship: The circuit board of the sub-eye unit (12) is fixed to the flexible PCB board (113) by solder ball welding or precision socket interface. The core processor (23) applies independent analog voltage signals to each MEMS actuator (112) through a set of high-precision digital-to-analog converters and high-voltage drive chips, via traces on the flexible PCB board (113).

[0044] Control and Calibration: Before leaving the factory, the system uses a laser interferometer to calibrate the voltage-displacement characteristics of each actuator and stores the calibration data table in the non-volatile memory of the core processor (23). After the algorithm calculates the target surface shape, the core processor (23) queries the calibration data table to convert the target displacement into a specific driving voltage value, thereby achieving accurate and rapid reconstruction of the substrate shape.

[0045] As an alternative embodiment, the deformable substrate (11) can also employ a microfluidic pressure chamber scheme. It consists of two layers of polydimethylsiloxane (PDMS) films bonded together, forming a network of 10x10 independent sealed microchambers. Each chamber is connected to a micro-solenoid valve, and a refractive index-matching fluid (such as fluorinated oil) is supplied by a shared micro-pump and reservoir. The core processor (23) controls the opening and closing duration of specific solenoid valves to inject or extract liquid into the corresponding chamber, thereby driving the upper flexible optical film to deform. This scheme offers a wider deformation range, but its response speed is slightly slower than the MEMS scheme.

[0046] 1.2 Selection and Integration of Sub-eye Unit (12) Variable focal length lens (121): Preferably an electrowetting liquid lens (such as the Corning Varioptic® A-39N0 series), with an aperture of 3.5mm and a focal length that can be continuously adjusted within a range of 5mm to 50mm via a 0-60V RMS voltage signal, with a response time of <10ms. It is mounted in the sub-lens via a standard M12x0.5 threaded interface.

[0047] Switchable beam splitter module (122): preferably a liquid crystal tunable filter. When a voltage of 0-5V is applied, its transmission band can be switched between 450-650nm (visible light mode) and 700-1000nm (near-infrared mode), with a switching time of <20ms. The filter is embedded in the optical path between the lens and the sensor in the form of an optical window.

[0048] Multi-band image sensor (123): Employs two independent sensors, including a 20-megapixel Sony IMX542 visible light sensor and a 640x512 resolution vanadium oxide uncooled infrared sensor, achieving a common optical path through precise structural alignment. Image data is transmitted via a MIPI CSI-2 interface and multi-channel synchronous acquisition is implemented by an FPGA, with a synchronization error of <1ms.

[0049] 2 System Calibration and Verification Methods To ensure the accuracy of intelligent control, the system needs to undergo the following calibrations: 2.1. Sub-eye Intrinsic and Extrinsic Parameter Calibration: Using Zhang Zhengyou's calibration method, multiple images of the standard chessboard calibration board were acquired at various focal lengths for each sub-eye, and their intrinsic parameter matrices (focal length, principal point) and distortion coefficients were calculated respectively. With the base in a planar state, the extrinsic parameters (rotation matrix and translation vector) of all sub-eyes relative to a unified world coordinate system were calculated by photographing the calibration object with known 3D coordinates.

[0050] 2.2. Joint calibration of “substrate morphology-field mapping”: In a large darkroom, a grid point light source target with known coordinates is set up. The deformable substrate (11) is controlled to transform into a series of predefined morphologies in sequence. Under each morphology, all sub-eyes simultaneously acquire target images, and through image recognition and bundle adjustment, the actual three-dimensional spatial coordinates and optical axis direction of the optical center of each sub-eye under the current substrate morphology are calculated in reverse. Finally, for each predefined substrate morphology, a “morphology parameter-sub-eye extrinsic parameter” lookup table (LUT) is generated and stored in the core processor (23).

[0051] 2.3. Control Instruction Generation and Mapping: After the algorithm decision, the core processor (23) performs the following steps to generate hardware control instructions: a) Coordinate mapping: Using the "base morphology-field mapping" lookup table, the image pixel coordinates are mapped to the physical sub-eye index and the required theoretical displacement.

[0052] b) Instruction generation: The theoretical displacement is compared with the MEMS actuator calibration data table to calculate the actual drive voltage value, which constitutes the base control instruction set U_base.

[0053] c) Synchronization control: The core processor (23) converts the target focal length value into a specific electrical drive signal (such as voltage) by querying the "focal length-drive signal" calibration curve of the variable focal length lens, and generates the sub-eye control instruction set U_lens_focal[].

[0054] 3. Core Algorithm and Workflow 3.1 Field of View-Resolution Trade-off Algorithm The field-of-view-resolution trade-off algorithm is one of the core optimization strategies operated by the core processor (23) in the intelligent control and processing unit (2) of this invention. Its fundamental purpose is to dynamically and intelligently break the inherent trade-off between field of view and resolution in traditional optical systems.

[0055] 3.1.1 Algorithm Input and Output Algorithm input: Task instructions: Abstract commands from operators or upper-level decision-making systems, such as MODE_SEARCH (wide-area search) and MODE_TRACK (target tracking).

[0056] Target detection report: Deep learning target detection results from the image processing module (22), including at least the target identifier, bounding box coordinates, detection confidence and target category.

[0057] System status: including the current curvature / morphological parameters of the deformable substrate (11) and the current focal length array of each sub-eye unit (12).

[0058] Algorithm output: Base control instruction set U_base: used to set the target shape of the deformable base (11).

[0059] Sub-eye control instruction set: includes the target focal length value array U_lens_focal[] and the target spectral mode array U_spectral_mode[] for each sub-eye unit.

[0060] 3.1.2 Algorithm Workflow and Decision Logic This algorithm is a continuously running state machine, and its main control flow is as follows: Figure 2 As shown, the detailed configuration process is as follows: Figure 3 As shown.

[0061] State 1: Wide-area search mode Triggering conditions: System startup or receipt of the MODE_SEARCH command.

[0062] Decision-making logic: The primary goal is to cover the largest possible field of view.

[0063] Controlling actions: Substrate control: Instruct the deformable substrate (11) to form a convex spherical surface with large curvature (e.g., radius of curvature R_search = 500 mm).

[0064] Sub-eye control: Set the focal length of all sub-eye units (12) to a short focal length f_short (e.g., 5mm).

[0065] State 2: Partial gaze pattern Triggering condition: In wide-area search mode, the "visual salience" S of the detected target exceeds the high threshold Th_high.

[0066] Decision-making logic: Concentrate resources on key objectives to achieve the highest resolution.

[0067] Controlling actions: Target region mapping and sub-eye grouping: Calculate the projected region of the target on the physical array and determine the set of sub-eyes G covering the region.

[0068] Focal length reconstruction: Switch the focal length of the sub-eyes in set G to a long focal length f_long (e.g., 50mm).

[0069] Substrate morphology reconstruction: drive local flattening of the substrate in the region where the G set is located to eliminate perspective distortion and optimize the optical path.

[0070] State 3: Collaborative Tracking Mode Triggering condition: The target has been locked, but its "visual salience" S is at a moderate level.

[0071] Decision-making logic: to make a smooth transition or to robustly track uncertain objectives.

[0072] Controlling actions: Motion prediction: Predict the location P of the target at the next moment.

[0073] Resource pre-allocation: The "local gaze configuration" step is performed with the predicted location P as the center, but with more moderate control parameters.

[0074] Key technical details: Visual saliency calculation: The overall score of visual saliency S is calculated using the following formula: S = α*confidence + β*threat_class + γ*(bbox_size / image_size) + δ*proximity_to_center. This saliency score S is used to guide the core processor (23) in generating cooperative control instructions for the deformable substrate (11) and the sub-eye unit (12).

[0075] in: confidence is the target detection confidence output by the image processing module (22).

[0076] `threat_class` represents the threat level of the target. In a preferred embodiment, this parameter is automatically mapped by the system based on the identified target category. A "target category - threat level" mapping table (e.g., missile -> 1.0, drone -> 0.8, bird -> 0.1) is pre-stored in the core processor (23), and is automatically queried and assigned a value after the target is identified.

[0077] bbox_size / image_size is the ratio of the area of ​​the target bounding box to the area of ​​the entire image, reflecting the apparent size of the target.

[0078] proximity_to_center represents the proximity of the target location relative to the center of the image.

[0079] α, β, γ, δ are weighting coefficients pre-calibrated through experiments, used to balance the influence of each factor.

[0080] Coordinate transformation model: A dynamic nonlinear mapping from image pixel coordinates (u, v) to the physical sub-eye index (i, j) and the theoretical basis displacement Δz is achieved through a pre-calibrated "basis morphology-field-of-view mapping lookup table." As a preferred embodiment, this mapping process is implemented using bicubic spline interpolation. Specifically: The lookup table is viewed as discrete sampling points, which stores the ideal imaging center pixel coordinates (U_ij, V_ij) corresponding to each sub-eye index (i,j) for a specific base morphology, as well as the displacement Z_ij required to achieve the current morphology.

[0081] When the algorithm decision requires reconstructing a target region, the core processor (23) first determines the pixel coordinates (u_target, v_target) of the target on the image.

[0082] Then, the nearest neighbor 4x4 grid points enclosing (u_target, v_target) are found in the lookup table.

[0083] Using the bicubic spline interpolation formula, the weights of the physical sub-eye indices covered by the target point are calculated, and the theoretical displacement Δz required to align the optical centers of these sub-eyes with the target is calculated in reverse. This interpolation method can ensure the continuity and smoothness of the displacement changes, avoiding abrupt changes during array reconstruction.

[0084] Control loop: The entire algorithm operates in a high-frequency "perception-decision-execution-feedback" closed loop.

[0085] 3.2 Depth-of-Field Extension Algorithm The depth-of-field extension algorithm is used to solve the problem of simultaneous clear imaging of distant and near targets in a scene. Its core idea is to actively set different sub-eyes at different focusing distances, making them "dedicated sensors" for different depth-of-field ranges, and then obtain a "fully clear" image through image fusion.

[0086] 3.2.1 Workflow Phase 1: Focal Plane Planning and Sub-Eye Resource Allocation With prior information on scene depth: Analyze the depth map, determine the positions of N focal planes (e.g., 500m, 1700m, 5800m, 20000m), and evenly distribute sub-eyes to these focal planes.

[0087] No scene depth prior information ("blind" extended depth of field): triggered by initial detection, the sub-eye subset is controlled to perform focus stack scanning, the target distance is located according to the image sharpness, and then sub-eye resources are allocated.

[0088] Phase Two: Multi-Focal Plane Image Acquisition and Preprocessing Each sub-eye acquires images in parallel according to its assigned focusing distance.

[0089] The image processing module (22) independently completes the stitching and correction of the sub-eye image of each focal plane, generating N panoramic images I_1, I_2, ..., I_N that are clear at different distances.

[0090] Phase 3: Intelligent Fusion of Multi-Focal Plane Images Generating a sharpness map: For each input multi-focal panoramic image I_k, calculate the local sharpness value at each pixel location (x, y) to generate a sharpness map S_k(x, y). As a preferred embodiment, variance based on the Laplacian operator is used as the sharpness evaluation function. The specific steps are as follows: Convert the image I_k to grayscale (if it is a color image).

[0091] A grayscale image is convolved using a discrete two-dimensional Laplacian operator (with a kernel such as [[0,1,0],[1,-4,1],[0,1,0]]) to obtain a Laplacian response map L(x, y). The Laplacian operator produces higher response values ​​at image edges and details.

[0092] Within a local window W centered at pixel (x, y) (e.g., 5x5 or 7x7 pixels in size), the variance σ²(x, y) of the Laplacian response is calculated.

[0093] The variance value σ²(x, y) is assigned to the sharpness map S_k(x, y). The higher the variance, the more dramatic the image gradient change in that local area, and the sharper the image.

[0094] Multi-scale fusion: Using multi-scale fusion techniques such as Laplacian pyramid, the weighted sub-bands of each scale are reconstructed to obtain the final fully clear fused image I_fused.

[0095] 3.3 Intelligent Band Fusion Strategy The intelligent band fusion strategy aims to address the inherent limitations of single-spectral-band imaging in complex environments. Its core idea is to dynamically adjust the acquisition weights and fusion methods of each band's data by analyzing the environmental context and target characteristics in real time.

[0096] 3.3.1 Multi-layered converged architecture This strategy is a multi-level fusion architecture that follows a fusion paradigm of "data level → feature level → decision level".

[0097] First layer: Data-level fusion Environmental context analysis: Based on environmental perception data (lighting, weather) and task instructions, query the pre-set environment-band performance lookup table to determine the basic fusion mode (such as FUSION_IR_ENHANCED).

[0098] Dynamic spectral control: Based on the basic mode, the instruction U_spectral is generated to control the working mode of the switchable spectral module (122).

[0099] Pixel-level adaptive fusion: An adaptive weighted fusion method based on multi-scale decomposition (such as Laplacian pyramid) is adopted to dynamically calculate the fusion weights at different scales based on local gradients or contrast.

[0100] Second layer: Feature-level fusion Dual-band feature extraction: A two-stream neural network is used to extract feature maps F_vis and F_ir from the visible light image I_vis and the thermal infrared image I_ir, respectively.

[0101] Adaptive Feature Fusion: Introduces an attention-guided feature fusion module.

[0102] Channel attention: Dynamically generate fusion weights [α_vis, α_ir] for visible light and infrared streams based on the environment context encoding e_env.

[0103] Spatial attention: Based on the feature response intensities of the two streams, a spatial weight map M_attention is generated to achieve adaptive fusion in space. The fused features F_fused_final are then fed into subsequent network layers.

[0104] Third level: Decision-level integration Independent detection and confidence calibration: The pure visible light model and the pure infrared model are detected independently, and their detection confidence is calibrated according to the environmental context.

[0105] Decision Arbitration: Arbitrate the two sets of test results, adopt the result with high confidence consistency or the more reliable result under the current environment, and output the final target report.

[0106] 3.3.2 Algorithm Collaboration The intelligent band fusion strategy works in deep collaboration with the field-of-view-resolution tradeoff and depth-of-field extension algorithms. For example, in wide-area searches in hazy weather, the band fusion strategy first sets the system to infrared enhancement mode; the depth-of-field extension algorithm then intervenes to focus; when the target is locked, the field-of-view-resolution tradeoff algorithm instructs the local sub-eye to switch to telephoto for staring; at the same time, the band fusion strategy dynamically adjusts the fusion parameters in the local staring mode to assist in fine recognition.

[0107] Example

[0108] Example 1: Air Defense Early Warning System Based on MEMS Actuators and Liquid Lenses refer to Figure 1 , Figure 2 and Figure 3 This embodiment provides a high-performance optical detection system for air defense of key areas.

[0109] The dynamic variable array subsystem (1) employs the aforementioned 16x16 MEMS deformable substrate (11), on which 16 sub-eye elements (12) are integrated in a 4x4 matrix. The sub-eye elements employ the preferred components described in Section 1.2.

[0110] Intelligent control and processing unit (2): The core processor (23) is a combination of high-performance FPGA and AI chip, and is pre-installed with the three core algorithms described in Section 6.

[0111] Workflow (intercepting cruise missiles): 1. Wide-area search: The system is initialized with a convex spherical base (R=500mm) and all sub-eyes are short focal length (5mm) to perform 120° field of view panoramic monitoring.

[0112] 2. Target detection: The image processing module (22) identifies high-speed suspected missile points at the edge of the field of view.

[0113] 3. Cooperative Reconfiguration: The core processor (23) initiates the field-of-view-resolution tradeoff algorithm. Within milliseconds: - The actuators on the corresponding region of the MEMS substrate (11) are displaced to locally "flatten" the array surface.

[0114] - The synchronous command instructs the liquid lenses of the four sub-eyes in this area to switch the focal length to 50mm (telephoto).

[0115] - Instructs its switchable beam splitter module (122) to enhance near-infrared band reception.

[0116] 4. Precise positioning: A "local high-resolution staring telescope" is formed to confirm that the target is a cruise missile, calculate its three-dimensional coordinates, velocity and trajectory, and transmit them to the interception system through the data interface (33).

[0117] Example 2: Border Monitoring System Based on Microfluidic Substrates and Multi-Algorithm Fusion Based on Example 1, this example is optimized for mountainous and foggy environments.

[0118] Dynamic variable array subsystem (1): The deformable substrate (11) adopts a microfluidic pneumatic chamber array to support greater curvature deformation. The sub-eye's beam splitting module (122) supports three bands: visible light, short-wave infrared, and long-wave infrared.

[0119] Intelligent control and processing unit (2): Enhanced depth-of-field extension algorithm and intelligent band fusion strategy.

[0120] Workflow: 1. Routine monitoring: Run the depth-of-field extension algorithm, control the sub-eyes to focus at 500m, 2km and 5km respectively, and fuse them to obtain a fully clear panoramic image.

[0121] 2. Adaptation to severe weather: When smog is detected, the intelligent fusion strategy is triggered: - Instructs all sub-eyes to prioritize switching their beam splitting modules to the long-wave infrared band.

[0122] - In image processing, dynamically increase the fusion weights of infrared image features.

[0123] This strategy ensures that the system can maintain sufficient detection range and recognition rate even in adverse weather conditions.

[0124] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A biomimetic compound eye optical detection system with a dynamically variable array, characterized in that, include: The dynamic variable array subsystem (1) includes: Deformable substrate (11), wherein the deformable substrate (11) is a microelectromechanical system (MEMS) actuator array or a microfluidic pressure chamber array, is configured to dynamically switch between at least two forms of a plane, a convex spherical surface, and a concave spherical surface in response to an electronic control signal; Sub-eye unit array (12) is fixedly integrated on the deformable substrate (11) in an array form; wherein each sub-eye unit includes: The variable focal length lens (121) is configured to continuously change its focal length without mechanical movement in response to an electronically controlled signal; A switchable beam splitter (122) is disposed behind the optical path of the variable focal length lens (121) and is configured to dynamically select the optical band it transmits in response to an electronic control signal. A multi-band image sensor (123) is used to receive light transmitted through the switchable beam splitter (122) and convert it into an electrical signal; The intelligent control and processing unit (2) is electrically connected to the dynamically variable array subsystem (1); characterized in that the intelligent control and processing unit (2) is configured as follows: Based on task instructions and / or environmental perception data, collaborative control instructions are generated for the coordinated control of the shape of the deformable substrate (11), the focal length of each of the variable focal length lenses (121), and the operating state of each of the switchable beam splitting modules (122); signals from the multi-band image sensor (123) are received and processed to output at least one of a wide-area stitched image, target recognition information, and target three-dimensional spatial coordinates. It is also configured to execute at least one of a field-of-view-resolution tradeoff algorithm, a depth-of-field extension algorithm, and an intelligent band fusion strategy.

2. The system according to claim 1, characterized in that, The deformable substrate (11) is a microelectromechanical system (MEMS) actuator array.

3. The system according to claim 2, characterized in that, The deformable substrate (11) includes a silicon substrate (111), a MEMS actuator array (112) disposed on the silicon substrate (111), and a flexible PCB (113) covering the MEMS actuator array (112); the sub-eye unit array (12) is fixed on the flexible PCB (113).

4. The system according to claim 1, characterized in that, The deformable substrate (11) is a microfluidic pressure chamber array.

5. The system according to claim 1, characterized in that, The variable focal length lens (121) is an electrowetting liquid lens or an Alvarez lens pair.

6. The system according to claim 1, characterized in that, The switchable beam splitter (122) is a switchable filter based on liquid crystal or electrochromic materials.

7. The system according to claim 1, characterized in that, The multi-band image sensor (123) includes at least one visible light image sensor and one thermal infrared image sensor.

8. The system according to claim 1, characterized in that, The intelligent control and processing unit (2) includes an environmental perception module (21), an image processing module (22), and a core processor (23). The environmental sensing module (21) is used to acquire environmental sensing data; The image processing module (22) is used to process and identify targets in the image data acquired by the multi-band image sensor (123); The core processor (23) is used to generate the collaborative control instructions based on the task instructions, the environmental perception data and / or the target recognition results of the image processing module (22).

9. The system according to claim 1 or 8, characterized in that, The intelligent control and processing unit (2) is configured to generate the cooperative control instructions by one or any combination of the following methods: Perform field-of-view-resolution trade-off control: In wide-area search mode, control the deformable substrate (11) to form a convex spherical surface and set the variable focal length lens (121) to a short focal length state; and after identifying a specific target, control the deformable substrate (11) of the target projection area to be locally flattened, and simultaneously set the variable focal length lens (121) of the sub-eye unit covering the target area to a long focal length state; Perform depth-of-field extension control: actively set the variable focal length lenses (121) of different sub-eye units at different focal distances, and control the image processing module (22) to fuse the images acquired by these sub-eye units at different focal planes to generate a fused image with extended depth of field; Perform intelligent band fusion control: Based on environmental perception data, dynamically adjust the working band of the switchable beam splitter (122) and adaptively adjust the fusion weights of images acquired by image sensors of different bands at the data level, feature level, or decision level.

Images