Gaze fast tracking technology based on field of view modulation

Abstract

The application discloses a gaze fast tracking technology based on field of view modulation and belongs to the technical field of photoelectric detection and target tracking. The application comprises the following steps: separating composite light, collecting images of corresponding light through a sensor device, and performing quality evaluation on the images to obtain image quality indexes; a central processing module dynamically adjusts the weight of each waveband in a tracking loop based on the image quality indexes; a weighted sum of target miss distances of each waveband detected by the sensor device is obtained according to the weight; a deflection instruction is generated according to the comprehensive miss distance, and the deflection instruction is sent to a control driving module; and the angle is adjusted through the control driving module to drive a light beam deflection execution device. The application physically eliminates parallax through a common-aperture optical path, improves tracking accuracy, realizes environmental self-adaptation through dynamic weight control, ensures stable tracking in the whole period, greatly improves the system response speed through a low-inertia light beam deflection device, and realizes fast gaze locking on a high-speed maneuvering target.

Description

Technical Field

[0001] This invention relates to the field of photoelectric detection and target tracking technology, specifically to a gaze-based rapid tracking system and method based on field-of-view modulation. Background Technology

[0002] In the fields of air defense, security monitoring, aerial detection, and autonomous driving perception, in order to achieve stable detection of targets in all weather conditions (day, night, fog, rain, and snow), it is usually necessary to integrate information from multiple bands such as visible light, mid-wave infrared, and long-wave infrared.

[0003] Currently, mainstream multi-band photoelectric tracking systems (such as airborne photoelectric pods, ground-based photoelectric turntables, or vehicle-mounted multi-camera modules) typically employ a "multi-aperture parallel + mechanical gimbal" architecture. This system structure mainly includes: multiple independent imaging components, such as visible light cameras and infrared thermal imagers, physically mounted side-by-side on a two- or three-axis mechanical frame. Each component has an independent optical window (aperture). During operation, a mechanical motor drives the entire platform to rotate, causing the optical axes of all sensors to roughly point towards the same target. The acquired multiple images need to be corrected, scaled, and translated (registration algorithm) using image processing software to superimpose images from different perspectives before target recognition.

[0004] The existing technology has the following main problems: 1. Multi-sensor parallax problem: Due to the parallel arrangement of multiple apertures, the optical centers of each sensor do not coincide, resulting in physical parallax during imaging. The system must rely on complex software algorithms for image registration, which not only increases computational latency and makes it difficult to meet the real-time requirements of high-speed tracking, but also requires continuous adjustment of registration parameters when the target distance changes, which can easily lead to ghosting in the fused image and cause recognition errors.

[0005] 2. Limited system response speed: Due to the reliance on a large mechanical gimbal to rotate the entire load, the rotational inertia is large, and the angular acceleration of the system is limited, making it impossible to achieve rapid line-of-sight response and precise "staring" for targets with high-frequency vibration or rapid maneuvering.

[0006] 3. Insufficient environmental adaptability: Since the bands are usually simply superimposed or fused with fixed weights, they lack environmental adaptability. When the target suddenly enters the smoke or cloud layer (visible light failure) from the clear airspace, the system often cannot switch the tracking logic in time, resulting in the loss of the target.

[0007] To address the aforementioned issues, there is an urgent need for a gaze-tracking system and method based on field-of-view modulation to solve the problems associated with traditional methods. Summary of the Invention

[0008] The purpose of this invention is to provide a staring fast tracking technology based on field-of-view modulation, which improves tracking accuracy by physically eliminating parallax through a common aperture optical path; it achieves environmental adaptation through dynamic weight control, ensuring stable tracking at all times; and it significantly improves the system response speed by utilizing a low-inertia beam deflection device, thus realizing rapid staring lock-on of high-speed maneuvering targets.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A gaze-based rapid tracking system based on field-of-view modulation includes: a beam deflection actuator, a cascaded beam splitter and independent imaging device, sensor components, a central processing module, and a control and drive module. The beam deflection actuator is configured as the system's front-end common aperture inlet, used to receive composite light rays from the target scene and guide them to the cascaded beam splitter and independent imaging device by angle adjustment. The cascaded beam splitter and independent imaging device is equipped with a spectral splitting structure, used to separate the composite light rays into multiple independent spectral bands in physical space according to spectral wavelength characteristics, and transmit them to different optical path channels respectively. The sensor components include multiple sensing systems respectively disposed on the different optical path channels, each sensing system being equipped with an image sensor adapted to the spectral band of that channel, used to synchronously acquire image data of each spectral band. The central processing module is used to acquire image data and perform quality assessment, dynamically configure the weight parameters of each spectral band in the tracking loop based on the assessment results, and generate a comprehensive miss distance by combining the target miss distance calculated from each spectral band. The control and drive module is used to generate drive commands based on the comprehensive miss distance and drive the beam deflection actuator to perform closed-loop beam pointing correction.

[0010] Furthermore, the cascaded beam splitter and independent imaging device is configured with a cascaded beam splitter structure for separating composite light rays into multiple spectral bands according to their spectral characteristics, and for directionally transmitting the light signals of each spectral band to the corresponding sensing and detection channel. The beam splitter structure employs a dichroic mirror. The sensor device includes a sensing system that corresponds one-to-one with each spectral band.

[0011] Furthermore, the beam deflection actuator is a MEMS micromirror array, a mechanical galvanometer, a fast reflector, or a dual-beam wedge prism group.

[0012] This invention also provides a gaze-tracking fast method based on field-of-view modulation, applied to the aforementioned gaze-tracking fast system based on field-of-view modulation, comprising: Step 1: Deploy a gaze tracking system; Step 2: Receive composite light rays from the target scene through the beam deflection actuator, and guide the composite light rays to the cascaded beam splitter and independent imaging device; Step 3: The composite light is separated into multiple spectral bands by the cascaded beam splitter and independent imaging device, and the light signals of each spectral band are transmitted to the corresponding sensing and detection channels respectively. Step 4: The central processing module acquires the image data output from each sensor detection channel, performs quality assessment on the images of each spectral band, and generates corresponding image quality indicators. Step 5: The central processing module dynamically configures the weight parameters of each spectral band in the tracking loop based on the image quality index of each spectral band. Step 6: The central processing module performs a weighted fusion calculation based on the weight parameters and the target miss distances corresponding to each spectral band to generate a comprehensive miss distance. Step 7: Generate a beam deflection control command based on the comprehensive miss distance, and drive the beam deflection actuator to perform beam pointing correction by the control drive module.

[0013] Furthermore, the image quality metrics include, but are not limited to, any one or a combination of contrast, edge strength, signal-to-noise ratio, and target feature saliency.

[0014] Furthermore, the dynamic configuration of weight parameters in step 5 includes: increasing the weight parameter of a specific spectral band when the image quality index of that spectral band meets the preset advantage condition; decreasing the weight parameter of that spectral band and increasing the weight parameter of the other spectral bands when the image quality index of a specific spectral band deteriorates and the target features in the images of other spectral bands meet the sharpness criteria; wherein, the image quality deterioration criterion is that the current frame index value is lower than the product of the historical frame index average and the preset coefficient, and the target feature sharpness criteria include any one or a combination of hotspot saliency, contour energy, and edge response intensity.

[0015] Furthermore, the weighted fusion operation in step 6 includes: each sensing system calculating the off-target deviation between the target center and the field of view center within its own spectral band using an image recognition algorithm; the central processing module performing a weighted summation of the off-target deviations of each spectral band based on real-time weight parameters, and outputting the comprehensive off-target amount.

[0016] Furthermore, the generation of the beam deflection control command in step 7 includes: using the comprehensive miss distance as a feedback error input to the control algorithm module to generate an angle correction amount; the control drive module converts the angle correction amount into a drive signal and applies it to the beam deflection actuator according to the drive characteristic curve of the beam deflection actuator to achieve closed-loop correction of the beam direction.

[0017] In summary, the present invention has at least one of the following beneficial technical effects: 1. Eliminate parallax and improve tracking accuracy: Due to the use of a front-end common aperture beam deflection design, all wavelengths of light are reflected by the same deflection device, resulting in naturally parallel and physically aligned line-of-sight. Therefore, the system completely eliminates the complex image registration computational overhead of traditional multi-sensor systems, enabling more real-time response to changes in the position of high-speed moving targets without ghosting or fusion.

[0018] 2. Environmental Adaptability Enhances System Robustness: Because the system possesses multi-band sensing capabilities and employs a dynamic weight control strategy, it can dynamically adjust the weight of each band in tracking based on real-time image quality assessment results. When a sudden change in the environment (such as an aircraft entering clouds or a vehicle entering dense fog) causes a certain band to fail, the system can automatically switch to other effective bands to dominate tracking, thereby ensuring continuous staring capability in all weather conditions and all scenarios.

[0019] 3. Low inertia drive, improving system response speed: By employing miniaturized MEMS micromirrors or lightweight galvanometers and other ultra-low inertia beam deflection devices to replace traditional kilogram-class mechanical gimbals for fine line-of-sight adjustments, the system inertia is significantly reduced and the bandwidth is significantly improved. Therefore, it can achieve a more robust "staring" of targets with high-frequency jitter or instantaneous large maneuvers, effectively avoiding the overshoot and hysteresis phenomena common in mechanical gimbals. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 A schematic diagram of the staring fast tracking system structure; Figure 3 This is a schematic diagram of a specific embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0022] like Figure 2As shown, this invention provides a gaze-based rapid tracking system based on field-of-view modulation, comprising: a beam deflection actuator, a cascaded beam splitter and independent imaging device, sensor components, a central processing module, and a control and drive module; the beam deflection actuator is configured as the common aperture inlet at the front end of the system, used to receive composite light rays in the target scene, and guide the composite light rays to the cascaded beam splitter and independent imaging device by angle adjustment; the cascaded beam splitter and independent imaging device is configured with a spectral splitting structure, used to separate the composite light rays into multiple independent spectral bands of optical signals in physical space according to the spectral wavelength characteristics, and transmit them to different optical path channels respectively; the sensor components include multiple sensing systems respectively disposed on the different optical path channels, each sensing system being configured with an image sensor adapted to the spectral band of the channel, used to synchronously acquire image data of each spectral band; the central processing module is used to acquire image data and perform quality assessment, dynamically configure the weight parameters of each spectral band in the tracking loop according to the assessment results, and generate a comprehensive miss distance by combining the target miss distance calculated by each spectral band; The control and drive module is used to generate drive commands based on the comprehensive miss distance and drive the beam deflection actuator to perform closed-loop beam pointing correction.

[0023] The cascaded beam splitter and independent imaging device is configured with a cascaded beam splitter structure to separate composite light rays into multiple spectral bands according to their spectral characteristics, and to directionally transmit the light signals of each spectral band to the corresponding sensing and detection channel. The beam splitter structure uses a dichroic mirror. The sensor device includes a sensing system that corresponds one-to-one with each spectral band.

[0024] The beam deflection actuator is a MEMS micromirror array, a mechanical galvanometer, a fast reflector, or a dual-beam wedge prism group.

[0025] The present invention provides an embodiment as follows: 1. The cascaded beam splitter and independent imaging device includes a first dichroic mirror, a second dichroic mirror, and a plane mirror. The first dichroic mirror is used to reflect visible light and transmit light outside the visible light band. The second dichroic mirror is used to reflect long-wave infrared light and transmit mid-wave infrared light. The plane mirror is used to reflect mid-wave infrared light. The sensor device is provided with a first sensing system, a second sensing system, and a third sensing system corresponding to the first dichroic mirror, the second dichroic mirror, and the plane mirror, respectively. The first sensing system is a visible light CMOS sensor, used to generate visible light images based on visible light; the second sensing system is a long-wave infrared thermal imaging sensor, used to generate thermal imaging images based on long-wave infrared light; and the third sensing system is a mid-wave infrared FPA sensor, used to generate mid-wave images based on mid-wave infrared light.

[0026] like Figure 1As shown, the present invention also provides a gaze-tracking fast method based on field-of-view modulation, applied to the aforementioned gaze-tracking fast system based on field-of-view modulation, comprising: Step 1: Deploy a gaze tracking system; Step 2: Receive composite light rays from the target scene through the beam deflection actuator, and guide the composite light rays to the cascaded beam splitter and independent imaging device; Step 3: The composite light is separated into multiple spectral bands by the cascaded beam splitter and independent imaging device, and the light signals of each spectral band are transmitted to the corresponding sensing and detection channels respectively. Step 4: The central processing module acquires the image data output from each sensor detection channel, performs quality assessment on the images of each spectral band, and generates corresponding image quality indicators. Step 5: The central processing module dynamically configures the weight parameters of each spectral band in the tracking loop based on the image quality index of each spectral band. Step 6: The central processing module performs a weighted fusion calculation based on the weight parameters and the target miss distances corresponding to each spectral band to generate a comprehensive miss distance. Step 7: Generate a beam deflection control command based on the comprehensive miss distance, and drive the beam deflection actuator to perform beam pointing correction by the control drive module.

[0027] The image quality metrics include, but are not limited to, any one or a combination of contrast, edge strength, signal-to-noise ratio, and target feature saliency.

[0028] Step 5 involves dynamically configuring weight parameters as follows: when the image quality index of a specific spectral band meets the preset advantage conditions, the weight parameter of that spectral band is increased; when the image quality index of a specific spectral band deteriorates and the target features in the images of other spectral bands meet the sharpness criteria, the weight parameter of that spectral band is decreased and the weight parameters of the other spectral bands are increased; wherein, the image quality deterioration criterion is that the current frame index value is lower than the product of the historical frame index average and the preset coefficient, and the target feature sharpness criteria include any one or a combination of hotspot saliency, contour energy, and edge response intensity.

[0029] Step 6, the weighted fusion operation, includes: each sensing system calculating the off-target deviation between the target center and the field of view center within its own spectral band using an image recognition algorithm; the central processing module performing a weighted summation of the off-target deviations of each spectral band based on real-time weight parameters, and outputting the comprehensive off-target amount.

[0030] Step 7, generating the beam deflection control command, includes: using the comprehensive miss distance as a feedback error input to the control algorithm module to generate an angle correction amount; the control drive module, based on the drive characteristic curve of the beam deflection actuator, converts the angle correction amount into a drive signal and applies it to the beam deflection actuator to achieve closed-loop correction of the beam direction.

[0031] This invention provides a schematic flowchart of a specific embodiment, as follows: Figure 3 As shown; The following describes the specific usage process of the gaze tracking system, specifically steps 2 and 3: 1. Capture and reflection of composite rays (front-end common aperture stage): (1) Incident: The composite light rays (including visible light, mid-wave infrared and long-wave infrared spectra) from the target scene first reach the front end of the system, namely the beam deflection actuator; (2) Reflection and pointing: At this time, the beam deflection actuator is at a specific deflection angle, which is determined by the control and drive module at the back end. After the composite light comes into contact with the mirror of the beam deflection actuator, it is reflected as a whole towards the subsequent cascaded beam splitter and independent imaging device according to the law of reflection of light. Since all wavelengths of light strike the same mirror, their propagation paths after reflection are strictly locked in spatial geometry, forming a parallel composite beam that enters the downstream.

[0032] 2. Cascaded beam splitting and independent imaging (back-end beam splitting stage): (1) First-stage separation: The composite beam propagates to the first dichroic mirror (DM1); The coating characteristics on DM1 cause visible light (such as 400-700nm) to be reflected, change direction and enter the first mirror group. After being focused, an electrical signal is generated on the photosensitive surface of the first sensing system, which can be used to generate a visible light image. Meanwhile, infrared light (>700nm) passes through DM1 and continues to propagate along the original straight line.

[0033] (2) Second-stage separation: The transmitted infrared beam reaches the second dichroic mirror (DM2).

[0034] The coating characteristics on DM2 cause long-wave infrared light (such as 8-14μm) to be reflected and enter the second mirror group, generating an electrical signal on the second sensing system, i.e., a thermal imaging image.

[0035] The remaining mid-wave infrared (e.g., 1-5μm) passes through DM2, is folded 90 degrees by the plane mirror, and finally enters the third mirror group, generating an electrical signal on the third sensing system, i.e., the mid-wave image.

[0036] 3. Signal convergence: The three digital video streams (visible light image, thermal imaging image, and medium wave image) output from the first, second, and third sensing systems are simultaneously transmitted to the central processing module via a high-speed data bus (such as a MIPI or LVDS interface) in preparation for the next step of algorithm processing.

[0037] 4. Summarize the overall work process: (1) Common aperture inlet and beam deflection actuator: The beam deflection actuator is set as the only optical entry point (common aperture) for the entire system.

[0038] The preferred device is a MEMS micromirror array, which takes advantage of its small size and fast response. Since light of all wavelengths passes through the same mirror and is reflected at the same time, according to the law of reflection, the reflection angles are completely consistent. Therefore, the images acquired by each sensor are strictly parallel in physical space and there is no parallax.

[0039] (2) Spectral beam splitting section (spectral splitting strategy): Located downstream of the optical path of the beam deflection actuator, it consists of cascaded dichroic mirrors and is responsible for separating the composite beam according to wavelength. Its specific structure is as described above. Here, following physical characteristics, the dichroic mirror is usually designed to transmit a specific sensor working center band and reflect the other bands.

[0040] (3) Light path direction: As described above, the specifics will not be repeated here.

[0041] (3) Processing and Control Section: Central Processing Module: This module can be implemented using FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application-Specific Integrated Circuit) or GPU to meet the needs of high-concurrency processing of multiple images.

[0042] Control and drive module: Receives instructions from the central processing module and generates voltage or current signals to drive the beam deflection actuator.

[0043] In step 4, the image quality metrics include contrast, edge strength, and signal-to-noise ratio (SNR), and the calculation process is as follows: 1. Contrast Used to determine whether an image is grayish or completely white (such as being obscured by smog). This invention provides an embodiment, for example, using root mean square contrast (RMS Contrast) for calculation, specifically: (1) In the formula, It is the first The brightness of each pixel It is the average brightness of the entire image. If the calculation result... A very low value (e.g., below 10) indicates that the image's grayscale distribution is highly concentrated, possibly obscured by smoke, and contains little effective information.

[0044] 2. Edge strength Used to determine whether an image is clear and whether the outline of the target can be seen; This invention provides an embodiment that extracts gradients, for example, using the Sobel or Laplacian operators. A convolution operation is performed on the image, and the gradient value (rate of change) of each pixel is calculated. The number of pixels with gradient values ​​greater than a certain threshold across the entire image is counted. A large number indicates rich image texture and clear outlines; a very small number indicates a blurry image or unclear targets.

[0045] 3. Signal-to-noise ratio Used to determine whether a target signal is obscured by noise at night or in low light conditions, the specific calculation formula is as follows: (2) In the formula, The average brightness of the target area. The SNR value represents the standard deviation of the brightness of the background area. The higher the SNR value, the more prominent the target is and the more reliable the tracking is.

[0046] In step 5, the central processing module dynamically adjusts the weights of each band in the tracking loop based on image quality metrics, specifically as follows: Scene A (Sunny Day / Clear Texture): If the contrast of the visible light image is greater than the preset threshold, the weight of the visible light channel is increased, for example, the weight of the visible light channel is set to 0.8, and the weight of the infrared channel is decreased, for example, it is set to 0.2. The high resolution of visible light is used to identify target details, such as aircraft models and license plates. Scenario B: If the contrast of the visible light image drops sharply and the hot spots or outlines are clear, then reduce the weight of the visible light channel to 0 and increase the weight of the infrared channel to 1. The section on the sudden drop in contrast in visible light images explains that the system maintains a time-sliding window, recording the most recent... Average visible light contrast of frames (e.g., 10 frames) ; Drop criterion: Contrast of the current frame A sudden drop is defined as one that meets the following conditions: (3) in, This is the drop factor (for example, 0.4 means that the contrast instantly drops to less than 40% of its original value). This usually corresponds to the moment when an aircraft suddenly enters the clouds or when a ground vehicle is covered by smoke grenades.

[0047] Next, the hotspots or those with clear outlines will be explained as follows: 1. Clear hotspots (for long-wave infrared) Search for the area with the highest temperature in the thermal image; if the peak brightness of that area is... Significantly higher than the background noise level, i.e.: (4) in, The background mean. The background standard deviation, This is the significance threshold (e.g., 3.0, which is 3 times the signal-to-noise ratio).

[0048] 2. Clear outline (for mid-wave infrared) Calculate the sum of edge energies in a mid-wave image ,if Greater than the preset experience threshold If the outline is clear, then it is determined to be clear.

[0049] When the conditions are met (visible light drop == TRUE) AND (hotspot clarity == TRUE), the system executes: visible light weighting. Infrared weight .

[0050] In step 6, the central processing module performs a weighted summation of the target miss distances detected by the weighted sensors in each band to obtain the comprehensive miss distance, specifically: The first, second, and third sensing systems respectively calculate the deviation of the target center from the center of the field of view, i.e., the miss distance, using image recognition algorithms, as follows: Visible light deviation: ; Long-wave infrared deviation: ; Mid-wave infrared deviation: ; Obtain the weights of the visible light channel and the infrared channel, i.e., the weight coefficients calculated by the aforementioned logic. (and satisfy) ); Based on the hotspot intensity of long waves and the texture clarity of mid waves, Dynamic splitting to and .

[0051] Total infrared weight (i.e., 1 - ) Allocation principle: Advantages of long-wave infrared (LWIR): Extremely sensitive to high-heat targets (such as engine nozzles), and has a high signal-to-noise ratio.

[0052] Mid-wave infrared (MWIR) advantages: excellent fog penetration performance and better texture edge features than long-wave infrared.

[0053] Calculation formula: The allocation coefficient α (between 0 and 1) is set based on the mid-wave edge energy Eedge and the long-wave hotspot intensity I. max The normalized ratio is determined as follows: (5) , Assuming the index is normalized, the final weights of each sub-band are: (6) (7) Alternatively, calculations can be omitted, and a preset fixed ratio (such as equal division) can be used.

[0054] Long-wave hotspot intensity refers to the numerical value of the brightest point in a long-wave infrared (LWIR) image. Long-wave infrared (8-14 μm) is most sensitive to temperature.

[0055] Mid-wave edge energy refers to the sum of the sharpness of all object outlines in a mid-wave infrared (MWIR) image. Physical basis: Mid-wave infrared (3-5 μm) is a very special band. It can sense heat and reveals the details of objects more easily than long-wave infrared. Especially in humid or foggy conditions, mid-wave infrared's fog-penetrating ability is generally better than short-wave and visible light. In mid-wave images, you can often see the shape of an aircraft wing and the edges of a vehicle, not just a heat spot.

[0056] The overall miss distance (ΔX) is calculated based on deviation and weight. final ΔY final ),for: ΔX final =W vis ·Δx vis +W lw ·Δx lw +W mw ·Δx mw (8) ΔY final =W vis ·Δy vis +W lw ·Δy lw +Wmw ·Δy mw (9) The output is a single two-dimensional coordinate vector (ΔX). final ΔY final This represents the deviation of the target's perceived actual location by the system. For example, if visible light is lost (weight 0) and infrared (weight 1) detects the target 10 pixels to the left, the overall result will be 10 pixels to the left.

[0057] In step 7, a deflection command is generated based on the overall miss distance and sent to the control drive module. The control drive module then drives the beam deflection actuator to adjust the angle. Specifically: The central processing module inputs the comprehensive miss distance as an error signal to the PID controller, where: Proportional term (P): Quickly generates a reverse correction amount based on the current error magnitude (larger error, larger correction force).

[0058] Integral term (I): Eliminates steady-state error, ensuring that the target can eventually stabilize precisely at the center, rather than fluctuating around the center.

[0059] Differential term (D): Predicts the trend of error change and prevents overshoot (prevents overcorrection).

[0060] The PID controller calculates the required angle correction. (Pitch and yaw angles); The control and drive module converts the angle correction amount into a drive voltage value (e.g., 5.2V applied to the X-axis and 3.1V applied to the Y-axis) based on the voltage-angle characteristic curve (V-Angle Curve) of the beam deflection actuator. The drive voltage is then applied to the beam deflection actuator, such as the comb driver or piezoelectric actuator of the MEMS micromirror, to drive the beam deflection actuator to adjust its angle. This changes the reflection angle of the incident light, causing the target's imaging position on the image to move towards the center of the field of view. When the next frame is acquired, the miss distance will decrease until it becomes zero, thus achieving staring lock.

[0061] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A gaze-tracking fast tracking system based on field-of-view modulation, characterized in that, include: Beam deflection actuator, cascaded beam splitter and independent imaging device, sensor device, central processing module and control drive module; The beam deflection actuator is configured as the common aperture inlet at the front end of the system, used to receive composite light rays from the target scene and guide the composite light rays to the cascaded beam splitter and independent imaging device by angle adjustment; the cascaded beam splitter and independent imaging device is configured with a spectral splitting structure, used to separate the composite light rays into multiple independent spectral bands of optical signals in physical space according to the spectral wavelength characteristics, and transmit them to different optical path channels respectively; the sensor device includes multiple sensing systems respectively set on the different optical path channels, each sensing system is configured with an image sensor adapted to the spectral band of the channel, used to synchronously acquire image data of each spectral band; the central processing module is used to acquire image data and perform quality assessment, dynamically configure the weight parameters of each spectral band in the tracking loop according to the assessment results, and generate a comprehensive miss distance by combining the target miss distance calculated by each spectral band; the control drive module is used to generate drive commands according to the comprehensive miss distance to drive the beam deflection actuator to perform closed-loop beam pointing correction.

2. The gaze-tracking fast tracking system based on field-of-view modulation according to claim 1, characterized in that, The cascaded beam splitter and independent imaging device is configured with a cascaded beam splitter structure to separate composite light rays into multiple spectral bands according to their spectral characteristics, and to directionally transmit the light signals of each spectral band to the corresponding sensing and detection channel. The beam splitter structure uses a dichroic mirror. The sensor device includes a sensing system that corresponds one-to-one with each spectral band.

3. The gaze-tracking fast tracking system based on field-of-view modulation according to claim 1, characterized in that, The beam deflection actuator is a MEMS micromirror array, a mechanical galvanometer, a fast reflector, or a dual-beam wedge prism group.

4. A gaze-tracking fast method based on field-of-view modulation, applied to the gaze-tracking fast system based on field-of-view modulation as described in any one of claims 1-3, characterized in that, include: Step 1: Deploy a gaze tracking system; Step 2: Receive composite light rays from the target scene through the beam deflection actuator, and guide the composite light rays to the cascaded beam splitter and independent imaging device; Step 3: The composite light is separated into multiple spectral bands by the cascaded beam splitter and independent imaging device, and the light signals of each spectral band are transmitted to the corresponding sensing and detection channels respectively. Step 4: The central processing module acquires the image data output from each sensor detection channel, performs quality assessment on the images of each spectral band, and generates corresponding image quality indicators. Step 5: The central processing module dynamically configures the weight parameters of each spectral band in the tracking loop based on the image quality index of each spectral band. Step 6: The central processing module performs a weighted fusion calculation based on the weight parameters and the target miss distances corresponding to each spectral band to generate a comprehensive miss distance. Step 7: Generate a beam deflection control command based on the comprehensive miss distance, and drive the beam deflection actuator to perform beam pointing correction by the control drive module.

5. The gaze-based fast tracking method based on field-of-view modulation according to claim 1, wherein the image quality indicators include, but are not limited to, any one or a combination of contrast, edge strength, signal-to-noise ratio, and target feature saliency.

6. The gaze-based fast tracking method based on field-of-view modulation according to claim 5, characterized in that, Step 5 involves dynamically configuring weight parameters as follows: when the image quality index of a specific spectral band meets the preset advantage conditions, the weight parameter of that spectral band is increased; when the image quality index of a specific spectral band deteriorates and the target features in the images of other spectral bands meet the sharpness criteria, the weight parameter of that spectral band is decreased and the weight parameters of the other spectral bands are increased; wherein, the image quality deterioration criterion is that the current frame index value is lower than the product of the historical frame index average and the preset coefficient, and the target feature sharpness criteria include any one or a combination of hotspot saliency, contour energy, and edge response intensity.

7. The gaze-based fast tracking method based on field-of-view modulation according to claim 6, characterized in that, Step 6, the weighted fusion operation, includes: each sensing system calculating the off-target deviation between the target center and the field of view center within its own spectral band using an image recognition algorithm; the central processing module performing a weighted summation of the off-target deviations of each spectral band based on real-time weight parameters, and outputting the comprehensive off-target amount.

8. The gaze-based fast tracking method based on field-of-view modulation according to claim 7, characterized in that, Step 7, generating the beam deflection control command, includes: using the comprehensive miss distance as a feedback error input to the control algorithm module to generate an angle correction amount; the control drive module, based on the drive characteristic curve of the beam deflection actuator, converts the angle correction amount into a drive signal and applies it to the beam deflection actuator to achieve closed-loop correction of the beam direction.

Images