Remote multispectral day and night monitoring method and system for adaptive communication

Abstract

The invention discloses a long-distance multispectral day and night monitoring method and system for adaptive communication. The method comprises the following steps: acquiring environment illumination brightness, time information and a brightness rapid change event, judging a monitoring working state based on the information, controlling a thermal imaging unit to be started or closed in a dawn, daytime, shady fog, dusk or night mode, driving a time multiplexing coding optical element (TMCA) to perform time-sharing gating between white light and near-infrared light wavebands according to a spectrum time slot, and performing time-sharing gating between the white light and near-infrared light wavebands according to a spectrum time slot. And the spectrum time ratio is dynamically adjusted based on the ROI imaging quality. And the system evaluates the quality of a communication link, adaptively selects FSO or RF link transmission, preferentially uploads ROI data when the bandwidth is limited, and caches and supplementarily transmits other data. According to the method, multispectral imaging is realized through the TMCA, and the imaging quality of a key area is improved by combining the high-speed illumination detection of the event camera and the ROI sub-exposure mechanism.

Description

Technical Field

[0001] This invention relates to monitoring technology, and in particular to a long-distance multispectral day and night monitoring method and system with adaptive communication. Background Technology

[0002] Existing long-range surveillance systems generally employ a three-channel combination of visible white light, near-infrared (NIR), and thermal imaging for day and night monitoring. The switching between day and night channels typically relies on an average image brightness threshold. This simple threshold determination is prone to misjudgment during transitional periods such as dawn and dusk, leading to switching lag or frequent jitter and slow response times. Furthermore, current ROI (Region of Interest) exposure typically still uses a full-frame evaluation and adjustment model, without independent exposure optimization for key areas. This results in underexposure and low recognition rates of important target areas in backlighting or low-light conditions. Additionally, existing systems lack intelligent scheduling mechanisms for communication link switching, failing to adjust data upload strategies based on real-time link quality.

[0003] In summary, the existing technology has the following shortcomings: the spectral channel switching is not intelligent and reliable enough, the communication link utilization is not optimized, the imaging effect in key areas is poor, and the multispectral full-frame acquisition consumes too much bandwidth and storage resources, so there is an urgent need for a new technical solution to improve it. Summary of the Invention

[0004] In view of the problems existing in the prior art, the purpose of this invention is to provide a long-distance multispectral day and night monitoring method and system with adaptive communication, so as to at least partially solve the above-mentioned problems.

[0005] In a first aspect, embodiments of this application provide a long-distance multispectral day-night monitoring method with adaptive communication, comprising the following steps:

[0006] S1) Collect environmental characterization information, including at least ambient light intensity and time information, and receive event information indicating rapid changes in characterization brightness;

[0007] S2) Determine the monitoring status based on the environmental characterization information and event information;

[0008] S3) According to the monitoring working status, control the start or stop of the thermal imaging unit. At the same time, within a single image frame period, drive the TMCA element to perform time-division gating between the white light and infrared light spectral bands by controlling the order and proportion of the time slots of each spectral channel, and dynamically adjust the time ratio of the two spectral bands according to the imaging quality of the ROI area.

[0009] S4) Evaluate the link quality of the communication link, adaptively select the transmission link and determine the upload load priority, so that when the transmission capacity is limited, the data related to ROI is uploaded first and the remaining data is cached and retransmitted.

[0010] S5) At the receiving end, the multispectral data is demixed and / or fused, and the monitoring results are output.

[0011] Specifically, determining the monitoring working mode includes judging the ambient light intensity and obtaining time information, making a comprehensive judgment based on the ambient light intensity and time information, with the ambient light intensity having the highest priority.

[0012] The monitoring modes include dawn mode, daytime mode, fog mode, dusk mode, or night mode.

[0013] In Dawn and Fog modes, the TMCA element sequentially performs exposures in two time slots: white light and near-infrared, to acquire images, while the thermal imaging unit is activated. In Daytime mode, the TMCA element uses the white light channel for full-frame exposure, and the thermal imaging unit is disabled. In Dusk mode, the TMCA element sequentially performs exposures in two time slots: near-infrared and white light, to acquire images, while the thermal imaging unit is activated. In Night mode, the TMCA element uses the near-infrared channel for full-frame exposure, and the thermal imaging unit is enabled.

[0014] The method of dynamically adjusting the time proportion of spectral bands according to the imaging quality of the ROI region specifically includes dynamically adjusting the time proportion of different spectra according to the monitoring working mode, and tilting the time slot proportion towards the main spectrum in the next sampling cycle. The main spectrum is the first sequential spectrum in the current monitoring working mode.

[0015] Secondly, this application provides an adaptive communication long-range multispectral day and night monitoring system, comprising:

[0016] Information acquisition module: Collects environmental characterization information, including at least ambient light intensity and time information;

[0017] Imaging Unit 1: Includes a visible light camera and a near-infrared camera, with the optical axes of each camera coaxially aligned with the same monitoring field of view; an adaptive communication multispectral day / night monitoring system with a code aperture, characterized in that it includes:

[0018] Imaging Unit 2: Includes a thermal infrared imaging device;

[0019] Event camera: Used to detect scene brightness and dynamic changes in events;

[0020] TMCA element: capable of time-division gating between the white light and infrared light spectral bands within a single frame;

[0021] The main control unit is used to receive data from the imaging unit and the environmental characterization information, to determine the monitoring mode, to control the time-division gating of the TMCA spectrum, and to schedule the communication link.

[0022] The communication module includes an FSO and an RF link, which can automatically switch according to link quality. Thirdly, this application also provides an electronic device, including: a processor;

[0023] Memory used to store the processor's executable instructions;

[0024] The processor is configured to execute the instructions to implement the steps of the method of the first aspect described above.

[0025] Beneficial effects

[0026] First, by utilizing TMCA element technology to achieve temporal switching exposure of different spectral channels within a single frame, multispectral compressed sensing imaging is achieved. This allows for the acquisition of multispectral fused images with less hardware and ensures high-quality reconstruction output. Compared to traditional fixed aperture or filter switching schemes, TMCA encoding offers superior performance and higher reconstruction quality in compressed spectral imaging and light field imaging. Combined with visible light and infrared imaging methods, richly detailed color video can be acquired during the day, and heat source targets can still be clearly monitored at night or in low-light environments, enabling 24-hour uninterrupted monitoring.

[0027] Secondly, this invention utilizes an event camera to quickly detect changes in ambient light brightness. Once a drastic change in lighting occurs (such as the sky rapidly darkening or brightening, or lights suddenly turning on or off), the system can instantly trigger a switch between day / night or special weather modes, significantly improving the accuracy and speed of response during day-night transitions and sudden smog events, and avoiding the delays and misjudgments caused by traditional average brightness judgment.

[0028] Furthermore, this invention can improve the imaging quality of the ROI area by dynamically adjusting the slot ratio based on the imaging quality of the ROI area while ensuring panoramic monitoring.

[0029] Finally, the adaptive scheduling strategy for communication links in this invention ensures robust system operation under different communication conditions: when the free-space optical communication (FSO) link weakens due to weather or distance factors, the system automatically switches to the radio frequency (RF) link or reduces the amount of data uploaded to at least transmit critical ROI images and cache the full amount of data locally; once the link is restored or bandwidth is sufficient, the cached data is retransmitted. This mechanism ensures that the most important monitoring information is retained and transmitted even under unstable communication conditions, greatly improving the robustness and reliability of the system.

[0030] In summary, through the above-mentioned innovative design, this invention can effectively overcome the shortcomings of existing technologies and achieve better image quality, higher data transmission efficiency, and more stable communication assurance in all-weather, multispectral long-distance monitoring applications. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the monitoring method in the embodiment;

[0032] Figure 2 This is a block diagram of the monitoring system in the embodiment, illustrating imaging unit 1, imaging unit 2, event camera, TMCA element, communication module, main control unit and central server.

[0033] Figure 3 This is a schematic diagram of the electronic device in Embodiment 2. Detailed Implementation

[0034] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0035] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "front", "rear", etc., indicate the orientation or positional relationship based on the description of the structure of this invention shown in the accompanying drawings. They are only for the convenience of describing this invention and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0036] Furthermore, unless otherwise explicitly specified and limited, the terms "installation" and "connection" should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two structures. Those skilled in the art can understand the specific meaning of the above terms in this invention by considering the overall concept of the invention and the specific context of the solution.

[0037] Example 1

[0038] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0039] like Figure 2 As shown, the hardware structure of the monitoring system of the present invention includes the following main modules:

[0040] Imaging Unit 1: Includes a visible light camera and a near-infrared camera. The optical axes of all cameras are coaxially aligned to the same monitoring field of view. Coaxial configuration and multispectral imaging synchronization are achieved through optical beam splitters or filters. The visible light camera is used to acquire color or white light images during the day; the near-infrared camera is used to acquire near-infrared images and enhance details under low light and hazy conditions.

[0041] Imaging Unit 2: Thermal Imaging Device. The thermal imaging device is used to acquire thermal infrared images of targets at night or when visibility is obstructed.

[0042] The TMCA element (Time-Multiplexed Coded Aperture) is located in the optical path of the imaging unit and is constructed using a programmable grating, a liquid crystal spatial light modulator (SLM), or other variable optical elements. This coded aperture is controlled by the main control unit and can switch between different coded patterns or transmission spectral bands according to a predetermined time slot sequence within each exposure frame, thereby achieving time-division gated exposure of the white light and near-infrared channels.

[0043] Based on the determined mode and the established TMCA time slot schedule, the main control unit sends control commands to the coded aperture and each imaging sensor to synchronously execute multispectral exposure. Within each frame period, the sensors in each spectral channel will activate exposure and acquire light signals in their respective time slots, while the coded aperture will display corresponding optical coding patterns or filtering states in different time slots, thus enabling the sensors to acquire modulated multispectral images. For Regions of Interest (ROIs), the main control unit can control the sensors to read pixels in that region individually in a windowed manner for additional exposure acquisition (e.g., utilizing the multi-ROI readout function of a high-speed CMOS image sensor). Through this series of operations, the system obtains an image data stream containing full-frame multispectral information and enhanced ROI regions.

[0044] Event camera: As an auxiliary sensing channel, it employs an independent event sensor (such as a dynamic vision sensor, DVS) to continuously monitor pixel-level brightness changes in the scene. When ambient lighting changes drastically, the event camera outputs a high-frequency event stream (event signal), transmitting the amplitude and timing of the change to the main control unit. With a microsecond-level response speed, the event camera can provide crucial information from fleeting changes imperceptible to traditional frame-based cameras, driving the system's rapid mode switching logic.

[0045] The main control unit, composed of a high-performance processor or FPGA+ARM embedded chip, is the core of the system's control and computation. It integrates modules such as ROI region detection algorithms, event triggering logic, TMCA element control programs, and communication link evaluation and scheduling algorithms. On one hand, the main control unit receives event signals from the event camera and various sensor data (ambient brightness values, time signals), and uses software algorithms to determine the current monitoring scene's mode (dawn, daytime, foggy weather, dusk, or night). On the other hand, based on the determined mode, it controls the imaging unit and TMCA elements to execute corresponding multispectral exposure combinations (including full-frame exposure and ROI sub-region enhancement exposure). Simultaneously, the main control unit acquires link quality parameters provided by the communication module in real time, intelligently decides the image data upload strategy, selects to send via FSO or RF link, and can write secondary data to a local cache. The main control unit can also perform basic image preprocessing, such as simple decoding of TMCA encoded frames or fusion of ROI sub-frames, to reduce the load on the central server.

[0046] The communication module includes one free-space optical communication (FSO) transceiver and one radio frequency (RF) transceiver, both connected to the main control unit. The FSO communication unit can transmit image data at high speed within line-of-sight using directional laser or infrared beams, but is susceptible to weather conditions. The RF communication unit transmits data via radio frequency channels (such as microwave or cellular networks), offering relatively lower bandwidth but higher stability. The main control unit evaluates the quality of the two links based on real-time signal strength, bit error rate, and other indicators fed back from the communication modules, automatically selecting the currently prioritized upload link. When the FSO link is clear and reliable, the system prioritizes its high-bandwidth data transmission; if fog, rain, or other factors degrade the FSO link quality, the system automatically switches or redundantly transmits critical data via the RF link to ensure timely delivery of important image data to the central server.

[0047] Local cache: Composed of high-speed non-volatile memory (such as SSD hard drives or industrial SD cards), connected to the main control unit, it is used to temporarily store acquired image data when the communication link is interrupted or bandwidth is severely insufficient. When the link is poor, the main control unit writes the temporarily untransmittable full-frame multispectral images and secondary data into the local cache for storage and marks the event time. When the link is restored, the main control unit can retransmit the cached data to the central server to ensure that the monitoring data is not lost.

[0048] Central Server: Deployed as a backend server in the monitoring center, it includes a high-performance processor and corresponding software algorithm modules. The central server connects to the communication module of the monitoring system via a network interface, receiving monitoring image data sent from the front end. The server executes TMCA-encoded image decoding and multispectral image fusion algorithms, restoring coded frames from different spectral channels into clear images or fused images of each band. Furthermore, the central server runs target detection and recognition algorithms to identify and track important targets (such as personnel, vehicles, and drones), and generates alarm information according to preset rules, sending it to users. The central server can also store and display the decoded monitoring video stream for security personnel to review and analyze.

[0049] The software control process of the system of the present invention takes time T as a cycle. Preferably, the selectable cycle of T is [100ms-500ms]. It continuously collects environmental information and adjusts the monitoring mode and data transmission strategy.

[0050] See Figure 1 The flowchart shown below illustrates the following logic:

[0051] Step S1: Collect environmental characterization information, including at least ambient light intensity and time information, and receive event information indicating rapid changes in brightness.

[0052] The main control unit periodically reads information such as the illuminance value (Lux) and current timestamp from the ambient light sensor. In this embodiment, the acquisition period is set to 100 milliseconds to ensure timely detection of environmental changes.

[0053] The main control unit monitors the output of the event camera. If the event camera detects a sudden change in ambient brightness and generates an event signal between any two acquisition cycles, the main control unit immediately initiates the FastSwitch process. This process has the highest priority and can adjust the system's operating mode immediately before the end of the traditional frame cycle to cope with sudden changes in lighting or visibility, thereby improving response speed.

[0054] Step S2: Determine the monitoring status based on the environmental characterization information and event information.

[0055] After no emergency event is detected or the event triggering process is completed, the main control unit determines which monitoring mode to enter based on the current ambient brightness and time information. Preset modes include: Dawn, Day, Overcast, Dusk, and Night. Dawn and Dusk modes are primarily for the sunrise and sunset transition periods, while Overcast mode is for daytime conditions with low light or dense fog. If the brightness and time match the conditions for Dawn mode, it switches to Dawn mode; if it is daytime but the brightness is below a threshold, it enters Overcast mode; if the brightness conditions correspond to nighttime, it enters Night mode, and so on. Determining the monitoring status specifically involves judging the ambient light intensity and acquiring time information, making a comprehensive judgment based on both ambient light intensity and time information, with ambient light intensity having the highest priority.

[0056] Specifically: Step 1: Determine the ambient brightness

[0057] The main control unit reads the light sensor data (in lx) in real time and compares it with preset thresholds: Brightness > L_day → enters initial selection of daytime mode; L_dawn < Brightness ≤ = L_day → enters initial selection of dawn / dusk mode; Brightness ≤ = L_dawn → enters initial selection of nighttime mode. Preferably, L_day is set to 1000 and L_dawn is set to 300. This step roughly classifies the light levels into three levels, ensuring that the first step of mode selection is based on the actual environmental conditions measured physically.

[0058] Step 2: Time Information Verification and Correction

[0059] The system clock or GPS time synchronization is invoked to read the current time. If the brightness determination is inconsistent with the time range (e.g., brightness is lower than L_day but higher than L_dawn at noon, possibly indicating fog / heavy rain), then the system enters overcast mode. If the brightness determination is consistent with the time range, for example, brightness is 500 and time is 6:00, then the system enters dawn mode; if time is 18:00, then the system enters dusk mode. During nighttime periods but with relatively high brightness (e.g., in areas with strong lighting), dusk mode can still be entered to maintain some white light imaging and reduce reliance on thermal imaging. If the brightness is lower than L_dawn, then the system directly enters night mode regardless of time information; that is, brightness information has the highest priority when entering the determination mode.

[0060] Step 3: Final Mode Determination

[0061] Based on the results of the previous three steps, enter one of the following modes: Dawn, Day, Overcast, Dusk, or Night.

[0062] Step S3: According to the monitoring working status, control the start or stop of the thermal imaging unit. At the same time, within a single image frame period, drive the TMCA element to perform time-division gating between the white light and infrared light spectral bands by controlling the order and proportion of the time slots of each spectral channel. And dynamically adjust the time ratio of the two spectral bands according to the imaging quality of the ROI area.

[0063] Once the mode is determined, TMCA is invoked to dynamically adjust the time slot order and length of each spectral channel. The mode determination logic can be represented as a lookup table or a decision tree, as shown in Table 1.

[0064] Table 1 below provides examples of the time range, brightness conditions, and mode names used for each monitoring period, as well as the spectral exposure sequence strategy executed by the TMCA element under that mode:

[0065] Table 1

[0066]

[0067] After entering different modes, the system will control the coded aperture and the photosensitive devices of each channel to perform exposure according to the spectral slot priority order corresponding to that mode. For example, in dawn mode, as the sky becomes brighter, it is set to perform white light and near-infrared two-stage exposures sequentially within each frame, and thermal imaging exposure is activated; while in dusk mode, due to the reduced effect of visible light, near-infrared imaging exposure is prioritized, supplemented by a short white light exposure, and thermal imaging exposure is activated. In night mode, due to the extremely low brightness, visible light supplementation is usually turned off, and only near-infrared imaging and thermal imaging are activated to avoid affecting the natural presentation of the target in low-light environments. Preferably, the default slot ratio of the TMCA element in imaging unit 1 is 1:1. For example, in dawn mode, the slot ratio of white light to the slot time ratio of near-infrared light is set to 1:1, and this ratio setting can be adjusted according to the quality of the imaging results.

[0068] At the beginning of each sampling period, the main control unit acquires the monitoring image from the previous period. After preprocessing such as denoising and grayscale normalization, it uses algorithms such as target detection, motion segmentation, or edge enhancement to determine one or more regions of interest (ROIs) and records their location, size, and category information. For ROIs that have been identified in the previous sampling period and still exist, the main control unit calculates their imaging quality indicators, including but not limited to: sharpness (high-frequency energy), local contrast / signal-to-noise ratio, exposure status (overexposed / underexposed pixel ratio), and motion blur index.

[0069] If the imaging quality indicators are unsatisfactory, the slot time ratio will be dynamically adjusted according to the different modes. In the next sampling cycle, the total slot ratio will be adjusted in small steps (±10%) from the default 1:1 ratio towards the main spectrum. However, the ratio of any spectrum must not be lower than 10% of the total slot to maintain the continuity of basic information. For example, in dawn mode or fog mode, the proportion of white light will be increased, and in dusk mode, the proportion of near-infrared light will be increased.

[0070] To further illustrate the effectiveness of the method of the present invention, the system working process is described below in conjunction with typical scenarios:

[0071] In a dawn scene (e.g., 05:30, ambient illuminance approximately 500 Lux): The sky is gradually brightening, but the overall brightness is still low, prompting the system to enter Dawn mode. The TMCA element sequentially executes exposure acquisition in two slots: white light and near-infrared, ensuring the capture of visible light details while acquiring infrared thermal information. The slot time ratio for white light and near-infrared light is set to 1:1. Due to the rapid brightness change, the event camera continuously outputs event trigger signals during sunrise, and the system adjusts the exposure strategy in real time to avoid misjudging it as daytime mode. In this scenario, a fast and accurate switch from night mode to dawn mode is achieved, avoiding the repeated jitter that may occur with traditional average brightness-based solutions at the beginning of daylight.

[0072] In clear daytime scenes (e.g., 12:00, illuminance > 5000 Lux): The system is in Day mode. Visible light is ample and the target is clear; the TMCA element primarily uses the white light channel for full-frame exposure.

[0073] Hazy / overcast scene (9:30 AM, illuminance approximately 800 Lux): Due to the significantly reduced ground illumination and visibility caused by haze, the system detects that the current brightness is below 1000 Lux and determines to enter Overcast / haze mode. The TMCA element sequentially arranges white light → near-infrared exposure within each frame, while thermal imaging is activated to supplement image information. Compared to clear daytime scenes, near-infrared and thermal imaging components are added to penetrate the haze and obtain more image details.

[0074] Dusk scene (e.g., 7:00 PM, illuminance approximately 400 Lux): As darkness falls, the system enters Dusk mode. At this time, visible light imaging quality rapidly decreases, and the TMCA element is set to prioritize near-infrared light, followed by white light, with the ratio of the two set to 1:1 by default. The event camera generates a continuous event stream in response to the dimming light, triggering the system to maintain Dusk mode promptly and prevent premature shutdown of white light by accidentally switching to Night mode.

[0075] Nighttime Scene (23:30, ambient illuminance approximately 100 Lux): Under low light conditions at night, the system operates in Night mode. Monitoring primarily relies on thermal imaging and the near-infrared channel. The TMCA element locks onto the near-infrared, the thermal imaging sensor captures the temperature distribution of the scene, and the near-infrared camera captures contour details in low light.

[0076] Step S4: Evaluate the link quality of the communication link, adaptively select the transmission link and determine the upload load priority, so that when the transmission capacity is limited, the data related to ROI is uploaded first and the remaining data is cached and retransmitted.

[0077] After each frame of data is acquired, the main control unit immediately obtains the status information of the current communication module's FSO and RF links, including the FSO received signal power, channel stability, and the RF link's bandwidth usage and latency. Then, it determines the data upload scheme for this frame according to a predetermined strategy: if the communication environment is good (e.g., a clear and stable FSO link), the complete fused multispectral image and related data are uploaded to the central server via the high-speed link; if a degraded FSO link quality or insufficient available bandwidth is detected, the main control unit selects a fallback strategy, extracting and uploading only key images of the ROI region (e.g., thermal imaging ROI sub-images or fused ROI sub-images), while saving the complete original multispectral frame data to a local cache. In the event of a complete interruption (e.g., FSO is blocked and the RF signal is weak), the system enters emergency mode, temporarily refraining from uploading any data, storing only key frames and all data locally, and continuously waiting for the link to recover.

[0078] Based on the above decisions, the main control unit controls the communication module to select the appropriate link to send data. For example, under normal circumstances, the FSO link is used to transmit high-definition full-frame images, while when switching to the RF link, only compressed ROI images or alarm information are sent.

[0079] All transmitted data is copied locally to prevent loss. Untransmitted data is queued in a local cache. When link conditions improve, the master control unit will gradually transmit the untransmitted data in the cache to the central server according to FIFO order or importance priority principle to achieve final data integrity.

[0080] Step S5: At the receiving end, the multispectral data is demixed and / or fused, and the monitoring results are output.

[0081] The receiving end first identifies the spectral type corresponding to each image time slice based on the time-encoded information transmitted with the data, and separates different spectral subframes within the same frame period. Then, spatial registration is performed on the different spectral subframes, and white light, near-infrared, and thermal imaging data are fused to obtain a monitoring image containing multispectral information. When bandwidth is limited, the Region of Interest (ROI) in the fused result is output first to ensure the imaging quality of critical areas; other areas can be delayed for supplementary transmission or transmitted at reduced resolution. The final output fused monitoring result can be used for real-time display and subsequent storage.

[0082] The above control process is executed continuously in a loop, enabling the system to adjust its working status in real time according to environmental changes and communication conditions, achieving efficient and adaptive all-weather monitoring.

[0083] The pseudocode example is as follows:

[0084] Loop every 100ms:

[0085] Read lux, time, dist = sensor data

[0086] if event camera.detect():

[0087] Execute FastSwitch() / / Fast mode switching

[0088] mode = Determine the current mode (lux, time, dist)

[0089] schedule = Generate TMCA time series plan (mode, ROI)

[0090] Execute TMCA exposure (schedule)

[0091] comm_status = Check communication link status()

[0092] upload_plan = Determines the upload strategy (comm_status, full frame data, ROI data)

[0093] Perform data upload (upload_plan)

[0094] Example 2

[0095] An electronic device, such as Figure 3 As shown, it includes a memory storing executable program code and a processor coupled to the memory; wherein the processor calls the executable program code stored in the memory to execute the method steps disclosed in the above embodiments.

[0096] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A long-distance multispectral day and night monitoring method with adaptive communication, characterized in that: The monitoring method includes the following steps: S1) Collect environmental characterization information, including at least ambient light intensity and time information, and receive event information indicating rapid changes in characterization brightness; S2) Determine the monitoring status based on the environmental characterization information and event information; S3) According to the monitoring working status, control the start or stop of the thermal imaging unit. At the same time, within a single image frame period, drive the TMCA element to perform time-division gating between the white light and infrared light spectral bands by controlling the order and proportion of the time slots of each spectral channel, and dynamically adjust the time ratio of the two spectral bands according to the imaging quality of the ROI area. S4) Evaluate the link quality of the communication link, adaptively select the transmission link and determine the upload load priority, so that when the transmission capacity is limited, the data related to ROI is uploaded first and the remaining data is cached and retransmitted. S5) At the receiving end, the multispectral data is demixed and / or fused, and the monitoring results are output.

2. The method according to claim 1, characterized in that: The monitoring modes include dawn mode, daytime mode, fog mode, dusk mode, or night mode.

3. The method according to claim 2, characterized in that: Determining the monitoring working mode specifically includes judging the ambient light intensity and obtaining time information. The judgment is made based on the ambient light intensity and time information, with the ambient light intensity having the highest priority.

4. The method according to claim 2, characterized in that: In dawn mode and fog mode, the TMCA element sequentially performs exposure acquisition in two time slots: white light and near-infrared, while the thermal imaging unit is activated.

5. The method according to claim 2, characterized in that: In daytime mode, the TMCA element uses the white light channel for full-frame exposure, and the thermal imaging unit is turned off.

6. The method according to claim 2, characterized in that: In Dusk Mode, the TMCA element sequentially performs two time slots of exposure to acquire images: near-infrared and white light, while the thermal imaging unit is activated.

7. The method according to claim 2, characterized in that: In night mode, the TMCA element uses the near-infrared channel for full-frame exposure, and the thermal imaging unit is activated.

8. The method according to claim 1, characterized in that: The method of dynamically adjusting the time ratio of the two spectral bands based on the imaging quality of the ROI region specifically includes, if the imaging quality index is unqualified, dynamically adjusting the time ratio of different spectra according to the monitoring working mode, and tilting the time slot ratio towards the main spectrum in the next sampling cycle, wherein the main spectrum is the first sequential spectrum in the current monitoring working mode.

9. A long-range multispectral day and night monitoring system with adaptive communication, characterized in that, include: The information acquisition module collects environmental characterization information, including at least ambient light intensity and time information. Imaging unit one includes a visible light camera and a near-infrared camera, with the optical axes of each camera coaxially aligned with the same monitoring field of view; Imaging unit two includes a thermal infrared imaging device; Event camera: Used to detect scene brightness and dynamic changes in events; The TMCA element is capable of time-division gating between the white light and infrared light spectral bands within a single frame. The main control unit is used to receive data from the imaging unit and the environmental characterization information, to determine the monitoring mode, to control the time-division gating of the TMCA spectrum according to the monitoring mode, and to schedule the communication link. The communication module, including FSO and RF links, can automatically switch based on link quality.

Images