Multifunctional multispectral portable observation system and method

Abstract

The application provides a multifunctional multispectral portable observation system, comprising: a housing including an upper housing and a lower housing; a multispectral imaging module; a visual module; the visual module comprises an OLED (Organic Light Emitting Diode) display module; the laser ranging module comprises a laser ranging mounting bracket, a transmitting antenna and a receiving antenna; a visible light imaging module; an infrared light imaging module; the intelligent processing module is used for identifying and tracking the camouflage target; an ARM embedded module; the battery pack is used for supplying power to the observation system; the multispectral imaging module and the visual module are mounted on the upper shell; the intelligent processing module, the battery pack, the laser ranging module, the visible light imaging module, the infrared light imaging module and the ARM embedded module are installed on the lower shell. The technical scheme provided by the invention has a multifunctional observation mode, can adopt different modes to realize effective observation of the camouflage target, and adapts to various complex observation requirements.

Description

Technical Field

[0001] The present application relates to the technical field of observation instruments, and in particular to a multifunctional multispectral portable observation system and method. Background Art

[0002] In modern warfare, driven by information technology, high-precision military reconnaissance and target recognition technologies have become crucial factors in determining victory or defeat in war. Rapidly and accurately identifying key enemy military targets and providing key information such as their type and location provides the information foundation for subsequent precision strikes. This allows for effective strikes against enemy combat systems in a shorter timeframe and with lower resource consumption, seizing the initiative on the battlefield. In the field of military reconnaissance, optical reconnaissance technology, with its advantages of high resolution and intuitive imaging, has become a widely used and important tool. High-resolution imaging and infrared imaging are particularly common.

[0003] However, existing single-mode imaging observation technologies have significant limitations. While visible light imaging technology can achieve relatively high resolution, it is highly susceptible to interference from environmental factors such as weather and lighting. Image quality degrades significantly in rain, fog, and at night, and camouflaged targets are difficult to identify. While infrared imaging technology offers all-weather monitoring capabilities and can detect targets using their thermal radiation characteristics, it faces challenges due to the rapid development of infrared stealth technology. Enemies employ infrared stealth materials and thermal radiation suppression devices to significantly reduce the detectability of targets in the infrared band, hindering the effective application of infrared imaging technology. As modern warfare demands ever-increasing operational efficiency and precision, single-mode observation technology can no longer meet increasingly complex observation needs. Summary of the Invention

[0004] In view of this, the present application provides a multifunctional multispectral portable observation system and method to solve the technical problem that the observation system in the prior art cannot meet the increasingly complex observation needs.

[0005] In a first aspect, the present application provides a multifunctional multispectral portable observation system, comprising:

[0006] a housing, the housing comprising an upper housing and a lower housing;

[0007] Multispectral imaging module;

[0008] Visual module; the visual module includes an OLED display module;

[0009] A laser ranging module, comprising a laser ranging mounting bracket, a transmitting antenna, and a receiving antenna;

[0010] Visible light imaging module;

[0011] Infrared imaging module;

[0012] Intelligent processing module for identifying and tracking camouflaged targets;

[0013] ARM embedded module;

[0014] A battery pack, used to power the observation system;

[0015] Installing the multispectral imaging module and the visual module on the upper shell;

[0016] The intelligent processing module, battery pack, laser ranging module, visible light imaging module, infrared light imaging module and ARM embedded module are installed on the lower shell.

[0017] Preferably, the multispectral imaging module includes a multispectral image processing board, a multispectral sensor and a multispectral lens assembly;

[0018] The multispectral sensor is used to divide the visible light band into multiple different bands to distinguish the spectral characteristics of the environment and the target radiation;

[0019] The multi-spectral image processor is used to detect camouflaged targets through spectral feature response, image signal enhancement, and filtering algorithms.

[0020] Preferably, the visible light imaging module comprises a visible light mounting bracket, a visible light image sensor and a visible light lens assembly;

[0021] The infrared light imaging module includes an infrared light mounting bracket, an infrared light image sensor and an infrared light lens assembly.

[0022] Preferably, the visible light lens assembly and the infrared light lens assembly are continuously zoomable lens assemblies.

[0023] Preferably, the multifunctional multispectral portable observation system further includes: a GPS positioning component; the GPS positioning component includes a radome and a GPS positioning antenna; and the GPS positioning component is fixed to the upper shell.

[0024] Preferably, the multifunctional multispectral portable observation system further includes: a compass orientation module; the compass orientation module includes an electronic compass and a compass mounting bracket, and the electronic compass is fixedly connected to the lower shell via the compass mounting bracket.

[0025] Preferably, the visual module further includes a diopter adjustment mechanism and an eyepiece assembly; wherein, the diopter threshold adjusted by the diopter adjustment mechanism is (-5 to +5), and the eyepiece assembly is adjustable in magnification from 1 to 20 times.

[0026] In a second aspect, the present application provides a multifunctional multispectral portable observation method, comprising:

[0027] Start the GPS positioning component to obtain the current location information, calibrate the azimuth and elevation angles of the observation system through the compass orientation module, and load and adjust the working parameters of each module according to the observation mode;

[0028] The observation data of the working modules are collected synchronously, and the collected multi-source data are temporally and spatially aligned and fused; wherein the working module is at least one of a laser ranging module and a visible light imaging module, an infrared imaging module, and a multispectral imaging module; the temporal and spatial alignment of the collected multi-source data includes aligning image data, distance data, and orientation data according to time, and correcting and eliminating image pixel offset; the fusion of the collected multi-source data includes mapping the distance data to the image data, superimposing GPS information and compass data, and generating georeferenced image cube data;

[0029] Targets in the observation environment are identified based on the generated geo-referenced image cube data, and a structured observation report is generated.

[0030] Preferably, the multifunctional multispectral portable observation method further includes:

[0031] A spectral-spatial dual path is used to construct a Transformer-based multimodal fusion observation model to identify targets in the observation environment;

[0032] The spectral curve of each pixel is convolved to extract the global spectral features, and each band image is independently divided into blocks to extract the local spatial features. In addition, a learnable band weight parameter is added to the input layer, and the fusion layer interacts the dual-path features through cross-attention.

[0033] T fused =CrossAttn(Q=T spatial ,K / V=T spectral )

[0034] T spectral =Transformer(Conv1D(I pirel ∈R B ))

[0035] Among them, T fused is a dual-path feature, T spatial is the spatial feature, T spectral is the spectral feature, Q, K, V are the query, key, and value in the cross-attention mechanism CrossAttn, Conv is the convolutional layer, D is the feature dimension, and I pirel ∈R B is the learnable band weight parameter;

[0036] Using a multi-scale feature pyramid, the underlying features are fused with small target features through upsampling, and Deformable Attention is used instead of standard Attention to focus on the target area:

[0037]

[0038] Among them, p is the reference point coordinate, Δp k is the learnable offset of the k-th sampling point, is the attention weight.

[0039] Preferably, the multifunctional multispectral portable observation method further includes:

[0040] Automatically switches the main imaging mode according to the ambient light intensity and dynamically adjusts the exposure parameters based on the reflectivity of the multispectral band;

[0041] Extract texture information from visible light imaging and spectral features from multispectral images, generate classified images, fuse infrared thermal data with laser ranging results, identify the spatial depth of target distribution, and output interactive composite data layers.

[0042] The multifunctional multispectral portable observation system and method provided in this application have at least the following beneficial effects:

[0043] The observation system provided in this application has the advantages of integration, modularization, miniaturization, low power consumption, easy operation, and visualization.

[0044] The observation system provided in this application adopts a single-platform multi-optical axis stabilization mechanism to fix the visible light axis, infrared optical axis, laser emission optical axis, laser receiving optical axis and magnetic axis on a stable platform with a bridge mechanism. The optical axis stability can reach 0.1 milliradians, thereby ensuring that the product's ranging, orientation and other performance have good accessibility.

[0045] The observation system provided in this application can optionally use visible light imaging detection and infrared imaging detection to achieve observation of distant targets during the day and at night. Multispectral imaging detection can effectively detect standard camouflage, with a detection range of up to 4km, meeting the requirements of individual reconnaissance and camouflage exposure, meeting diverse detection needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. It should be noted that the drawings described below are schematic and should not be understood as limiting the present application in any way. In the drawings:

[0047] Figure 1 A schematic exploded view of the structure of a multifunctional multispectral portable observation system in one embodiment of the present application is shown.

[0048] 1. Upper shell; 2-1. Antenna cover; 2-2. GPS positioning antenna; 3-1. Multispectral image processing board; 3-2. Multispectral sensor; 3-3. Multispectral lens assembly; 4-1. Diopter adjustment mechanism; 4-2. Eyepiece assembly; 4-3. OLED display module; 5. Intelligent processing module; 6. Battery pack; 7-1. Electronic compass; 7-2. Compass mounting bracket; 8-1. Laser ranging mounting bracket; 8-2. Transmitting antenna; 8-3. Receiving antenna; 9-1. Visible light mounting bracket; 9-2. Visible light image sensor; 9-3. Visible light lens assembly; 10. Lower shell; 11-1. Infrared light mounting bracket; 11-2. Infrared image sensor; 11-3. Infrared lens assembly; 12. ARM embedded module. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0050] Example

[0051] In a first aspect, the present application provides a multifunctional multispectral portable observation system.

[0052] See also Figure 1 As shown in the figure, the multifunctional multispectral portable observation system includes:

[0053] The housing comprises an upper housing 1 and a lower housing 10;

[0054] Multispectral imaging module; the multispectral imaging module includes a multispectral image processing board 3-1, a multispectral sensor 3-2, and a multispectral lens assembly 3-3; the multispectral sensor is used to divide the visible light band into multiple different bands to distinguish the spectral characteristics of the environment and the target radiation; the multispectral image processor is used to detect camouflaged targets through spectral characteristic response, image signal enhancement, and filtering algorithms;

[0055] Visual module 4; the visual module includes an OLED display module 4-3; the OLED display module converts visible light, infrared, and multi-spectral video signals into visual image signals for human eye observation.

[0056] Laser ranging module, the laser ranging module includes a laser ranging mounting bracket 8-1, a transmitting antenna 8-2 and a receiving antenna 8-3;

[0057] Visible light imaging module; the visible light imaging module includes a visible light mounting bracket 9-1, a visible light image sensor 9-2, and a visible light lens assembly 9-3; used to convert and process visible light signals and output them to a display screen;

[0058] Infrared imaging module; the infrared imaging module includes an infrared mounting bracket 11-1, an infrared image sensor 11-2, and an infrared lens assembly 11-3; used to convert and process thermal radiation light signals and output them to a display screen;

[0059] Intelligent processing module 5;

[0060] ARM embedded module 12;

[0061] Battery pack 6, used to power the observation system;

[0062] A multispectral imaging module and a visual module are installed on the upper shell;

[0063] An intelligent processing module, a battery pack, a laser ranging module, a visible light imaging module, an infrared light imaging module and an ARM embedded module are installed on the lower shell.

[0064] In this embodiment, the observation system utilizes multi-channel detection modes, including visible light imaging, infrared night vision, and multispectral imaging, to achieve daytime and nighttime target search and measurement. The infrared imaging module segments the infrared radiation of different objects, as well as the infrared characteristics of camouflage or obstructions, and highlights them through edge effects, image enhancement, and other algorithms, enabling nighttime exposure. Multispectral imaging analyzes the spectral characteristics of camouflage and obstructions during the day, deriving spectral differences between camouflage and obstructions and natural objects through image algorithms. Image enhancement algorithms then highlight camouflage and obstructions within the scene, facilitating timely detection and exposure.

[0065] When a target or camouflage is found, the intelligent processing module identifies and tracks the target, issues an early warning based on specific threatening weapons, and measures the distance to the target by controlling the laser ranging module. At the same time, the ARM embedded module collects information such as the target's direction, distance, and own position, and calculates the target's position.

[0066] The observation system provided in this application has the advantages of integration, modularization, miniaturization, low power consumption, easy operation, and visualization.

[0067] The observation system provided in this application adopts a single-platform multi-optical axis stabilization mechanism to fix the visible light axis, infrared optical axis, laser emission optical axis, laser receiving optical axis and magnetic axis on a stable platform with a bridge mechanism. The optical axis stability can reach 0.1 milliradians, thereby ensuring that the product's ranging, orientation and other performance have good accessibility.

[0068] The observation system provided in this application can optionally use visible light imaging detection and infrared imaging detection to achieve observation of distant targets during the day and at night. Multispectral imaging detection can effectively detect standard camouflage, with a detection range of up to 4km, meeting the requirements of individual reconnaissance and camouflage exposure, meeting diverse detection needs.

[0069] Preferably, the visible light lens assembly and the infrared light lens assembly are continuously zoomable lens assemblies to achieve clear imaging in different ranges.

[0070] In this embodiment, dual closed-loop control technology, both speed and position, is employed to achieve high-precision control of the zoom lens assembly. Real-time focus compensation is achieved by collecting compensation values ​​for each fixed field of view of the lens, fitting this data to a focus compensation curve, and then using the servo control component to perform real-time compensation based on this curve. This ensures clear images throughout the zoom process, demonstrating a high degree of intelligence.

[0071] Preferably, the multifunctional multispectral portable observation system further includes: a GPS positioning component; the GPS positioning component includes a radome 2-1 and a GPS positioning antenna 2-2; and the GPS positioning component is fixed to the upper shell.

[0072] In this embodiment, the antenna cover is used to protect the GPS positioning antenna from environmental erosion and has the function of transmitting electromagnetic waves. The GPS positioning antenna is used to search for and receive Beidou satellite signals to achieve its own positioning function.

[0073] Preferably, the multifunctional multispectral portable observation system further includes a compass orientation module, which includes an electronic compass 7-1 and a compass mounting bracket 7-2, wherein the electronic compass is fixedly connected to the lower housing via the compass mounting bracket, and is used to measure the target azimuth.

[0074] GPS and an electronic compass are used to collect the observation system's position and attitude information, respectively. This information is then combined with range information to determine the target's coordinates when detecting targets. This improves the observation and operational efficiency of the observation system, further enhancing its applicability. Using image reconstruction and restoration technology, inertial attitude processing, and target direction guidance, the system implements intelligent recognition and tracking, enabling rapid capture of both threatening and hidden targets, making the combat environment highly transparent and providing operational advantages.

[0075] Preferably, the visual module further includes a diopter adjustment mechanism 4-1 and an eyepiece assembly 4-2. The diopter adjustment mechanism has a diopter threshold of (-5 to +5) to accommodate people with different vision, and the eyepiece assembly has an adjustable magnification of 1-20 times. This improves the observation efficiency and usage efficiency of the observation system and further enhances the applicability of the observation system.

[0076] In a second aspect, the present application provides a multifunctional multispectral portable observation method, comprising:

[0077] Start the GPS positioning component to obtain the current location information, calibrate the azimuth and elevation angles of the observation system through the compass orientation module, and load and adjust the working parameters of each module according to the observation mode;

[0078] The observation data of the working modules are collected synchronously, and the collected multi-source data are temporally and spatially aligned and fused; wherein the working module is at least one of a laser ranging module and a visible light imaging module, an infrared imaging module, and a multispectral imaging module; temporally and spatially aligning the collected multi-source data includes aligning image data, distance data, and orientation data according to time, and correcting and eliminating image pixel offsets; fusing the collected multi-source data includes mapping the distance data to the image data, superimposing GPS information and compass data, and generating georeferenced image cube data;

[0079] The targets in the observation environment are identified based on the generated geo-referenced image cube data, and a structured observation report is generated.

[0080] In this embodiment, upon system startup, the GPS positioning component is first activated, receiving satellite signals to obtain accurate current geographic location information, including latitude, longitude, and altitude. Simultaneously, the compass orientation module begins operating, calibrating the azimuth and elevation angles of the observation system to ensure directional accuracy. Based on the preset observation mode, the system automatically loads and adjusts operating parameters for modules such as the laser ranging module, visible light imaging module, infrared imaging module, and multispectral imaging module, such as the laser ranging module's measurement frequency, the imaging module's resolution, and exposure time, to meet the needs of different observation scenarios.

[0081] When each working module operates according to the set parameters, the system synchronously collects the observation data generated by it. The multi-source data collected covers image data (from visible light, infrared light, and multispectral imaging modules), distance data (from the laser ranging module), and orientation data (from the compass orientation module). In order to effectively integrate and utilize these multi-source data, the system first performs spatiotemporal registration operations, aligning different types of data according to timestamps to ensure that the image data, distance data, and orientation data at the same moment correspond to each other, and uses algorithm correction to eliminate the offset of image pixels caused by factors such as device jitter and angular deviation. Next, the system maps the distance data to the image data, superimposes GPS information and compass data, and fuses the multi-source data to generate georeferenced image cube data, which contains rich spatial, geographic, and spectral information.

[0082] Based on the generated georeferenced image cube data, the system uses advanced image recognition algorithms to identify targets in the observation environment. It can distinguish between different types of targets and extract target characteristics such as shape, size, location, and spectral characteristics. Finally, the system generates a clear and comprehensive structured observation report based on the recognition results, allowing users to intuitively understand the targets in the observation environment.

[0083] The coordinated work of the GPS positioning component and the compass orientation module ensures the accurate positioning and direction calibration of the observation system in space, providing a precise geographic and azimuth reference for subsequent observation data, effectively avoiding observation deviations caused by position and direction errors, and greatly improving the reliability and accuracy of observation data.

[0084] The system simultaneously collects data from various modules, including laser ranging and imaging, and organically integrates these heterogeneous data sources through spatiotemporal registration and fusion technologies. This processing method not only resolves temporal and spatial inconsistencies between data, but also fully leverages the strengths of each module's data to generate georeferenced image cube data containing rich information. This provides more comprehensive and accurate data support for target recognition, improving the system's adaptability to complex observation environments and its data processing capabilities.

[0085] Target identification is performed based on georeferenced image cube data, and structured observation reports are generated, enabling intelligent processing and presentation of observation results. Target information within the observation environment can be quickly obtained without complex data parsing and analysis, greatly improving work efficiency and reducing the time and errors associated with manual analysis.

[0086] The georeferenced image cube data generated by multi-source data fusion not only contains information from a single module but also comprehensively reflects the various characteristics of the observed environment. For example, by fusing visible light and infrared images, at night or in low-light conditions, the system can utilize both the texture details of the visible light image and the thermal radiation information of the infrared image to more comprehensively perceive the environment and identify hidden targets or subtle details, greatly enhancing the system's perception of complex environments.

[0087] The system achieves a high degree of automation, from positioning and orientation, parameter adjustment, to data collection, processing, and report generation. Throughout the observation process, frequent human intervention is unnecessary, reducing manpower input and the risk of human error. It also ensures long-term, stable, and efficient operation, making it particularly suitable for scenarios requiring continuous observation or in harsh environments where extended human presence is difficult.

[0088] The generated structured observation reports have a unified and standardized format, with clearly organized and categorized data, facilitating data storage, retrieval, and analysis management. Furthermore, this standardized reporting format facilitates data sharing and interaction across departments, institutions, or systems, promoting the flow and integration of information and facilitating multi-team collaboration and cross-disciplinary research.

[0089] During target recognition, the advanced image recognition algorithm employed has been continuously optimized to quickly and accurately identify targets from geo-referenced image cube data. Compared to traditional algorithms, this algorithm offers higher accuracy in complex backgrounds and situations involving similar targets. It also reduces processing time and improves target recognition efficiency, enabling timely detection and response to changes in observed targets, meeting the requirements of applications with demanding real-time performance.

[0090] It can be seen that the observation method provided in this application effectively improves the accuracy of the observation results and the adaptability to meet the needs of complex observation environments, and improves the utilization efficiency of the observation system.

[0091] Preferably, the multifunctional multispectral portable observation method further includes:

[0092] A spectral-spatial dual path is used to construct a Transformer-based multimodal fusion observation model to identify targets in the observation environment;

[0093] The spectral curve of each pixel is convolved to extract the global spectral features, and each band image is independently divided into blocks to extract the local spatial features. In addition, a learnable band weight parameter is added to the input layer, and the fusion layer interacts the dual-path features through cross-attention.

[0094] T fused =CrossAttn(Q=T spatial ,K / V=Tspectral )

[0095] T spectral =Transformer(Conv1D(I pirel ∈R B ))

[0096] Among them, T fused is a dual-path feature, T spatial is the spatial feature, T spectral is the spectral feature, Q, K, V are the query, key, and value in the cross-attention mechanism CrossAttn, Conv is the convolutional layer, D is the feature dimension, and I pirel ∈R B is the learnable band weight parameter;

[0097] Using a multi-scale feature pyramid, the underlying features are fused with small target features through upsampling, and Deformable Attention is used instead of standard Attention to focus on the target area:

[0098]

[0099] Among them, p is the reference point coordinate, Δp k is the learnable offset of the k-th sampling point, is the attention weight.

[0100] This implementation addresses the challenges of high-dimensional redundancy in the spectral dimension, loss of spatial detail, and inaccurate small-target detection in traditional multispectral observations through an innovative architecture that decouples and interacts spectral and spatial features. It also combines the local feature extraction capabilities of convolutional neural networks (CNNs) with the global dependency modeling advantages of Transformers to form a cross-modal feature enhancement mechanism.

[0101] In this implementation, a convolution operation is performed on the spectral curve of each pixel, leveraging the powerful local feature extraction capabilities of a convolutional neural network (CNN) to extract global spectral features. The convolution operation captures local patterns and trends in the spectral curve, converting the spectral data into representative feature vectors that contain information about the spectral characteristics of the target substance.

[0102] Each band image is independently divided into multiple local regions. Each local region contains image information at a specific spatial location. By processing these blocks, local spatial features are extracted. This block-based processing method can better capture the spatial structure and texture information of the target in the image.

[0103] Learnable band weight parameters are added to the input layer. These parameters adaptively adjust the importance of each band based on data characteristics. Different bands contribute differently to target recognition. By learning these weights, the model can more effectively utilize band information that is valuable for target recognition and suppress interference from noise and redundant information.

[0104] At the fusion layer, a cross-attention mechanism enables interaction between the two-path features. This mechanism allows spectral and spatial features to perceive and utilize each other, fully exploiting the complementary relationship between the two. Spectral features provide spatial features with clues about the target's material properties, while spatial features provide spectral features with information about the target's location and morphology. The fusion of the two forms a more comprehensive representation of target features.

[0105] Using a multi-scale feature pyramid structure, the model's underlying features have high resolution, capturing detailed information in the image and making it suitable for detecting small objects. Upsampling fuses the underlying features with small object features, enabling the model to effectively extract target features at different scales. Furthermore, Deformable Attention replaces standard Attention. Deformable Attention adaptively adjusts the focus to the target area, reducing attention to irrelevant background information, thereby more accurately extracting target features.

[0106] In this implementation, global spectral features and local spatial features are extracted separately through a dual spectral-spatial pathway, fully leveraging the spectral and spatial information in the data. The fusion of these two features forms a richer and more comprehensive target feature description. Combined with a cross-attention mechanism, this effectively exploits the complementarity of the two features, enabling the model to more accurately identify targets. Furthermore, Deformable Attention focuses on the target area, avoiding background interference, further improving recognition accuracy and enabling accurate target identification even in complex observation environments. The learnable band weight parameters in the input layer adaptively adjust band importance based on different datasets and application scenarios. This enables the model to better utilize effective information, reduce the impact of noise, and enhance generalization when dealing with observational data from diverse sources and characteristics. A multi-scale feature pyramid, combined with upsampling of underlying features and fusion of small target features, effectively addresses the difficulty in identifying small targets due to low resolution. The underlying high-resolution features provide rich, detailed information for small target recognition. After upsampling and fusion, the model can more clearly capture the features of small targets, significantly improving their recognition rate. Compared to traditional complex models, this model, through rational module design and optimization strategies, reduces unnecessary computation while ensuring recognition accuracy. For example, compared to standard attention, Deformable Attention reduces computation on irrelevant areas, improving computational efficiency and making it more suitable for real-time scenarios in practical applications.

[0107] It further improves the accuracy of observation results and the adaptability to meet the needs of complex observation environments, and improves the efficiency of observation system use.

[0108] Preferably, the multifunctional multispectral portable observation method further includes:

[0109] Automatically switches the main imaging mode according to the ambient light intensity and dynamically adjusts the exposure parameters based on the reflectivity of the multispectral band;

[0110] Extract texture information from visible light imaging and spectral features from multispectral images, generate classified images, fuse infrared thermal data with laser ranging results, identify the spatial depth of target distribution, and output interactive composite data layers.

[0111] In this implementation, the system automatically switches between main imaging modes based on ambient light intensity and dynamically adjusts exposure parameters based on multispectral reflectance, enabling it to acquire high-quality image data under a wide range of lighting conditions. Automatically switching to infrared imaging mode in low-light environments ensures unrestricted observation. Intelligent adjustment of multispectral exposure parameters effectively avoids over- or underexposure. This not only improves image clarity and accuracy but also provides a higher-quality data foundation for subsequent analysis tasks such as target identification and object classification, enhancing the system's applicability and reliability in diverse environments.

[0112] By extracting texture information from visible light imaging and spectral features from multispectral images to generate classified images, and fusing infrared thermal data with laser ranging results to identify target spatial depth and output interactive composite data layers, the system achieves in-depth mining and efficient utilization of observation data. The classified images intuitively present the distribution of features, while the composite data layers provide rich target details. Users can access the required data through interactive operations, greatly enhancing data visualization and interactivity. This helps users understand the observation scene more quickly and accurately, providing more comprehensive and intuitive data support for subsequent decision-making, significantly enhancing the data's application value.

[0113] It further improves the accuracy of observation results and the adaptability to meet the needs of complex observation environments, and improves the efficiency of observation system use.

[0114] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] Although the embodiments of the present application are described in conjunction with the accompanying drawings, this should not be construed as limiting the scope of the patent application. It should be noted that, for those skilled in the art, other variations or modifications may be made based on the above description without departing from the concept of the present application. It is not necessary and impossible to list all embodiments here. Obvious variations or modifications derived therefrom remain within the scope of protection created by the present application. Therefore, the scope of protection of the patent application shall be based on the appended claims.

Claims

1. A multifunctional multispectral portable observation system, characterized in that: include: a housing, the housing comprising an upper housing and a lower housing; Multispectral imaging module; Visual module; The visual module includes an OLED display module; A laser ranging module, comprising a laser ranging mounting bracket, a transmitting antenna, and a receiving antenna; Visible light imaging module; Infrared imaging module; Intelligent processing module for identifying and tracking camouflaged targets; ARM embedded module; A battery pack, used to power the observation system; Installing the multispectral imaging module and the visual module on the upper shell; The intelligent processing module, battery pack, laser ranging module, visible light imaging module, infrared light imaging module and ARM embedded module are installed on the lower shell.

2. The multifunctional multispectral portable observation system according to claim 1, characterized in that: The multispectral imaging module includes a multispectral image processing board, a multispectral sensor and a multispectral lens assembly; The multispectral sensor is used to divide the visible light band into multiple different bands to distinguish the spectral characteristics of the environment and the target radiation; The multi-spectral image processor is used to detect camouflaged targets through spectral feature response, image signal enhancement, and filtering algorithms.

3. The multifunctional multispectral portable observation system according to claim 2, characterized in that: The visible light imaging module includes a visible light mounting bracket, a visible light image sensor and a visible light lens assembly; The infrared light imaging module includes an infrared light mounting bracket, an infrared light image sensor and an infrared light lens assembly.

4. The multifunctional multispectral portable observation system according to claim 3, characterized in that: The visible light lens assembly and the infrared light lens assembly are continuously zoomable lens assemblies.

5. The multifunctional multispectral portable observation system according to claim 4, characterized in that: Also includes: GPS positioning assembly; the GPS positioning assembly includes a radome and a GPS positioning antenna; the GPS positioning assembly is fixed to the upper shell.

6. The multifunctional multispectral portable observation system according to claim 5, characterized in that: Also includes: Compass orientation module; The compass orientation module includes an electronic compass and a compass mounting bracket. The electronic compass is fixedly connected to the lower housing via the compass mounting bracket.

7. The multifunctional multispectral portable observation system according to claim 6, characterized in that: The visual module also includes a diopter adjustment mechanism and an eyepiece assembly; wherein, the diopter threshold adjusted by the diopter adjustment mechanism is (-5 to +5), and the eyepiece assembly is adjustable to 1-20 times magnification.

8. A multifunctional multispectral portable observation method, characterized in that: include: Start the GPS positioning component to obtain the current location information, calibrate the azimuth and elevation angles of the observation system through the compass orientation module, and load and adjust the working parameters of each module according to the observation mode; Synchronously collect observation data from the working modules, and perform spatiotemporal registration and fusion on the collected multi-source data; wherein the working module is at least one of a laser ranging module and a visible light imaging module, an infrared imaging module, and a multispectral imaging module; performing spatiotemporal registration on the collected multi-source data includes aligning image data, distance data, and orientation data according to time, and correcting and eliminating image pixel offsets; and performing fusion on the collected multi-source data includes mapping the distance data to the image data, superimposing GPS information and compass data, and generating georeferenced image cube data. Targets in the observation environment are identified based on the generated geo-referenced image cube data, and a structured observation report is generated.

9. The multifunctional multispectral portable observation method according to claim 8, characterized in that: Also includes: A spectral-spatial dual path is used to construct a Transformer-based multimodal fusion observation model to identify targets in the observation environment; The spectral curve of each pixel is convolved to extract the global spectral features, and each band image is independently divided into blocks to extract local spatial features. A learnable band weight parameter is added to the input layer, and the fusion layer interacts the dual-path features through cross-attention. T fused =CrossAttn(Q=T spatial ,K / V=T spectral ) T spectral =Transformer(Conv1D(I pirel ∈R B )) Among them, T fused is a dual-path feature, T spatial is the spatial feature, T spectral is the spectral feature, Q, K, V are the query, key, and value in the cross-attention mechanism CrossAttn, Conv is the convolutional layer, D is the feature dimension, and I pirel ∈R B is the learnable band weight parameter; Using a multi-scale feature pyramid, the underlying features are fused with small target features through upsampling, and DeformableAttention is used instead of standard Attention to focus on the target area: Among them, p is the reference point coordinate, Δp k is the learnable offset of the k-th sampling point, is the attention weight.

10. The multifunctional multispectral portable observation method according to claim 9, characterized in that: Also includes: Automatically switches the main imaging mode according to the ambient light intensity and dynamically adjusts the exposure parameters based on the reflectivity of the multispectral band; Extract texture information from visible light imaging and spectral features from multispectral images, generate classified images, fuse infrared thermal data with laser ranging results, identify the spatial depth of target distribution, and output interactive composite data layers.

Images