Body-aware intelligent monitoring instrument and method for monitoring targets in the qinghai-tibet plateau region

Abstract

This application discloses an embodied intelligent monitoring instrument and method for monitoring targets in the Qinghai-Tibet Plateau region, relating to the field of integrated ecological environment and meteorological observation. The method includes a sensing subsystem, an action subsystem, and a decision subsystem. The sensing subsystem is used to acquire environmental sensing data of the monitoring targets in the Qinghai-Tibet Plateau region. The action subsystem is used to receive control commands and change the spatial orientation of the action subsystem and the sensing subsystem on the action subsystem. The decision subsystem is communicatively connected to the sensing subsystem and the action subsystem, respectively, and is used to process the environmental sensing data of the monitoring targets in the Qinghai-Tibet Plateau region acquired by the sensing subsystem, generate control commands of the action subsystem, and the three-dimensional environmental field of the monitoring targets in the Qinghai-Tibet Plateau region. This application improves the efficient, accurate, and adaptive monitoring capabilities of multiple monitoring targets in the special environment of the Qinghai-Tibet Plateau region through an integrated embodied intelligent framework of "sensing-decision-action".

Description

Technical Field

[0001] This application relates to the field of integrated ecological environment and meteorological observation, and in particular to an embodied intelligent monitoring instrument and method for monitoring targets in the Qinghai-Tibet Plateau region. Background Technology

[0002] As the "Roof of the World" and the "Water Tower of Asia," the Qinghai-Tibet Plateau's ecological environment (such as snow-capped mountains, glaciers, lakes, and vegetation) and meteorological processes (such as cumulonimbus clouds) have a crucial impact on regional and even global climate. Currently, monitoring this region faces significant challenges.

[0003] Currently, the unique characteristics of the plateau environment present certain technical bottlenecks: the Qinghai-Tibet Plateau has complex terrain, harsh environment, and scarce infrastructure. Traditional single-point observations are difficult to capture spatial heterogeneity; ground-based radar is obstructed by terrain, making network construction difficult and costly; although satellite remote sensing has wide coverage, its spatiotemporal resolution is low (especially for rapidly evolving processes such as cumulonimbus clouds), and it is easily affected by cloud interference, resulting in limited accuracy of inversion products. Summary of the Invention

[0004] The purpose of this application is to provide an embodied intelligent monitoring instrument and method for monitoring targets in the Qinghai-Tibet Plateau region. Through an integrated embodied intelligent framework of "perception-decision-action", it can monitor multiple key targets in the Qinghai-Tibet Plateau in a unified platform, adaptively and with high precision.

[0005] To achieve the above objectives, this application provides the following solution: Firstly, this application provides an embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region, comprising: a perception subsystem, an action subsystem, and a decision-making subsystem.

[0006] The sensing subsystem includes a multispectral camera array, a laser active detector, and a multi-parameter profile monitoring sensor. The sensing subsystem is used to acquire environmental sensing data of monitoring targets in the Qinghai-Tibet Plateau region. The monitoring targets include cumulonimbus clouds, snow-capped mountains, glaciers, lakes, or vegetation. The action subsystem includes a first-level gimbal and a second-level gimbal. The second-level gimbal and the sensing subsystem are both mounted on the first-level gimbal. The multispectral camera array and laser active detector of the sensing subsystem are mounted on the second-level gimbal. The action subsystem is used to receive control commands and change the spatial orientation of the action subsystem and the sensing subsystem on the action subsystem. The decision subsystem is communicatively connected to the perception subsystem and the action subsystem, respectively, and is used to process the environmental perception data of the monitoring targets in the Qinghai-Tibet Plateau region acquired by the perception subsystem, generate control commands for the action subsystem, and the three-dimensional environmental field of the monitoring targets in the Qinghai-Tibet Plateau region.

[0007] In one embodiment, the decision subsystem includes a data acquisition module, a data fusion module, a data analysis module, and a control output module; The data acquisition module is configured to receive environmental perception data of the monitoring targets in the Qinghai-Tibet Plateau region acquired by the perception subsystem. The environmental perception data includes raw images acquired by a multispectral camera array, lidar point cloud data acquired by a laser active detector, and profile data acquired by a multi-parameter profile monitoring sensor. The data analysis module is configured to identify and locate the monitoring target and divide it into grids based on environmental perception data, so as to obtain the location of the monitoring target and each grid after division. The data fusion module is configured to perform three-dimensional reconstruction based on environmental perception data to obtain the three-dimensional environmental field of the monitoring target in the Qinghai-Tibet Plateau region; The control output module is configured to generate control commands for the action subsystem based on the identification and positioning results of the monitored target and the grid division results, and then send the control commands to the action subsystem.

[0008] In one embodiment, the multispectral camera array includes a wide-angle visible light camera, a telephoto visible light camera, a short-wave infrared camera, and a long-wave infrared camera.

[0009] Secondly, this application provides an embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region, executed by the aforementioned embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region, comprising: The sensing subsystem performs its first data collection, acquiring the first environmental sensing data of the monitoring targets in the Qinghai-Tibet Plateau region. The decision subsystem identifies and locates the target based on the first environmental perception data, obtains the location of the monitored target, and generates a first control command based on the location of the monitored target. The first-level pan-tilt unit points to the location of the monitored target according to the first control command; When the first-level gimbal points to the location of the monitoring target according to the first control command, the sensing subsystem performs a second data acquisition to obtain the second environmental sensing data of the monitoring target in the Qinghai-Tibet Plateau region. The decision subsystem divides the grid based on the second environmental perception data to obtain the divided grids; and generates second control commands for each divided grid. The second-level gimbal points sequentially to each of the divided grids according to the second control command; When the second-level gimbal points to each grid according to the second control command, the sensing subsystem performs a third data acquisition to obtain the third environmental sensing data of the monitoring targets in the Qinghai-Tibet Plateau region; The decision subsystem performs three-dimensional reconstruction based on the third environmental perception data to obtain the three-dimensional environmental field of the monitoring targets in the Qinghai-Tibet Plateau region.

[0010] In one embodiment, a deep learning model or image processing algorithm is used to identify and locate the first environmental perception data. The deep learning model includes a CNN (Convolutional Neural Networks) model or a Transformer model, and the image processing algorithm includes an edge detection algorithm or a threshold segmentation algorithm.

[0011] In one embodiment, the grid division includes uniformly dividing the circumscribed rectangular region of the monitoring target into an M×N grid or performing non-uniform grid division according to the shape of the monitoring target.

[0012] In one embodiment, three-dimensional reconstruction is performed based on third-party environmental perception data to obtain the three-dimensional environmental field of the monitoring target in the Qinghai-Tibet Plateau region, specifically including: Spatial interpolation is performed on the lidar point cloud data and profile data in the third environmental perception data to obtain spatial interpolation data; Based on the attitude angles of the first-level gimbal and the second-level gimbal in the action subsystem, the spatially interpolated data is geometrically corrected in the vertical direction to obtain the corrected data. The original images in the third environmental perception data are stitched together and the viewpoint is corrected to obtain the corrected image; The correction data and the corrected image are fused to obtain the initial three-dimensional field; The initial three-dimensional field is spatially interpolated in the horizontal direction to obtain the three-dimensional environment field.

[0013] In one embodiment, the spatial interpolation employs Kriging interpolation or inverse distance weighted interpolation.

[0014] In one embodiment, feature point matching and perspective transformation are used for viewpoint correction.

[0015] In one embodiment, the feature point matching operator is a SIFT, SURF, or ORB operator.

[0016] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides an embodied intelligent monitoring instrument and method for monitoring targets in the Qinghai-Tibet Plateau region. The perception subsystem acquires multi-dimensional and multi-scale environmental information through a combination of multispectral camera arrays, active laser detectors, and multi-parameter profile monitoring sensors, improving the comprehensiveness and accuracy of monitoring. The action subsystem adopts a two-level gimbal structure, which can flexibly adjust the spatial orientation of the perception subsystem to achieve autonomous tracking and scanning monitoring of dynamic targets (such as cumulonimbus clouds) or wide-area static targets (such as glaciers, lakes, and vegetation), improving monitoring efficiency and range. The decision-making subsystem can process multimodal environmental perception data in real time, automatically generate control commands to drive the action subsystem, and construct a three-dimensional environmental field of the monitored targets, providing high-precision spatiotemporal information for scientific research and disaster early warning. Through the integrated embodied intelligent framework of "perception-decision-action", it can cope with extreme conditions such as high altitude, low temperature, strong radiation, and complex terrain, improving the efficient, accurate, and adaptive monitoring capabilities of various monitoring targets in the special environment of the Qinghai-Tibet Plateau, and achieving continuous and stable monitoring of various monitoring targets such as cumulonimbus clouds, snow-capped mountains, glaciers, lakes, and vegetation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of an embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region, according to one embodiment of this application. Figure 2 This is a schematic diagram of the functional modules of an embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region, according to one embodiment of this application. Figure 3 A flowchart illustrating an embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region, provided as an embodiment of this application; Figure 4 This is a schematic diagram for interpolation processing.

[0019] Figure label: 1-Multispectral camera array; 2-Active laser detector; 3-Multi-parameter profile monitoring sensor; 4-First-stage gimbal; 5-Second-stage gimbal. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] First, the technical terms used in this application will be explained.

[0023] Currently, monitoring of the Qinghai-Tibet Plateau region still faces many challenges: The diversity of monitoring targets and the fragmentation of monitoring methods: Existing monitoring systems are usually designed for a single objective. For example, weather stations are used to monitor meteorological elements, camera stations are used to monitor glacier changes, and satellite remote sensing is used to monitor large-scale land cover. These systems are independent of each other, and their data formats and spatiotemporal references are not uniform. This makes it impossible to conduct coordinated observation and correlation analysis of elements such as "mountains-water-forest-fields-lakes-grasslands-clouds", and it is difficult to reveal the mechanisms of their interactions.

[0024] The passive nature and lack of intelligence in observation: Traditional observation equipment mostly operates in a fixed mode, lacking the closed-loop capability of "perception-decision-action". It cannot proactively adjust the observation strategy when a target (such as a rapidly developing cumulonimbus cloud) appears, and cannot achieve a seamless switch from "wide-area general survey" to "detailed investigation of key targets", resulting in low value density of observation data.

[0025] Therefore, there is an urgent need in this field for an intelligent integrated monitoring solution that can unify platforms, adapt, and coordinately monitor multiple key targets (ecological and environmental elements and meteorological processes) on the Qinghai-Tibet Plateau with high precision.

[0026] like Figures 1-2 As shown, the embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region provided in this application is based on embodied intelligence technology and includes: a perception subsystem, an action subsystem, and a decision-making subsystem.

[0027] The sensing subsystem includes a multispectral camera array 1, a laser active detector 2, and a multi-parameter profile monitoring sensor 3. The sensing subsystem is used to acquire environmental sensing data of monitoring targets in the Qinghai-Tibet Plateau region. The monitoring targets include cumulonimbus clouds, snow-capped mountains, glaciers, lakes, or vegetation.

[0028] The action subsystem includes a first-level gimbal 4 and a second-level gimbal 5. The second-level gimbal 5 and the sensing subsystem are both installed on the first-level gimbal 4. The multispectral camera array 1 and the laser active detector 2 of the sensing subsystem are installed above the second-level gimbal 5. The action subsystem is used to receive control commands and change the spatial orientation of the action subsystem and the sensing subsystem on the action subsystem.

[0029] The decision subsystem is communicatively connected to the perception subsystem and the action subsystem, respectively, and is used to process the environmental perception data of the monitoring targets in the Qinghai-Tibet Plateau region acquired by the perception subsystem, generate control commands for the action subsystem, and the three-dimensional environmental field of the monitoring targets in the Qinghai-Tibet Plateau region.

[0030] The perception subsystem of this application acquires multi-dimensional and multi-scale environmental information through a combination of a multispectral camera array 1, a laser active detector 2, and a multi-parameter profile monitoring sensor 3, thereby improving the comprehensiveness and accuracy of monitoring. The action subsystem adopts a two-level gimbal structure, which can flexibly adjust the spatial orientation of the perception subsystem to achieve autonomous tracking and scanning monitoring of dynamic targets (such as cumulonimbus clouds) or wide-area static targets (such as glaciers, lakes, and vegetation), thereby improving monitoring efficiency and range. The decision-making subsystem can process multimodal environmental perception data in real time, automatically generate control commands to drive the action subsystem, and construct a three-dimensional environmental field of the monitored targets, providing high-precision spatiotemporal information for scientific research and disaster early warning. Through the embodied intelligent framework integrating "perception-decision-action", it can cope with extreme conditions such as high altitude, low temperature, strong radiation, and complex terrain, improve the efficient, accurate, and adaptive monitoring capabilities of multiple monitoring targets in the special environment of the Qinghai-Tibet Plateau, and achieve continuous and stable monitoring of multiple monitoring targets such as cumulonimbus clouds, snow-capped mountains, glaciers, lakes, and vegetation.

[0031] In another exemplary embodiment of this application, the decision subsystem includes a data acquisition module, a data fusion module, a data analysis module, and a control output module.

[0032] The data acquisition module is configured to receive environmental perception data of the monitoring targets in the Qinghai-Tibet Plateau region acquired by the perception subsystem. The environmental perception data includes raw images acquired by the multispectral camera array 1, lidar point cloud data acquired by the laser active detector 2, and profile data acquired by the multi-parameter profile monitoring sensor 3.

[0033] The data analysis module is configured to identify and locate monitoring targets and divide them into grids based on environmental perception data, thereby obtaining the location of the monitoring targets and the divided grids.

[0034] The data fusion module is configured to perform three-dimensional reconstruction based on environmental perception data to obtain the three-dimensional environmental field of the monitoring target in the Qinghai-Tibet Plateau region.

[0035] The control output module is configured to generate control commands for the action subsystem based on the identification and positioning results of the monitored target and the grid division results, and then send the control commands to the action subsystem.

[0036] In another exemplary embodiment of this application, the multispectral camera array 1 includes a wide-angle visible light camera, a telephoto visible light camera, a short-wave infrared camera, and a long-wave infrared camera.

[0037] This embodied intelligent monitoring instrument adopts an integrated hardware and software design, specifically including: a perception subsystem, an action subsystem, and a decision-making subsystem. The perception subsystem is used to acquire multimodal environmental data; the action subsystem is used to change the spatial orientation of the perception subsystem; the decision-making subsystem is communicatively connected to both the perception and action subsystems; the action subsystem includes a first-level gimbal 4 and a second-level gimbal 5. The first-level gimbal 4 carries all components of the perception subsystem, and the second-level gimbal 5 is mounted on the first-level gimbal 4 and carries some of the high-resolution sensor components of the perception subsystem; the decision-making subsystem is configured to control the perception and action subsystems to execute an embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region.

[0038] The perception subsystem includes a biomimetic eagle-eye multispectral camera array (wide-angle visible light, long-focus visible light, short-wave infrared, and long-wave infrared cameras), a laser active detector 2 (1064nm laser emission, APD reception, and echo processing module), and a multi-parameter profile monitoring sensor 3 (multi-wavelength laser, beam splitting reception, and echo inversion module). All sensors are mounted on the action subsystem.

[0039] Action Subsystem: Composed of a two-stage gimbal structure. First-stage gimbal 4 (slow-speed gimbal): Serves as the base, supporting the entire perception subsystem. It is responsible for large-scale, low-speed attitude adjustments to achieve coarse target alignment. Second-stage gimbal 5 (high-speed gimbal): Nested above the first-stage gimbal 4, it moves independently. It houses a biomimetic eagle-eye multispectral camera array and high-precision sensors such as the laser active detector 2. It is responsible for small-scale, high-speed, and precise pointing to achieve fine-grained grid scanning. Position and Connection: The two gimbals are connected in series via a mechanical structure and are both connected to the decision subsystem via cables / buses to receive control commands.

[0040] Decision Subsystem: Includes the edge computer and the decision software running on it. Software modules include: Data Acquisition Module: Asynchronously receives data from all sensors. Data Fusion Module: Performs spatiotemporal registration and multi-source data fusion. Data Analysis Module: Executes core algorithms such as target recognition, localization, mesh generation, and 3D reconstruction. Control Output Module: Generates and sends control commands to the two-stage gimbal.

[0041] System connectivity: All sensors in the perception subsystem and the two-level gimbals in the action subsystem are connected to the edge computer in the decision subsystem, forming a complete "perception-decision-action" closed-loop feedback system.

[0042] Optionally, the sensing subsystem is used to sense various parameters of cumulonimbus clouds, including a biomimetic eagle-eye multispectral camera array, a laser active detector 2, and a multi-parameter profile monitoring sensor 3. The biomimetic eagle-eye multispectral camera array includes a wide-angle visible light camera, a telephoto visible light camera, a short-wave infrared camera, and a long-wave infrared camera, outputting two-dimensional morphology, humidity, and temperature information of the cumulonimbus clouds. The laser active detector 2 includes a 1064nm infrared laser emitting module, an APD (avalanche diode) receiving module, and an echo signal processing module, outputting information such as cloud base height and cloud thickness of the cumulonimbus clouds. The multi-parameter profile monitoring sensor 3 includes a multi-wavelength high-repetition-rate high-energy laser, a spectrometer receiving module, and a multi-wavelength echo acquisition and inversion module, outputting profile information such as temperature, humidity, and particle size distribution of the cumulonimbus clouds.

[0043] The action subsystem is used to change the monitoring direction of the perception subsystem. It includes two-stage gimbals. The first-stage gimbal 4 is a slow-speed gimbal, which carries the hardware modules of the entire perception subsystem. The second-stage gimbal 5 is a high-speed gimbal, which carries the biomimetic eagle eye multispectral camera array and laser active detector 2. The second-stage gimbal 5 is located on top of the first-stage gimbal 4.

[0044] The decision subsystem is used to process the data of the perception subsystem and to issue control commands to the action subsystem based on the processing results. It includes an edge computer and decision software running on it. The decision software includes a data acquisition module, a data fusion module, a data analysis module, and a control output module.

[0045] All sensors of the perception subsystem and the two-stage gimbal of the action subsystem are connected to the decision subsystem and are controlled by it.

[0046] like Figure 3As shown, this application also provides an embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region, executed by the aforementioned embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region, including the following steps 201 to 208.

[0047] Step 201: The sensing subsystem performs its first data collection to acquire the first environmental sensing data of the monitoring targets in the Qinghai-Tibet Plateau region.

[0048] Step 202: The decision subsystem identifies and locates the target based on the first environmental perception data, obtains the location of the monitored target, and generates a first control command based on the location of the monitored target.

[0049] Step 203: The first-level gimbal 4 points to the location of the monitored target according to the first control command.

[0050] Step 204: When the first-level gimbal 4 points to the location of the monitoring target according to the first control command, the sensing subsystem performs a second data acquisition to obtain the second environmental sensing data of the monitoring target in the Qinghai-Tibet Plateau region.

[0051] Step 205: The decision subsystem divides the grid according to the second environmental perception data to obtain each grid after division; and generates a second control command for each grid after division.

[0052] Step 206: The second-level gimbal 5 points to each of the divided grids in sequence according to the second control command.

[0053] Step 207: When the second-level gimbal 5 points to each grid according to the second control command, the sensing subsystem performs the third data acquisition to obtain the third environmental sensing data of the monitoring targets in the Qinghai-Tibet Plateau region.

[0054] Step 208: The decision subsystem performs three-dimensional reconstruction based on the third environmental perception data to obtain the three-dimensional environmental field of the monitoring target in the Qinghai-Tibet Plateau region.

[0055] This application, through steps 201-203, obtains the location of the monitoring target from wide-area, multimodal first environmental perception data, thereby pointing the first-level gimbal 4 to the location of the monitoring target, achieving rapid initial screening and approximate positioning of the monitoring target. This avoids the time and energy waste of manual searching or blind scanning, greatly improving the initial response speed of monitoring operations, and is particularly suitable for tracking rapidly changing cumulonimbus clouds or locating glaciers, lakes, and vegetation over a large area. Through steps 204-206, the second environmental perception data obtained from the coarse pointing of the first-level gimbal 4 is used to divide the data into grids, thereby pointing the first-level gimbal 4 to each of the divided grids. After the first-level gimbal 4 completes the large-angle coarse pointing, the second-level gimbal 5 performs fine pointing within a small area, at a high frequency, and with high precision. Combining the advantages of two types of gimbals facilitates the acquisition of high-quality data. Through steps 207-208, based on the third environmental perception data obtained from the second-level gimbal 5, high-precision and high-resolution data are provided for 3D reconstruction, resulting in the 3D environmental field of the monitoring target in the Qinghai-Tibet Plateau region. Through the closed-loop operation of "perception-decision-action," no manual intervention is required, realizing full-process automation from "finding the target" to "producing a 3D model." The physical capabilities of the mechanical system (two-level gimbal) and the decision-making logic of the software system (three-level acquisition strategy) are deeply integrated, forming a set of efficient, accurate, and adaptive monitoring solutions for the vast, complex, and harsh environment of the Qinghai-Tibet Plateau, greatly improving the intelligence level and practical value of environmental perception and data acquisition.

[0056] In another exemplary embodiment of this application, a deep learning model or an image processing algorithm is used to identify and locate the first environmental perception data. The deep learning model includes a CNN model or a Transformer model, and the image processing algorithm includes an edge detection algorithm or a threshold segmentation algorithm.

[0057] In another exemplary embodiment of this application, the grid division includes uniformly dividing the circumscribed rectangular region of the monitoring target into an M×N grid or performing non-uniform grid division according to the shape of the monitoring target.

[0058] In another exemplary embodiment of this application, the spatial interpolation employs Kriging interpolation or inverse distance weighted interpolation.

[0059] In another exemplary embodiment of this application, feature point matching and perspective transformation are used for viewpoint correction.

[0060] In another exemplary embodiment of this application, the feature point matching operator is a SIFT, SURF, or ORB operator.

[0061] Optionally, the embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region includes: performing a wide-area scan through a sensing subsystem to acquire initial multimodal sensing data; identifying and locating at least one monitoring target based on the initial multimodal sensing data; controlling a first-level gimbal 4 of the action subsystem to point at the monitoring target; after the first-level gimbal 4 points at the monitoring target, acquiring medium-resolution regional data of the monitoring target through the sensing subsystem; dividing the region of the monitoring target into grids based on the medium-resolution regional data; controlling a second-level gimbal 5 of the action subsystem to sequentially point at each grid after division; acquiring high-resolution data of the corresponding grid through a high-resolution sensor component of the sensing subsystem when the second-level gimbal 5 points at each grid; and fusing the high-resolution data of all grids to generate three-dimensional environmental field information of the monitoring target.

[0062] This embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region is applicable to a variety of monitoring objects (including cumulonimbus clouds, snow-capped mountains, glaciers, lakes, or vegetation), and includes the following steps: S1: Wide-area sensing and data acquisition The perception subsystem operates in a default scanning mode or a fixed posture to acquire wide-area, multimodal initial perception data (or first environmental perception data).

[0063] The initial sensing data (or first environmental sensing data) includes, but is not limited to: visible light images (acquired by a wide-angle visible light camera): used for target shape, texture, and contour recognition; multispectral / hyperspectral images (acquired by a short-wave infrared camera): used for material composition recognition (such as water vapor content, chlorophyll content, and snow and ice coverage); lidar point cloud data (acquired by a lidar active detector 2): used to obtain distance, cloud height, and terrain elevation; and profile data (acquired by a multi-parameter profile monitoring sensor 3): used to obtain the vertical distribution of parameters such as temperature, humidity, and aerosols.

[0064] Step S1 is the "raw accumulation" of data, which obtains a wide-area, multimodal, but potentially low-resolution "panoramic snapshot," providing a data foundation for subsequent target selection. The data is cached after being uniformly timestamped and spatially referenced, and is not immediately deleted for future correlation analysis.

[0065] S2: Target Identification and Initial Localization The decision subsystem performs fusion analysis on the initial perception data (or first environmental perception data) obtained in step S1, identifies the monitoring targets of interest (such as cumulonimbus clouds, snow mountain outlines, glacier fronts, lake boundaries, and vegetation areas), and determines their initial spatial locations.

[0066] Specific methods: Input: Multimodal data stream in S1. Processing: 1) Feature extraction: Extract potential target regions from visible light images using pre-defined deep learning models (such as CNN, Transformer) or traditional image processing algorithms (such as edge detection, threshold segmentation). 2) Multimodal verification: Verify using other modal data. For example, after identifying a cloud region, calculate its average water vapor content using shortwave infrared data; after identifying a suspected glacier region, verify its elevation and surface roughness using lidar data. 3) Target determination: Determine the target based on predefined criteria related to the target type. For example, cumulonimbus cloud criteria: Tall, full texture features in the visible light image, and shortwave infrared water vapor content ≥ threshold W1. Glacier / snow peak criteria: Bright, continuous region in the visible light image, elevation retrieved by lidar ≥ threshold H1, and shortwave infrared spectral features consistent with ice and snow characteristics. Lake criteria: Dark, smooth, closed region in the visible light image, and extremely low near-infrared reflectivity. Output: The type label of the monitored target and its initial bounding box or centroid position in the wide-angle camera coordinate system (i.e., the initial center position that the first-stage gimbal 4 needs to point to). Data from different targets can be processed either independently using dedicated models or simultaneously through a multi-task learning model to achieve synergy. For example, when identifying cumulonimbus clouds, temperature changes in the glacier area downwind can be monitored simultaneously.

[0067] S3: Precise Target Location and Processing The decision subsystem further analyzes the identified targets and calculates the precise pointing position used to guide the gimbal for detailed observation.

[0068] Processed parameters: The parameters processed in this step are common and are usually the spatial distribution characteristics of the target. For example, for cumulonimbus clouds, it might be the region with the largest gradient or centroid in the water vapor content map. For glaciers, it might be the center of the advancing glacier tongue. For lakes, it might be a temperature anomaly region (such as a thermal pollution point) or an algal bloom region. Processed result: One or more key location coordinates. These coordinates can be more precise locations extracted from the initial region in step S2 through clustering algorithms (such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise)), centroid calculation, or gradient analysis.

[0069] S4: Primary gimbal coarse direction The decision subsystem controls the first-level (slow) gimbal of the action subsystem to roughly align the entire perception subsystem platform with the precise pointing position calculated in step S3. This step completes the first action loop from "target detection" to "target alignment".

[0070] S5: Medium-resolution scanning of the target area After the first-level gimbal 4 is repositioned, the perception subsystem uses its onboard sensors suitable for medium-resolution scanning (such as a wide-angle visible light camera and a multi-parameter profile monitoring sensor 3) to perceive the target area again.

[0071] The differences and focus between Step S1 and Step S5 are as follows: Step S1 is a "wide-area census," with low resolution, and its objective is "what is there." Step S5 is a "detailed regional survey," conducted under the premise that the target's existence is known and has been initially aligned. It focuses on the target itself and its surrounding area, acquiring data with higher spatial resolution (due to closer distance or more accurate pointing), and its objective is "what is the current state of the target," providing a basis for the next step of grid division. The data from Step S1 is retained as background and reference and will not be overwritten.

[0072] S6: Target Area Grid Decomposition The decision subsystem divides the target area into several grids based on the higher resolution target area data (such as visible light images) obtained in step S5.

[0073] General steps: Mesh generation can employ general algorithms, such as regular mesh generation (uniformly dividing the target's circumscribed rectangular region into an M×N grid) or irregular mesh generation (adaptively dividing based on the target's shape). The granularity of the mesh can be dynamically adjusted according to the target's size, distance from the instrument, and required observation accuracy.

[0074] S7: Secondary gimbal precision pointing Based on the grid division results from step S6, the decision subsystem controls the second-level (high-speed) gimbal of the action subsystem, ensuring that it points precisely and sequentially to the center of each grid. The second-level gimbal 5 is equipped with higher-precision sensors.

[0075] S8: High-resolution grid data acquisition When the second-level gimbal 5 is pointed at each grid, the sensing subsystem uses its onboard high-resolution sensors (such as a long-range visible light camera, a short-wave infrared camera, a long-wave infrared camera, and a laser active detector 2) to collect detailed data of that grid.

[0076] Differences and focus from step S1: Steps S1 and S5 are scanning of "areas", while step S8 is a fine measurement of "points".

[0077] The sensors used in step S8 typically have narrower field of view and higher spatial / spectral resolution, designed to acquire the finest physical and chemical parameters within each grid cell. For example, a telephoto camera can reveal cloud microstructures, while the laser active detector 2 can obtain precise distance information for that grid cell. The data from steps S1 and S5 are retained as information at different levels and fused with the data from step S8 to form a multi-scale observation dataset.

[0078] S9: Multi-source data fusion and 3D environment field reconstruction The decision subsystem fuses the multi-source data from all grids collected in step S8 to generate three-dimensional environmental field information of the object to be monitored.

[0079] The parameters covered by the three-dimensional environmental field include: in addition to three-dimensional morphology, temperature, and humidity fields, they can also include: reflectivity field (inverted from visible light / multispectral images), effective particle radius field (inverted from multispectral / LiDAR data), optical thickness field (inverted from LiDAR and infrared data), material phase distribution field (such as water, ice, and mixed phases, determined by multispectral and temperature data), leaf area index (LAI), vegetation cover (for vegetation), and surface velocity field (for glaciers or lakes, tracked through continuous images).

[0080] Specifically, using cumulonimbus clouds as the monitoring target: S1: Wide-angle visible light camera and short-wave infrared camera perform all-day sky imaging. S2: Identify cloud areas; combining with the short-wave infrared water vapor channel, when the water vapor content > W1, it is determined to be a cumulonimbus cloud, and its initial range is located. S3: Cluster the water vapor map, find the largest connected region, and use its centroid as the precise pointing location. S4-S5: The first-level gimbal 4 points to this centroid, and the wide-angle camera and multi-parameter profiler scan the cumulonimbus cloud. S6: Based on the visible light image from S5, the cumulonimbus cloud region is divided into a 10x10 grid. S7-S8: The second-level gimbal 5 points to each grid sequentially, and the telephoto camera, long-wave / short-wave infrared camera, and lidar respectively acquire high-resolution images, temperature / humidity, and cloud height and thickness data. S9: Fuse all data to generate the 3D morphology, temperature, humidity, and condensate phase field of the cumulonimbus cloud.

[0081] In one specific implementation, the process for embodied intelligent monitoring of cumulonimbus clouds in the Qinghai-Tibet Plateau region includes: The first step is cumulonimbus cloud identification. The decision subsystem's data acquisition module reads image data acquired by the wide-angle visible light camera and short-wave infrared images acquired by the short-wave infrared camera in the perception subsystem. The data analysis module analyzes the two types of images to determine whether the observed cloud is a cumulonimbus cloud. The cumulonimbus cloud determination is based on the following: the built-in recognition module of the decision subsystem identifies the cloud region in the acquired visible light image; then, it further confirms the average water content in the region through short-wave infrared water vapor channel imaging. When the average water content is greater than or equal to a set content threshold, the region is marked as a cumulonimbus cloud. The control output module then outputs control commands to the action subsystem. The action subsystem controls the first-level gimbal 4 (low-speed gimbal) in the action subsystem to point to the center of the cumulonimbus cloud according to the commands. The method for obtaining the center position is as follows: Clustering is performed using the peak position of the water vapor channel obtained by the shortwave red camera as the center. The diameters of each cluster are sorted, and the center position of the cluster with the largest diameter is taken as the center point, with the diameter of that cluster as the center position.

[0082] The second step is cumulonimbus cloud tracking. The decision subsystem data acquisition module acquires image data of the cumulonimbus cloud region from the wide-angle camera in the perception subsystem, and acquires multi-parameter profile data collected by the multi-parameter profile monitoring sensor 3. The decision subsystem data analysis module divides the cumulonimbus cloud region into grids.

[0083] The third step involves cumulonimbus cloud data acquisition. The decision subsystem control output module controls the second-level gimbal 5 (high-speed gimbal) in the action subsystem to point to each grid according to the division results, thereby acquiring long-focus camera images, short-wave infrared images, long-wave infrared images, and laser active detector 2 data of the cumulonimbus clouds in the current grid area.

[0084] The fourth step involves cumulonimbus cloud data processing. The data fusion module of the decision subsystem fuses the data obtained in step three to obtain the three-dimensional morphology, temperature, and humidity field information of the cumulonimbus cloud. The method for generating the three-dimensional morphology is as follows: The cloud thickness data acquired by the laser active detector 2 of the perception subsystem is interpolated to improve distance resolution; the cloud thickness data is vertically corrected according to the angle between the two-stage gimbal pointing and the horizontal plane of the gimbal base; the telephoto camera grid image is stitched and the viewing angle is corrected; the image and cloud thickness data are fused; and the horizontal cloud thickness data is interpolated to fill in gaps.

[0085] The method for generating a three-dimensional temperature field is as follows: The temperature data acquired by the multi-parameter profile monitoring sensor 3 of the aforementioned sensing subsystem is interpolated to improve the resolution of the distance distribution; the temperature data is vertically corrected according to the angle between the two-stage gimbal pointing and the horizontal plane of the gimbal base; long-wave infrared grid image stitching and viewing angle correction are performed; long-wave infrared images and temperature data are fused; and horizontal temperature data is interpolated to fill in gaps.

[0086] The method for generating a three-dimensional humidity field is as follows: The humidity data acquired by the multi-parameter profile monitoring sensor 3 of the aforementioned sensing subsystem is interpolated to improve the resolution in distance distribution; the humidity data is vertically corrected based on the angle between the two-stage gimbal pointing and the horizontal plane of the gimbal base; short-wave infrared grid image stitching and viewing angle correction are performed; short-wave infrared images and humidity data are fused; and horizontal humidity data is interpolated to fill in gaps.

[0087] Specifically, with snow-capped mountains / glaciers as the monitoring targets: S1 / S2 identifies areas with low brightness temperature and high elevation; S3 locates the glacier tongue leading edge or snow line position; S8 focuses on lidar measurement of topographic elevation and shortwave infrared measurement of snow and ice coverage; S9 generates a high-precision digital elevation model (DEM) and a three-dimensional dynamic map of snow and ice melting.

[0088] Specifically, with lakes as the monitoring target: S1 / S2 identifies water body boundaries; S3 locates temperature anomaly zones or algal bloom zones; S8 focuses on measuring lake surface temperature with long-wave infrared and water quality parameters with short-wave infrared / multispectral; S9 generates a three-dimensional distribution field of lake surface temperature and chlorophyll concentration.

[0089] Specifically, the monitoring targets are trees (vegetation): S1 / S2 identify areas with high vegetation indices (such as NDVI (Normalized Difference Vegetation Index)); S3 locates forest gaps or specific tree species communities; S8 focuses on multispectral cameras and high-resolution lidar; S9 generates three-dimensional vegetation canopy structure and biomass distribution maps.

[0090] In another exemplary embodiment of this application, a three-dimensional reconstruction is performed based on third environmental perception data to obtain a three-dimensional environmental field of the monitoring target in the Qinghai-Tibet Plateau region, which is replaced by the following steps 301 to 305:

[0091] Step 301: Spatial interpolation is performed on the lidar point cloud data and profile data in the third environmental perception data to obtain spatial interpolation data; Step 302: Based on the attitude angles of the first-level gimbal 4 and the second-level gimbal 5 in the action subsystem, perform vertical geometric correction on the spatially interpolated data to obtain the corrected data. Step 303: The original image in the third environmental perception data is stitched together and the viewpoint is corrected to obtain the corrected image; Step 304: Fuse the correction data with the corrected image to obtain the initial three-dimensional field; Step 305: Perform spatial interpolation in the horizontal direction on the initial three-dimensional field to obtain the three-dimensional environment field.

[0092] The method for generating the three-dimensional environmental field (information) in the aforementioned embodied intelligent monitoring method includes: 1. Acquiring environmental data: The environmental data includes, but is not limited to, cloud thickness, temperature, humidity, aerosol backscattering coefficient, and ground elevation, and is collected by the laser active detector 2 and the multi-parameter profile monitoring sensor 3.

[0093] 2. Data Interpolation: Interpolating sparse point or profile data improves spatial resolution. Kriging interpolation or Inverse Distance Weighting (IDW) interpolation can be used.

[0094] The formula for calculating inverse distance weighted interpolation is: .

[0095] .

[0096] in, coordinates The interpolation result at that point, Let be the observation value of the i-th known sample point, and let the coordinates of this sample point be . , Let i be the weight of the i-th known sample point. The total number of sample points. For distance, It is an exponential parameter.

[0097] 3. Vertical Correction: Geometric correction is performed on the measured values ​​based on the gimbal's pitch angle α and horizontal angle β. The sensor measurement value L is in the line-of-sight direction and needs to be corrected to the vertical height Z.

[0098] The calculation formula is: α) (Assuming the horizontal plane is the reference).

[0099] like Figure 4 As shown, the formula for calculating interpolation Li is: Where α is the pitch angle of the gimbal, β is the horizontal angle, and L is the value within the grid acquired by the sensor, which is interpolated and compensated using an exponential distribution.

[0100] 4. Image Stitching and Viewpoint Correction: The images of each grid are stitched together to form a complete target image, and distortions caused by perspective and gimbal angle are corrected. Feature point matching (such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), ORB (Oriented Fast and Rotated BRIEF)) and perspective transformation can be used.

[0101] The perspective transformation matrix is: Where H is a 3x3 homography matrix, Input point (original coordinates) The output point (transformed homogeneous coordinates).

[0102] 5. Data Fusion: The corrected image (reflecting texture and radiometric information) is fused with the corrected environmental data (reflecting physical parameters) in a unified spatial coordinate system. For example, each image pixel is assigned a depth value (from LiDAR) and a temperature value (from profiler interpolation) to construct a preliminary three-dimensional field.

[0103] The formal expression is as follows: For any point P(x,y,z) in space, its attribute set is Attr(P)={RGB, T, RH, ...}, which comes from the mapping f_sensor:P→value of different sensors. Here, Attr(P) is the attribute set of the spatial point P, RGB is the color information from the image, T is the temperature, RH is the humidity, f_sensor is the sensor mapping function, and value is the measured value.

[0104] 6. Data Fusion: Gap Data Filling: For the gaps that still exist after fusion (such as occluded parts), spatial interpolation is performed using the surrounding data (such as the interpolation method in step 2) to generate a complete and continuous three-dimensional environment field.

[0105] Optionally, generating three-dimensional environmental field information includes: acquiring raw environmental data (or third-party environmental sensing data) collected by the sensing subsystem; performing spatial interpolation on the raw environmental data (or third-party environmental sensing data) to obtain high-resolution environmental data (or spatially interpolated data); performing vertical geometric correction on the high-resolution environmental data according to the gimbal attitude angle of the action subsystem to obtain vertically corrected high-resolution environmental data (or corrected data); stitching and viewing angle correction on the grid images collected by the sensing subsystem to obtain a corrected grid image (or corrected image); fusing the corrected grid image (or corrected image) with the vertically corrected high-resolution environmental data (or corrected data) to obtain an initial three-dimensional environmental field (or initial three-dimensional field); and performing horizontal interpolation on the missing data in the initial three-dimensional environmental field to obtain complete three-dimensional environmental field information.

[0106] This application relates to an embodied intelligent monitoring instrument and method applicable to the complex environment of the Qinghai-Tibet Plateau, which enables adaptive and refined monitoring of multiple targets (including but not limited to cumulonimbus clouds, snow-capped mountains, glaciers, lakes, and vegetation). The core of this application lies in the realization of collaborative control of multiple heterogeneous sensors on a general platform through an integrated embodied intelligent framework of "perception-decision-action". This allows for adaptive and automated monitoring of multiple targets such as cumulonimbus clouds, snow-capped mountains, glaciers, lakes, and trees, from discovery, identification, and localization to fine 3D reconstruction.

[0107] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region, characterized in that: include: The sensing subsystem includes a multispectral camera array, a laser active detector, and a multi-parameter profile monitoring sensor. The sensing subsystem is used to acquire environmental sensing data of monitoring targets in the Qinghai-Tibet Plateau region. The monitoring targets include cumulonimbus clouds, snow-capped mountains, glaciers, lakes, or vegetation. The action subsystem includes a first-level gimbal and a second-level gimbal. The second-level gimbal and the sensing subsystem are both mounted on the first-level gimbal. The multispectral camera array and laser active detector of the sensing subsystem are mounted on the second-level gimbal. The action subsystem is used to receive control commands and change the spatial orientation of the action subsystem and the sensing subsystem on the action subsystem. The decision subsystem is communicatively connected to the perception subsystem and the action subsystem, respectively, and is used to process the environmental perception data of the monitoring targets in the Qinghai-Tibet Plateau region acquired by the perception subsystem, generate control commands for the action subsystem, and the three-dimensional environmental field of the monitoring targets in the Qinghai-Tibet Plateau region.

2. The embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region according to claim 1, characterized in that, The decision-making subsystem includes a data acquisition module, a data fusion module, a data analysis module, and a control output module; The data acquisition module is configured to receive environmental perception data of the monitoring targets in the Qinghai-Tibet Plateau region acquired by the perception subsystem. The environmental perception data includes raw images acquired by a multispectral camera array, lidar point cloud data acquired by a laser active detector, and profile data acquired by a multi-parameter profile monitoring sensor. The data analysis module is configured to identify and locate monitoring targets and divide them into grids based on environmental perception data, thereby obtaining the location of the monitoring targets and the grids after division. The data fusion module is configured to perform three-dimensional reconstruction based on environmental perception data to obtain the three-dimensional environmental field of the monitoring target in the Qinghai-Tibet Plateau region; The control output module is configured to generate control commands for the action subsystem based on the identification and positioning results of the monitored targets and the grid division results, and then send the control commands to the action subsystem.

3. The embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region according to claim 1, characterized in that, The multispectral camera array includes a wide-angle visible light camera, a telephoto visible light camera, a short-wave infrared camera, and a long-wave infrared camera.

4. An embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region, executed by the embodied intelligent monitoring instrument for monitoring targets in the Qinghai-Tibet Plateau region as described in any one of claims 1-3, characterized in that, include: The sensing subsystem performs its first data collection, acquiring the first environmental sensing data of the monitoring targets in the Qinghai-Tibet Plateau region. The decision subsystem identifies and locates the target based on the first environmental perception data, obtains the location of the monitored target, and generates a first control command based on the location of the monitored target. The first-level pan-tilt unit points to the location of the monitored target according to the first control command; When the first-level gimbal points to the location of the monitoring target according to the first control command, the sensing subsystem performs a second data acquisition to obtain the second environmental sensing data of the monitoring target in the Qinghai-Tibet Plateau region. The decision subsystem divides the grid based on the second environmental perception data, resulting in the divided grids. The second control command is then generated for each of the divided grids; The second-level gimbal points sequentially to each of the divided grids according to the second control command; When the second-level gimbal points to each grid according to the second control command, the sensing subsystem performs a third data acquisition to obtain the third environmental sensing data of the monitoring targets in the Qinghai-Tibet Plateau region; The decision subsystem performs three-dimensional reconstruction based on the third environmental perception data to obtain the three-dimensional environmental field of the monitoring targets in the Qinghai-Tibet Plateau region.

5. The embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region according to claim 4, characterized in that, The first environmental perception data is identified and located using a deep learning model or an image processing algorithm. The deep learning model includes a CNN model or a Transformer model, and the image processing algorithm includes an edge detection algorithm or a threshold segmentation algorithm.

6. The embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region according to claim 4, characterized in that, The grid division includes uniformly dividing the outer rectangular area of ​​the monitoring target into an M×N grid or performing non-uniform grid division according to the shape of the monitoring target.

7. The embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region according to claim 4, characterized in that, Based on the third-party environmental perception data, a three-dimensional reconstruction is performed to obtain the three-dimensional environmental field of the monitoring targets in the Qinghai-Tibet Plateau region, specifically including: Spatial interpolation is performed on the lidar point cloud data and profile data in the third environmental perception data to obtain spatial interpolation data; Based on the attitude angles of the first-level gimbal and the second-level gimbal in the action subsystem, the spatially interpolated data is geometrically corrected in the vertical direction to obtain the corrected data. The original images in the third environmental perception data are stitched together and the viewpoint is corrected to obtain the corrected image; The correction data and the corrected image are fused to obtain the initial three-dimensional field; The initial three-dimensional field is spatially interpolated in the horizontal direction to obtain the three-dimensional environment field.

8. The embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region according to claim 7, characterized in that, The spatial interpolation uses Kriging interpolation or inverse distance weighted interpolation.

9. The embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region according to claim 7, characterized in that, Feature point matching and perspective transformation are used for viewpoint correction.

10. The embodied intelligent monitoring method for monitoring targets in the Qinghai-Tibet Plateau region according to claim 9, characterized in that, The feature point matching operator uses SIFT, SURF, or ORB operators.

Images