Ecological environment monitoring processing method and device, storage medium and equipment

Abstract

The application discloses an ecological environment monitoring processing method and device, a storage medium and equipment, relates to the technical field of big data processing, and mainly aims to solve the problem of low processing efficiency of existing ecological environment monitoring. It comprises the following steps: acquiring multi-modal data collected by an ecological environment; performing fusion processing on the multi-modal data based on a multi-modal basic model to obtain an environmental state vector after fusion, wherein the multi-modal basic model comprises a visual feature encoding unit, a time sequence feature encoding unit, a numerical feature encoding unit and a cross-modal fusion unit; performing abnormal monitoring on the environmental state vector based on an abnormal monitoring strategy to obtain a monitoring result, and generating monitoring disposal information based on the monitoring result and an environmental constraint condition, wherein the monitoring disposal information is used to represent an environmental disposal strategy executable based on the monitoring result.

Description

Technical Field

[0001] This application relates to the field of big data processing technology, and in particular to a processing method, apparatus, storage medium and equipment for ecological environment monitoring. Background Technology

[0002] Ecological and environmental protection has become a major global challenge. The health of various ecosystems such as forests, wetlands, rivers, and oceans directly affects the stability of the global carbon cycle, biodiversity, and the sustainable development of human society. Efficiently carrying out ecological and environmental monitoring and pollution emergency response is a core requirement for ensuring ecological security and promoting green development.

[0003] Currently, remote sensing technology, drones, amphibious robots, and various types of sensing equipment are widely used, enabling real-time acquisition of multimodal ecological and environmental data such as images, videos, gas concentrations, and water quality indicators, providing hardware support for comprehensive, dynamic, and refined monitoring. In the process of pollution incident identification, decentralized technologies such as sensor monitoring, drone data collection, and on-site manual or robotic execution can be used for human monitoring of pollution incidents. However, multi-source, multimodal ecological data are complex and heterogeneous in format, lacking efficient integration and unified management mechanisms. The value of this data is difficult to fully exploit, and the tasks of monitoring, data collection, and execution are scattered and independent. There is no collaborative scheduling system for terminal equipment such as sensors, drones, and robots, and it is impossible to quickly assess pollution incidents and generate accurate response plans. Emergency response heavily relies on on-site manual operations, resulting in low response efficiency and poor coordination, thus leading to low processing efficiency in ecological and environmental monitoring. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, storage medium and equipment for ecological environment monitoring, the main purpose of which is to solve the problem of low processing efficiency in existing ecological environment monitoring.

[0005] According to one aspect of this application, a method for processing ecological environment monitoring data is provided, comprising: Acquire multimodal data collected on the ecological environment; The multimodal data is fused based on a multimodal base model to obtain a fused environmental state vector. The multimodal base model includes a visual feature coding unit, a temporal feature coding unit, a numerical feature coding unit, and a cross-modal fusion unit. Anomaly monitoring is performed on the environmental state vector based on the anomaly monitoring strategy to obtain monitoring results. Based on the monitoring results and environmental constraints, monitoring and disposal information is generated. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results.

[0006] Furthermore, before fusing the multimodal data based on the multimodal fundamental model to obtain the fused environmental state vector, the method further includes: A multimodal base model is constructed, comprising a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit. The visual feature encoding unit is used to extract spatial semantic information from image data and video data. The temporal feature encoding unit is used to extract temporal evolution features from sensor data. The numerical feature encoding unit is used to process the input modal data. The cross-modal fusion unit is used to map different modalities to a unified semantic space and generate a fused environmental state representation. The multimodal base model is trained based on environmental sample data to obtain a fully trained multimodal base model.

[0007] Furthermore, the fusion processing of the multimodal data based on the multimodal fundamental model to obtain the fused environmental state vector includes: The environmental sample data is feature extracted by the visual feature encoding unit, the temporal feature encoding unit, and the numerical feature encoding unit, respectively. The extracted modal features are mapped to a unified semantic representation space through the cross-modal fusion unit, and the modal feature weights are determined by using attention mechanisms and task relevance. The mapped modal features and their weights are fused to obtain the fused environmental state vector.

[0008] Furthermore, the anomaly monitoring strategy is used to perform anomaly monitoring on the environmental state vector, and the monitoring results obtained include: In response to a monitoring task instruction, monitoring task information is acquired, and an anomaly monitoring strategy is determined based on the monitoring task information. The anomaly monitoring strategy includes at least one of a spatiotemporal consistency analysis strategy, a statistical threshold judgment strategy, a change trend comparison strategy, a multimodal anomaly scoring strategy, and a knowledge base reasoning strategy. The monitoring task information is used to characterize the task information for selecting the anomaly monitoring strategy. The environmental state vectors are matched based on the anomaly monitoring strategy to obtain monitoring results.

[0009] Furthermore, the generation of monitoring and disposal information based on the monitoring results and environmental constraints includes: Environmental constraints are determined based on the level information in the monitoring results. These environmental constraints include time constraints, resource constraints, and operational constraints. The monitoring and disposal information that matches the environmental constraints and the level information is determined. The monitoring and disposal information includes pollution type matching information, environmental constraint assessment information, resource availability analysis information, disposal priority ranking information, and disposal strategy information.

[0010] Furthermore, the method also includes: The monitoring and handling information is processed using a large language model to generate task description text, and the task description text is then output. Task planning information is generated based on the task description text, and task handling instructions are triggered through the task planning information.

[0011] Furthermore, the acquisition of multimodal data collected on the ecological environment includes: Sensing data is acquired through multimodal sensors, including optical sensors, chemical sensors, physical sensors, and position sensors. The multimodal data is generated after preprocessing the sensor data.

[0012] According to another aspect of this application, an ecological environment monitoring processing device is provided, comprising: The acquisition module is used to acquire multimodal data collected from the ecological environment. The fusion module is used to perform fusion processing on the multimodal data based on the multimodal basic model to obtain the fused environmental state vector. The multimodal basic model includes a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit. The generation module is used to perform anomaly monitoring on the environmental state vector based on the anomaly monitoring strategy, obtain monitoring results, and generate monitoring and disposal information based on the monitoring results and environmental constraints. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results.

[0013] Furthermore, the device also includes: The building module is used to construct a multimodal base model including a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit. The visual feature encoding unit is used to extract spatial semantic information from image data and video data. The temporal feature encoding unit is used to extract temporal evolution features from sensor data. The numerical feature encoding unit is used to process the input modal data. The cross-modal fusion unit is used to map different modalities to a unified semantic space and generate a fused environmental state representation. The training module is used to train the multimodal base model based on environmental sample data to obtain a multimodal base model that has completed model training.

[0014] Furthermore, the fusion module is specifically used to extract features from the environmental sample data through the visual feature encoding unit, the temporal feature encoding unit, and the numerical feature encoding unit respectively; to map the extracted modal features to a unified semantic representation space through the cross-modal fusion unit, and to determine the modal feature weights using an attention mechanism and task relevance; and to fuse the mapped modal features and the modal feature weights to obtain the fused environmental state vector.

[0015] Furthermore, the generation module is specifically used to respond to the monitoring task instruction, acquire monitoring task information, and determine an anomaly monitoring strategy based on the monitoring task information. The anomaly monitoring strategy includes at least one of the following: spatiotemporal consistency analysis strategy, statistical threshold judgment strategy, change trend comparison strategy, multimodal anomaly scoring strategy, and knowledge base reasoning strategy. The monitoring task information is used to characterize the task information for selecting the anomaly monitoring strategy. Based on the anomaly monitoring strategy, the environmental state vector is matched to obtain the monitoring result.

[0016] Furthermore, the generation module is specifically used to determine environmental constraints based on the level information in the monitoring results, the environmental constraints including time constraints, resource constraints, and operational constraints; and to determine monitoring and disposal information that matches the environmental constraints and the level information, the monitoring and disposal information including pollution type matching information, environmental constraint assessment information, resource availability analysis information, disposal priority ranking information, and disposal strategy information.

[0017] Furthermore, the device also includes: The output module is used to process the monitoring and handling information through a large language model, generate task description text, and output the task description text. The triggering module is used to generate task planning information based on the task description text, and to trigger task handling instructions through the task planning information.

[0018] Furthermore, the acquisition module is specifically used to collect sensing data through multimodal sensors, including optical sensors, chemical sensors, physical sensors, and position sensors; and to generate the multimodal data after preprocessing the sensing data.

[0019] According to another aspect of this application, a storage medium is provided, wherein at least one executable instruction is stored therein, the executable instruction causing a processor to perform an operation corresponding to the above-described ecological environment monitoring processing method.

[0020] According to another aspect of this application, a terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described ecological environment monitoring processing method.

[0021] By employing the above technical solutions, the technical solutions provided in the embodiments of this application have at least the following advantages: This application provides a method, apparatus, storage medium, and device for ecological environment monitoring. Compared with the prior art, the embodiments of this application acquire multimodal data collected from the ecological environment; fuse the multimodal data based on a multimodal basic model to obtain a fused environmental state vector. The multimodal basic model includes a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit; perform anomaly monitoring on the environmental state vector based on anomaly monitoring strategies to obtain monitoring results; and generate monitoring and disposal information based on the monitoring results and environmental constraints. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results. By comprehensively monitoring multimodal environmental data through a large model, the application achieves the comprehensive purpose of automatic identification of environmental anomalies, risk warning, and disposal suggestion generation, thereby improving the accuracy and response efficiency of pollution identification and greatly enhancing the automatic disposal capability and fault tolerance capability under continuous environmental pollution. The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a method for ecological environment monitoring provided in an embodiment of this application is shown. Figure 2 This paper shows a block diagram of a processing device for ecological environment monitoring provided in an embodiment of this application; Figure 3 A schematic diagram of the structure of a terminal provided in an embodiment of this application is shown. Detailed Implementation

[0023] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0026] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0027] Based on this, in one embodiment, the present invention provides a method for monitoring the ecological environment. Taking the application of this method to computer equipment such as servers as an example, the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0028] This application provides a method for processing ecological environment monitoring, such as... Figure 1As shown, the method includes: 101. Acquire multimodal data collected on the ecological environment.

[0029] In this embodiment, the executing entity, as the current execution end, can interact with multiple detection devices of different modalities to collect multimodal data of the ecological environment. At this time, the multimodal data includes, but is not limited to, multimodal environmental data such as images, videos, gas concentrations, water quality parameters, geographical locations, and meteorological information. This embodiment does not make specific limitations.

[0030] In another embodiment of this application, for further definition and explanation, the step of acquiring multimodal data collected on the ecological environment includes: Sensor data is acquired using multimodal sensors; The multimodal data is generated after preprocessing the sensor data.

[0031] To improve the accuracy of monitoring multimodal data on the ecological environment, the current execution end collects sensor data through multimodal sensors. These multimodal sensors include optical sensors, chemical sensors, physical sensors, and position sensors. Specifically, optical sensors can be visible light cameras or infrared imagers to acquire image and video data; chemical sensors can be gas sensors or water quality sensors to collect data such as gas concentration and water quality parameters; physical sensors can be temperature and humidity sensors or wind speed and direction sensors to collect data such as temperature and humidity; and position sensors can be BeiDou / GPS positioning devices to collect data such as geographical location and meteorological information. This application does not impose specific limitations on these embodiments.

[0032] It should be noted that, in order to ensure the validity of the sensor data, the sensor data can be preprocessed, including but not limited to data cleaning, data imputation and data anomaly filtering. This application embodiment does not make specific limitations.

[0033] 102. Based on the multimodal basic model, the multimodal data is fused to obtain the fused environmental state vector.

[0034] In this embodiment, after the current execution end obtains multimodal data, it fuses the multimodal data based on a multimodal base model to obtain an environmental state vector. The multimodal base model includes a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit; that is, the multimodal data is fused sequentially through these units. Furthermore, the multimodal base model can be built based on the Qwen-VL architecture or the InternVL architecture. Each unit in the multimodal base model can be built based on different neural network models, including but not limited to the Transformer architecture, the ResNet architecture, and temporal convolutional networks. This embodiment does not impose specific limitations on these models.

[0035] In another embodiment of this application, for further definition and explanation, before the step of fusing the multimodal data based on a multimodal fundamental model to obtain the fused environmental state vector, the method further includes: Construct a multimodal basic model that includes visual feature coding units, temporal feature coding units, numerical feature coding units, and cross-modal fusion units; The multimodal base model is trained based on environmental sample data to obtain a fully trained multimodal base model.

[0036] To achieve the fusion of multimodal data using intelligent models, a multimodal base model is pre-built at the current execution end. Here, the visual feature encoding unit extracts spatial semantic information from image and video data, the temporal feature encoding unit extracts temporal evolution features from sensor data, the numerical feature encoding unit processes the input modal data, and the cross-modal fusion unit maps different modalities to a unified semantic space and generates a fused environmental state representation. Environmental sample data includes image samples, video samples, gas concentration samples, water quality parameter samples, and location samples, etc., to train the multimodal base model based on this environmental sample data. During model training, the visual feature encoding unit extracts spatial semantic information from images and videos, which can be implemented using a pre-trained visual neural network architecture, including but not limited to visual Transformer architectures or convolutional neural network architectures such as ViT, Swin Transformer, ResNet, and EfficientNet, to learn and output a sequence of visual feature vectors. The temporal feature encoding unit extracts temporal evolution features from sensor data and can be implemented using temporal modeling networks, including but not limited to temporal convolutional networks (TCNs), recurrent neural networks, or Transformer encoders such as LSTM and GRU, to learn and capture temporal dependencies. The numerical feature encoding unit characterizes continuous variables such as gas concentration and water quality parameters, converting numerical data into embedded vectors through feature mapping networks, such as multilayer perceptrons (MLPs) and Fourier feature maps. The cross-modal fusion unit maps different modalities to a unified semantic space and forms a fused environmental state representation. It can employ multi-head attention mechanisms and cross-modal Transformer architectures, such as contrastive learning architectures and dual-tower fusion architectures, to achieve semantic alignment and fusion of features from different modalities.

[0037] In another embodiment of this application, for further definition and explanation, the step of fusing the multimodal data based on a multimodal fundamental model to obtain a fused environmental state vector includes: The environmental sample data is feature extracted by the visual feature encoding unit, the temporal feature encoding unit, and the numerical feature encoding unit, respectively. The extracted modal features are mapped to a unified semantic representation space through the cross-modal fusion unit, and the modal feature weights are determined by using attention mechanisms and task relevance. The mapped modal features and their weights are fused to obtain the fused environmental state vector.

[0038] To achieve multimodal data fusion and improve the accuracy of environmental anomaly monitoring, the current execution end performs fusion processing as follows: First, the environmental sample data is extracted using the visual feature encoding unit, the temporal feature encoding unit, and the numerical feature encoding unit, respectively. That is, multimodal environmental data is sequentially input into these units for feature extraction, resulting in modal features. Then, the extracted modal features are mapped to a unified semantic representation space using a cross-modal fusion unit. Attention mechanisms and task relevance are used to determine the modal feature weights. During mapping, a cross-modal alignment mechanism can be used to map different modal features to a unified semantic representation space, such as using a contrastive loss function or a triplet loss function. This embodiment does not impose specific limitations on this approach. Simultaneously, an attention weighting mechanism is used to adjust the weights of each modal feature based on task relevance. Task relevance here represents the contribution of the modal feature to the prediction of the current task and can be calculated based on similarity or relevance scoring functions. Finally, the fused environmental state vector is output. This embodiment does not impose specific limitations on this approach either.

[0039] 103. Based on the anomaly monitoring strategy, perform anomaly monitoring on the environmental state vector, obtain monitoring results, and generate monitoring and handling information based on the monitoring results and environmental constraints.

[0040] In this embodiment, the anomaly monitoring strategy is used to characterize the specific method for judging the environmental state vector. The anomaly monitoring strategy includes at least one of the following: spatiotemporal consistency analysis strategy, statistical threshold judgment strategy, trend comparison strategy, multimodal anomaly scoring strategy, and knowledge base reasoning strategy, so as to monitor from multiple dimensions and meet the diverse needs of monitoring. Furthermore, after obtaining the monitoring results, monitoring and disposal information is generated using environmental constraints. At this time, the monitoring and disposal information is used to characterize the environmental disposal strategy that can be executed based on the monitoring results. The environmental constraints are used to characterize the maximum limit on which the corresponding disposal operation can be performed on the environment. The environmental constraints include time constraints, resource constraints, and operational constraints. For example, the time constraint is 1 month, the resource constraint is region a, and the operational constraint is operation x. This embodiment does not make specific limitations.

[0041] In another embodiment of this application, for further definition and explanation, the step of performing anomaly monitoring on the environmental state vector based on an anomaly monitoring strategy to obtain monitoring results includes: In response to a monitoring task instruction, the system acquires monitoring task information and determines an anomaly monitoring strategy based on the monitoring task information. The environmental state vectors are matched based on the anomaly monitoring strategy to obtain monitoring results.

[0042] To meet the monitoring requirements of high accuracy, high effectiveness, and high flexibility, when the current execution end performs anomaly monitoring, it can first receive a monitoring task instruction triggered by the system, such as being triggered according to a specific time, a specific detection result, or manually, in order to obtain monitoring task information. At this time, the monitoring task information is used to characterize the task information for selecting the anomaly monitoring strategy. It can be manually set or determined through random or polling methods, so as to select at least one as the judgment basis from the spatiotemporal consistency analysis strategy, statistical threshold judgment strategy, change trend comparison strategy, multimodal anomaly scoring strategy, and knowledge base reasoning strategy.

[0043] In some embodiments, the spatiotemporal consistency analysis strategy is used to compare monitoring data from different time points at the same spatial location over time to detect temporal anomalies, and simultaneously perform spatial clustering analysis on data from different spatial locations at the same time point to identify spatially anomalous regions. The statistical threshold judgment strategy is used to establish normal range statistical models for each parameter based on historical data, such as Gaussian distribution models and quantile interval models, triggering anomaly alarms when real-time data exceeds preset thresholds. The trend comparison strategy is used to identify anomalous targets, such as oil spills, garbage, and dead fish, using target detection models on image and video data, and to detect new anomalous targets through frame-by-frame comparison or historical benchmark comparison. The multimodal anomaly scoring strategy is used to calculate anomaly scores by comprehensively analyzing multimodal data, using weighted fusion methods or machine learning classifiers for comprehensive judgment, such as random forests and gradient boosting trees. The knowledge base inference strategy is used to match detected anomalous features with historical cases in the ecological environment knowledge base based on similarity to infer possible pollution types and sources. When determining the above-mentioned anomaly monitoring strategy, one or more strategies can be combined based on task information. Through the joint judgment of the above methods, the abnormal areas, abnormal time windows and abnormal associated objects in the time series data of images, videos and sensors can be comprehensively analyzed, and the pollution category, suspected pollution source location and confidence level can be output, such as water eutrophication, industrial wastewater discharge, oil spill, solid waste dumping, etc. The embodiments of this application do not make specific limitations.

[0044] In another embodiment of this application, for further definition and explanation, the step of generating monitoring and disposal information based on the monitoring results and environmental constraints includes: Environmental constraints are determined based on the level information in the monitoring results. Determine monitoring and response information that matches the environmental constraints and the level information.

[0045] To generate effective disposal information for operators to execute, thereby reducing errors in human judgment, increasing the intensity of ecological environment monitoring and treatment, and ensuring ecological environment safety, the current execution end first determines environmental constraints based on the level information in the monitoring results. Specifically, when matching environmental state vectors based on anomaly monitoring strategies to obtain monitoring results, the monitoring results may include pollution category, suspected pollution source location, and confidence level, and a ranking system is constructed. For example, the higher the pollution category and confidence level, the higher the level; this application embodiment does not impose specific limitations. Furthermore, based on a preset mapping relationship, time constraints, resource constraints, and operational constraints corresponding to the level information are determined, so as to determine monitoring and disposal information matching the environmental constraints and level information based on the preset disposal mapping relationship. The monitoring and disposal information includes pollution type matching information, environmental constraint assessment information, resource availability analysis information, disposal priority ranking information, and disposal strategy information. Pollution type matching information is used to characterize the retrieval of corresponding disposal plan templates from the knowledge base based on the identified pollution type. Environmental constraint assessment information is used to characterize the assessment of the impact of current environmental conditions on disposal operations. Current environmental conditions may include terrain, weather, distance, accessibility, etc. Resource availability analysis information is used to characterize the type, quantity, location, and power or load status of available embodied intelligent agents. Disposal priority ranking information is used to characterize the determination of disposal priorities based on factors such as pollution severity, diffusion risk, and ecological sensitivity. Disposal strategy information is used to characterize the generation of the optimal disposal strategy by comprehensively considering the above factors, including selecting appropriate embodied intelligent agent types, determining operational methods such as adsorption, salvage, sampling, and isolation, and planning the operational sequence, etc. This application's embodiments do not impose specific limitations.

[0046] It should be noted that when determining the monitoring and handling information that matches the environmental constraints and the level information based on the preset handling mapping relationship, the corresponding selection can be made based on the preset handling mapping relationship, or it can be determined based on the analysis of the intelligent agent, such as the large language model. This application embodiment does not make specific limitations.

[0047] In another embodiment of this application, for further definition and explanation, the steps also include: The monitoring and handling information is processed using a large language model to generate task description text, and the task description text is then output. Task planning information is generated based on the task description text, and task handling instructions are triggered through the task planning information.

[0048] To ensure the readability and usability of monitoring and response information, the current execution terminal processes the monitoring and response information using a large language model after receiving it, generating task description text and outputting it. Based on this task description text, task planning information is then generated, and task response instructions are triggered through the task planning information. The task description text includes a structured task description with fields such as task objective, work object, work area, priority, constraints, required tools, and completion criteria. The task objective clarifies the final state to be achieved, such as completing water quality sampling in area XX or removing oil pollution from the water surface in area XX. The work object specifies the target object of the operation, such as water area near a certain coordinate point or detected abnormal pollutants. The work area specifies the spatial scope of the operation, such as GPS coordinate range or polygonal area. The priority indicates the urgency of the task, such as urgent, high, medium, or low. Constraints list operational limitations, such as battery life limits, weather conditions, and safety distance requirements. Required tools specify the type of end-point tool, such as sampling bottles, retrieval nets, or absorbent materials. The completion criteria are used to define the conditions for determining the completion of the task, such as the number of samples reaching N or the pollutant coverage rate dropping below X%. This application does not impose specific limitations on these criteria in its embodiments.

[0049] It should be noted that when generating task planning information based on task description text, a large-scale task planning model can be used to parse the structured task description into task-level objectives that can be directly used for planning. Specifically, the large-scale task planning model performs semantic parsing, objective decomposition, and constraint extraction on the task description text, transforming high-level disposal objectives into a sequence of executable sub-tasks, i.e., task planning information. At this point, the large-scale task planning model possesses core capabilities such as natural language understanding, task decomposition, dependency reasoning, and constraint propagation. Natural language understanding is used to parse the structured task description, extracting task objectives, constraints, and completion criteria; task decomposition is used to break down high-level objectives into atomic-level sub-task sequences; dependency reasoning is used to identify the sequential, parallel, and resource dependencies between sub-tasks; and constraint propagation is used to decompose and propagate global constraints to each sub-task. Preferably, the large-scale task planning model can employ a general-purpose large-scale language model with natural language understanding and reasoning capabilities, such as the GPT series, Qwen series, or GLM series models, or an industry-specific model fine-tuned for ecological and environmental disposal tasks. This application does not impose specific limitations on this model.

[0050] Furthermore, during task parsing, a structured task description text is first input. The semantics of the task objective are extracted through feedforward inference of the model. Then, based on task templates in the task knowledge base, the objective is decomposed into several sub-tasks. Subsequently, the inference module identifies the temporal dependencies, resource competition, and constraint inheritance relationships between sub-tasks, ultimately outputting a task-level objective and its dependency graph as task planning information. Simultaneously, a candidate action space is initialized. Here, the action space refers to the set of actions that an embodied agent can execute under given environment and constraints. It is a predefined, finite set of actions, including standard action units such as navigation actions (e.g., route adjustment, hovering, approaching, returning), operational actions (e.g., inspection, sampling, cleaning, dropping, grasping), and safety actions (e.g., obstacle avoidance, abort, emergency landing). The role of the action space is to constrain the abstract task objective into specific executable operations, ensuring that the planning results are feasible and providing a searchable space of action nodes for path planning algorithms. Through action sequence combinations, complex tasks are implemented, providing a discrete action selection space for policy optimization methods such as reinforcement learning, and a standardized action abstraction layer for heterogeneous agents.

[0051] In some embodiments, during task allocation and path planning, the sub-task allocation scheme, work area division scheme, and travel path of multiple embodied intelligent agents are jointly optimized based on historical knowledge base data, current environmental state, and agent state. The task allocation strategy refers to determining the sub-tasks each embodied intelligent agent is responsible for, specifically including task-agent matching (selecting appropriate agent types based on task types), load balancing (evenly distributing tasks among multiple agents of the same type), and coordination sequence (determining the temporal coordination relationship of multi-agent collaborative work). The path strategy refers to determining the travel route of each agent from the starting point to the work area and the work trajectory within the area. Optimization objectives may include minimizing total execution time, minimizing energy consumption, maximizing coverage, and minimizing repetitive tasks. Constraints may include device type compatibility, remaining battery power, load capacity, spatial distance, task urgency, risk level, environmental obstacles, and weather conditions, which are not specifically limited in this application. Preferably, some optimization methods may employ heuristic search, graph optimization, cost function minimization, or learning strategies. Heuristic search employs algorithms such as Dijkstra's to find the shortest path, combined with a task priority heuristic function for pruning. Graph optimization models task allocation and path planning as graph optimization problems, such as the Traveling Salesman Problem (TSP) and the Vehicle Routing Problem (VRP), and solves them using genetic algorithms, simulated annealing, or branch and bound algorithms. Cost function minimization defines a comprehensive cost function, and the optimal allocation is found using gradient descent or numerical optimization methods. The learning-based policy trains a policy network based on historical data, taking the current state (task queue, agent state, environment state) as input, and outputting the optimal task allocation and path decision. Through iterative updates using the above methods, overall execution time is reduced, repetitive tasks are minimized, and processing efficiency is improved. A centralized scheduling mechanism is adopted to efficiently schedule and coordinate multiple embodied agents, effectively avoiding task conflicts and resource waste. Finally, the planning results are encapsulated into standardized task instructions, i.e., task processing instructions, and sent to the corresponding embodied agents in real time via a message bus. Simultaneously, execution monitoring is initiated to track key indicators such as task progress, pose, battery level, and sensor anomalies in real time. This application does not impose specific limitations on the embodiments.

[0052] In this embodiment, to achieve automated environmental protection operations, devices with intelligent agent execution capabilities can be used to execute task instructions, i.e., to direct embodied intelligent agents to carry out ecological and environmental pollution control operations according to task instructions. Embodied intelligent agents include, but are not limited to, drones, ground inspection robots, surface robots (unmanned surface vessels), underwater robots (ROVs / AUVs), and composite mobile platforms with end-effector mechanisms. Drones are used for aerial inspections, large-scale monitoring, target tracking, and aerial deployment; ground inspection robots are used for ground inspections, pollutant identification, and solid waste cleanup; surface robots are used for surface inspections, floating debris cleanup, and surface sampling; underwater robots (ROVs / AUVs) are used for underwater inspections, underwater sampling, and underwater pollutant detection; and composite mobile platforms are multifunctional platforms with end-effector mechanisms, capable of carrying different end-effector tools to perform diverse operations. This embodiment does not impose specific limitations on these capabilities. Furthermore, specific operation types may include pollution investigation, waste cleanup, water sampling, soil sampling, and ecological intervention, with the specific operation method adapted to the end-effector tool and operation mode. During execution, multimodal data is collected in real time by airborne sensors, preprocessed at the edge, and then uploaded to the decision-making center. In case of unforeseen circumstances, task instructions are dynamically adjusted using an edge-cloud collaborative approach. Minor anomalies are handled locally on the edge and executed immediately, while significant deviations or task failures are re-planned by the cloud and sent back to the terminal to maintain operational continuity and improve task completion rates. Finally, the raw data returned by the embodied intelligent agent can be cleaned, aligned, and anonymized before being input into a large model for continuous optimization of the strategy model. The processed data is also synchronously written into the ecological environment knowledge base for subsequent learning, case retrieval, and model iteration, thus forming a closed-loop adaptive optimization mechanism.

[0053] This application provides a method for processing ecological environment monitoring. Compared with the prior art, this application acquires multimodal data collected from the ecological environment; fuses the multimodal data based on a multimodal basic model to obtain a fused environmental state vector. The multimodal basic model includes a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit; performs anomaly monitoring on the environmental state vector based on anomaly monitoring strategies to obtain monitoring results; and generates monitoring and disposal information based on the monitoring results and environmental constraints. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results. By comprehensively monitoring multimodal environmental data through a large model, the method achieves the comprehensive purpose of automatic identification of environmental anomalies, risk warning, and disposal suggestion generation, thereby improving the accuracy and response efficiency of pollution identification and greatly enhancing the automatic disposal capability and fault tolerance capability under continuous environmental pollution. Furthermore, as a response to the above Figure 1The implementation of the method shown in this application provides a processing device for ecological environment monitoring, such as... Figure 2 As shown, the device includes: Module 21 is used to acquire multimodal data collected from the ecological environment; The fusion module 22 is used to perform fusion processing on the multimodal data based on the multimodal basic model to obtain the fused environmental state vector. The multimodal basic model includes a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit. The generation module 23 is used to perform anomaly monitoring on the environmental state vector based on the anomaly monitoring strategy, obtain monitoring results, and generate monitoring and disposal information based on the monitoring results and environmental constraints. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results.

[0054] Furthermore, the device also includes: The building module is used to construct a multimodal base model including a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit. The visual feature encoding unit is used to extract spatial semantic information from image data and video data. The temporal feature encoding unit is used to extract temporal evolution features from sensor data. The numerical feature encoding unit is used to process the input modal data. The cross-modal fusion unit is used to map different modalities to a unified semantic space and generate a fused environmental state representation. The training module is used to train the multimodal base model based on environmental sample data to obtain a multimodal base model that has completed model training.

[0055] Furthermore, the fusion module is specifically used to extract features from the environmental sample data through the visual feature encoding unit, the temporal feature encoding unit, and the numerical feature encoding unit respectively; to map the extracted modal features to a unified semantic representation space through the cross-modal fusion unit, and to determine the modal feature weights using an attention mechanism and task relevance; and to fuse the mapped modal features and the modal feature weights to obtain the fused environmental state vector.

[0056] Furthermore, the generation module is specifically used to respond to the monitoring task instruction, acquire monitoring task information, and determine an anomaly monitoring strategy based on the monitoring task information. The anomaly monitoring strategy includes at least one of the following: spatiotemporal consistency analysis strategy, statistical threshold judgment strategy, change trend comparison strategy, multimodal anomaly scoring strategy, and knowledge base reasoning strategy. The monitoring task information is used to characterize the task information for selecting the anomaly monitoring strategy. Based on the anomaly monitoring strategy, the environmental state vector is matched to obtain the monitoring result.

[0057] Furthermore, the generation module is specifically used to determine environmental constraints based on the level information in the monitoring results, the environmental constraints including time constraints, resource constraints, and operational constraints; and to determine monitoring and disposal information that matches the environmental constraints and the level information, the monitoring and disposal information including pollution type matching information, environmental constraint assessment information, resource availability analysis information, disposal priority ranking information, and disposal strategy information.

[0058] Furthermore, the device also includes: The output module is used to process the monitoring and handling information through a large language model, generate task description text, and output the task description text. The triggering module is used to generate task planning information based on the task description text, and to trigger task handling instructions through the task planning information.

[0059] Furthermore, the acquisition module is specifically used to collect sensing data through multimodal sensors, including optical sensors, chemical sensors, physical sensors, and position sensors; acquire image data and video data through a positioning device; and generate the multimodal data after preprocessing the sensing data.

[0060] This application provides a processing device for ecological environment monitoring. Compared with the prior art, this application acquires multimodal data collected from the ecological environment; fuses the multimodal data based on a multimodal basic model to obtain a fused environmental state vector. The multimodal basic model includes a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit; performs anomaly monitoring on the environmental state vector based on anomaly monitoring strategies to obtain monitoring results, and generates monitoring and disposal information based on the monitoring results and environmental constraints. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results. By comprehensively monitoring multimodal environmental data through a large model, the device achieves the comprehensive purpose of automatic identification of environmental anomalies, risk warning, and disposal suggestion generation, thereby improving the accuracy and response efficiency of pollution identification and greatly enhancing the automatic disposal capability and fault tolerance capability under continuous environmental pollution.

[0061] According to one embodiment of this application, a storage medium is provided, the storage medium storing at least one executable instruction, which can execute the processing method for ecological environment monitoring in any of the above method embodiments.

[0062] Figure 3The diagram shows a structural schematic of a terminal according to one embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the terminal.

[0063] like Figure 3 As shown, the terminal may include: a processor 302, a communications interface 304, a memory 306, and a communications bus 308.

[0064] The processor 302, communication interface 304, and memory 306 communicate with each other via communication bus 308.

[0065] Communication interface 304 is used to communicate with other network elements such as clients or other servers.

[0066] The processor 302 is used to execute program 310, specifically to execute the relevant steps in the above-described ecological environment monitoring processing method embodiment.

[0067] Specifically, program 310 may include program code that includes computer operation instructions.

[0068] Processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The terminal includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0069] Memory 306 is used to store program 310. Memory 306 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0070] Specifically, program 310 can be used to cause processor 302 to perform the following operations: Acquire multimodal data collected on the ecological environment; The multimodal data is fused based on a multimodal base model to obtain a fused environmental state vector. The multimodal base model includes a visual feature coding unit, a temporal feature coding unit, a numerical feature coding unit, and a cross-modal fusion unit. Anomaly monitoring is performed on the environmental state vector based on the anomaly monitoring strategy to obtain monitoring results. Based on the monitoring results and environmental constraints, monitoring and disposal information is generated. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results.

[0071] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0072] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for processing data in ecological environment monitoring, characterized in that, include: Acquire multimodal data collected on the ecological environment; The multimodal data is fused based on a multimodal base model to obtain a fused environmental state vector. The multimodal base model includes a visual feature coding unit, a temporal feature coding unit, a numerical feature coding unit, and a cross-modal fusion unit. Anomaly monitoring is performed on the environmental state vector based on the anomaly monitoring strategy to obtain monitoring results. Based on the monitoring results and environmental constraints, monitoring and disposal information is generated. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results.

2. The method according to claim 1, characterized in that, Before fusing the multimodal data based on the multimodal fundamental model to obtain the fused environmental state vector, the method further includes: A multimodal base model is constructed, comprising a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit. The visual feature encoding unit is used to extract spatial semantic information from image data and video data. The temporal feature encoding unit is used to extract temporal evolution features from sensor data. The numerical feature encoding unit is used to process the input modal data. The cross-modal fusion unit is used to map different modalities to a unified semantic space and generate a fused environmental state representation. The multimodal base model is trained based on environmental sample data to obtain a fully trained multimodal base model.

3. The method according to claim 2, characterized in that, The multimodal data is fused based on a multimodal fundamental model to obtain a fused environmental state vector, which includes: The environmental sample data is feature extracted by the visual feature encoding unit, the temporal feature encoding unit, and the numerical feature encoding unit, respectively. The extracted modal features are mapped to a unified semantic representation space through the cross-modal fusion unit, and the modal feature weights are determined by using attention mechanisms and task relevance. The mapped modal features and their weights are fused to obtain the fused environmental state vector.

4. The method according to claim 3, characterized in that, The anomaly monitoring strategy is used to perform anomaly monitoring on the environmental state vector, and the monitoring results obtained include: In response to a monitoring task instruction, monitoring task information is acquired, and an anomaly monitoring strategy is determined based on the monitoring task information. The anomaly monitoring strategy includes at least one of a spatiotemporal consistency analysis strategy, a statistical threshold judgment strategy, a change trend comparison strategy, a multimodal anomaly scoring strategy, and a knowledge base reasoning strategy. The monitoring task information is used to characterize the task information for selecting the anomaly monitoring strategy. The environmental state vectors are matched based on the anomaly monitoring strategy to obtain monitoring results.

5. The method according to claim 4, characterized in that, The generation of monitoring and disposal information based on the monitoring results and environmental constraints includes: Environmental constraints are determined based on the level information in the monitoring results. These environmental constraints include time constraints, resource constraints, and operational constraints. The monitoring and disposal information that matches the environmental constraints and the level information is determined. The monitoring and disposal information includes pollution type matching information, environmental constraint assessment information, resource availability analysis information, disposal priority ranking information, and disposal strategy information.

6. The method according to claim 5, characterized in that, The method further includes: The monitoring and handling information is processed using a large language model to generate task description text, and the task description text is then output. Task planning information is generated based on the task description text, and task handling instructions are triggered through the task planning information.

7. The method according to any one of claims 1-6, characterized in that, The acquisition of multimodal data collected on the ecological environment includes: Sensing data is acquired through multimodal sensors, including optical sensors, chemical sensors, physical sensors, and position sensors. The multimodal data is generated after preprocessing the sensor data.

8. A processing device for ecological environment monitoring, characterized in that, include: The acquisition module is used to acquire multimodal data collected from the ecological environment. The fusion module is used to perform fusion processing on the multimodal data based on the multimodal basic model to obtain the fused environmental state vector. The multimodal basic model includes a visual feature encoding unit, a temporal feature encoding unit, a numerical feature encoding unit, and a cross-modal fusion unit. The generation module is used to perform anomaly monitoring on the environmental state vector based on the anomaly monitoring strategy, obtain monitoring results, and generate monitoring and disposal information based on the monitoring results and environmental constraints. The monitoring and disposal information is used to characterize the environmental disposal strategies that can be executed based on the monitoring results.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 1.

Images