A green ecological monitoring and management system based on a knowledge graph

Abstract

The application discloses a kind of green ecological monitoring management systems based on knowledge graph, including multimodal data reading module: generate first preprocessed data and second preprocessed data;Multimodal feature extraction module: first preprocessed data is input to ShuffleNet model, and second preprocessed data is input to RoBERTa model;Ecological space topology construction module: constructs ecological space topology atlas;Space vector generation module: based on two-stage space mapping mechanism, generates visual space vector and text space vector;Space probability model construction module: constructs space probability model;Cross-modal space inference and matching module: execute cross-modal causal consistency inference;Ecological knowledge graph generation module: realize the knowledge graph storage and update of multimodal entity pair.The application realizes higher precision multimodal fusion and more complete knowledge graph construction effect in complex ecological monitoring scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent question-answering technology, and in particular to a green ecological monitoring and management system based on knowledge graphs. Background Technology

[0002] With the continuous strengthening of ecological and environmental protection efforts, various regions have deployed cameras, drones, and mobile inspection terminals to collect images, videos, and text records of ecologically sensitive areas such as rivers, shorelines, and wetlands. However, existing ecological monitoring technologies mainly rely on single-modal data analysis. For example, image detection focuses on identifying visual targets such as sewage outlets, floating objects, and shoreline damage, while text analysis focuses on extracting event descriptions from inspection records or reports. This type of monitoring method based on single-modal data cannot achieve a unified understanding of images, videos, and text records, nor can it correlate and match the same ecological entity across different data sources. This results in scattered monitoring results and isolated information, failing to form a complete ecological monitoring knowledge system.

[0003] Existing multimodal fusion technologies are mostly used in general scenarios, such as text and image retrieval or visual semantic matching, but lack modeling of the spatial structural characteristics of ecological environment monitoring scenarios. Ecological monitoring areas typically have clear geographical structures, such as rivers having a flow sequence, shorelines having left and right bank distinctions, and wetlands having fixed boundaries; however, existing technologies lack the ability to structure these geographical elements into a computable spatial topology model, and also lack the ability to map visual entities and text descriptions into a unified spatial structure. Traditional methods can only use geographical coordinates for simple distance judgments, and cannot utilize ecological spatial patterns such as upstream-downstream, left-bank-right-bank, and adjacency relationships, resulting in ambiguous spatial positioning and unclear cross-modal correspondences.

[0004] Ecological events exhibit clear causal chain characteristics; for example, sewage discharge often leads to downstream water anomalies, and shoreline damage can cause harm to surrounding vegetation. However, existing monitoring technologies often only perform entity recognition or event extraction, lacking causal relationship inference mechanisms based on ecological laws. They cannot determine whether visual detection results and textual events belong to the same ecological event chain, resulting in low accuracy in cross-modal association. Furthermore, existing technologies do not provide a comprehensive inference mechanism based on topological structure, spatial vectors, probabilistic models, and causal rules, making it difficult to support unified management of ecological entities and the construction of knowledge graphs.

[0005] Therefore, how to provide a green ecological monitoring and management system based on knowledge graphs is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a green ecological monitoring and management system based on a knowledge graph. This invention comprehensively employs multimodal deep learning models, ecological spatial topology modeling, spatial vector representation, spatial probability inference, and ecological causal relationship inference methods to uniformly process and fuse multi-source ecological monitoring data, including images, videos, and text. By constructing an ecological spatial topology graph composed of river channel topology nodes, left and right bank topology nodes, and wetland topology nodes, visual detection results are converted into spatial vector representations, and spatial descriptive words in the text are structured into text spatial vectors. Simultaneously, spatial probability models are used to quantify spatial fuzzy relationships, and cross-modal association and consistency inference between visual entities and text entities are achieved based on ecological causal relationship rules. Finally, unified ecological entity nodes are generated and written into the ecological knowledge graph. This invention can achieve accurate alignment of multimodal ecological information, structured expression of complex spatial relationships, and causal inference of ecological events, possessing advantages such as high intelligence, strong expressive power, and wide applicability.

[0007] A knowledge graph-based green ecological monitoring and management system according to an embodiment of the present invention includes: Multimodal data reading module: used to read image data and video data as first input data, text data as second input data, and preprocess the first input data and second input data to generate first preprocessed data and second preprocessed data; Multimodal feature extraction module: used to input the first preprocessed data into the ShuffleNet model to obtain visual entity and pixel position information; and input the second preprocessed data into the RoBERTa model to obtain text entity, text event and spatial semantic structured results; Ecological space topology construction module: used to construct an ecological space topology map based on data from the monitoring area, forming a set of topology nodes through river topology nodes, left bank topology nodes, right bank topology nodes, and wetland topology nodes; Spatial Vector Generation Module: Used to convert the pixel coordinates of visual entities into geographic coordinates and perform spatial overlay judgment; based on a two-stage spatial mapping mechanism, it calculates direction and distance information according to spatial orientation relationship and vertical distance to generate visual spatial vectors, and generates text spatial vectors according to the topological correspondence of spatial descriptive words. Spatial probability model construction module: used to construct a spatial probability model corresponding to spatial descriptors by assigning initial weight values ​​to a set of topological nodes as candidate spatial region units, and obtaining spatial probability features; Cross-modal spatial inference and matching module: It is used to perform cross-modal causal consistency inference based on visual spatial vectors, text spatial vectors and spatial probability features, calculate the comprehensive matching score of candidate cross-modal entity pairs, and filter out multimodal entity pairs; Ecological knowledge graph generation module: It is used to establish unified ecological entity nodes for the selected multimodal entity pairs, and record multimodal alignment relationship edges, visual entity nodes and text entity nodes in the ecological knowledge graph, so as to realize the knowledge graph storage and updating of multimodal entity pairs.

[0008] Optionally, modules can be integrated using the following methods: Step 1: Acquire multimodal ecological monitoring data of the monitoring area, using the collected images and videos as the first input data and the text data as the second input data; Step 2: Input the first input data into the ShuffleNet model to obtain visual entities and their corresponding pixel location information; input the second input data into the RoBERTa model to obtain text entities, text events, and spatial semantic structured results. Step 3: Based on the river segmentation, shoreline segmentation, and wetland zoning of the monitoring area, construct an ecological spatial topology map to form a set of topological nodes; Step 4: Convert the pixel location information of the visual entity into geographic coordinates, and map the geographic coordinates to the topology nodes; Step 5: Based on the two-stage spatial mapping mechanism, calculate direction and distance information according to spatial orientation relationship and vertical distance, generate visual spatial vector, and generate text spatial vector according to the topological correspondence of spatial descriptive words; Step 6: Construct a spatial probability model. Input the pixel position information of the visual entity into the spatial probability model to obtain the spatial probability features of the visual entity under the corresponding text space description, and perform cross-modal causal consistency inference on the visual entity and the text entity. Step 7: Based on visual spatial vectors, text spatial vectors, spatial probability features, and causal consistency results, perform spatial consistency matching on visual entities and text entities within the same monitoring area to obtain multimodal entity pairs, and write the multimodal entity pairs into the ecological knowledge graph.

[0009] Optionally, step one specifically includes: Raw image and video monitoring data are received from cameras and front-end acquisition devices deployed within the monitoring area, and the raw image and video data are classified as the first input data. Receive text data related to the monitored area from the inspection terminal, monitoring platform and law enforcement management system, and classify the text data as the second input data; The first input data and the second input data are preprocessed respectively to obtain the first preprocessed data and the second preprocessed data.

[0010] Optionally, step two specifically involves: The first preprocessed data, grouped according to the collection time and monitoring area, is input into the pre-trained ShuffleNet model. In the ShuffleNet model, convolutional feature extraction and multi-scale feature mapping are performed on each frame of the image, and the output is the detection result containing the target category identifier and the target region boundary information. Based on the detection results, the target areas of water bodies, shorelines, vegetation, sewage outlets and floating objects are determined, and the pixel position information corresponding to each visual entity is extracted in the original image coordinate system. Sentence segmentation and paragraph segmentation are performed on the second preprocessed data, and the processed text sequence is input into the pre-trained RoBERTa model. In the RoBERTa model, word segmentation and encoding and context representation generation are performed on the text sequence. Text entity recognition, text event recognition and spatial descriptor recognition are performed on the context representation to obtain recognition results containing the positions of text entities, text events and spatial descriptors. Extract the event type, event trigger words, and text entity information related to the event from the text event; extract the spatial orientation description, relative position description, and distance description information corresponding to the spatial descriptive words. Organize text entities, text events, and spatial descriptors into a spatial semantic structured result.

[0011] Optionally, step three specifically includes: Obtain geographic vector data corresponding to the monitoring area, including river centerline vector data, shoreline vector data, and wetland boundary vector data; Based on the river centerline vector data, the river is divided into lengths along the river flow direction, and each river segment is defined as a river topology node. Based on the shoreline vector data, the shoreline within the monitoring area is divided into left and right banks. Each shoreline segment is defined as a shoreline topology node, where each left bank segment is defined as a left bank topology node and each right bank segment is defined as a right bank topology node. Based on the wetland boundary vector data, the wetlands within the monitoring area are divided into regions. Each equal-length segment forms a wetland region, and each wetland region is defined as a wetland topology node. Based on the flow direction information of the river centerline, upstream and downstream topological edges are established between adjacent river topological nodes in the order from upstream to downstream, and the relationship between upstream and downstream nodes is recorded as upstream and downstream relationship; based on the correspondence between left bank topological nodes and right bank topological nodes on the river cross section, left and right bank topological edges are established between corresponding left bank topological nodes and right bank topological nodes. Based on the geographical adjacency of river topology nodes, shoreline topology nodes, and wetland topology nodes, adjacency topology edges are established between adjacent nodes. The river channel topology nodes, shoreline topology nodes, and wetland topology nodes, along with their corresponding upstream and downstream topology edges, left and right bank topology edges, and adjacent topology edges, are stored in the ecological space topology graph to form a set of topology nodes.

[0012] Optionally, step four specifically includes: Based on the camera's external and internal parameters, the center pixel coordinates of each visual entity are converted into geographic coordinates in the monitoring area coordinate system. The geographic coordinates of visual entities are spatially overlaid with the spatial range of each topological node for judgment: visual entities that fall within the spatial range of the river topological node are marked as visual entities within the river. Visual entities that fall within the spatial range of the left bank topology node or the right bank topology node are marked as shoreline visual entities. Visual entities falling within the spatial range of wetland topological nodes are marked as wetland visual entities; Optionally, the two-stage spatial mapping mechanism is specifically as follows: Based on the geographic coordinates of the visual entity, a projection operation is performed on the centerline of the river channel of the corresponding river channel topology node to determine the projection position of the visual entity along the centerline of the river channel. Based on the distance between the projection position and the starting point of the river centerline, as well as the total length of the river centerline, the relative positional proportion information of the visual entity along the river flow direction is obtained. Based on the spatial orientation and vertical distance between the geographic coordinates of the visual entity and the centerline of the river, calculate the direction and distance information of the visual entity relative to the centerline of the river.

[0013] The topological node identifier to which the visual entity belongs, the geographical coordinates of the visual entity, the relative position ratio information along the river flow direction, and the direction and distance information relative to the river centerline are combined and encoded to generate a visual spatial vector corresponding to the visual entity. Based on the spatial range of the river topology nodes, left bank topology nodes, right bank topology nodes, and wetland topology nodes in the ecological spatial topology map, the spatial descriptive words describing the river segment location are mapped to the corresponding river topology nodes. Map the spatial descriptors describing the shoreline location to the corresponding left bank or right bank topology nodes; map the spatial descriptors describing the region to the corresponding wetland topology nodes; Record the topology node identifier corresponding to each spatial descriptor; Based on the spatial orientation description, the spatial orientation description is converted into flow direction and lateral direction indicators; based on the relative position description, the relative position description is converted into position level indicators. Based on the distance description, convert the distance description into a preset distance level identifier; The topological correspondences are combined according to the preset field order to generate an initial text space vector corresponding to each spatial descriptor. The topology correspondence includes topology node identifier, flow direction identifier, lateral direction identifier, location level identifier, and distance level identifier; Perform unified vector encoding on the initial text space vector to generate a new text space vector.

[0014] Optionally, step six specifically includes: The set of topological nodes is used as the candidate spatial region unit; Based on spatial descriptors, an initial weight value is assigned to each candidate spatial region unit to construct a weight matrix; The weight matrix is ​​then normalized to generate node probability distributions, forming an independent spatial probability model for each spatial descriptor. The geographic coordinates of visual entities are mapped to the corresponding topological nodes to the spatial probability model, and the corresponding probability values ​​are obtained as spatial probability features. Pre-establish a set of ecological causal relationship rules; Based on the visual spatial vector of visual entities, the text spatial vector of text entities, and spatial probability features, visual entities and text entities belonging to the same monitoring area and the same time window are selected as candidate cross-modal entity pairs. For each candidate cross-modal entity pair, retrieve rule entries from the ecological causal relationship rule set that match the premise with the visual entity category, text entity category, text event type, and spatial relationship between the visual entity and the text entity. If a matching rule entry exists, generate causal consistency results for the corresponding candidate cross-modal entity pairs.

[0015] Optionally, step seven specifically includes: For each candidate cross-modal entity pair, the spatial vector similarity value is calculated based on the vector similarity between the visual spatial vector and the text spatial vector; Based on the causal consistency results, extract the causal matching score corresponding to the current candidate cross-modal entity pair; combine the spatial vector similarity value, spatial probability value and causal matching score according to the preset weight coefficient to generate the comprehensive matching score of the candidate cross-modal entity pair; The comprehensive matching score is compared with the preset matching threshold, and the candidate cross-modal entity pairs whose comprehensive matching score is greater than or equal to the matching threshold are determined as multimodal entity pairs that pass the spatial consistency matching. In the ecological knowledge graph, a unified ecological entity node is established for each multimodal entity pair; Establish multimodal alignment edges between the visual entities and text entities corresponding to the current multimodal entity pair and the unified ecological entity nodes respectively; Unified ecological entity nodes, multimodal alignment relationship edges, and associated visual and text entity nodes are written or updated into the ecological knowledge graph to complete the graph storage of multimodal entity pairs.

[0016] The beneficial effects of this invention are: This invention achieves structured representation and intelligent fusion of ecological information in monitored areas by introducing multimodal deep learning models, ecological spatial topology modeling, and cross-modal inference mechanisms, yielding significant technical results. First, by combining the ShuffleNet and RoBERTa models, this invention enables the simultaneous recognition and structured representation of visual and textual entities, significantly improving the processing capability and information extraction accuracy of multi-source data in ecological monitoring. Second, this invention constructs an ecological spatial topology map composed of river channel topology nodes, left and right bank topology nodes, and wetland topology nodes, enabling standardized and computable representation of spatial structures in ecological scenarios, providing a stable basis for subsequent spatial mapping and inference. Furthermore, by mapping visual entities to topology nodes and generating visual spatial vectors, while simultaneously converting spatial descriptive words into text spatial vectors, this invention achieves a unified vectorized representation of visual spatial information and text spatial semantics, effectively solving the problem of inconsistent spatial semantics between different modalities. Further, this invention constructs a spatial probability model to quantify the fuzziness and uncertainty in spatial descriptions, and combines ecological causal relationship rules to achieve causal consistency inference between visual and textual entities, significantly improving the accuracy and reliability of cross-modal matching. Ultimately, this invention achieves precise screening of multimodal entity pairs by integrating spatial vector similarity, spatial probability features, and causal matching scores, and incorporates them into an ecological knowledge graph, realizing unified storage and intelligent fusion of ecological information. Therefore, this invention can achieve higher-precision multimodal fusion, stronger spatial semantic understanding, and more complete knowledge graph construction in complex ecological monitoring scenarios. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the structure of a knowledge graph-based green ecological monitoring and management system proposed in this invention; Figure 2 This is an overall flowchart of a knowledge graph-based green ecological monitoring and management method proposed in this invention; Figure 3This is a schematic diagram of the spatial probability model and cross-modal causal consistency inference structure of a green ecological monitoring and management system based on knowledge graph proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figures 1-3 A knowledge graph-based green ecological monitoring and management system includes: Multimodal data reading module: used to read image data and video data as first input data, text data as second input data, and preprocess the first input data and second input data to generate first preprocessed data and second preprocessed data; Multimodal feature extraction module: used to input the first preprocessed data into the ShuffleNet model to obtain visual entity and pixel position information; and input the second preprocessed data into the RoBERTa model to obtain text entity, text event and spatial semantic structured results; Ecological space topology construction module: used to construct an ecological space topology map based on data from the monitoring area, forming a set of topology nodes through river topology nodes, left bank topology nodes, right bank topology nodes, and wetland topology nodes; Spatial Vector Generation Module: Used to convert the pixel coordinates of visual entities into geographic coordinates and perform spatial overlay judgment; based on a two-stage spatial mapping mechanism, it calculates direction and distance information according to spatial orientation relationship and vertical distance to generate visual spatial vectors, and generates text spatial vectors according to the topological correspondence of spatial descriptive words. Spatial probability model construction module: used to construct a spatial probability model corresponding to spatial descriptors by assigning initial weight values ​​to a set of topological nodes as candidate spatial region units, and obtaining spatial probability features; Cross-modal spatial inference and matching module: It is used to perform cross-modal causal consistency inference based on visual spatial vectors, text spatial vectors and spatial probability features, calculate the comprehensive matching score of candidate cross-modal entity pairs, and filter out multimodal entity pairs; Ecological knowledge graph generation module: It is used to establish unified ecological entity nodes for the selected multimodal entity pairs, and record multimodal alignment relationship edges, visual entity nodes and text entity nodes in the ecological knowledge graph, so as to realize the knowledge graph storage and updating of multimodal entity pairs.

[0020] In this embodiment, step one specifically includes: Determine the monitoring area and obtain the spatial range identification information corresponding to the monitoring area; The system receives raw image and video monitoring data from cameras and front-end acquisition devices deployed within the monitoring area, marks the acquisition time and monitoring area of ​​the raw image and video monitoring data, and categorizes the raw image and video data as the first input data. The system receives text data related to the monitoring area from the inspection terminal, monitoring platform and law enforcement management system, including inspection record text, monitoring report text and law enforcement record text. The text data is marked with the collection time and the monitoring area, and the text data is classified as the second input data. The first input data is preprocessed by format unification, resolution standardization and frame extraction to obtain the first preprocessed data; The second input data is preprocessed by unifying the encoding format, segmenting, and dividing the fields to obtain the second preprocessed data. The first and second preprocessed data are stored in the multimodal ecological monitoring data storage module, and the correspondence between the first and second preprocessed data and the monitoring area is established.

[0021] Step two in this embodiment specifically refers to: The first preprocessed data is grouped according to the collection time and monitoring area, and the grouped first preprocessed data is input into the pre-trained ShuffleNet model. In the ShuffleNet model, convolutional feature extraction and multi-scale feature mapping are performed on each frame of image. Candidate target regions are generated and classified using the feature mapping, and the detection results containing target category identifiers and target region boundary information are output. Perform the following processing in the ShuffleNet model: Perform a channel partitioning operation on each input frame of image to divide the image feature map into multiple channel groups; Perform grouped convolution operations on each channel group to generate corresponding local feature maps; Perform channel rearrangement operation on the local feature mapping of multiple channel groups, and recombine the rearranged features into a unified feature matrix in a predetermined order; Perform layer-by-layer convolution, pooling and activation operations on the feature matrix after channel rearrangement to obtain an intermediate feature map containing multi-scale information; Feature map downsampling is performed on the intermediate feature map to standardize the feature maps of different spatial scales to a preset scale set; Perform candidate region generation operations on feature maps at each scale, and determine the initial position and range of candidate target regions based on feature response intensity; Perform a local feature cropping operation on each candidate target region to extract the local feature blocks corresponding to the candidate region; The local feature blocks are input into the classification sub-network to generate target category identifiers for candidate regions; The local feature blocks are input into the boundary regression subnetwork to generate the bounding box coordinate information of the candidate regions. The target category identifiers and bounding box coordinates of the candidate regions are filtered and merged, low-confidence targets are deleted, and visual entities that meet the confidence conditions are retained; Output the target category identifier, the coordinates of the top left corner, the bottom right corner, the center point, and the region size information for each visual entity.

[0022] Based on the detection results, the target areas of water bodies, shorelines, vegetation, sewage outlets and floating objects are determined, and the pixel position information corresponding to each visual entity is extracted in the original image coordinate system. Sentence segmentation and paragraph segmentation are performed on the second preprocessed data, and the processed text sequence is input into the pre-trained RoBERTa model. In the RoBERTa model, word segmentation and encoding and context representation generation are performed on the text sequence. Text entity recognition, text event recognition and spatial descriptor recognition are performed on the context representation to obtain recognition results containing the positions of text entities, text events and spatial descriptors. Perform the following processing in the RoBERTa model: Perform character-level or word-level segmentation on the input text sequence to divide the text into multiple continuous basic text units; Perform vocabulary mapping processing on each text unit, converting the text unit into a corresponding vocabulary index sequence; Position encoding is performed on the vocabulary index sequence to add corresponding sequence position information to each text unit; The vocabulary index and positional encoding are combined to form an input vector sequence, which is then input into the RoBERTa encoding layer. Perform multi-layer self-attention operations and multi-layer feedforward network operations on the input vector sequence to generate the corresponding context representation matrix; Sequence labeling is performed on the context representation matrix to identify entity boundary positions in the text and extract the start and end positions of text entities; Based on a pre-defined event type vocabulary, trigger word recognition is performed on the context representation matrix to determine the triggering location and event type of the text event; Perform spatial descriptor recognition on the context representation matrix, and determine the location range of the spatial descriptors based on the annotation results of spatial orientation words and distance words; Based on the text entity recognition results, text event recognition results, and spatial descriptor recognition results, the text content, entity category identifier, and location range in the original text of each entity are recorded as text entity information; Record the trigger word, event type, and the location of the text entities involved in each event as text event information; Record the spatial orientation description, distance description, and relative position description corresponding to each spatial descriptor as spatial descriptor information; The text entity information, text event information, and spatial descriptor information are merged to generate a spatial semantic structured result, and an index relationship is established with the corresponding second preprocessed data.

[0023] Based on the recognition results, extract the text content, entity category identifier, and start and end positions of the text entities in the original text; extract the event type, event trigger words, and text entity information related to the event; and extract the spatial orientation description, relative position description, and distance description information corresponding to the spatial descriptive words. Text entities, text events, and spatial descriptors are organized into spatial semantic structured results, and an index association is established between the spatial semantic structured results and the corresponding first and second preprocessed data.

[0024] Step three in this embodiment specifically refers to: Obtain geographic vector data corresponding to the monitoring area, including river centerline vector data, shoreline vector data, and wetland boundary vector data; The river is divided into lengths along the river flow direction based on the river centerline vector data. The river centerline is divided into N river segments according to the preset segment length, and each river segment is defined as a river topology node. Based on the shoreline vector data, the shoreline within the monitoring area is divided into left and right banks. According to the spatial relative position between the river centerline and the shoreline, the shoreline is divided into left bank segments and right bank segments. Each shoreline segment is defined as a shoreline topology node. Specifically, each left bank segment is defined as a left bank topology node, and each right bank segment is defined as a right bank topology node. Based on the wetland boundary vector data, the wetlands within the monitoring area are divided into regions. The outer contour line of the wetland boundary is divided into M equal-length segments according to the total boundary length. Each equal-length segment forms a wetland region, and each wetland region is defined as a wetland topology node. Each river channel topology node, shoreline topology node, and wetland topology node is assigned a unique node identifier, and the node type, node spatial range, and correspondence with the monitoring area are recorded. Based on the flow direction information of the river centerline, upstream and downstream topological edges are established between adjacent river topological nodes in the order from upstream to downstream, and the relationship between upstream and downstream nodes is recorded as upstream and downstream relationship. Based on the correspondence between the left bank topological nodes and the right bank topological nodes on the river channel cross section, left and right bank topological edges are established between the corresponding left bank topological nodes and right bank topological nodes, and the association relationship between the left bank topological nodes and the right bank topological nodes is recorded as the left and right bank relationship. Based on the adjacency relationship of river topology nodes, shoreline topology nodes and wetland topology nodes in geospatial space, adjacency topology edges are established between adjacent nodes, and the association relationship between river nodes and adjacent shoreline nodes and wetland nodes is recorded as adjacency relationship. River topology nodes, shoreline topology nodes, and wetland topology nodes, along with their corresponding upstream and downstream topology edges, left and right bank topology edges, and adjacent topology edges, are stored in the ecological space topology map storage structure to form a set of topology nodes.

[0025] In this embodiment, step four specifically refers to: The external and internal parameters of the camera corresponding to the visual entity are obtained. The external parameters include the camera's installation position, attitude information, and shooting height. The internal parameters include imaging resolution and imaging optical parameters. Based on external and internal parameters, the center pixel coordinates of each visual entity are converted into geographic coordinates in the monitoring area coordinate system, and the geographic coordinates of the visual entities are aligned with the geographic coordinate system of the monitoring area. Based on the spatial extent of river topological nodes, left bank topological nodes, right bank topological nodes, and wetland topological nodes in the ecological spatial topology map, the geographic coordinates of visual entities are spatially overlaid with the spatial extent of each topological node for judgment: Visual entities falling within the spatial range of the river channel topology nodes are marked as visual entities within the river channel. Visual entities that fall within the spatial range of the left bank topology node or the right bank topology node are marked as shoreline visual entities. Visual entities that fall within the spatial range of wetland topological nodes are marked as wetland visual entities.

[0026] Step five in this embodiment specifically refers to: Based on the geographic coordinates of the visual entity, a projection operation is performed on the centerline of the river channel of the corresponding river channel topology node to determine the projection position of the visual entity along the centerline of the river channel. Based on the distance between the projection position and the starting point of the river centerline, as well as the total length of the river centerline, the relative positional proportion information of the visual entity along the river flow direction is obtained. Based on the spatial orientation and vertical distance between the geographic coordinates of the visual entity and the centerline of the river, calculate the orientation and distance information of the visual entity relative to the centerline of the river. The positional relationship of a visual entity relative to the centerline of the river channel in the lateral direction (left bank and right bank) and the flow direction (upstream and downstream).

[0027] Vertical distance is the minimum point-to-line distance from the geographic coordinates of a visual entity to each segment of the broken line along the centerline of the river channel.

[0028] The directional information includes lateral direction indicators indicating whether the visual entity is located to the left or right of the river centerline, and flow direction indicators indicating whether the visual entity is located upstream or downstream of the river. The lateral direction indicators are obtained by calculating the dot product of the displacement vector between the geographical coordinates of the visual entity and the nearest point on the river centerline and the normal direction vector of the river centerline. The flow direction indicators are obtained by calculating the ratio of the distance between the projection position of the visual entity along the river centerline and the starting point of the river centerline.

[0029] The distance information includes the shortest distance between the visual entity and the river centerline, and the projected distance of the visual entity along the river centerline. The shortest distance is obtained by calculating the minimum point-to-line distance from the geographic coordinates of the visual entity to each segment of the broken line along the river centerline. The projected distance is obtained by calculating the cumulative length of the broken line segment from the projection point of the visual entity to the starting point of the river centerline.

[0030] The topological node identifier to which the visual entity belongs, the geographical coordinates of the visual entity, the relative position ratio information along the river flow direction, and the direction and distance information relative to the river centerline are combined and encoded to generate a visual spatial vector corresponding to the visual entity.

[0031] The visual spatial vectors and their corresponding visual entities are then indexed and associated in the multimodal ecological monitoring data storage module.

[0032] Read spatial descriptors and associated text entities and text events from the spatial semantic structuring results, and obtain the spatial orientation description, relative position description and distance description corresponding to each spatial descriptor; Based on the spatial range of the river topology nodes, left bank topology nodes, right bank topology nodes, and wetland topology nodes in the ecological spatial topology map, the spatial descriptive words describing the river segment location are mapped to the corresponding river topology nodes. The description of the river section includes "upstream", "middlestream" and "downstream"; Map the spatial descriptors describing the shoreline location to the corresponding left bank topology nodes or right bank topology nodes; The shoreline location description includes "left bank" and "right bank"; Map the spatial descriptors of the region to the corresponding wetland topology nodes; The area description includes "within the wetland" and "at the edge of the wetland"; Record the topology node identifier corresponding to each spatial descriptor; Based on the upstream or downstream side description and the left bank or right bank side description contained in the spatial orientation description, the spatial orientation description is converted into flow direction identifier and lateral direction identifier. Based on the location descriptions included in the relative location description, such as upstream section, midstream section, downstream section, middle section of the river channel, and near the river mouth, the relative location description is converted into a location level identifier. Based on distance words such as "near", "nearby", and "far away" contained in the distance description, the distance description is converted into a preset distance level identifier; The topology node identifier, flow direction identifier, lateral direction identifier, location level identifier, distance level identifier, and text entity identifier and text event identifier associated with the spatial descriptor are combined according to the preset field order to generate an initial text spatial vector corresponding to each spatial descriptor. Perform unified vector encoding on the initial text space vector to generate a new text space vector.

[0033] The unified vector coding process includes numerical coding of flow direction identifiers, lateral direction identifiers, location level identifiers, and distance level identifiers, normalizing the numerical coding results, and adjusting the coding results to a text space vector representation consistent with the visual space vector dimension through vector dimension transformation. The unified encoded text spatial vectors are indexed and associated with the corresponding spatial descriptors, text entities, and text events in the multimodal ecological monitoring data storage module.

[0034] This implementation achieves a unified expression of visual and textual spatial semantics by constructing spatial vectors for both visual entities and textual spatial descriptors. On the visual side, based on the geographic coordinates of the visual entity, its projected position along the river centerline, relative position ratio, direction information, and distance information are calculated, and a visual spatial vector is generated by combining this with the identifier of its respective topological node. On the text side, based on the mapping relationship between spatial descriptors and topological nodes, spatial orientation descriptions, relative position descriptions, and distance descriptions are extracted, converted into direction identifiers, position level identifiers, and distance level identifiers, and then encoded using a unified vector system to obtain the textual spatial vector. Through this process, standardized expression of visual and textual information within the same spatial vector system is achieved, improving the accuracy and stability of cross-modal spatial semantic alignment.

[0035] In this embodiment, step six specifically refers to: Using the set of topological nodes as candidate spatial region units, an initial weight value is assigned to each candidate spatial region unit, and the initial weight values ​​of each candidate spatial region unit are normalized to construct a spatial probability model corresponding to the spatial descriptor. Input the geographic coordinates of the visual entity into the spatial probability model of the corresponding spatial descriptor, calculate the probability value of the visual entity falling into each candidate spatial region unit, and take the probability value of the candidate spatial region unit associated with the spatial descriptor as the spatial probability feature of the current visual entity under the corresponding text spatial description. A set of ecological causal relationship rules is pre-established. The set of ecological causal relationship rules includes rule entries with the premise of visual entity category, text entity category, text event type, and spatial relationship between visual entity and text entity as the premise and ecological causal relationship identifier as the result. The set of ecological causal relationship rules is specifically as follows: The ecological causal relationship rule set is a rule base constructed based on the categories of visual entities, text entities, text event types, and spatial relationship conditions between visual entities and text entities within the monitoring area. It is used to perform cross-modal causal consistency inference on visual entities and text entities in step six.

[0036] The rule set is stored in the form of a list of rule entries, and each rule entry includes: Visual entity category conditions: Visual entity categories include water body visual entities, shoreline visual entities, vegetation visual entities, sewage outlet visual entities, and floating object visual entities; Text entity category conditions: Text entity categories include water body text entities, shoreline text entities, vegetation text entities, sewage outlet text entities, pollutant text entities, and facility text entities; Text event type conditions: Text event types include sewage discharge events, water quality anomaly events, floating object events, shoreline damage events, vegetation damage events, etc. Spatial relationship conditions: Spatial relationship conditions include the following between visual entities and text entities: flow direction relationship (upstream or downstream), lateral direction relationship (left bank or right bank), projection distance relationship (whether the distance segment is within the preset range), and topological node adjacency relationship (whether they belong to adjacent topological nodes).

[0037] The result item for each rule entry is an ecological causal relationship identifier, which is used to indicate that when the above conditions are met, the visual entity and the text entity may belong to the same ecological event or have a causal relationship.

[0038] Based on the visual spatial vector of visual entities, the text spatial vector of text entities, and spatial probability features, visual entities and text entities belonging to the same monitoring area and the same time window are selected as candidate cross-modal entity pairs. For each candidate cross-modal entity pair, retrieve rule entries from the ecological causal relationship rule set that match the premise with the visual entity category, text entity category, text event type, and spatial relationship between the visual entity and the text entity. If matching rule entries exist, generate causal consistency identifiers and causal matching scores for the corresponding candidate cross-modal entity pairs, and use the causal consistency identifiers and causal matching scores as the causal consistency results of the candidate cross-modal entity pairs.

[0039] The process of generating causal consistency identifiers and causal matching scores for the corresponding candidate cross-modal entity pairs is as follows: If a matching rule entry exists, the causal relationship identifier contained in the matching rule entry is read from the ecological causal relationship rule set, and the causal relationship identifier is used as the causal consistency identifier of the candidate cross-modal entity pair. Each of the visual entity category conditions, text entity category conditions, text event type conditions, and spatial relationship conditions in the matching rule entries is scored, and the scoring results are combined according to preset weight coefficients to obtain the intermediate causal score. Normalize the intermediate causal scores to generate causal matching scores; When the same candidate cross-modal entity pair matches multiple rule entries, the highest score is selected from the causal matching scores corresponding to all matching rule entries as the causal matching score of the candidate cross-modal entity pair.

[0040] This implementation method achieves cross-modal causal consistency inference for visual and text entities by constructing a spatial probability model and ecological causal relationship rules. The system uses a set of topological nodes as candidate spatial region units, generates a spatial probability model through weight allocation and normalization, and obtains corresponding spatial probability features using the geographic coordinates of visual entities. Simultaneously, it pre-establishes a set of ecological causal relationship rules consisting of visual entity categories, text entity categories, text event types, and spatial relationship conditions, using the causal relationship identifier of each rule entry as the causal consistency identifier. In candidate cross-modal entity pairs, causal relationships are determined through rule matching, and each condition is scored according to preset weights. After normalization, a causal matching score is formed, enabling a quantitative assessment of the causal consistency of cross-modal entities.

[0041] Step seven in this embodiment specifically refers to: For each candidate cross-modal entity pair, the spatial vector similarity value is calculated based on the vector similarity between the visual spatial vector and the text spatial vector; The calculation method for the spatial vector similarity value is as follows: calculate the difference between the encoding values ​​of each corresponding dimension in the visual spatial vector and the text spatial vector; perform absolute value operation on the difference of each dimension; sum the absolute differences of all dimensions according to the preset weight coefficient; perform reverse normalization on the weighted sum value to obtain the spatial vector similarity value within the preset interval.

[0042] Extract spatial probability values ​​associated with the current candidate cross-modal entity pair based on the spatial probability features of the visual entity; Extract the causal matching score corresponding to the current candidate cross-modal entity pair based on the causal consistency results; The spatial vector similarity value, spatial probability value, and causal matching score are combined according to preset weight coefficients to generate a comprehensive matching score for candidate cross-modal entity pairs. The overall matching score is compared with a preset matching threshold. Candidate cross-modal entity pairs with an overall matching score greater than or equal to the matching threshold are determined as multimodal entity pairs that pass spatial consistency matching, while candidate cross-modal entity pairs with an overall matching score less than the matching threshold are determined as entity pairs that fail matching. Assign a multimodal entity pair identifier to each multimodal entity pair that passes spatial consistency matching, and record the corresponding visual entity identifier, text entity identifier, comprehensive matching score, spatial vector similarity value, spatial probability value and causal matching score; In the ecological knowledge graph, a unified ecological entity node is established for each multimodal entity pair; Multimodal alignment edges will be established between the visual entity and text entity corresponding to the current multimodal entity pair and the unified ecological entity node, respectively. The multimodal entity pair identifier, its comprehensive matching score, spatial vector similarity value, spatial probability value and causal matching score will be recorded in the multimodal alignment edge. Unified ecological entity nodes, multimodal alignment relationship edges, and associated visual and text entity nodes are written or updated into the ecological knowledge graph to complete the graph storage of multimodal entity pairs.

[0043] Example 1: To verify the feasibility of this invention, it was applied to a continuous monitoring task in a typical river and lake ecological monitoring area. The monitoring area is approximately 12.6 km long and consists of the main river channel, left and right bank shorelines, and multiple wetlands. A total of 24 fixed cameras were installed within the area, with 16 deployed along both sides of the river and 8 deployed around the wetland areas. Each camera captures images and short video clips at a frequency of 10 seconds per frame and uploads them simultaneously to the monitoring platform. Meanwhile, local patrol personnel submit approximately 120-200 patrol records, monitoring logs, and enforcement records daily through the patrol terminal system. This embodiment aims to verify the multimodal ecological monitoring method proposed in this invention through a real monitoring task, including core components such as visual entity extraction, text spatial semantic parsing, ecological spatial topology construction, cross-modal spatial alignment, and ecological knowledge graph generation.

[0044] In a real-world scenario, 24 fixed cameras generated approximately 56,400 frames of image data and 1,380 short video clips over a 30-day period. After analysis by the ShuffleNet model in this invention, approximately 215,700 visual entities were detected in the images, including approximately 101,900 water entities, approximately 52,800 shoreline entities, approximately 38,400 vegetation entities, approximately 3,100 sewage outlet entities, and approximately 19,500 floating object entities. The bounding box coordinates, center point coordinates, and region size information of all visual entities in the original image coordinate system were obtained and combined with the camera parameters to calculate their geographic coordinates.

[0045] Regarding text data, the inspection terminal and monitoring platform generated approximately 5,240 text data entries within the same period, including 3,820 inspection record texts, 960 monitoring report texts, and 460 law enforcement record texts. The RoBERTa model performed entity recognition, event recognition, and spatial descriptor extraction on the text, yielding approximately 37,500 text entities, approximately 2,700 text events, and approximately 15,800 spatial descriptors.

[0046] For example, in texts such as “turbid water appears near the sewage outlet on the left bank”, this invention can automatically identify “sewage outlet” as a text entity, “turbid water” as a text entity, and “sewage discharge event” as a text event, while extracting spatial semantic descriptions such as “left bank” and “nearby”.

[0047] In its implementation, this invention first constructs an ecological spatial topology map based on the vector data of the river centerline, shoreline, and wetland boundaries in the actual monitoring area. In the actual monitoring area, the river centerline is divided into 207 river topology nodes according to a 50-meter length rule; 422 shoreline topology nodes are generated on each of the left and right banks; and 32 wetland topology nodes are generated according to an equal-length division rule for the wetland boundaries. Subsequently, 207 upstream and downstream topological edges are established according to the river flow direction, 211 left and right bank topological edges are established according to the correspondence between the left and right banks, and 503 adjacent topological edges are established according to the spatial relationships of adjacent areas. This ultimately forms an ecological spatial topology structure with 661 nodes and a total of 921 topological edges.

[0048] The system then mapped each of the 215,700 visual entities to the topology nodes.

[0049] A floating visual entity P-0892 in a certain image frame has geographic coordinates (x=72.4165, y=13.9520). After projection, it falls into the spatial range of the river channel topology node RD-041 at a vertical distance of 4.2 meters, with the orientation labeled "right bank—downstream side," and a relative position ratio of 0.21. It was successfully labeled as a visual spatial vector. A total of 215,700 visual spatial vectors and 15,800 corresponding text spatial vectors were generated within 30 days.

[0050] During a monitoring operation, the inspection text described an event as "a large amount of white garbage floating near the right bank of the middle reaches." This invention automatically maps "middle reaches" to river nodes RD-103~RD-115, "right bank" to corresponding shoreline nodes RA-102~RA-115, and "nearby" to a distance level identifier D2, generating a text spatial vector. The system instantly invokes a spatial probability model to match the geographical location of visual entities with the node probability distribution. When the event occurred, the camera captured 12 floating object visual entities. The system calculated their location probabilities and found that three of these entities had a probability exceeding 0.82 of falling into the aforementioned area. These three entities successfully matched the text event "floating object event" in the causal rule base, with causal matching scores exceeding 0.9. Ultimately, all three were confirmed as cross-modal consistent entity pairs and written into the ecological knowledge graph.

[0051] In this 30-day period, the present invention successfully established approximately 14,600 cross-modal consistent entity pairs and identified 365 ecological events, including 41 sewage discharge events, 86 water quality anomaly events, 103 floating debris events, 34 shoreline damage events, and 101 vegetation damage events. The system automatically generated approximately 52,000 ecological knowledge graph nodes and approximately 118,000 relationship edges. Specific experimental data are shown in Table 1. Table 1. Statistical Table of Multimodal Data Fusion in Ecological Monitoring Area

[0052] As shown in Table 1, compared with the traditional manual retrieval matching method, the present invention achieves a 38.5% improvement in text-image alignment accuracy, a 53.7% improvement in event recognition efficiency, and a reduction of approximately 61.2% in spatial positioning error compared with the traditional method.

[0053] Throughout the practical application process, this invention effectively solved the problems of "vague text descriptions that cannot be located, isolated image data that cannot be queried, and lack of causal chain support for events" in traditional ecological monitoring, and achieved quantifiable monitoring results in real-world scenarios.

[0054] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A green ecological monitoring and management system based on knowledge graphs, characterized in that, include: Multimodal data reading module: used to read image data and video data as first input data, text data as second input data, and preprocess the first input data and second input data to generate first preprocessed data and second preprocessed data; Multimodal feature extraction module: used to input the first preprocessed data into the ShuffleNet model to obtain visual entity and pixel position information; and input the second preprocessed data into the RoBERTa model to obtain text entity, text event and spatial semantic structured results; Ecological space topology construction module: used to construct an ecological space topology map based on data from the monitoring area, forming a set of topology nodes through river topology nodes, left bank topology nodes, right bank topology nodes, and wetland topology nodes; Spatial Vector Generation Module: Used to convert the pixel coordinates of visual entities into geographic coordinates and perform spatial overlay judgment; based on a two-stage spatial mapping mechanism, it calculates direction and distance information according to spatial orientation relationship and vertical distance to generate visual spatial vectors, and generates text spatial vectors according to the topological correspondence of spatial descriptive words. Spatial probability model construction module: used to construct a spatial probability model corresponding to spatial descriptors by assigning initial weight values ​​to a set of topological nodes as candidate spatial region units, and obtaining spatial probability features; Cross-modal spatial inference and matching module: It is used to perform cross-modal causal consistency inference based on visual spatial vectors, text spatial vectors and spatial probability features, calculate the comprehensive matching score of candidate cross-modal entity pairs, and filter out multimodal entity pairs; Ecological knowledge graph generation module: It is used to establish unified ecological entity nodes for the selected multimodal entity pairs, and record multimodal alignment relationship edges, visual entity nodes and text entity nodes in the ecological knowledge graph to realize the knowledge graph storage of multimodal entity pairs.

2. The knowledge graph-based green ecological monitoring and management system according to claim 1, characterized in that, The modules are connected in the following way: Step 1: Acquire multimodal ecological monitoring data of the monitoring area, using the collected images and videos as the first input data and the text data as the second input data; Step 2: Input the first input data into the ShuffleNet model to obtain visual entities and their corresponding pixel location information; input the second input data into the RoBERTa model to obtain text entities, text events, and spatial semantic structured results. Step 3: Based on the river segmentation, shoreline segmentation, and wetland zoning of the monitoring area, construct an ecological spatial topology map to form a set of topological nodes; Step 4: Convert the pixel location information of the visual entity into geographic coordinates, and map the geographic coordinates to the topology nodes; Step 5: Based on the two-stage spatial mapping mechanism, calculate direction and distance information according to spatial orientation relationship and vertical distance, generate visual spatial vector, and generate text spatial vector according to the topological correspondence of spatial descriptive words; Step 6: Construct a spatial probability model. Input the pixel position information of the visual entity into the spatial probability model to obtain the spatial probability features of the visual entity under the corresponding text space description, and perform cross-modal causal consistency inference on the visual entity and the text entity. Step 7: Based on visual spatial vectors, text spatial vectors, spatial probability features, and causal consistency results, perform spatial consistency matching on visual entities and text entities within the same monitoring area to obtain multimodal entity pairs, and write the multimodal entity pairs into the ecological knowledge graph.

3. The knowledge graph-based green ecological monitoring and management system according to claim 2, characterized in that, Step one specifically involves: Raw image and video monitoring data are received from cameras and front-end acquisition devices deployed within the monitoring area, and the raw image and video data are classified as the first input data. Receive text data related to the monitored area from the inspection terminal, monitoring platform and law enforcement management system, and classify the text data as the second input data; The first input data and the second input data are preprocessed respectively to obtain the first preprocessed data and the second preprocessed data.

4. The knowledge graph-based green ecological monitoring and management system according to claim 2, characterized in that, Step two specifically involves: The first preprocessed data, grouped according to the collection time and monitoring area, is input into the pre-trained ShuffleNet model. In the ShuffleNet model, convolutional feature extraction and multi-scale feature mapping are performed on each frame of the image, and the output is the detection result containing the target category identifier and the target region boundary information. Based on the detection results, the target areas of water bodies, shorelines, vegetation, sewage outlets and floating objects are determined, and the pixel position information corresponding to each visual entity is extracted in the original image coordinate system. Sentence segmentation and paragraph segmentation are performed on the second preprocessed data, and the processed text sequence is input into the pre-trained RoBERTa model. In the RoBERTa model, word segmentation and encoding and context representation generation are performed on the text sequence. Text entity recognition, text event recognition and spatial descriptor recognition are performed on the context representation to obtain recognition results containing the positions of text entities, text events and spatial descriptors. Extract the event type, event trigger words, and text entity information related to the event from the text event; extract the spatial orientation description, relative position description, and distance description information corresponding to the spatial descriptive words. Organize text entities, text events, and spatial descriptors into a spatial semantic structured result.

5. A knowledge graph-based green ecological monitoring and management system according to claim 2, characterized in that, Step three specifically involves: Obtain geographic vector data corresponding to the monitoring area, including river centerline vector data, shoreline vector data, and wetland boundary vector data; Based on the river centerline vector data, the river is divided into lengths along the river flow direction, and each river segment is defined as a river topology node. Based on the shoreline vector data, the shoreline within the monitoring area is divided into left and right banks. Each shoreline segment is defined as a shoreline topology node, where each left bank segment is defined as a left bank topology node and each right bank segment is defined as a right bank topology node. Based on the wetland boundary vector data, the wetlands within the monitoring area are divided into regions. Each equal-length segment forms a wetland region, and each wetland region is defined as a wetland topology node. Based on the flow direction information of the river centerline, upstream and downstream topological edges are established between adjacent river topological nodes in the order from upstream to downstream, and the relationship between upstream and downstream nodes is recorded as upstream and downstream relationship; based on the correspondence between left bank topological nodes and right bank topological nodes on the river cross section, left and right bank topological edges are established between corresponding left bank topological nodes and right bank topological nodes. Based on the geographical adjacency of river topology nodes, shoreline topology nodes, and wetland topology nodes, adjacency topology edges are established between adjacent nodes. The river channel topology nodes, shoreline topology nodes, and wetland topology nodes, along with their corresponding upstream and downstream topology edges, left and right bank topology edges, and adjacent topology edges, are stored in the ecological space topology map to form a set of topology nodes.

6. A knowledge graph-based green ecological monitoring and management system according to claim 2, characterized in that, Step four specifically involves: Based on the camera's external and internal parameters, the center pixel coordinates of each visual entity are converted into geographic coordinates in the monitoring area coordinate system. The geographic coordinates of visual entities are spatially overlaid with the spatial range of each topological node for judgment: visual entities that fall within the spatial range of the river topological node are marked as visual entities within the river. Visual entities that fall within the spatial range of the left bank topology node or the right bank topology node are marked as shoreline visual entities. Visual entities that fall within the spatial range of wetland topological nodes are marked as wetland visual entities.

7. A knowledge graph-based green ecological monitoring and management system according to claim 2, characterized in that, The two-stage spatial mapping mechanism is specifically as follows: Based on the geographic coordinates of the visual entity, a projection operation is performed on the centerline of the river channel of the corresponding river channel topology node to determine the projection position of the visual entity along the centerline of the river channel. Based on the distance between the projection position and the starting point of the river centerline, as well as the total length of the river centerline, the relative positional proportion information of the visual entity along the river flow direction is obtained. Based on the spatial orientation and vertical distance between the geographic coordinates of the visual entity and the centerline of the river, calculate the orientation and distance information of the visual entity relative to the centerline of the river. The topological node identifier to which the visual entity belongs, the geographical coordinates of the visual entity, the relative position ratio information along the river flow direction, and the direction and distance information relative to the river centerline are combined and encoded to generate a visual spatial vector corresponding to the visual entity. Based on the spatial range of the river topology nodes, left bank topology nodes, right bank topology nodes, and wetland topology nodes in the ecological spatial topology map, the spatial descriptive words describing the river segment location are mapped to the corresponding river topology nodes. Map the spatial descriptors describing the shoreline location to the corresponding left bank or right bank topology nodes; map the spatial descriptors describing the region to the corresponding wetland topology nodes; Record the topology node identifier corresponding to each spatial descriptor; Based on the spatial orientation description, the spatial orientation description is converted into flow direction and lateral direction indicators; based on the relative position description, the relative position description is converted into position level indicators. Based on the distance description, convert the distance description into a preset distance level identifier; The topological correspondences are combined according to the preset field order to generate an initial text space vector corresponding to each spatial descriptor. The topology correspondence includes topology node identifier, flow direction identifier, lateral direction identifier, location level identifier, and distance level identifier; Perform unified vector encoding on the initial text space vector to generate a new text space vector.

8. A knowledge graph-based green ecological monitoring and management system according to claim 2, characterized in that, Step six specifically involves: The set of topological nodes is used as the candidate spatial region unit; Based on spatial descriptors, an initial weight value is assigned to each candidate spatial region unit to construct a weight matrix; The weight matrix is ​​then normalized to generate node probability distributions, forming an independent spatial probability model for each spatial descriptor. The geographic coordinates of visual entities are mapped to the corresponding topological nodes to the spatial probability model, and the corresponding probability values ​​are obtained as spatial probability features. Pre-establish a set of ecological causal relationship rules; Based on the visual spatial vector of visual entities, the text spatial vector of text entities, and spatial probability features, visual entities and text entities belonging to the same monitoring area and the same time window are selected as candidate cross-modal entity pairs. For each candidate cross-modal entity pair, retrieve rule entries from the ecological causal relationship rule set that match the premise with the visual entity category, text entity category, text event type, and spatial relationship between the visual entity and the text entity. If a matching rule entry exists, generate causal consistency results for the corresponding candidate cross-modal entity pairs.

9. A knowledge graph-based green ecological monitoring and management system according to claim 2, characterized in that, Step seven specifically involves: For each candidate cross-modal entity pair, the spatial vector similarity value is calculated based on the vector similarity between the visual spatial vector and the text spatial vector; Extract the causal matching score corresponding to the current candidate cross-modal entity pair based on the causal consistency results; The spatial vector similarity value, spatial probability value, and causal matching score are combined according to preset weight coefficients to generate a comprehensive matching score for candidate cross-modal entity pairs. The comprehensive matching score is compared with the preset matching threshold, and the candidate cross-modal entity pairs whose comprehensive matching score is greater than or equal to the matching threshold are determined as multimodal entity pairs that pass the spatial consistency matching. In the ecological knowledge graph, a unified ecological entity node is established for each multimodal entity pair; Establish multimodal alignment edges between the visual entities and text entities corresponding to the current multimodal entity pair and the unified ecological entity nodes respectively; Unified ecological entity nodes, multimodal alignment relationship edges, and associated visual and text entity nodes are written into the ecological knowledge graph to complete the graph storage of multimodal entity pairs.

Images