Information retrieval method based on multi-modal operation and maintenance knowledge base

Abstract

The invention discloses an information retrieval method based on a multi-modal operation and maintenance knowledge base, particularly relates to the technical field of information retrieval, and comprises the following steps: starting from multi-modal operation and maintenance data acquisition and preprocessing, performing feature extraction on each modal data, and mapping to a unified semantic vector space to realize alignment; and constructing a knowledge index for supporting vector retrieval, and converting the multi-modal query of the user into vectors in the same space for matching. During retrieval, a result is returned according to vector similarity, a dynamic risk scoring mechanism is introduced, and finally, a differentiated multi-level dynamic response strategy is adaptively executed according to different intervals where risk scores are located; through multi-modal fusion and unified semantic index, panoramic retrieval of operation and maintenance knowledge is realized, and comprehensiveness and accuracy of problem diagnosis are greatly improved; through dynamic risk scoring and a multi-level response strategy, sensitive information leakage is intelligently prevented while efficient service is achieved, and efficiency and safety are balanced.

Description

Technical Field

[0001] This invention relates to the field of information retrieval technology, and more specifically, to an information retrieval method for a multimodal operation and maintenance knowledge base. Background Technology

[0002] In existing technologies, multimodal information retrieval effectively enhances the ability to associate different types of information through cross-modal semantic alignment, which is of positive significance. However, when this technology is applied to professional fields such as operations and maintenance that involve sensitive data and strict compliance requirements, its inherent technical characteristics may introduce new and more insidious security challenges.

[0003] Specifically, to achieve efficient retrieval, multimodal models need to bring the semantic representations of different forms of data (such as text and images) describing the same event or entity closer together in a unified vector space. While this process is technically reasonable, it may inadvertently dismantle the original security boundaries of the data based on content sensitivity. For example, a highly classified fault analysis report (containing details of core system vulnerabilities) and a related ordinary data center environment photo are closely linked in the semantic space. Although their original access permissions are different, this close association based on deep semantics itself constitutes a potential information bridge.

[0004] Traditional access control models based on identity and static permissions cannot identify and manage this new type of risk derived from algorithmic semantic associations. Attackers or unauthorized personnel may start with low-class or public data (such as a simple photo of a device), repeatedly and systematically perform similarity searches, observe and analyze the metadata (such as the source system, associated event time) and other peripheral information of the returned results, and use reasoning puzzles to gradually and indirectly infer high-class sensitive information. This "side-channel" information leakage path stems from the model's exposure of deep data associations rather than direct unauthorized access, making it highly covert and potentially leading to a substantial "compliance collapse" despite compliance with surface-level access control rules. Existing solutions lack an inherent mechanism for real-time assessment and adaptive adjustment of this dynamic risk arising from semantic associations during the retrieval process. Therefore, this invention proposes an information retrieval method based on a multimodal operations and maintenance knowledge base to address the aforementioned problems. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution: An information retrieval method based on a multimodal operation and maintenance knowledge base includes the following steps: Step S1: Collect and preprocess multimodal operation and maintenance data; Step S2: Using a pre-trained feature extraction model, extract features from the data of each modality to obtain the feature representation of each modality's data; Step S3: Through cross-modal contrastive learning, the feature representations of each modality data are mapped to a unified semantic vector space for alignment, forming a multimodal vector representation; Step S4: Associate and store the multimodal vector representations with the corresponding original data to construct a unified knowledge index that supports vector similarity retrieval; Step S5: Receive a query request submitted by the user. The query request can be at least one of natural language text, image, or time-series data. Input the query request into steps S2 and S3 to obtain its query vector representation in the unified semantic vector space. Step S6: Based on the query vector representation, perform an approximate nearest neighbor search in the unified knowledge index, and execute a multi-level dynamic response strategy to return information based on the search results and the dynamic risk score of the query vector representation. The dynamic risk score is obtained through fuzzy logic inference based on the semantic distance between the query vector representation and the high-density information cluster, the degree of intent anomaly of the query request, and the entropy of cross-query session association inference. The multi-level dynamic response strategy adaptively triggers different levels of retrieval result adjustment actions according to the different risk intervals in which the dynamic risk score is located.

[0006] In a preferred embodiment, the multimodal operation and maintenance data includes text data, image data, and time series data. The following operations are performed during preprocessing: Perform word segmentation and stop word removal on the text data, and identify and normalize the key entities. The image data is size-standardized, and the device identification and status information are manually or automatically labeled based on the image content; Outlier handling, missing value imputation, and standardization scaling operations are performed on time series data to form regular time series segments.

[0007] The pre-trained feature extraction model is used to extract features from the data of each modality, specifically including: The preprocessed text data is input into a pre-trained natural language processing model to extract deep semantic feature representations of the text. The preprocessed image data is input into a pre-trained convolutional neural network model to extract visual feature representations of the image; The preprocessed time series data is input into a pre-trained recurrent neural network model to extract dynamic pattern feature representations of the time series data.

[0008] In a preferred embodiment, cross-modal contrastive learning refers to: Construct multimodal training sample pairs. Each training sample pair contains different modal data describing the same operation and maintenance event as positive samples, and randomly combines data describing different operation and maintenance events as negative samples. The feature representations of each modality are input into a projection network and mapped to a shared semantic vector space to obtain the initial multimodal vector representation; By optimizing the contrastive learning loss function, the model is trained with the goal of reducing the distance between positive samples and negative samples in the semantic vector space and increasing the distance between them. The model is trained until the loss function converges to below a predetermined threshold, thus completing feature alignment and forming the final aligned multimodal vector representation.

[0009] In a preferred embodiment, a unified knowledge index for support vector similarity retrieval is constructed through the following steps: The final aligned multimodal vector representation is used as an index vector and stored in a dedicated vector database; Each index vector is associated with the original multimodal operation and maintenance data that generated the vector, and its corresponding metadata information is stored. The metadata information includes at least the data source, data type and permission identifier. An index structure for approximate nearest neighbor search is built for the index vectors in the vector database. Vector similarity is measured by calculating the cosine similarity between vectors, and finally the construction of a unified knowledge index is completed.

[0010] In a preferred embodiment, the semantic distance is calculated through the following steps: From the unified knowledge index, the multimodal vector representations corresponding to the original data with high-level access control in all associated metadata information are selected to form a high-level initial vector set. An unsupervised clustering analysis is performed on the high-level initial vector set using a clustering algorithm to group semantically similar vectors into the same cluster, forming at least one high-level information cluster. The mean of all vectors in each high-level information cluster is calculated to obtain the centroid vector representing the central semantics of the cluster. After obtaining the query vector representation, calculate the geometric distance between the query vector representation and the centroid vector of each high-density information cluster obtained in step one; the geometric distance is calculated using Euclidean distance or cosine distance measurement; traverse all high-density information clusters to obtain a set of basic distance values; From a set of baseline distance values, the smallest value is selected and recorded as the minimum baseline distance. The minimum baseline distance is then input into a predefined monotonically decreasing function for mapping transformation. This function is designed so that the smaller the input distance value, the larger the output mapping value. The mapping value is then normalized so that it falls within a numerical range between zero and one, ultimately yielding a semantic distance risk component that represents the degree of semantic proximity. The larger the value of this component, the closer the query is to high-density information in the semantic space, and the higher the baseline risk.

[0011] In a preferred embodiment, the intent anomaly degree is calculated through the following steps: Extracting behavioral features of the current query, specifically including: counting the number of times the current query occurs within a preset time unit as the instantaneous frequency; identifying the preset category to which the current query request belongs; and obtaining the coordinates of the current query vector in a unified semantic vector space; Retrieve the user's historical behavior baseline, which includes: the average frequency of each category of the user's historical queries within the same preset time unit; and several high-frequency regions and their boundary ranges that are ranked first and exceed the preset standard, obtained by clustering algorithms based on the set of historical query vector coordinates. The deviation of the current query behavior from the historical behavior baseline is calculated as follows: Calculate the absolute difference between the instantaneous frequency of the current query category and the historical average frequency of that category, and divide the difference by the latter to obtain the frequency deviation ratio; calculate the shortest geometric distance from the current query vector coordinates to the boundary of the historical high-frequency region corresponding to its category. If the coordinates are located inside the region, the distance is zero; multiply the frequency deviation ratio and the shortest geometric distance by preset weight coefficients and then add them together to obtain the deviation value. The calculated deviation values ​​are normalized to obtain the intentional abnormality risk component, which is between zero and one.

[0012] In a preferred embodiment, the associative inference entropy is calculated through the following steps: Within a single user session, the core topic identifiers of the results returned by a preset number of recent queries are cached in chronological order to form a topic sequence. Analyze the topic sequence, and based on the preset sensitive topic association map, check whether there is a direct or indirect association path between any two topic identifiers in the sequence, and record all identified association paths; For each identified association path, a strength value is assigned. The assignment rules are as follows: the base strength value is equal to the number of topic tags involved in the path; if all associations in the path are direct associations, then multiply by the first reinforcement coefficient; if the path contains indirect associations, then multiply by the second reinforcement coefficient; the first reinforcement coefficient is greater than the second reinforcement coefficient. The comprehensive risk assessment value is calculated as follows: Paths whose endpoint topics belong to a preset list of sensitive topics are selected from all related paths; the strength of each selected path is assigned and summed to obtain the total strength value; the total strength value is input into a preset saturation function for mapping, and the mapping result is normalized. The final output is the association inference entropy risk component between zero and one.

[0013] In a preferred embodiment, the dynamic risk score is obtained through reasoning using a fuzzy logic device, specifically: The semantic distance risk component, the intention anomaly risk component, and the association reasoning entropy risk component are converted into fuzzy set representations of low, medium, and high risk levels respectively according to the preset membership function. The fuzzy set representation is input into a preset fuzzy rule base for reasoning. The fuzzy rule base consists of multiple conditional statements, which define the mapping relationship between different combinations of input fuzzy sets and output risk level fuzzy sets. The fuzzy set of output risk levels obtained from the inference is defuzzified. The centroid of the fuzzy set is calculated to obtain a numerical value, which is then normalized and used as the final dynamic risk score.

[0014] The technical effects and advantages of this invention are as follows: This invention systematically collects and integrates text, image, and time-series data from heterogeneous sources such as logs, monitoring charts, topology screenshots, and performance curves. Through cross-modal alignment and unified indexing, it establishes, for the first time in the operations and maintenance (O&M) field, a semantic knowledge system capable of comprehensively understanding multiple information forms. This method fundamentally changes the limitations of traditional O&M knowledge bases that rely on single text keyword matching. It enables searches for any fault phenomenon or operational question to automatically link relevant historical fault reports, solution documents, monitoring curve screenshots from that time, and topology change records. This provides O&M personnel with panoramic, evidence-rich decision support, significantly improving the efficiency and accuracy of root cause analysis for complex problems.

[0015] This invention introduces a dynamic risk intelligent assessment and control mechanism based on multi-dimensional behavioral and semantic awareness. This mechanism does not rely on static permission tags, but rather generates a dynamic risk score by calculating the semantic proximity between the query content and core sensitive information in real time, analyzing the deviation between the user's current behavior and historical patterns, and inferring their potential information puzzle intent within the session. Based on this score, a multi-layered response strategy is triggered, transforming the retrieval system from a passive query tool to a proactive security collaborator. The system can intelligently and dynamically adjust the granularity and authenticity of information output according to the level of risk, ensuring daily operational efficiency while building an effective defense-in-depth against internal information probing and advanced persistent threats, significantly improving the security of the enterprise's core operational knowledge assets. Attached Figure Description

[0016] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a schematic diagram of an information retrieval method based on a multimodal operation and maintenance knowledge base in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0018] Reference Figure 1 The following examples were obtained: Example 1: An information retrieval method for an operation and maintenance knowledge base based on multimodality, comprising the following steps: Step S1: Collect and preprocess multimodal operation and maintenance data; the significance of this step is to lay a high-quality data foundation for the entire information retrieval system. By systematically collecting heterogeneous and scattered text, image and time-series data from the operation and maintenance site, and cleaning, labeling and formatting them, the original messy information is transformed into standardized data that can be directly processed and learned by machine learning models, thus solving the initial problem of isolated existence of multi-source operation and maintenance data.

[0019] Step S2: Using a pre-trained feature extraction model, features are extracted from the data of each modality to obtain the feature representation of each modality. The significance of this step is to transform the pre-processed raw data into digital features containing deep semantics. By calling an expert model pre-trained on a large general dataset, semantic information is extracted from text, visual patterns are extracted from images, and dynamic patterns are extracted from time-series curves. This achieves a high degree of abstraction and compressed expression of the essential information of each modality, creating conditions for subsequent cross-modal association.

[0020] Step S3: Through cross-modal contrastive learning, the feature representations of each modality data are mapped to a unified semantic vector space for alignment, forming a multimodal vector representation. The significance of this step is to construct a universal semantic understanding core that can understand multimodal associations. It breaks down the representation barriers between different modal data, enabling different forms of information such as text, images, and curves describing the same operation and maintenance event to obtain similar representations in the vector space. This establishes a unified standard for measuring "meaning" in mathematics, which is the key to achieving retrieval based on "semantics" rather than "keywords".

[0021] Step S4: Associate and store the multimodal vector representations with the corresponding original data to build a unified knowledge index that supports vector similarity retrieval. The significance of this step is to create a structured and efficiently queryable dynamic knowledge memory. It persistently associates the abstract vector representations with specific original knowledge items (documents, images, data sources) and builds a dedicated index, solidifying the understanding ability of the intelligent model into a knowledge base that can be queried in real time, realizing the transformation from "model reasoning" to "knowledge service".

[0022] Step S5: Receive the query request submitted by the user. The query request can be at least one of natural language text, image, or time-series data. Input the query request into steps S2 and S3 to obtain its query vector representation in a unified semantic vector space. The significance of this step is to achieve accurate understanding and digitization of the user's diverse and convenient query intentions. No matter how the user asks the question, the system can use the same intelligent processing pipeline used when building the knowledge base to transform the query content into vector coordinates in the same semantic space, thereby enabling the user's query and the knowledge in the database to be comparable in a unified dimension.

[0023] Step S6: Based on the query vector representation, perform an approximate nearest neighbor search in the unified knowledge index, and execute a multi-level dynamic response strategy to return information based on the search results and the dynamic risk score of the query vector representation. The dynamic risk score is derived through fuzzy logic inference based on the semantic distance between the query vector representation and the high-density information cluster, the degree of intent anomaly in the query request, and the entropy of cross-query session association. The multi-level dynamic response strategy adaptively triggers different levels of retrieval result adjustment actions according to the different risk ranges of the dynamic risk score. This step is significant in completing a closed loop of intelligent retrieval and secure, controllable decision-making output. First, similarity matching is performed in the vector space to initially filter semantically relevant information from the knowledge base. Then, by introducing a dynamic risk scoring mechanism, the system conducts real-time and intelligent assessment of security risks such as the potential leakage of sensitive information that may result from this retrieval behavior. Finally, the strategy for returning information is adaptively adjusted according to different risk levels. This enables the system not only to accurately find answers but also to intelligently determine "how many answers should be returned" and "in what form," improving operational efficiency while dynamically ensuring information security and compliance, achieving a balance between intelligence and security.

[0024] In one specific implementation, multimodal O&M data includes text data, image data, and time-series data. The collection and preprocessing of multimodal O&M data is designed as a multi-stage, pipelined data governance process, aiming to transform raw, heterogeneous O&M raw materials into high-quality, structured, model-readable input. For text data preprocessing, the system extracts raw text streams from log files, fault tickets, and technical documents. Preprocessing not only performs basic word segmentation and general stop word removal but also focuses on semantic cleansing and structuring in the O&M domain. For example, for identified key entities such as IP addresses (e.g., 192.168.1.1), server hostnames, and specific error codes (e.g., ORA-07445), verification and normalization are performed by querying a pre-configured configuration management database, mapping different expressions (e.g., "DB01 host" and "database-server-01") to standard unique identifiers. This eliminates ambiguity and alias interference in the text, laying the foundation for subsequent accurate semantic understanding.

[0025] For image data preprocessing, raw images from surveillance cameras, device panel screenshots, and network topology diagrams undergo strict size normalization, scaling or cropping all images to a fixed resolution to fit the input requirements of subsequent visual models. More importantly, content annotation is performed: by combining object detection algorithms with manual verification, bounding boxes and status classification labels are assigned to key devices in the images (such as specific models of switches and server indicator panels), dashboard readings, and status indicators (such as constantly lit, flashing, and off lights). For example, three hard drive indicator lights identified in a server panel image might be labeled as "constantly lit green," "flashing green," and "off," respectively.

[0026] For time-series data preprocessing, the objects of processing are time-series indicators from the monitoring system, such as CPU utilization, memory usage, and network throughput. Preprocessing sequentially includes outlier handling, missing value imputation, and standardized scaling. Outlier handling employs a statistical distribution threshold-based method to identify and smooth extreme points caused by momentary fluctuations; missing values ​​are appropriately filled using linear interpolation or forward / backward imputation strategies to ensure the continuity of the sequence. Finally, all sequence segments are transformed into a uniform numerical range through max-min scaling or zero-mean standardization, making indicators with different dimensions and baselines comparable and facilitating the model's capture of their relative change patterns.

[0027] Data units that have undergone independent preprocessing for each modality are aligned and associated across modalities based on their metadata (such as timestamps and associated event ticket numbers). For example, a specific database failure event, its corresponding preprocessed error log text, a segment of the database server performance metric curve at the time of the failure, and an image of the data center rack status captured when the alarm is triggered, will be tagged with the same event identifier and packaged into a multimodal data sample. This refined preprocessing and associative encapsulation ensures that the input received by the subsequent feature extraction model is not only clean and standardized within each modality, but also highly aligned semantically and at the event level across modalities, thus providing a solid foundation for building a high-quality unified semantic vector space.

[0028] In one specific implementation, the feature extraction process for each modality of preprocessed data employs a deep neural network model pre-trained on a large-scale general dataset and specifically fine-tuned to ensure that the extracted features possess both a broad semantic understanding foundation and fit the specific context of the operations and maintenance domain. For the preprocessed text data with key entity normalization completed, it is input into a bidirectional encoder representation model based on the Transformer architecture for deep semantic feature extraction. This model is specifically configured with 12 encoder layers, each containing 12 self-attention heads, and the hidden layer dimension of the feedforward network is 768. Upon input, the text is converted into a sequence of terms and positional encoding is added. The model, through its multi-layer self-attention mechanism and fully connected layers, ultimately converges the semantic information of the entire sequence onto a specific classification label output, generating a dense 768-dimensional vector as the deep semantic feature representation of the text. For example, a log text describing "database master node CPU utilization exceeds 95% for five minutes" will, after passing through this model, be encoded into a fixed-dimensional numerical vector containing complex semantics such as "resource overload," "database component," and "persistent anomaly."

[0029] For preprocessed image data that has undergone size normalization and content annotation, a deep residual convolutional neural network model is input to extract visual feature representations. The model's specific structure includes 50 convolutional layers with cross-stage connections to mitigate the vanishing gradient problem. The input image is progressively processed by the bottom convolutional and pooling layers of the model, extracting hierarchical features from edges and textures to complex visual patterns. Finally, a global average pooling layer integrates the feature maps output from the last convolutional layer into a 2048-dimensional feature vector. For example, an image of a server front panel labeled with multiple "flashing red" status indicators, after processing by this model, will have its output feature vector encoding the visual layout, color distribution, and overall pattern of the status indicators for key devices in the image.

[0030] For preprocessed and normalized time-series data segments, a recurrent neural network model based on a gated recurrent unit structure is input to capture their dynamic pattern features. The model is configured with a two-layer stacked structure, each layer containing 128 hidden units, and bidirectional connections are set to simultaneously consider the dependencies between the preceding and following time series data. The sequence data is input sequentially time-step by time. The model's internal gating mechanism selectively remembers and forgets information, and finally, the bidirectional hidden states of the last time step are concatenated and linearly transformed to output a 256-dimensional feature vector. For example, a time-series segment with 100 time points reflecting a sudden increase in network latency, after processing by this model, can effectively characterize the steepness of the latency increase, the persistence of high-level fluctuations, and other dynamic trend information in its output feature vector.

[0031] To ensure that feature vectors extracted from different modalities can enter the subsequent unified alignment process, all feature representations generated in the above steps undergo standardization and dimensionality adaptation. L2 norm normalization unifies the scale of the 768-dimensional vector for text, the 2048-dimensional vector for images, and the 256-dimensional vector for time series data. Subsequently, each modality's feature vector is mapped to a common target dimension, such as 512 dimensions, through an independent, trainable fully connected projection layer. This not only unifies the input specifications for subsequent processing but also provides an adjustable adaptation interface for cross-modal contrastive learning, enabling features from heterogeneous data sources to be effectively compared and aligned in a new, learnable common semantic space.

[0032] In one specific implementation, the cross-modal contrastive learning process is designed as an end-to-end model training flow. Its core objective is to align the high-dimensional feature representations of different modalities into a unified, quantifiable low-dimensional semantic vector space through nonlinear mapping and optimization. This involves the refined construction of multimodal training sample pairs. Based on pre-processed and associated multimodal data sample packages, each package contains a text summary describing the same specific operational event, related image fragments, and key time-series curves. Different modalities from the same sample package are paired; for example, a text describing a "surge in network switch port error counts" is paired with a screenshot of an indicator light alarm on the corresponding switch panel, forming a positive sample. Simultaneously, an online hard negative sample mining strategy is employed. Heterogeneous data describing other unrelated events is randomly but selectively chosen from the current training batch and combined with a specific modality of positive samples to construct multiple negative samples. For example, the aforementioned alarm screenshot is paired with a text describing a "decline in database cache hit rate" as a negative sample. This construction method ensures that the model needs to learn to identify cross-modal data pairs that are truly semantically related.

[0033] A multi-layer projection network is constructed and initialized, receiving normalized feature vectors from feature extraction models of different modalities as input. The projection network consists of fully connected layers, activation function layers, and layer normalization layers in sequence, configured as a three-layer structure. The input dimension adapts to the dimension of the feature vectors of each modality, the intermediate hidden layers have a dimension of 1024, and finally all inputs are projected onto a 512-dimensional shared semantic vector space. For example, the 768-dimensional feature vector of text, the 2048-dimensional feature vector of images, and the 256-dimensional feature vector of time series are transformed into three 512-dimensional vectors through three structurally identical but parameter-independent projection network branches. These vectors serve as the initial multi-modal vector representation, enabling distance measurement within a unified space.

[0034] A contrastive learning loss function based on normalized temperature-scale cross-entropy is defined and optimized. The core computation of this loss function involves calculating the cosine similarity between positive sample vectors for each sample pair within a batch, and simultaneously calculating the cosine similarity between that sample and all other negative sample vectors within the batch. The goal of the loss function is to maximize the similarity score between positive sample pairs and minimize the similarity score between negative sample pairs. During training, backpropagation and an adaptive moment estimation optimizer are used to continuously adjust some parameters of the projection network and even the front-end feature extraction model, driving the model to update its mapping method. This results in a gradual decrease in the distance between text vectors and image vectors describing the same event in the shared semantic space, while a gradual increase in the distance between them and image vectors describing unrelated events.

[0035] The training process is iterative. After each complete training cycle, the model's alignment performance is evaluated on a separately prepared validation dataset. Validation metrics include not only the decrease in the loss function value but also the quantitative calculation of alignment accuracy. For example, a batch of randomly selected positive sample pairs is used to calculate whether their vector similarity is higher than that of randomly selected negative sample pairs. When the loss function value no longer decreases significantly over multiple consecutive training cycles and remains consistently below a pre-set threshold, while the alignment accuracy on the validation set reaches a preset standard, the model is considered to have converged. At this point, the final projection network parameters are saved, solidifying the feature alignment capability. Subsequently, any new operational data can be mapped into this 512-dimensional semantic space through this process, generating a final aligned multimodal vector representation with cross-modal comparability, laying the foundation for subsequent unified retrieval.

[0036] In one specific implementation, constructing a unified knowledge index for support vector similarity retrieval is a process of persistently storing and structurally organizing intelligently aligned knowledge vectors. Its goal is to lay the physical foundation for subsequent efficient and accurate semantic retrieval. The process involves performing normalized data import of the index vectors. The final aligned multimodal vector representations generated after cross-modal contrastive learning—that is, each fixed-dimensional dense vector representing a specific operational event or knowledge point—are imported in batches into a vector database specifically optimized for high-dimensional vectors. During this process, to ensure vector quality and consistency, all vectors to be imported undergo a unified L2 norm normalization process, making the Euclidean length of each vector 1. This process is crucial because it allows subsequent cosine similarity measurements to be directly converted into vector inner product operations, greatly improving retrieval computation efficiency. For example, after normalization, all 512-dimensional vectors representing different fault scenarios have a unified modulus, and similarity comparison is entirely determined by the angle between the vectors.

[0037] A precise association is established between vectors and raw data and multidimensional metadata. In the vector database, a unique identifier is created for each index vector entering the database, and this identifier is used as the primary key to establish a strong link with the raw multimodal operation and maintenance data in a relational table. This raw data includes cleaned text fragments, labeled image storage paths, and regularized time-series data files. Simultaneously, a carefully designed metadata record is created and associated with the vector. This record not only includes basic fields such as data source, data type, and permission identifier, but also extends to structured information such as the event timestamp, the physical device identifier for data collection, and the business level classification to which the knowledge entry belongs. For example, a vector describing "core database master-slave failover" would have metadata recording its source as "production environment monitoring system A," data type as "text-image-time-series composite," permission identifier as "advanced," and business classification as "database-high availability."

[0038] This paper constructs an efficient approximate nearest neighbor search index structure for massive datasets. Within the vector database, a hierarchical navigable small-world graph algorithm is used to build the internal index for all normalized index vectors. The algorithm first generates an initial graph structure through random projection, where each vector is a node. Then, through iterative optimization, connecting edges are established between similar vector nodes, forming a network with "highway" and "local road" characteristics. Key parameters for constructing this index include the number of nearest neighbors during construction, the number of graph layers, and the size of the dynamic candidate set during search. This index structure allows for querying without precise calculations (i.e., brute-force search) against every vector in the database. Instead, it operates like navigation on a map, starting from several entry points and quickly approximating the region most similar to the query vector along connecting edges, returning near-optimal results in sub-second time, achieving a balance between retrieval efficiency and accuracy.

[0039] After the index structure is built, a series of integrity checks and performance benchmark tests are performed. This includes verifying the integrity of all ingested vectors and their associated metadata and original data links to ensure no data loss or misalignment. Next, a set of preset query vector samples is used to test the index for retrieval, evaluating its recall and response latency to ensure it meets preset performance metrics. Only when all checks and tests pass is the unified knowledge index considered complete and ready for service. At this point, a knowledge infrastructure deeply integrating semantic vectors, original multimodal evidence, and rich contextual metadata, with millisecond-level similarity retrieval capabilities, is ready to provide stable and efficient data support for upper-layer intelligent retrieval applications.

[0040] In one specific implementation, the calculation of semantic distance risk components is a refined process that integrates static knowledge structure analysis and dynamic query awareness. Its goal is to provide a fundamental quantitative indicator for subsequent comprehensive risk assessment, measuring the proximity of query content to the core areas of sensitive knowledge. Offline construction and centroid calculation of high-density information clusters are performed. This step is carried out periodically in the background and is not triggered by a single query. Specifically, from the completed unified knowledge index, based on the permission identifier field in the metadata records, the multimodal vector representations corresponding to all data entries marked "high-density" are automatically filtered out, forming a high-density initial vector set. Subsequently, a density-based clustering algorithm is used to analyze this set. This algorithm can automatically discover densely distributed regions in the vector space and aggregate semantically similar vectors into the same cluster, while effectively filtering discrete noise points, ultimately forming several stable high-density information clusters. For each generated cluster, the arithmetic mean of all its member vectors is calculated to obtain a new vector, called the centroid vector of the cluster. This centroid vector mathematically represents the core of the high-density semantic topic. For example, all the documents and topology diagrams related to the core database user list and permission configuration may be clustered into a cluster, and their centroids encode the central semantics of the sensitive topic of "core database permission configuration".

[0041] After receiving a real-time query online and generating a query vector representation, the system initiates targeted distance calculations. It sequentially calculates the geometric distance between the query vector representation and the centroid vectors of each high-density information cluster pre-calculated and stored in the first step. Cosine distance is preferred as the metric because it focuses on measuring differences in direction and is insensitive to the absolute length of the vectors, making it well-suited for evaluating the similarity between normalized semantic vectors distributed on a unit hypersphere. The calculation iterates through all high-density information clusters, generating a list of basic distance values. Each value in the list represents the degree of deviation of the current query from a specific high-density semantic core. For example, a query vector about "database connection pool parameter adjustment" might be far from the centroid of the "core database permission configuration" cluster but close to the centroid of the "database performance tuning" cluster.

[0042] Based on the calculated list of base distances, minimum distances are extracted and nonlinear risk mapping is performed. From the set of base distance values ​​obtained above, the smallest value is selected and recorded as the minimum base distance. This minimum base distance intuitively reflects which high-density information core the current query vector is closest to in the semantic space. Subsequently, this minimum base distance is input into a predefined exponential decay function for transformation. The mathematical property of this function is that the smaller the input value, the larger the output value, and the rate of change increases nonlinearly. For example, a function of the form y = e^(-λ*x) can be used, where x is the minimum base distance and λ is a sensitivity coefficient greater than zero. Through this function, a smaller geometric distance is mapped to a larger risk median, which amplifies the risk gradient when approaching a sensitive core.

[0043] The intermediate risk value obtained from the previous nonlinear mapping step is fed into a max-min normalization processor. This processor sets a reasonable mapping interval based on the minimum baseline distance distribution range observed in historical query data, linearly scaling the intermediate risk value to a closed interval between zero and one. The final output scalar value between zero and one is the semantic distance risk component. The larger this score, the closer the query vector is to the core of at least one high-density information cluster in the semantic space, and the higher the baseline risk value of its potential intent to access sensitive information. This component will serve as one of the core inputs, joining other subsequently calculated components in the comprehensive risk assessment model.

[0044] In one specific implementation, the calculation of the intent anomaly risk component is a dynamic, user profile-based behavioral deviation analysis process. It aims to quantify the deviation between a single query request and a user's long-term stable behavioral pattern, thereby capturing potential abnormal or probing intents. Multi-dimensional behavioral characteristics of the current query session are extracted in real time, including three aspects: First, using minutes as a preset time unit, the cumulative number of similar queries initiated by the current user within that time window is counted, recorded as the instantaneous frequency; second, based on the matching of the query content with a preset classification system, the category to which the current query request belongs is identified. This classification system covers operational scenarios such as fault diagnosis, performance analysis, configuration queries, and topology browsing; third, the precise coordinates of the current query in a unified semantic vector space are obtained from the query processing flow. This is a high-dimensional vector that encodes the deep semantic content of this query. For example, if a user submits three consecutive queries about "port error count of core switch A" within the past minute, its instantaneous frequency is 3, the category is identified as "network device fault diagnosis," and its query vector corresponds to a coordinate point in the semantic space regarding the error state of that specific device.

[0045] The system dynamically retrieves and calculates the user's historical behavior baseline, which consists of two parts: The first part calculates the average frequency of query requests belonging to various preset categories within the same minute time unit, based on the user's historical query logs over the past thirty days, forming a frequency distribution overview representing their habitual operational rhythm. The second part, for each query category, collects the set of all query vector coordinates generated by the user's historical queries and analyzes this set using a density-based clustering algorithm, specifically a noisy density-based spatial clustering application algorithm. This algorithm can automatically discover densely distributed regions of data points in the vector space and identify these regions as semantic "high-frequency areas" that the user habitually explores within that category, while filtering out isolated and accidental query points. For each identified density cluster, the convex hull algorithm is used to calculate its boundary range, thus clearly defining the semantic spatial activity range of the user's historical behavior. For example, for the "network device fault diagnosis" category, the user's baseline might show an average frequency of 0.5 times per minute, and its semantic high-frequency areas might be concentrated in the vector subspace describing issues such as "port status" and "link packet loss."

[0046] Based on a refined comparison of current features and historical baselines, a comprehensive deviation value is calculated. This calculation involves two parallel paths followed by fusion: Path one calculates the frequency deviation ratio, which is the absolute value of the difference between the instantaneous frequency of the current query's category and the historical average frequency of that category, divided by the historical average frequency to obtain a ratio representing the degree of frequency anomaly. If the historical average frequency is zero, a smoothing technique is applied. Path two calculates the semantic spatial location deviation, which is the shortest Euclidean distance from the current query vector coordinates to the boundary of the convex hull of the historical high-frequency region corresponding to its category. If the current coordinates are located inside the convex hull, this distance is zero, indicating that the semantics of the current query do not exceed the historical common range. Finally, the frequency deviation ratio and the shortest geometric distance, two indicators with different dimensions, are multiplied by pre-set weighting coefficients based on historical behavior analysis, and then summed to obtain the final deviation value. For example, a user suddenly queries "port error count of core switch A" at a frequency of 3 times per minute, while its baseline frequency is 0.5 times, which is a large frequency deviation ratio. At the same time, if its query vector happens to be near or even outside the high-frequency area boundary of "network device fault diagnosis", the geometric distance value is also large. After weighted summation, a significant deviation value will be generated.

[0047] The deviation score is transformed into a standardized risk component. Since direct comparison of deviation scores has limited meaning due to varying behavioral fluctuations among users, a max-min normalization method based on individual user historical data is employed. Specifically, the deviation score calculated in this instance is compared with the historical maximum and minimum deviation scores calculated from all query sessions performed by the user over a past period. This is then mapped to a closed interval between zero and one using a linear scaling formula. The resulting scalar value is the intent anomaly risk component. The closer this score is to one, the greater the difference between the current query behavior and the user's historical patterns in terms of frequency and semantic content. This indicates a higher degree of anomalous intent and potential risk, potentially suggesting information probing, privilege abuse, or other atypical operational intentions. This component is one of the key dynamic indicators for assessing overall retrieval risk.

[0048] In one specific implementation, the density-based spatial clustering algorithm with noise is a mature clustering method in the prior art. In this invention, it is creatively applied to mine behavioral patterns from user historical query vectors to construct a personalized semantic activity baseline. This invention does not propose a new core algorithm, but rather deeply integrates it with the specific scenario of operation and maintenance knowledge retrieval, and sets key parameters adapted to this scenario. Specifically, the two core parameters of the algorithm—neighborhood radius and minimum number of points—are calibrated based on the characteristics of user query behavior in the operation and maintenance scenario: the neighborhood radius is determined by analyzing the distribution of the distance between any vector in the historical query vector set and its k-th nearest neighbor to capture appropriate semantic relevance; the minimum number of points is set according to a certain percentage of the total historical queries of the user under this query category to ensure that the identified high-frequency regions have sufficient statistical significance, rather than being formed by chance. In this way, the algorithm can effectively divide the scattered, high-dimensional historical query vector coordinates of users into several density-connected semantic clusters (i.e., "high-frequency regions") and isolated query points regarded as "noise", thereby accurately outlining the semantic subspace boundaries that users habitually explore under specific categories, providing a dynamic and personalized reference benchmark for subsequent calculation of intent anomaly degree.

[0049] The weighting coefficients used in this invention to fuse frequency deviation ratio and semantic spatial location deviation are not arbitrary or fixed, but rather based on domain knowledge analysis of normal operational behavior and potential exploratory behavior of maintenance personnel. Specifically, the frequency deviation ratio primarily captures sudden changes in short-term behavioral intensity, such as intensive searches for the same topic within a short period. This may be a strong signal, but sometimes it also stems from normal emergency troubleshooting. Semantic spatial location deviation, on the other hand, captures substantial topic shifts in query content, such as suddenly querying a system configuration area never before explored. This may be a more covert and intentional exploratory behavior. The allocation of weights needs to balance the sensitivity of these two signals with the risk of false alarms. In a preferred embodiment, the initial values ​​of the weights are obtained by backfitting confirmed normal and abnormal session samples from historical audit logs; that is, finding a set of weights that results in generally low overall deviation values ​​for normal sessions and significantly higher values ​​for abnormal sessions. Furthermore, the weights can be designed to be dynamically adjustable, adjusting based on the risk level of the query category itself: for query categories with high risk levels (such as "permission configuration"), the weight of semantic spatial location deviation will be assigned a higher value to more rigorously examine the anomalies of the content; for regular monitoring categories (such as "performance metric viewing"), the weight of frequency deviation ratio will be relatively higher to pay more attention to anomalies in operational rhythm. This domain knowledge-based and data-driven weight setting method ensures that the calculated deviation values ​​can more accurately and reasonably reflect the true level of anomalies in user behavior intent, thereby improving the accuracy of the final risk component assessment.

[0050] In one specific implementation, the calculation of the associative inference entropy risk component is a session-level risk perception process that simulates potential intelligence puzzle-solving behavior. It aims to quantify the systemic risk of a user logically approaching sensitive information through a series of seemingly disparate queries. This involves constructing and caching a sequence of query topics within the session. Within a user's single search session, the system dynamically caches the core topic identifiers of the most relevant search results returned by the most recent preset number of queries (e.g., the last 10) in chronological order. These core topic identifiers are not simple keywords, but standardized semantic tags processed through entity linking and concept normalization. They are extracted from the content and metadata of the search results and used to summarize the core topic of the result. For example, a query about "server load" might return a core topic identifier of "CPU overload alarm," while another query about "database connectivity" might return an identifier of "connection pool exhausted." These identifiers are arranged sequentially to form a topic sequence reflecting the user's current session exploration path.

[0051] Based on a pre-defined knowledge graph, the system mines topic-related paths. It maintains a pre-defined sensitive topic-related graph for the operations and maintenance domain. This graph uses topic identifiers as nodes and semantic logical relationships between topics (such as "cause," "depends on," or "belongs to the same system") as edges. The system analyzes the topic sequence, checking whether any two topic identifiers in the sequence (especially those that are temporally adjacent or close in interval) have a path connected by one or more edges in the graph. All discovered paths are recorded. A "direct association" means that the two topics are directly connected by an edge, while an "indirect association" means that they need to be connected through one or more intermediate topic nodes.

[0052] The discovered association paths are assigned refined strength values, with each path's strength determined by a base value and a reinforcement coefficient. The base strength value equals the number of topic identifiers linked by the path, reflecting the length of the inference chain. Subsequently, reinforcement coefficients are applied based on the association nature of the path: if every association in the path is direct, it is multiplied by a first reinforcement coefficient (e.g., 1.2); if the path contains at least one indirect association, it is multiplied by a second reinforcement coefficient (e.g., 1.0). The first reinforcement coefficient is greater than the second because inference chains consisting entirely of direct associations have stronger logical coherence and intent direction, thus receiving a higher risk weight. Both coefficients are positive, ensuring that the strength assignment is positive and reasonably reflects the association quality.

[0053] The system filters all related paths to the endpoint topic from a pre-defined list of sensitive topics (such as "core system topology details," "privileged account list," and "undisclosed vulnerability information"). The strength of these filtered paths is assigned and accumulated to obtain the total inference strength value representing the sensitive information presented in the current session. To handle cases where the total strength value might be too high and map it to a reasonable range, this total value is input into a pre-defined saturation function for transformation, such as an sigmoid function. This function is characterized by an almost linear increase in output when the input value is small; as the input value exceeds a certain threshold, the output growth rate gradually decreases and approaches an upper limit, effectively preventing extreme values ​​from dominating risk assessment. Finally, the output of the saturation function is normalized to obtain an associative inference entropy risk component between zero and one. The higher this component value, the more it indicates that the user, through a series of queries in the current session, has constructed multiple logically related paths pointing to sensitive topics, exhibiting a highly systematic behavior pattern and potential information gathering intent, posing a significant risk.

[0054] In one specific implementation, the dynamic risk score is obtained through reasoning using a fuzzy logic device. This process transforms precise numerical risk components into fuzzy assessments that conform to human decision-making, and then uses a set of interpretable rules for comprehensive judgment, ultimately outputting a precise score value. The three precise input risk components are fuzzified, transforming them into fuzzy set representations of linguistic variables. The universe of discourse (zero to one) is defined for the semantic distance risk component, the intention anomaly risk component, and the associative reasoning entropy risk component, respectively. Membership functions for low, medium, and high linguistic values ​​are designed for each component. These functions are typically defined using triangular or trapezoidal functions. For example, for the semantic distance risk component, a value below 0.3 indicates a high degree of "low" risk; around 0.5 indicates the highest degree of "medium" risk; and above 0.7 indicates a high degree of "high" risk. Assuming the semantic distance risk component of the current input is 0.8, the intention anomaly risk component is 0.6, and the association reasoning entropy risk component is 0.4, after the membership function is calculated, their respective membership degrees to "low", "medium", and "high" will constitute a set of fuzzy inputs, such as the semantic distance risk component being {low: 0.0, medium: 0.4, high: 0.8}.

[0055] The fuzzified input set is fed into a pre-defined fuzzy rule base for parallel inference. This rule base consists of a series of "if-then" conditional statements built based on knowledge of the operations and maintenance security domain. Each statement's antecedent is a combination of the linguistic values ​​of three input variables, and its consequent is the linguistic value of the output variable (i.e., the overall risk level). For example, a typical rule might be: "If the semantic distance risk is high and the intent anomaly is high, then the overall risk is extremely high." Another rule might be: "If the semantic distance risk is medium and the associative reasoning entropy risk is high, then the overall risk is high." The system iterates through all rules, calculating the matching degree of the current fuzzy input to the antecedent of each rule (usually through minimum or product operations), and then prunes or scales the output fuzzy set of the corresponding consequent based on the matching degree. The output fuzzy sets of all activated rules are then aggregated (usually through maximum operations) into a total fuzzy set relating to the overall risk level.

[0056] The aggregated output fuzzy set is defuzzified to transform it into a precise numerical value. This invention employs the centroid method, a widely used defuzzification technique. Specifically, the centroid of the entire output fuzzy set is calculated, i.e., the weighted average position of the region defined by the fuzzy set on the horizontal axis is calculated. This requires defining the membership functions of the output risk levels (e.g., low, medium, high, very high) on the same output universe of discourse from zero to one. By calculating the weighted area of ​​each activated output linguistic value fuzzy set, the centroid coordinates of the total area are finally obtained. For example, the aggregated output fuzzy set may have a high membership degree in the range of 0.6 to 0.9, and the centroid method may yield a precise value of 0.75. This value represents the unnormalized raw score of the comprehensive risk assessment.

[0057] The raw score obtained from defuzzification undergoes final normalization and calibration to generate a dynamic risk score. Since the output universe of discourse is defined between zero and one, the defuzzification result is usually within this range, resulting in a dynamic risk score between zero and one. This score intuitively reflects the overall risk level of the current query session, with zero representing extremely low risk and one representing extremely high risk, providing a direct and clear quantitative basis for subsequent implementation of multi-level dynamic response strategies.

[0058] A multi-level dynamic response strategy refers to a system that automatically selects and executes control and intervention actions based on the specific numerical range of the dynamically calculated risk score in real time. The core of this strategy lies in the dynamic matching and adaptive escalation / de-escalation between response measures and perceived risk. This strategy divides the risk score from zero to one into continuous intervals using predefined thresholds, with each interval corresponding to a set of preset control logic. The following is a detailed explanation using specific examples: Suppose that, after fuzzy logic inference, the dynamic risk score calculated for a certain query session is 0.75 (belonging to a preset high-risk range, such as 0.7 to 1.0). The system will automatically trigger the highest-level response strategy, namely "Dynamic Sandbox Mirroring and Reverse Source Tracing". Specifically, the system will not return any real operational knowledge entries from the real unified knowledge index. Instead, it will utilize generative artificial intelligence technology to generate a logically consistent but completely fictitious subset of the "sandbox mirror knowledge base" in real time, based on the global data distribution and statistical patterns of the real knowledge base. For example, for a user querying "port configuration of core switch A", the system will not return a real configuration document or topology diagram, but will generate a technical document with a standardized format, but with fictitious switch model, port number, and IP address information. Simultaneously, a highly covert reverse source tracing program is activated, which records in detail all subsequent queries by the user in this session, their interaction trajectory in the sandbox, and the logical chains they attempt to construct, and attempts to correlate their network identity and behavioral fingerprints for security analysis.

[0059] If the dynamic risk score is 0.45 (belonging to the preset medium-risk range, such as 0.3 to 0.7), the system will trigger a "proactive injection of decoy knowledge and vector space perturbation" strategy. The system will retrieve and return some highly relevant real results from the real index, but will simultaneously mix in "decoy knowledge entries" synthesized by an adversarial generative network. For example, when returning a real monitoring report on database performance, a record containing a fictitious internal IP address range and incorrect configuration parameters will be embedded. In addition, the system will perform a small, random directional perturbation on the query vector, causing the user to submit extremely similar queries later, resulting in the inability to stably reproduce the same batch of results due to vector representation drift, thereby thwarting the user's attempt to piece together intelligence through multiple queries.

[0060] If the dynamic risk score is 0.15 (belonging to the preset low-risk range, such as 0.0 to 0.3), the system will execute a "dynamic result desensitization and attribution obfuscation" strategy. The system will normally return the most relevant results retrieved from the real index, but will dynamically generalize the metadata and specific details accompanying the results. For example, it might return a real fault solution document, but replace the precise timestamp "2023-10-01 14:05:00" with "last week afternoon," and replace the specific hostname "DB-PROD-01" with "a certain production database server." The specific degree and wording of each generalization will vary slightly, greatly increasing the difficulty for attackers to accurately reconstruct the facts or correlate other intelligence while ensuring information availability. This strategy, driven by dynamic scoring, achieves a progressive defense from "mild obfuscation" to "active deception" and then to "complete isolation and monitoring," enabling the system to respond to various risk scenarios, from ordinary queries to advanced persistent threats, in an intelligent, covert, and dynamically adaptable manner.

[0061] The above algorithms or formulas are all dimensionless and numerical calculations, and the results are obtained by software simulation based on a large amount of collected data to obtain the most recent real-world results. The preset parameters are set by those skilled in the art according to the actual situation.

[0062] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0063] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0064] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0065] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An information retrieval method for an operation and maintenance knowledge base based on multimodality, characterized in that, Includes the following steps: Step S1: Collect and preprocess multimodal operation and maintenance data; Step S2: Using a pre-trained feature extraction model, extract features from the data of each modality to obtain the feature representation of each modality's data; Step S3: Through cross-modal contrastive learning, the feature representations of each modality data are mapped to a unified semantic vector space for alignment, forming a multimodal vector representation; Step S4: Associate and store the multimodal vector representations with the corresponding original data to construct a unified knowledge index that supports vector similarity retrieval; Step S5: Receive the query request submitted by the user, input the query request into steps S2 and S3, and obtain its query vector representation in the unified semantic vector space; Step S6: Based on the query vector representation, perform an approximate nearest neighbor search in the unified knowledge index, and execute a multi-level dynamic response strategy to return information based on the search results and the dynamic risk score of the query vector representation. Among them, the dynamic risk score is obtained by reasoning through a fuzzy logic device based on the semantic distance between the query vector representation and the high-density information cluster, the degree of intent anomaly of the query request, and the entropy of cross-query session association reasoning. The multi-level dynamic response strategy adaptively triggers different levels of search result adjustment actions based on the different risk ranges in which the dynamic risk score is located.

2. The information retrieval method for a multimodal operation and maintenance knowledge base according to claim 1, characterized in that, Multimodal operation and maintenance data includes text data, image data, and time series data. The following operations are performed during preprocessing: Perform word segmentation and stop word removal on the text data, and identify and normalize the key entities. The image data is size-standardized, and the device identification and status information are manually or automatically labeled based on the image content; Outlier handling, missing value imputation, and standardization scaling operations are performed on time series data to form regular time series segments.

3. The information retrieval method for a multimodal operation and maintenance knowledge base according to claim 2, characterized in that, The pre-trained feature extraction model is used to extract features from the data of each modality, specifically including: The preprocessed text data is input into a pre-trained natural language processing model to extract deep semantic feature representations of the text. The preprocessed image data is input into a pre-trained convolutional neural network model to extract visual feature representations of the image; The preprocessed time series data is input into a pre-trained recurrent neural network model to extract dynamic pattern feature representations of the time series data.

4. The information retrieval method for a multimodal operation and maintenance knowledge base according to claim 3, characterized in that, Cross-modal contrastive learning refers to: Construct multimodal training sample pairs. Each training sample pair contains different modal data describing the same operation and maintenance event as positive samples, and randomly combines data describing different operation and maintenance events as negative samples. The feature representations of each modality are input into a projection network and mapped to a shared semantic vector space to obtain the initial multimodal vector representation; By optimizing the contrastive learning loss function, the model is trained with the goal of reducing the distance between positive samples and negative samples in the semantic vector space and increasing the distance between them. The model is trained until the loss function converges to below a predetermined threshold, thus completing feature alignment and forming the final aligned multimodal vector representation.

5. The information retrieval method for a multimodal operation and maintenance knowledge base according to claim 4, characterized in that, A unified knowledge index for support vector similarity retrieval is constructed through the following steps: The final aligned multimodal vector representation is used as an index vector and stored in a dedicated vector database; Each index vector is associated with the original multimodal operation and maintenance data that generated the vector, and its corresponding metadata information is stored. The metadata information includes at least the data source, data type and permission identifier. An index structure for approximate nearest neighbor search is built for the index vectors in the vector database. Vector similarity is measured by calculating the cosine similarity between vectors, and finally the construction of a unified knowledge index is completed.

6. The information retrieval method for a multimodal operation and maintenance knowledge base according to claim 5, characterized in that, Semantic distance is calculated through the following steps: From the unified knowledge index, the multimodal vector representations corresponding to the original data with the permission level marked as high-security in all associated metadata information are selected to form a high-security initial vector set; Unsupervised clustering analysis was performed on the high-density initial vector set using a clustering algorithm, and vectors with similar semantics were grouped into the same cluster to form at least one high-density information cluster. Calculate the mean of all vectors in each high-density information cluster to obtain the centroid vector representing the central semantics of that cluster; After obtaining the query vector representation, calculate the geometric distance between the query vector representation and the centroid vector of each high-density information cluster obtained in step one; the geometric distance is calculated using Euclidean distance or cosine distance measurement; traverse all high-density information clusters to obtain a set of basic distance values; From a set of base distance values, select the smallest value and record it as the minimum base distance; input the minimum base distance into a predefined monotonically decreasing function for mapping transformation; The mapping values ​​are normalized to fall within a numerical range between zero and one, ultimately yielding the semantic distance risk component that represents the degree of semantic proximity.

7. The information retrieval method for a multimodal operation and maintenance knowledge base according to claim 5, characterized in that, The degree of intent anomaly is calculated through the following steps: Extracting behavioral features of the current query, specifically including: counting the number of times the current query occurs within a preset time unit as the instantaneous frequency; identifying the preset category to which the current query request belongs; and obtaining the coordinates of the current query vector in a unified semantic vector space; Retrieve the user's historical behavior baseline, which includes: the average frequency of each category of the user's historical queries within the same preset time unit; and several high-frequency regions and their boundary ranges that are ranked first and exceed the preset standard, obtained by clustering algorithms based on the set of historical query vector coordinates. The deviation of the current query behavior from the historical behavior baseline is calculated as follows: Calculate the absolute difference between the instantaneous frequency of the current query category and the historical average frequency of that category, and divide the difference by the latter to obtain the frequency deviation ratio; calculate the shortest geometric distance from the current query vector coordinates to the boundary of the historical high-frequency region corresponding to its category. If the coordinates are located inside the region, the distance is zero; multiply the frequency deviation ratio and the shortest geometric distance by preset weight coefficients and then add them together to obtain the deviation value. The calculated deviation values ​​are normalized to obtain the intentional abnormality risk component, which is between zero and one.

8. The information retrieval method for a multimodal operation and maintenance knowledge base according to claim 5, characterized in that, The entropy of inference is calculated through the following steps: Within a single user session, the core topic identifiers of the results returned by a preset number of recent queries are cached in chronological order to form a topic sequence. Analyze the topic sequence, and based on the preset sensitive topic association map, check whether there is a direct or indirect association path between any two topic identifiers in the sequence, and record all identified association paths; For each identified association path, a strength value is assigned. The assignment rules are as follows: the base strength value is equal to the number of topic tags involved in the path; if all associations in the path are direct associations, then multiply by the first reinforcement coefficient; if the path contains indirect associations, then multiply by the second reinforcement coefficient; the first reinforcement coefficient is greater than the second reinforcement coefficient. The comprehensive risk assessment value is calculated as follows: Paths whose endpoint topics belong to a preset list of sensitive topics are selected from all related paths; the strength of each selected path is assigned and summed to obtain the total strength value; the total strength value is input into a preset saturation function for mapping, and the mapping result is normalized. The final output is the association inference entropy risk component between zero and one.

9. The information retrieval method for a multimodal operation and maintenance knowledge base according to claim 8, characterized in that, The dynamic risk score is obtained through reasoning using a fuzzy logic device, specifically: The semantic distance risk component, the intention anomaly risk component, and the association reasoning entropy risk component are converted into fuzzy set representations of low, medium, and high risk levels respectively according to the preset membership function. The fuzzy set representation is input into a preset fuzzy rule base for reasoning. The fuzzy rule base consists of multiple conditional statements, which define the mapping relationship between different combinations of input fuzzy sets and output risk level fuzzy sets. The fuzzy set of output risk levels obtained from the inference is defuzzified. The centroid of the fuzzy set is calculated to obtain a numerical value, which is then normalized and used as the final dynamic risk score.

Images