Data processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a data processing method and device, electronic equipment and a storage medium. The method comprises the steps that multiple types of to-be-processed data are acquired; the multiple types of data to be processed comprise at least two of document data, audio data and image data; the to-be-processed data are processed in a corresponding data processing mode, corresponding structured data are obtained, and the structured data comprise information associated with the entity; constructing a knowledge graph and a vector database based on the structured data; based on the obtained query information, determining a target query intention of a target entity in the query information, and based on the target query intention, determining a target query strategy; and querying from the knowledge graph and / or the vector database based on the target query strategy to obtain a query result. The method can improve the accuracy and reliability of knowledge retrieval.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus, electronic device and storage medium. Background Technology

[0002] As digital transformation continues, enterprises accumulate a large amount of unstructured text data during equipment operation and maintenance, technology management, and experience transfer. This includes document fragments such as technical manuals, maintenance reports, expert interview records, and on-site meeting minutes. Currently, enterprises store these document fragments in various document, database, and file system formats. However, this storage method easily leads to problems such as information fragmentation and context breaks, resulting in low accuracy of query results. Summary of the Invention

[0003] This application provides a data processing method, apparatus, electronic device, and storage medium to solve the problem of low accuracy of query results in the aforementioned knowledge base.

[0004] To achieve the above objectives, according to a first aspect of this application, a data processing method is provided, the method comprising: Acquire multiple types of data to be processed; the multiple types of data to be processed include at least two of document data, audio data, and image data. The data to be processed is processed using the corresponding data processing method to obtain the corresponding structured data, which includes information associated with entities; A knowledge graph and a vector database are constructed based on the structured data. The knowledge graph includes multiple nodes and edges between nodes. The nodes correspond to the entities, and the edges are used to indicate the relationships between the entities corresponding to the nodes. The vector database includes feature vectors obtained by feature extraction from the structured data. There is a mapping relationship between the node corresponding to the same entity in the knowledge graph and the feature vector corresponding to it in the vector database. Based on the obtained query information, the target query intent of the target entity in the query information is determined, and based on the target query intent, a target query strategy is determined; the target query intent includes one of relational reasoning intent, content retrieval intent, and fuzzy reasoning intent; If the target query intent includes relational reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph; if the target query intent includes content retrieval intent, the target query strategy includes querying the candidate feature vector associated with the target entity in the vector database; if the target query intent includes fuzzy reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph and querying the candidate feature vector associated with the target entity in the vector database. Based on the target query strategy, queries are performed from the knowledge graph and / or the vector database to obtain query results.

[0005] Optionally, at least one node in the knowledge graph corresponds to multiple edges, and each edge corresponds to a relation type; if the target query intent includes fuzzy reasoning intent, the target query strategy further includes querying the knowledge graph first and then querying the vector database. The step of querying the knowledge graph and / or vector database based on the target query strategy to obtain query results includes: Based on the query information, determine the target relationship type corresponding to the target entity; Based on the target relationship type and the query information, a subgraph associated with the target entity is obtained by querying the knowledge graph; based on the nodes contained in the subgraph, a candidate feature vector with a mapping relationship with the nodes contained in the subgraph is obtained by querying the vector database. or, Based on the target relationship type and the query information, a subgraph associated with the target entity is obtained by querying the knowledge graph; an enhanced query vector is generated based on the subgraph and the query information, and a candidate feature vector matching the enhanced query vector is obtained by querying the vector database based on the enhanced query vector. The query result is determined based on the subgraph and the candidate feature vector.

[0006] Optionally, determining the query result based on the subgraph and the candidate feature vector includes: Determine the correlation between the subgraph and the target entity, and the similarity between each candidate feature vector and the target entity; Based on the structural features of the subgraph, a first weight corresponding to the subgraph and a second weight corresponding to the candidate feature vector are determined. For each candidate feature vector, a weighted average is performed based on the correlation degree of the subgraph and the first weight, as well as the similarity of the candidate feature vector and the second weight, to obtain the matching score value of the candidate feature vector; Based on the matching score of each of the candidate feature vectors, the target feature vector is determined from the plurality of candidate feature vectors; The query result is obtained based on the target feature vector and the subgraph.

[0007] Optionally, processing the data to be processed using a corresponding data processing method to obtain corresponding structured data includes: The data to be processed is processed in a manner corresponding to the type of the data to be processed to obtain standardized data; The standardized data is evaluated to obtain at least one of sensitivity information and complexity information of the standardized data; wherein, the sensitivity information includes at least one of sensitivity score, sensitive entity, sensitivity type and severity level corresponding to the sensitive entity, and sensitivity level label of the data to be processed to which the standardized data belongs; the complexity information includes complexity. If the sensitivity information meets preset conditions, and / or the complexity is greater than a preset complexity threshold, the standardized data is format-converted to obtain converted standardized data, and the converted standardized data is structured to obtain corresponding structured data. The preset conditions include: the sensitivity score exceeds a preset score threshold, and / or the number of sensitive entities with a severity level greater than a preset level threshold exceeds a preset quantity threshold, and / or the standardized data contains sensitive entities with a target severity level and / or a target sensitivity type, and / or the sensitivity level is marked as a preset sensitivity level mark.

[0008] Optionally, the data to be processed is document data, and the process of processing the data to be processed in a manner corresponding to the type of the data to be processed to obtain standardized data includes: The document data is parsed and extracted to obtain at least one first content block; Based on the at least one first content block, standardized data corresponding to the document data is obtained, wherein each first content block has at least one attribute information for describing the first content block, the attribute information of the first content block includes the type information of the first content block and the hierarchy information, and the type information of the first content block includes at least one of title, list, table or paragraph.

[0009] Optionally, the data to be processed is audio data, and the process of processing the data to be processed in a manner corresponding to the type of the data to be processed to obtain standardized data includes: The transcribed text obtained from the audio data is segmented to obtain at least one speech unit; The at least one discourse unit is identified to obtain the question text and the answer text; By associating the question text and the answer text, at least one question-answer pair can be obtained; Based on the at least one question-answer pair, standardized data corresponding to the audio data is obtained.

[0010] Optionally, the data to be processed is image data, and the process of processing the data to be processed in a manner corresponding to the type of the data to be processed to obtain standardized data includes: The image data is processed to obtain text description information that describes the content of the image data; The text description information is parsed and extracted to obtain at least one second content block; Based on the at least one second content block, standardized data corresponding to the image data is obtained, wherein each second content block has at least one attribute information for describing the second content block, and the attribute information of the second content block includes the type information of the second content block, which includes at least one of title, list, table or paragraph.

[0011] According to a second aspect of this application, embodiments of this application also provide a data processing apparatus, the apparatus comprising: The acquisition module is used to acquire multiple types of data to be processed; the multiple types of data to be processed include at least two of document data, audio data, and image data. The processing module is used to process the data to be processed using a corresponding data processing method to obtain corresponding structured data, wherein the structured data includes information associated with entities; A knowledge graph and a vector database are constructed based on the structured data. The knowledge graph includes multiple nodes and edges between nodes. The nodes correspond to the entities, and the edges are used to indicate the relationships between the entities corresponding to the nodes. The vector database includes feature vectors obtained by feature extraction from the structured data. There is a mapping relationship between the node corresponding to the same entity in the knowledge graph and the feature vector corresponding to it in the vector database. Based on the obtained query information, the target query intent of the target entity in the query information is determined, and based on the target query intent, a target query strategy is determined; the target query intent includes one of relational reasoning intent, content retrieval intent, and fuzzy reasoning intent; If the target query intent includes relational reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph; if the target query intent includes content retrieval intent, the target query strategy includes querying the candidate feature vector associated with the target entity in the vector database; if the target query intent includes fuzzy reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph and querying the candidate feature vector associated with the target entity in the vector database. Based on the target query strategy, queries are performed from the knowledge graph and / or the vector database to obtain query results.

[0012] According to a third aspect of this application, embodiments of this application also provide an electronic device, comprising: A memory on which computer programs are stored; A processor is configured to execute the computer program in the memory to implement the steps of any of the methods provided in the embodiments of this application.

[0013] According to a fourth aspect of this application, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods provided in embodiments of this application.

[0014] Some embodiments of this application include at least the following beneficial effects: By processing various types of data in a targeted manner to obtain structured data of different types of data, the problem of information loss caused by single-type data can be overcome; and by constructing a knowledge graph and a vector database through the structured data, the knowledge graph provides structured knowledge information, facilitating logical reasoning and relational queries, while the vector database stores unstructured or semi-structured feature vectors, facilitating similarity searches and semantic matching; by associating the two through mapping relationships, structured queries and semantic retrieval can be achieved, improving the comprehensiveness and accuracy of knowledge retrieval; and by determining the user's target query intent and selecting to call at least one of the knowledge graph and vector database according to the target query strategy determined by the target query intent, it helps to meet the diverse needs of practical applications, while providing the most relevant query results and improving the information retrieval effect in complex knowledge scenarios.

[0015] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] To gain a more complete understanding of this application and its beneficial effects, the following description will be based on the accompanying drawings, in which the same reference numerals denote the same parts in the following description.

[0018] Figure 1 This is an application scenario diagram of the data processing method shown in some embodiments of this application; Figure 2 This is an exemplary flowchart of a data processing method according to some embodiments of this application; Figure 3 These are exemplary schematic diagrams of a knowledge management system according to some embodiments of this application; Figure 4 These are exemplary schematic diagrams illustrating Markdown document processing according to some embodiments of this application; Figure 5 This is a schematic diagram of the structure of an electronic device according to some embodiments of this application. Detailed Implementation

[0019] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0020] To facilitate understanding of the implementation schemes provided in this application, the relevant application background of the data processing method provided in this application will be explained first.

[0021] To facilitate understanding, the main terms used in this application will be explained first.

[0022] Structured data refers to pre-organized, structured or semi-structured knowledge information from power equipment operation and maintenance procedures, technical manuals, historical work orders, or expert experience, which can be accessed by the system. Structured data is used to describe equipment status, fault phenomena, handling procedures, safety regulations, or related logic. This structured data directly affects the accuracy and reliability of knowledge retrieval.

[0023] Knowledge Query: Knowledge query refers to the process of retrieving and integrating relevant information from the power equipment knowledge system based on a user's natural language question (such as "How to handle a 10kV circuit breaker tripping?") to generate an accurate and actionable answer. Knowledge query involves understanding and matching multi-dimensional information such as equipment entities, fault types, and operating procedures, aiming to quickly locate the knowledge content most relevant to the current problem.

[0024] Query results: Query results are the process of filtering a subset of knowledge highly relevant to the current scenario from the knowledge base based on the user's target query intent. This can narrow the search scope, provide high-value information, and avoid interference from irrelevant knowledge, thereby improving the accuracy and practicality of the answer.

[0025] A knowledge base refers to a systematic collection of knowledge describing phenomena, causes, handling measures, safety constraints, equipment parameters, and association rules involved in the operation and maintenance of power equipment. For example, the content of a knowledge base may include: equipment type and model definitions, typical fault phenomena and causes, standard handling procedures and steps, safe operating boundary conditions, topological relationships between equipment, historical case experience, regulatory references, or other information. Of course, the content of the knowledge base can be flexibly configured according to actual business needs, and this embodiment of the application does not limit this.

[0026] Large Language Models (LLMs) are a type of artificial intelligence technique based on deep learning. They are trained on large-scale datasets to generate natural language text or understand the meaning of spoken text. These models learn and simulate the complex rules of human language through layered neural network structures, achieving near-human-level text generation capabilities.

[0027] Current power operation and maintenance knowledge query solutions in the industry suffer from poor accuracy, reliability, and scenario adaptability. This is primarily due to insufficient accuracy and contextual adaptation capabilities in knowledge recommendation. Furthermore, existing solutions are mostly limited to simple queries and struggle to handle real-world, complex, and dynamic scenarios.

[0028] Based on this, this application provides a data processing method that can significantly improve processing efficiency and accuracy when applied to power equipment operation and maintenance scenarios. Furthermore, the solution provided in this application can also be applied to various fields such as healthcare, finance, intelligent manufacturing, transportation, and education. Therefore, by configuring knowledge bases for different fields or application scenarios according to the solutions provided in the embodiments of this application, intelligent question answering in various scenarios can be achieved, avoiding the drawbacks of limited application scenarios.

[0029] The solution provided in this application can be applied to Figure 1The illustrated computer system uses a client / server (C / S) architecture. Figure 1 As shown, the computer system includes a terminal and a server.

[0030] Figure 1 The illustrated computer system application scenarios may include knowledge retrieval, intelligent diagnosis and operation and maintenance, or other scenarios, which are not limited to the embodiments of this application. The following examples illustrate this. Figure 1 The illustrated application scenario of the computer system is illustrated using intelligent operation and maintenance of power equipment as an example.

[0031] In power equipment operation and maintenance scenarios, data processing systems are deployed on servers. These server-side data processing systems support local or cross-site equipment knowledge queries. This system is used to retrieve and infer knowledge about various types of power equipment, assisting maintenance personnel in quickly handling anomalies. The server can be a cloud server, edge computing gateway, operation and maintenance management platform, or other product forms.

[0032] The terminal provides a user interface where users (e.g., maintenance personnel) input natural language questions (e.g., "What should I do if the SF6 pressure in my 10kV switchgear is low?"). Upon receiving the query request, the terminal forwards it to the server, initiating a knowledge query process. The server, through its deployed data processing system, executes the solution provided in this application to achieve intelligent query and answer generation. Specific implementation details are provided in the following method embodiment section and will not be elaborated upon here.

[0033] Furthermore, Figure 1 The illustrated computer system can be deployed on a single physical device, or... Figure 1 The illustrated computer system can be deployed on multiple physical devices, but this application does not limit this.

[0034] In some embodiments, the application scenario may also include, for example, networks, storage devices, etc. Networks may include any suitable wired or wireless networks that facilitate the exchange of information and / or data. Storage devices are used to store data, instructions, and / or any other information.

[0035] It is important to note that the application scenarios of the data processing methods are provided for illustrative purposes only and are not intended to limit the scope of this specification. Those skilled in the art can make various changes and modifications based on the descriptions in this specification. For example, application scenarios may also include databases, information sources, etc. Furthermore, application scenarios may be implemented on other devices to achieve similar or different functions. However, these changes and modifications will not depart from the scope of this specification.

[0036] Figure 2This is an exemplary flowchart of a data processing method according to some embodiments of this application. In some embodiments, process 200 may be executed based on an electronic device. Figure 2 As shown, process 200 includes the following steps.

[0037] Step 210: Obtain multiple types of data to be processed; the multiple types of data to be processed include at least two of the following: document data, audio data, and image data.

[0038] Data to be processed is the raw information used for construction. For example, various types of data to be processed may include, but are not limited to, technical documents, operating procedures, equipment manuals, historical work orders, inspection recordings, fault voice reports, equipment nameplate images, infrared thermal images, partial discharge maps, etc.

[0039] The document data refers to information recorded in text format. For example, document data may include, but is not limited to, PDF format operation and maintenance procedures for power equipment, Word-written fault analysis reports, Excel ledgers, HTML web page technical documents, and text log files. In some embodiments, document data can be uploaded by users through the terminal's user interface or obtained from data sources such as document management systems, mobile terminals, recording devices, and online monitoring platforms.

[0040] Audio data is information recorded in audio format. Examples of audio data include, but are not limited to, recordings of on-site inspection personnel describing equipment malfunctions, dispatch call logs, audio clips of abnormal equipment operation, and voice work orders. In some embodiments, audio data can be uploaded by the user through the terminal's user interface, or obtained through real-time microphone capture, importing audio files from a mobile terminal, retrieving data from a communication system interface, or exporting data from a storage device.

[0041] Image data is information recorded in two-dimensional or three-dimensional visual form. For example, image data may include, but is not limited to, photographs of the device's exterior, infrared thermal images, X-ray perspective views, close-up images of nameplates, and terminal block diagrams. In some embodiments, image data can be uploaded by the user through the terminal's user interface, or acquired through the output interface of the terminal's camera, infrared thermal imager, drone aerial photography equipment, online monitoring camera, or professional testing instrument.

[0042] Step 220: Process the data to be processed using the corresponding data processing method to obtain the corresponding structured data, which includes information related to entities.

[0043] Structured data is a processed form of information that is easy for machines to understand and associate. For example, structured data may include, but is not limited to, tree-structured data and dialog-based documents. In some embodiments, structured data can be obtained / determined in various ways.

[0044] The corresponding data processing methods are the methods of parsing, transforming, or recognizing specific types of data to be processed. For example, the corresponding data processing methods may include, but are not limited to: using Natural Language Processing (NLP) technology to extract text and perform semantic parsing on document data; using Automatic Speech Recognition (ASR) to transcribe audio data; and using Optical Character Recognition (OCR), object detection, or image classification models to recognize content on image data.

[0045] Entity association information is data content from which entities can be extracted. For example, entity association information is an object or concept with independent semantics. For example, entity association information may include, but is not limited to: specific equipment (e.g., 10kV vacuum circuit breaker), fault phenomena (e.g., mechanism jamming), operation actions (e.g., checking linkage), personnel roles (e.g., maintenance personnel). In some embodiments, entity labels can be extracted from structured data using a Named Entity Recognition (NAME) model, or entity labels can be identified by an LLM during the extraction phase based on a domain terminology dictionary and grammar rules.

[0046] It should be noted that structured data can also be obtained in any way known to those skilled in the art, and this application is an embodiment and does not limit this.

[0047] Step 230: Construct a knowledge graph and a vector database based on structured data. The knowledge graph includes multiple nodes and edges between nodes. Nodes correspond to entities, and edges are used to indicate the relationships between entities corresponding to nodes. The vector database includes feature vectors obtained by feature extraction from structured data. There is a mapping relationship between the node corresponding to the same entity in the knowledge graph and the corresponding feature vector in the vector database.

[0048] Knowledge graphs can be, but are not limited to, directed or undirected graphs composed of nodes and edges. In some embodiments, titles, list items, table rows, or triples extracted from LLM in tree-structured data can be converted into nodes and relationships in a graph database (such as Neo4j). Each node has metadata including entity type, confidence level, source identifier (e.g., document data ID of the source, chapter path or audio data ID, and recording timestamp). During the construction of the knowledge graph, identical entities from different data to be processed can also be merged to eliminate duplication and conflict, forming a unified knowledge representation.

[0049] For example, nodes can represent specific equipment models, fault phenomena, handling measures, and other concepts or objects, while edges can represent the relationships between connected nodes, such as hierarchical relationships and semantic relationships. Semantic relationships within associations represent the logical connections between entities, such as causal relationships, step-by-step relationships, complementary relationships, subordinate relationships, and compositional relationships. For instance, "rectification measures" and "rectification personnel" have a complementary relationship; combining them can provide a more complete description of the rectification information. Another example is the subordinate relationship between "safety accident" and "accident type." Hierarchical relationships represent the logical structure of entities within the data to be processed, such as first-level headings and second-level headings. For example, from the information that a circuit breaker tripping may lead to a power outage, we can identify two entities: the circuit breaker and the power outage, as well as the semantic relationship between these two entities (e.g., a causal relationship indicating the cause of the fault).

[0050] In some embodiments, the vector database may include, but is not limited to, a vector index library built on FAISS, Milvus, or Pinecone, wherein each piece of information stored in the vector index library corresponds to a feature vector of a document fragment (such as a question-answer pair, a paragraph of processing steps, or a summary block). In some embodiments, structured data may also be segmented into reasonable document fragments according to semantic boundaries, and embedded using a pre-trained embedding model (such as BGE-M3, embedding, etc.) to convert it into a fixed-dimensional feature vector. This feature vector is then associated with the original document, the document fragment ID, and the entity label to which it belongs and stored in the vector database.

[0051] A vector database is a database used to store, index, and query vectors. For example, a vector database can store feature vectors corresponding to a large number of document fragments. This vector database can support vector similarity-based retrieval, enabling the rapid finding of feature vectors semantically similar to the query information.

[0052] A feature vector is a vector representation of the content corresponding to structured data. In some embodiments, during the construction of a vector database, the structured data is further segmented into multiple semantically complete document fragments (such as a complete question-and-answer pair or a paragraph of troubleshooting steps), and feature vectors are generated by embedding each text fragment (e.g., mapped to an embedding vector). For example, feature vectors may include, but are not limited to, 768-dimensional or 1024-dimensional floating-point arrays output by an embedding model.

[0053] A mapping relationship refers to the correspondence between nodes in a knowledge graph and feature vectors in a vector database, used to record the relationship between the two. Specifically, constructing a mapping relationship can include: storing the node identifier corresponding to the same entity in the knowledge graph in the metadata of the vector database, and / or recording the identifier of the feature vector corresponding to the same entity in the vector database in the node attributes of the knowledge graph. For example, when there is a node in the knowledge graph representing the entity "10kV circuit breaker," a mapping relationship is established between this node and the feature vectors of technical documents, reports, and other data about 10kV circuit breakers stored in the vector database.

[0054] It should be understood that other methods can also be used to construct knowledge graphs and vector databases, and the above description of constructing knowledge graphs and vector databases should not be construed as a limitation on the embodiments of this application.

[0055] In some embodiments of this application, the knowledge graph and vector database are combined through the above mapping relationship to support multiple query methods, including relational queries based on the knowledge graph, semantic similarity queries based on the vector database, or hybrid queries, which can provide users with comprehensive and accurate knowledge recommendations.

[0056] Step 240: Based on the obtained query information, determine the target query intent of the target entity in the query information, and determine the target query strategy based on the target query intent; the target query intent includes one of relational reasoning intent, content retrieval intent, and fuzzy reasoning intent; Query information refers to information entered by the user to obtain knowledge content related to a specific question or topic. For example, query information may include, but is not limited to, questions in natural language, keyword combinations, or descriptions of fault phenomena. Alternatively, query information may also be audio data and / or image data entered by the user in the user interface of the terminal for querying.

[0057] The target entity is a concept or object in the query information. There can be one or more target entities, depending on the specific content of the query information.

[0058] In some embodiments, the query information can be analyzed using a named entity recognition model to output the identified target entities and the confidence level of each target entity.

[0059] Relational inference intent refers to a user's intention to obtain the relationship between a target entity and other entities. For example, the query information may contain keywords such as "reason," "impact," and "relationship." Content retrieval intent refers to a user's intention to obtain descriptive, factual, or experiential content surrounding a target entity. For example, the query information may include keywords such as "related cases" or "solutions." Fuzzy inference intent refers to a user's query intent that is relatively broad or complex. It should be noted that fuzzy inference intent can be a user's query information that simultaneously contains relational inference and content retrieval intents, or a query information that contains a broad and complex expression requiring multiple steps of analysis.

[0060] In some embodiments, the target query intent of the target entity in the query information can be determined in various ways. For example, one or more semantic elements related to the query information can be obtained; the semantic elements at least reflect the semantic features of the query information entered by the user in the current user interface. In some embodiments, the semantic elements may include, but are not limited to: the preprocessed lexical sequence of the query text currently entered by the user, the named entity recognition result (e.g., the target entity in the query information), syntactic structure features (e.g., interrogative word type, dependency relation), keywords contained in the query (e.g., cause, step, similar, etc.), and optionally, the user's historical query records (e.g., multiple queries related to circuit breaker tripping, clicking on the fault diagnosis function module, etc.).

[0061] In some embodiments, semantic elements can be input into a pre-trained intent recognition model, which may include, but is not limited to, deep neural networks (DNNs), recurrent neural networks (RNNs), convolutional neural networks (CNNs), or Transformer architectures, to output probability distributions of various query intents (e.g., [relational inference: 0.85, content retrieval: 0.3, fuzzy inference: 0.2]), and select the query intent with the highest probability as the final target query intent. In some embodiments, semantic elements can be encoded into vector form, for example, converted into vector representations through an embedding model. Based on the vector representations of semantic elements, a nearest neighbor search is performed in an intent text database to obtain the intent category that is semantically closest to the target query intent. The intent text database stores predefined standardized intent categories and their corresponding vector representations, and nearest neighbor search can be achieved based on metrics such as cosine similarity, Euclidean distance, or Manhattan distance.

[0062] The methods for determining the target query intent are only provided as examples; other methods may also be used.

[0063] Step 250: If the target query intent includes relational reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph; if the target query intent includes content retrieval intent, the target query strategy includes querying the candidate feature vector associated with the target entity in the vector database; if the target query intent includes fuzzy reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph and querying the candidate feature vector associated with the target entity in the vector database.

[0064] A subgraph is a local graph structure formed by taking the node corresponding to the target entity as the center and traversing predefined relational edges (such as cause and effect, steps, etc.) for several hops. For example, a subgraph may contain the target entity and / or other nodes and connecting edges that have direct or indirect relationships with the target entity. In some embodiments, the subgraph can be dynamically generated using a graph query language (such as Cypher).

[0065] Candidate feature vectors refer to feature vectors stored in a vector database that correspond to document fragments associated with the target entity. Document fragments can be question-and-answer pairs in structured data, paragraphs of fault handling steps, etc. In some embodiments, all corresponding feature vectors can be selected from the vector database as candidate feature vectors based on the target entity identifier and mapping relationship. Alternatively, an enhanced query vector can be generated by concatenating the target entity with the query statement, and a nearest neighbor search can be performed in the vector database to return the multiple feature vectors with the highest semantic similarity and their corresponding document fragments.

[0066] In some embodiments, when the target query intent includes fuzzy reasoning intent, queries can be performed in parallel or serial manner in the knowledge graph and vector database.

[0067] Step 260: Based on the target query strategy, query from the knowledge graph and / or vector database to obtain query results.

[0068] In some embodiments, prompts can be generated for the language model based on the query results, and the prompts can be input into the language model to obtain the answer content corresponding to the query results.

[0069] Prompt information refers to organized content containing retrieved knowledge, used to guide the language model in generating accurate and complete answers. For example, subgraphs can be converted into text descriptions (e.g., in the form of summary text), and document fragments corresponding to candidate feature vectors can be extracted and organized into structured prompt information based on preset prompt word templates.

[0070] In some embodiments, at least one node in the knowledge graph corresponds to multiple edges, and one edge corresponds to one relation type; if the target query intent includes fuzzy reasoning intent, the target query strategy also includes querying the knowledge graph first and then querying the vector database. Based on a target query strategy, queries are performed from knowledge graphs and / or vector databases to obtain query results, including: Determine the target relationship type corresponding to the target entity based on the query information; Based on the target relationship type and query information, a query is performed in the knowledge graph to obtain a subgraph associated with the target entity; based on the nodes contained in the subgraph, a query is performed in the vector database to obtain candidate feature vectors that have a mapping relationship with the nodes contained in the subgraph. or, Based on the target relationship type and query information, a subgraph associated with the target entity is obtained by querying the knowledge graph; an enhanced query vector is generated based on the subgraph and query information, and a candidate feature vector matching the enhanced query vector is obtained by querying the vector database. The query results are determined based on the subgraph and candidate feature vectors.

[0071] It is understandable that the relationship types of multiple edges corresponding to a node can be the same or different. For example, the relationship types corresponding to multiple edges connected to a node can be completely different, or partially or completely the same. When multiple relationship types are completely different, it means that the multiple edges connecting the node each correspond to a different relationship type. For example, the "10kV circuit breaker" node is connected to different nodes through different relationship types such as causal relationship and composition relationship. When multiple relationship types are partially or completely the same, it means that there are edges of the same relationship type between the node and multiple nodes. For example, the "10kV circuit breaker" node is connected to multiple causal relationship edges, and these causal relationship edges point to nodes such as "failure to operate," "maloperation," and "overheating," with each node corresponding to an entity representing a different fault phenomenon.

[0072] The target relation type refers to the relation type that is preferentially traversed during the knowledge graph query process. The target relation type may include one or more relation types to be queried. In some embodiments, the target relation type can be obtained by identifying the relation type of the query information. For example, a trained machine learning model can be used to process the query information and output the relation type identification result, which includes one or more identified relation types and the confidence level of that relation type. For example, the machine learning model that outputs the relation type identification result may include any one or a combination of Convolutional Neural Networks (CNN) models, Neural Networks (NN) models, or other custom model structures.

[0073] For example, for the query information "common faults and handling steps of 10kV circuit breakers", the target relation type is {causal relationship: 0.85, step relationship: 0.92, complementary relationship: 0.3}, which indicates that it is necessary to traverse the relation types of causal relationship and step relationship in the knowledge graph at the same time.

[0074] In an optional embodiment, when a query needs to be performed in the knowledge graph, the target query intent may also include sub-inference intents in the knowledge graph, with different sub-inference intents corresponding to different target relationship types. For example, for a sub-inference intent for fault diagnosis, the causal relationship of "phenomenon-cause-solution" is preferred; for a sub-inference intent for operation guidance, the step relationship of "equipment-operation steps-precautions" is preferred.

[0075] After obtaining the target query intent, if the target query intent is a fuzzy reasoning intent, the query order between the knowledge graph and the vector database, as well as the target relationship type in the knowledge graph, are determined based on the target query intent.

[0076] For example, in the scenario of power equipment maintenance, when the query information contains explicit names of power equipment (such as "main transformer", "circuit breaker", "GIS switchgear"), system status (such as "trip", "overload", "harmonic exceedance") and relational terms (such as "cause", "lead to", "belong to", "located in"), it is determined that the target query intent includes relational reasoning intent. Assuming that the target query intent includes relational reasoning intent, the target query strategy includes traversing the relationship types of nodes and edges in the knowledge graph to retrieve the matching fault paths or protection action sequences, thereby obtaining the query results.

[0077] For example, when the query information includes phrases such as "find historical cases similar to the current fault signal" or "technical documents for relays," it is determined that the target query intent includes content retrieval intent. For a target query intent that includes content retrieval intent, the target query strategy includes converting the query information into a query vector, using this query vector to perform a proximity search in a vector database to find the closest candidate feature vector, and then locating the relevant document fragment.

[0078] If the query information includes both relational reasoning and content retrieval requirements, or if the query information is broadly or complexly worded, then the target query intent is determined to include fuzzy reasoning intent. In this case, for a target query intent that includes fuzzy reasoning intent, the target query strategy includes using parallel or serial methods to query the knowledge graph and vector database.

[0079] In some embodiments, node identifiers corresponding to all nodes in the subgraph can be extracted. Based on these node identifiers, a matching query is performed in the metadata of the vector database to find all candidate feature vectors associated with these entities. These candidate feature vectors represent document fragments related to the entities contained in the subgraph.

[0080] Enhanced query vectors are feature vectors generated for vector database retrieval by incorporating knowledge related to the target entity from a knowledge graph, based on the original user query information. By introducing knowledge relevant to the target entity, the accuracy and relevance of vector matching are improved.

[0081] In some embodiments, the target entity and target relationship type are determined based on the user query information, and a subgraph associated with the target entity is retrieved from the knowledge graph. Subsequently, the node identifiers and relationship path-generated summary text (such as "the processing steps for mechanism jamming include checking the connecting rod and lubricating the drive shaft") contained in the subgraph are concatenated with the original query information. The concatenated information is then processed through an embedding model (such as BGE-M3) to obtain an enhanced query vector. Alternatively, the original query information is embedded to obtain a query vector, and the summary text is embedded to obtain a subgraph vector. The two vectors are then weighted and fused (such as 0.6 × query vector + 0.4 × subgraph vector) to obtain an enhanced query vector.

[0082] In some embodiments, an enhanced query vector can be generated by concatenating the nodes and query information contained in the subgraph. Based on the enhanced query vector, a retrieval is performed in the vector database to determine the reference feature vector that meets the matching conditions as the candidate feature vector.

[0083] The matching criteria refer to the judgment conditions used to determine candidate feature vectors. Matching criteria can include conditions such as the vector distance to the enhanced query vector being less than a distance threshold or the vector distance being the minimum. There are various methods for calculating vector distance, such as Euclidean distance and cosine distance.

[0084] In some embodiments, the target query strategy further includes querying the vector database first and then querying the knowledge graph; Based on a target query strategy, queries are performed from knowledge graphs and / or vector databases to obtain query results, including: Feature extraction is performed based on the query information to obtain the query vector corresponding to the query information. Candidate feature vectors that match the query vector are queried in the vector database. Based on the candidate feature vectors, a subgraph is obtained by querying the knowledge graph to obtain the nodes that have a mapping relationship with the candidate feature vectors.

[0085] For example, by using mapping relationships, the node identifier in the metadata of the candidate feature vector can be queried in the knowledge graph to obtain the subgraph consisting of the node corresponding to the node identifier and the relationship between the nodes.

[0086] By first querying the vector database and then the knowledge graph, the results of semantic similarity retrieval can be used to further supplement the relational reasoning results provided by the knowledge graph (such as the attributes or relationships corresponding to the target entity), thereby improving the accuracy of the retrieval.

[0087] It should be noted that the above description of the target query strategy querying from knowledge graphs and / or vector databases is not restrictive. The types of query strategies included in the target query strategy may include more or less content, and queries may be made from knowledge graphs and / or vector databases based on the target query strategy in other ways. This application embodiment does not limit this.

[0088] In some embodiments, the query results are determined based on the subgraph and candidate feature vectors, including: Determine the correlation between the subgraph and the target entity, and the similarity between each candidate feature vector and the target entity; Based on the structural features of the subgraph, determine the first weight corresponding to the subgraph and the second weight corresponding to the candidate feature vector; For each candidate feature vector, a weighted average is performed based on the correlation degree and first weight of the subgraph, as well as the similarity and second weight of the candidate feature vector, to obtain the matching score value of the candidate feature vector. Based on the matching score of each candidate feature vector, the target feature vector is determined from multiple candidate feature vectors; The query results are obtained based on the target feature vector and the subgraph.

[0089] Relevance reflects the degree of matching between the content of a subgraph and the target entity. For example, relevance can be obtained by calculating the semantic relevance between the entities corresponding to the nodes in the subgraph and the target entity. Alternatively, relevance can be determined based on the degree of overlap between the entities corresponding to the nodes in the subgraph and the entities in the query information.

[0090] The structural characteristics of a subgraph can be determined based on the total number of nodes, the total number of edges, and the number of entity types contained in the subgraph.

[0091] Understandably, different feature structures of subgraphs correspond to different first and second weights, and this correspondence can be determined based on experiments or experience. For example, the more nodes a subgraph has and the more entity types it contains, the higher its first weight and the lower the second weight of the candidate feature vector; conversely, the fewer nodes a subgraph has and the fewer entity types it contains, the lower its first weight and the higher the second weight of the candidate feature vector.

[0092] For each of the multiple candidate feature vectors, a weighted average is applied based on the relevance of its corresponding subgraph (with a first weight), and the similarity of the candidate feature vector itself (with a second weight), to obtain a matching score for each candidate feature vector. The matching score can be calculated using the weighted sum formula: Matching Score = (Relevance × First Weight) + (Similarity × Second Weight). For example, if a candidate feature vector has a subgraph relevance of 0.85, a first weight of 0.6, a similarity of 0.80, and a second weight of 0.4, then its matching score is (0.85 × 0.6) + (0.80 × 0.4) = 0.51 + 0.32 = 0.83. A higher score indicates a higher overall fit between the candidate feature vector and its associated knowledge path and the user's query intent.

[0093] In some embodiments, the matching score of each candidate feature vector may further include an adjustment factor. The adjustment factor may be determined based on factors such as the importance of the entity in the subgraph in the power field and the timeliness of the data to be processed corresponding to the subgraph. For example, the matching score of each candidate feature vector is = (relevance × first weight) + (similarity of candidate feature vector × second weight) + adjustment factor.

[0094] The target feature vector refers to one or more candidate feature vectors with the highest matching score, which are used to generate subsequent prompts. For example, a scoring threshold can be set (only candidate feature vectors with matching scores exceeding the threshold are selected) or multiple candidate feature vectors with the highest ranking can be selected (such as the top 5 candidate feature vectors with the highest matching scores) as the target feature vector.

[0095] In some embodiments, query results can be obtained based on the document fragment corresponding to the target feature vector and the summary text corresponding to the subgraph.

[0096] In some embodiments, prompt information can be generated by organizing the document fragment corresponding to the target feature vector and the summary text corresponding to the subgraph using a preset prompt word template. This prompt information is then input into a language model, allowing the language model to generate corresponding response content based on the query results as reference knowledge. For example, the preset prompt word template may include "the content of the question description" + "the content of the summary text" + "the document fragment corresponding to the target feature vector," etc.

[0097] A language model is a machine learning model trained using large amounts of text data, based on deep learning techniques. For example, a language model can be a Large Language Model (LLM). Exemplary large language models include, but are not limited to, BERT (Bidirectional Encoder Representation from Transformers), GPT (Generative Pre-Trained Transformer), XL-Net, and ChatGLM-6B.

[0098] In some embodiments, the data to be processed is processed using a corresponding data processing method to obtain corresponding structured data, including: The data to be processed is processed in a manner corresponding to the type of data to be processed, so as to obtain standardized data; The standardized data is evaluated to obtain at least one of the following: sensitivity information and complexity information. The sensitivity information includes at least one of the following: sensitivity score, sensitive entity, sensitivity type and severity level of the sensitive entity, and sensitivity level label of the data to be processed to which the standardized data belongs. The complexity information includes complexity. If the sensitivity information meets preset conditions and / or the complexity is greater than a preset complexity threshold, the standardized data is format-converted to obtain converted standardized data. The converted standardized data is then structured to obtain corresponding structured data. The preset conditions include: the sensitivity score exceeds a preset score threshold, and / or the number of sensitive entities with a severity level greater than a preset level threshold exceeds a preset quantity threshold, and / or the standardized data contains sensitive entities with a target severity level and / or a target sensitivity type, and / or the sensitivity level is marked with a preset sensitivity level mark.

[0099] Standardized data refers to data to be processed after standardization.

[0100] In some embodiments, data to be processed is acquired, and data standardization processing is performed on the data to be processed to obtain standardized data. Standardization processing includes, but is not limited to, format conversion, deduplication of redundant data, handling of missing values, and removal of irrelevant information.

[0101] It should be noted that the format conversion process specifically involves converting the data to be processed into XML or JSON format. The redundancy removal process involves using device name and connection relationship as unique identifiers to remove duplicate records from the data to be processed. The missing value handling process involves filling in numeric fields (such as rated power and voltage level) with average values ​​and removing records with empty values. The irrelevant information removal process involves clearing log and procedure notes or invalid fields from the data to be processed.

[0102] Sensitivity information and complexity information are sets of information output from the assessment process, reflecting potential risk content in standardized data. For example, assessment results may include, but are not limited to, the content of identified sensitive entities, the sensitivity type to which the sensitive entity belongs (such as "ID number" or "IP address"), the location information of the sensitive entity (such as the offset of the sensitive entity in the document fragment), confidence level and severity level (such as "high risk", "medium risk", or "low risk"), complexity, and the preset sensitivity level label of the data to be processed to which the standardized data belongs.

[0103] In some embodiments, the evaluation result can be determined through various methods such as language models or preset rules.

[0104] Sensitive entities are concepts or objects identified in the assessment results as posing a security or leakage risk. For example, sensitive entities may include, but are not limited to, personally identifiable information (such as name, ID number, contact information, etc.), internal company identifiers (such as equipment serial numbers, work order numbers, etc.), geographic location information, account passwords, financial data, or controlled technical parameters. In some embodiments, regular expressions can be used to match fixed-format phone numbers or email addresses, or a pre-trained entity recognition model can be used to detect sensitive fields such as names and phone numbers in standardized data.

[0105] Sensitive types of sensitive entities can include, but are not limited to, individuals, networks, authentication credentials, departments, etc.

[0106] The severity level of a sensitive entity is a quantitative or qualitative indicator that measures the degree of harm that the entity may cause. For example, the severity level of a sensitive entity may include, but is not limited to, three levels: "high," "medium," and "low," or a numerical score may be used to represent the risk intensity. In some embodiments, the level of a sensitive entity may be set according to its type (e.g., an ID number is considered a high level).

[0107] The location information of a sensitive entity is an identifier that indicates the specific location of that information within the standardized data. For example, the location of a sensitive entity may include, but is not limited to, the location offset of the sensitive entity, or a timestamp interval (for standardized data corresponding to audio data).

[0108] Confidence level is the reliability of determining whether a certain content is a sensitive entity. Confidence level can be expressed as a probability or percentage and is used to measure the credibility of the identification result. For example, confidence level can include, but is not limited to, the classification probability output by a machine learning model.

[0109] The preset sensitivity level label of the data to be processed to which the standardized data belongs can refer to the sensitivity level label indicated in the metadata of the data to be processed to which the standardized data belongs or in the source label.

[0110] The preset sensitivity level label can refer to the label or source label in the metadata of the data to be processed being "highly sensitive".

[0111] Preset conditions are used to determine whether standardized data needs to undergo format conversion or other processing. Preset conditions can include one or more, with different sensitivity or complexity information corresponding to different preset conditions.

[0112] The standardized data contains sensitive entities with target severity level and / or target sensitivity type. This can mean that multiple sensitive entities with specified target sensitivity type appear simultaneously in the standardized data, or that multiple sensitive entities with specified target severity level appear simultaneously in the standardized data, or that multiple sensitive entities with specified target severity level and target sensitivity type appear simultaneously in the standardized data.

[0113] For example, the situation in which standardized data contains sensitive entities with target severity level and / or target sensitivity type includes: the standardized data contains sensitive entities of sensitive type such as name, phone number, and address.

[0114] Format conversion is the process of de-identifying standardized data when preset conditions are met. For example, format conversion may include, but is not limited to, de-identifying and replacing sensitive entities (such as replacing a mobile phone number with "**"), deleting nodes, replacing content (such as replacing a specific location with "a certain area"), or converting to a preset storage format (such as TXT).

[0115] In some embodiments, when the sensitivity information meets preset conditions, the standardized data can be converted into a format to obtain the converted standardized data; or, when the complexity is greater than a preset complexity threshold, the standardized data can be converted into a format to obtain the converted standardized data; or, when the sensitivity information meets preset conditions and the complexity is greater than a preset complexity threshold, the standardized data can be converted into a format to obtain the converted standardized data.

[0116] In some embodiments, the sensitivity information may be determined to meet the preset conditions when the sensitivity score exceeds a preset score threshold; or, the sensitivity information may be determined to meet the preset conditions when the number of sensitive entities with a severity level greater than a preset level threshold exceeds a preset quantity threshold; or, the sensitivity information may be determined to meet the preset conditions when the standardized data contains sensitive entities with a target severity level and / or a target sensitivity type; or, the sensitivity information may be determined to meet the preset conditions when the sensitivity level is marked as a preset sensitivity level mark.

[0117] Among them, the preset complexity threshold, preset scoring threshold, preset level threshold, preset quantity threshold, target severity level, target sensitivity type, etc. can be system default values, experience values, human preset values, etc., or any combination thereof, and can be set according to actual needs. This application embodiment does not limit this.

[0118] In some embodiments, if the sensitivity information does not meet the preset conditions and the complexity is not greater than the preset complexity threshold, the standardized data is processed in a structured manner to obtain the structured data corresponding to the standardized data.

[0119] In some embodiments of this application, the security of standardized data used to construct knowledge graphs and vector databases can be ensured through evaluation and format conversion.

[0120] In some embodiments, the data to be processed is document data. The data to be processed is processed in a manner corresponding to the type of data to be processed to obtain standardized data, including: The document data is parsed and extracted to obtain at least one first content block; Based on at least one first content block, standardized data corresponding to the document data is obtained. Each first content block has at least one attribute information for describing the first content block. The attribute information of the first content block includes the type information and hierarchy information of the first content block. The type information of the first content block includes at least one of title, list, table or paragraph.

[0121] The first content block is the basic unit that constitutes the document data, corresponding to a text segment in the document data that has independent semantic or format characteristics.

[0122] The attribute information of a first content block refers to the set of metadata attached to each first content block to characterize its logical structure and semantic features. This attribute information includes at least type and hierarchy information, enabling subsequent processing modules to understand the functional positioning and semantic information of the first content block within the document. Specifically, "heading" indicates that the first content block is a chapter or section name; "list" indicates that the first content block is a collection of entries composed of bullet points or numbered items; "table" indicates that the first content block is a matrix of structured data; and "paragraph" indicates that the first content block is narrative text. Hierarchical information represents the logical structure of the first content block within the document data. For example, for a first content block of the heading category, its hierarchy is determined by the number of heading symbols (e.g., the number of # symbols in Markdown) or style hierarchy (e.g., "Heading 1" and "Heading 2" in Word). The hierarchy of first content blocks of non-heading categories (such as paragraphs or lists) can be determined based on the hierarchy of first content blocks of the parent heading category.

[0123] In some embodiments, parsing and extracting document data to obtain at least one first content block refers to analyzing input multi-format document data (such as PDF, Word, PPT) using a structure-aware document parsing engine, identifying block-level text fragments, and outputting a sequence of content blocks labeled with attribute information. In some embodiments, a deep learning-based hybrid model (such as LayoutLM or Donut) can be used, simultaneously utilizing text content, visual layout coordinates, and formatting tags to distinguish different types such as titles, body text, list items, and tables, and aggregating consecutive lines of the same type into a complete first content block. For example, a line starting with "## Fault Diagnosis Process" is identified as a first content block of type "Title" and its level is determined to be "2" based on the number of "#" characters; several subsequent lines starting with "-" are merged into a first content block of type "List" and inherit the level of its nearest preceding title.

[0124] In some embodiments, obtaining standardized data corresponding to document data based on at least one first content block means integrating all first content blocks and their attribute information into standardized data, which helps to obtain input for subsequent knowledge extraction.

[0125] Standardized data eliminates format noise (such as watermarks and irrelevant comments) in the original document data while fully preserving the logical structure of the document. This allows language models or rule engines to perform entity recognition and relation extraction based on the understanding that "a certain paragraph belongs to the second list item under the second section of Chapter 3", thereby improving the accuracy and interpretability of knowledge construction.

[0126] In some embodiments, standardized data can be semantically enhanced and semantically tagged to obtain structured data.

[0127] In some embodiments, the data to be processed is audio data. The data to be processed is processed in a manner corresponding to the type of data to be processed to obtain standardized data, including: The transcribed text obtained from the audio data is segmented to obtain at least one speech unit; Identify at least one discourse unit to obtain the question text and the answer text; By associating the question text with the answer text, at least one question-answer pair can be obtained; Based on at least one question-answer pair, standardized data corresponding to the audio data is obtained.

[0128] Transcribed text is the text output obtained by converting audio data. For example, transcribed text can include, but is not limited to, the text of audio data, speaker annotations with timestamps, etc.

[0129] In some embodiments, transcribed text can be obtained in a variety of ways. For example, a speech recognition model can be used to decode audio data to generate text content, and speaker separation technology can be used to distinguish and label different speakers to obtain transcribed text with timestamps and speaker annotations.

[0130] A discourse unit is the smallest segment of speech with independent semantics that is separated from transcribed text. One discourse unit can correspond to one continuous speech. For example, a discourse unit can include, but is not limited to, a declarative sentence, a question, etc.

[0131] Question text is one or more discourse units that are identified and categorized as interrogative statements, representing the questions raised by users or participants. For example, question text may include, but is not limited to, questions such as "How do I handle this fault?", "What was the cause of a similar situation last time?", and "Are there any recommended operating procedures?"

[0132] Answer text is one or more responsive discourse units that correspond to a specific question text, representing a solution, explanation, or suggestion to the question. For example, answer text may include, but is not limited to, expert solutions to specific faults, summaries of experience, parameter adjustment suggestions, or historical case information.

[0133] In some embodiments, the question and answer texts can be determined in various ways. For example, a text classification model can be used to determine the type of each utterance unit to obtain the question and answer texts. The text classification model can be a trained machine learning model, such as any one or a combination of Convolutional Neural Networks (CNN), Neural Networks (NN), or other custom model structures. In some embodiments, the text classification model can be trained based on a large number of labeled training samples using various feasible methods. For example, parameters can be updated using gradient descent. In some embodiments, the training samples include at least sample text. The training samples can be obtained based on historical data.

[0134] In some embodiments, the label may include the type corresponding to the training sample. Labels can be obtained through automatic or manual annotation.

[0135] A question-and-answer pair is a text set consisting of a question and its corresponding answer, representing an interactive question-and-answer session. For example, a question-and-answer pair may include, but is not limited to, ("What should I do if the switch won't close in winter?", "Check the lubrication of the magnetic control mechanism and replace the low-temperature grease if necessary.").

[0136] In some embodiments, the data to be processed is image data. The data to be processed is processed in a manner corresponding to the type of data to be processed to obtain standardized data, including: Image data is processed to obtain textual descriptions of the content of the image data. The text description information is parsed and extracted to obtain at least one second content block; Based on at least one second content block, standardized data corresponding to the image data is obtained, wherein each second content block has at least one attribute information for describing the second content block, and the attribute information of the second content block includes the type information of the second content block, which includes at least one of title, list, table or paragraph.

[0137] In some embodiments, semantic content can be automatically extracted from image data using artificial intelligence technology to generate natural language text, thus obtaining text description information. In some embodiments, text description information can also be obtained using any method well known to those skilled in the art, and this application embodiment does not impose any limitations on this.

[0138] In some embodiments, for text description information, a second content block can be obtained through parsing and extraction methods similar to those used for document data. For more information on parsing and extracting document data, please refer to the relevant descriptions above.

[0139] In some embodiments, standardized data (e.g., standardized text, standardized transcribed documents, hereinafter referred to as standardized data) is evaluated. Based on the evaluation results, the standardized data or the transformed standardized data is subjected to structured processing to obtain structured data corresponding to the data to be processed. A knowledge base is then constructed based on the structured data corresponding to the data to be processed. The knowledge base may include storage formats such as knowledge graphs and vector databases.

[0140] Structured processing of standardized data to obtain structured data refers to transforming the already obtained standardized data into an intermediate representation that can be directly understood by machines and used for downstream tasks (such as knowledge extraction, retrieval enhancement, and generation) through semantic parsing, data organization, and other methods. This structured processing is not a simple format conversion, but rather the identification, association, and reconstruction of implicit or explicit semantic units in the standardized data to generate a structured intermediate representation.

[0141] Specifically, when standardized data originates from document data, the standardized data contains multiple first content blocks. Each first content block carries type information (such as headings, paragraphs, lists, and tables) and hierarchy information (such as first-level headings and paragraphs under second-level headings). The structured processing of document data can include: constructing recursively nested tree-structured data (such as JSON objects) based on the attribute information of the first content blocks. In this structure, each node corresponds to a first content block, and the node attributes include its type, hierarchy, original text content, and list of child nodes.

[0142] When standardized data originates from audio data, it comprises question-and-answer pairs (e.g., question text and answer text). Optionally, standardized data may include speaker identifiers, timestamps, text type labels (e.g., question or answer), and key term tags. The structured processing of audio data includes: encapsulating each question-and-answer pair into a uniform semantic unit and labeling it with text type identifiers; simultaneously, based on domain terms, action verbs, or contextual topics within the question-and-answer pairs, further extracting entities, relationships, or action items, and clustering the various question-and-answer pairs (e.g., clustering multiple question-and-answer pairs involving the same technical topic into the same topic set). The resulting structured data can be represented as a set of objects with a fielded structure (e.g., a JSON array), each object containing a question field, an answer field, metadata fields (speaker, time, tags, etc.), and optional semantic enhancement fields (e.g., key entities, suggestion type, warning level, etc.).

[0143] When standardized data originates from image data, the standardized data contains multiple second content blocks and can be structured in a manner similar to that used for document data.

[0144] In some embodiments, the method further includes: Semantic understanding is performed on at least one type of newly added data to be processed to obtain updated information, which includes at least one of updated node information, updated edge information, or updated feature vector information. Based on the updated information, the knowledge graph and / or vector database are updated to obtain the updated knowledge graph and / or vector database.

[0145] Update information is a knowledge increment generated by semantic understanding of at least one type of new data to be processed, used to optimize knowledge graphs and / or vector databases. It reflects the knowledge content that needs to be added or adjusted. For example, update information may include, but is not limited to, newly added entity nodes (such as previously unidentified equipment models), newly added relationship edges (such as "temperature sensor failure" leading to "control system misjudgment"), supplementary information on the attribute information of existing nodes (such as adding rated current parameters to a circuit breaker node), or correction information on existing edge relationships (such as correcting a misjudged "parallel" relationship to a "sequential" relationship), newly added feature vectors, etc.

[0146] Updated node information is used to represent nodes that need to be added, modified, or deleted, and / or related information about node attributes.

[0147] The updated edge information is used to represent relevant information about the relationships between nodes that need to be added, adjusted, or removed.

[0148] The updated feature vector information is used to represent relevant information about feature vectors that need to be added, modified, or deleted.

[0149] In some embodiments of this application, the updated knowledge graph and / or vector database are the final optimized results, possessing higher completeness, accuracy, and semantic richness.

[0150] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0151] To better understand the above scheme, the data processing method will be explained below with a specific embodiment.

[0152] This application provides a knowledge management system, such as... Figure 3As shown, this knowledge management system includes a data preprocessing layer, a knowledge construction layer, a retrieval and reasoning layer, and a security monitoring layer. Each layer interacts with the others through predefined interfaces, enabling independent deployment and collaborative operation, thus improving the system's maintainability, scalability, and security.

[0153] The data preprocessing layer receives user-uploaded data and processes it accordingly to generate a structured or semi-structured intermediate representation. This data includes, but is not limited to, text files such as PDF, Word, PPT, and TXT, and audio files such as WAV, MP3, and M4A. The data preprocessing layer first performs modal recognition on the input data based on its file extension: for text files, the data enters the text processing channel; for audio files, the data enters the audio processing channel.

[0154] For user-uploaded text documents (also known as document data), the data preprocessing layer employs a hybrid parsing engine based on deep learning to process the input document data. This hybrid parsing engine integrates Optical Character Recognition (OCR) technology with document layout analysis models, such as the LayoutLM model. The engine can understand the visual layout information and semantic logical structure of the document data. Specifically, it can identify text elements (also known as first content blocks) such as titles, paragraphs, lists (e.g., bulleted lists), tables, headers, footers, and multi-column layouts, and establish the logical structural relationships (e.g., spatial and logical relationships) between these text elements. For example, the engine can identify a piece of text information located below the title "Chapter 3" and within a bulleted list in the right sidebar of the page, thus preserving the document's hierarchical structure and order. After processing, the hybrid parsing engine outputs a standardized text that retains the original document's logical structure and layout semantics. This standardized text can be in formats including, but not limited to, Markdown, XML, JSON, or custom markup languages, facilitating subsequent reading and utilization by the knowledge construction layer.

[0155] For user-uploaded audio files (also known as audio data), the data preprocessing layer first calls the open-source Automatic Speech Recognition (ASR) service Whisper to convert the input audio file into timestamped raw transcribed text. This transcription process preserves the time-series information of the speech signal, ensuring that each sentence is aligned with the time points in the original audio data. Subsequently, the data preprocessing layer uses deep learning-based speaker segmentation techniques, such as pre-trained models from the PyAnnote toolkit, to perform speakerprint analysis on the audio data, identify speech segments from different speakers, and label each utterance unit with a corresponding speaker tag in the transcribed text, achieving role differentiation in multi-speaker scenarios.

[0156] To further enhance the structuring of transcribed text, the data preprocessing layer can perform semantic enhancement steps on the transcribed text, transforming linear, unstructured transcribed text into standardized transcribed documents with explicit semantic structure. Specifically, this includes: Speech unit segmentation: Based on speech duration, text length, and punctuation distribution characteristics, continuous transcribed text is segmented into independent speech units. Subsequently, each speech unit is cleaned of noise using a predefined list of filler words (such as "uh," "um," "that," etc.) to remove redundant expressions and improve the readability and semantic purity of the text.

[0157] Discourse unit type classification: Combining syntactic rules, punctuation features, and acoustic intonation probability information output by ASR, discourse units with interrogative sentence patterns are identified and classified; at the same time, through keyword matching and a lightweight text classification model, declarative discourse units expressing opinions, claims, or conclusions are identified, thereby distinguishing different types of discourse units such as questions, answers, comments, and suggestions.

[0158] Dialogue relationship construction: Based on the temporal proximity, referential consistency, and co-occurrence of thematic keywords within a preset time window, semantic links between question-answer pairs are inferred and established. For successfully matched question and answer utterance units (also known as question texts), they are encapsulated into a unified question-answer pair structure. The question-answer pair structure can be encapsulated using XML or a similar markup language, and tags can be used to annotate the structure, thus forming structured question-answer pairs.

[0159] Semantic enhancement is performed: Semantic annotation is applied to structured question-and-answer pairs. Specifically, based on a pre-defined domain terminology dictionary, part-of-speech tagging results, and word frequency statistics methods, key technical terms in discourse units are automatically identified and labeled. Simultaneously, combining action verbs, suggestions, and other keywords with contextual information, sentence fragments representing specific actions, implementation suggestions, or risk warnings are identified and labeled. Finally, a standardized transcription document rich in semantic tags such as speaker roles, question-and-answer pair structure, core viewpoints, technical terms, actions, and suggestions is output. This standardized transcription document maintains consistency with the standardized text in data format, providing a high-quality input foundation for subsequent deep semantic understanding and knowledge extraction in large language models.

[0160] In some embodiments, before inputting standardized transcribed documents and standardized text into the knowledge construction layer, the standardized transcribed documents and standardized text can be evaluated. This evaluation may include sensitivity detection and complexity detection to identify sensitive entities contained in the standardized transcribed documents and standardized text, as well as complexity information. For example, features such as the heading depth, list nesting levels, and table complexity (number of rows and columns) of the Markdown document can be analyzed to output high-level, medium-level, and low-level complexity.

[0161] For example, a detection engine based on a combination of preset rules and keyword matching, such as the Presidio framework, can be used to automatically identify sensitive entities of various sensitive types. Multiple sensitive entities can be predefined, with different sensitivity types corresponding to different severity levels, such as three severity levels: High, Medium, and Low. High-level entities correspond to core business secrets, such as undisclosed product design parameters, strategic planning schemes, and algorithm model weights; Medium-level entities include personally identifiable information, such as phone numbers, email addresses, partial geographic address information, and employee IDs; Low-level entities include information that appears independently and is difficult to directly associate with individuals or confidential content, such as single names, city names, or non-sensitive timestamps. Each sensitive entity or its severity level is assigned a corresponding weight value for subsequent scoring calculations. For example, a high-risk entity might have a weight of 1.0, a medium-risk entity 0.5, and a low-risk entity 0.1. These weight values ​​can be configured and adjusted according to actual business needs.

[0162] For example, the detection engine can perform sensitivity detection based on regular expression pattern matching rules, pre-trained named entity recognition models, user-defined keyword dictionaries, and lightweight semantic classifiers. For example, standardized transcribed documents or standardized text can be used as the text to be detected. The detection engine then performs a comprehensive analysis of the text, outputting evaluation results such as identified sensitive entities and their attribute information. Attribute information may include the type of the sensitive entity, the identifier of the source data to be processed, location information (e.g., start and end offsets), and confidence level.

[0163] To further improve detection accuracy, low-confidence assessment results can be filtered based on a preset confidence threshold. For example, only assessment results with a confidence level greater than 0.8 can be retained to reduce the false alarm rate. After filtering, the assessment results are aggregated and analyzed: the frequency of occurrence of various sensitive entities is counted, as well as the number of severity levels. Based on this, the overall sensitivity score of the text to be detected is calculated. For example, the formula Sensitivity_Score=Σ(Weight_entity×Count_entity_type) is used to calculate the overall sensitivity score, where Weight_entity represents the weight value corresponding to each type of sensitive entity (such as ID card number, bank account number, core trade secrets, etc.), and Count_entity_type represents the frequency of occurrence of this type of sensitive entity in the document. The weighted results of all types of sensitive entities are summed to obtain the final overall sensitivity score. Another example is to use another formula Sensitivity_Score=Σ(Weight_severity_level×Count_level) for calculation, where Weight_severity_level represents the weight of the severity level divided by severity (such as High=1.0, Medium=0.5, Low=0.1), and Count_level represents the number of all sensitive entities belonging to this severity level.

[0164] In some embodiments, a normalization factor may be introduced, such as dividing the total number of sensitive entities by the total number of words in the document to obtain a density index of sensitive entities, and to determine whether there are specific high-risk sensitive entities, so as to obtain sensitivity information.

[0165] In some embodiments, a set of preset conditions may be configured to determine, based on the evaluation results, whether to convert the document from its original format (such as Markdown) to plain text (TXT) format for further processing.

[0166] For example, when any high-risk sensitive entity is detected, a format conversion is immediately triggered, converting the original format to TXT format to minimize the risk of sensitive entities being exposed during structured processing. Furthermore, if the number of high-risk entities exceeds a preset threshold N, or the overall sensitivity score exceeds a preset threshold, a format conversion operation is performed. In some embodiments, sensitive entity combinations are identified; for example, when a name, phone number, and address appear simultaneously in their corresponding context, even if a single sensitive entity is only of medium risk, the combination is considered high-risk, and a format conversion operation is performed. Additionally, if the metadata or source tag of the text to be detected is marked as "highly sensitive," the format conversion operation is performed directly to ensure the priority of business strategies.

[0167] To enhance the system's adaptability and maintainability, parameters such as the definition of sensitive entity types, severity level classification, weight assignment, detection confidence threshold, and preset conditions can all be dynamically adjusted through external configuration files, management interfaces, or application programming interfaces (APIs). Alternatively, an extension interface can be provided, allowing users to register custom identifiers, such as defining sensitive entities specific to a particular industry or enterprise using regular expressions, keyword lists, or simple logical rules, thereby enabling flexible expansion of assessment capabilities.

[0168] In some embodiments, structured processing may include: filtering noise from standardized text according to preset rules, such as removing non-semantic symbols from Markdown. The preset rules include structured symbol preservation rules, which are shown in Table 1 below: Table 1

[0169] In some embodiments, the preset rules may further include link and code block processing rules, which are shown in Table 2 below:

[0170] In some embodiments, for Markdown format data, structured processing may also include hierarchical information extraction: analyzing features such as the heading depth, the number of nested lists, and the complexity of tables (number of rows and columns) of the document data, and outputting a complexity rating of the document data, which may be one of three levels: high, medium, or low.

[0171] In some embodiments, if the document data is in Markdown format, the Markdown document data can be structurally parsed to generate an intermediate representation that reflects the hierarchical relationship of the original document, thereby providing structured support for the semantic understanding and generation tasks of the subsequent large language model.

[0172] In some embodiments, a nested relationship between headings can be constructed based on the hierarchical identifiers of the headings in the document data, forming a tree-structured data. Specifically, the document data can be traversed line by line, starting from the first line. When a line is detected to begin with a specific number of specific characters (#), it is determined that the line is a heading line, and its corresponding heading level is determined based on the number of specific characters. For example, one "#" corresponds to a first-level heading, two "##" correspond to a second-level heading, and so on. Subsequently, the text content under the heading in that line is extracted, and a new heading node is created based on this.

[0173] After creating a heading node, recursive processing is performed, including iterative processing of each line of content following that heading node. This involves treating all content at a lower level than the current heading node, as well as content that hasn't yet triggered a new heading node, as children of the current heading node. When a new heading at the same or higher level as the current heading is found, the current iteration ends, and the process returns to the previous level to continue building higher-level heading nodes. This method accurately restores the hierarchical nesting relationships in Markdown document data, forming a logically complete structure tree.

[0174] like Figure 4 As shown, for non-heading block-level elements in Markdown document data, context-based attribution is applied. For example, when a line is detected as a list item (e.g., starting with "-", "*", or a number followed by "."), it is identified as a list, and consecutive sibling list items are aggregated into a complete list content block. For ordinary paragraph text, cleaning processes are performed, including removing redundant whitespace characters and merging consecutive line breaks to improve text regularity. These non-heading content blocks are assigned to their nearest and highest-level valid heading node, serving as the node of that heading node, ensuring the integrity of the semantic context and avoiding breaks or isolation of structural information.

[0175] After completing the traversal and structure construction, the output is the structured data of the document (e.g., tree structure data). This structured data can be represented in JSON format, with each node configured with several attribute fields, including node type, text content, hierarchy information (only available for header nodes), and node information. Node information stores all subordinate content blocks belonging to the current node, thus expressing the hierarchical relationship. This structured data maps the logical structure of the original Markdown document data, preserving the hierarchical relationship while eliminating interference from formatting symbols.

[0176] Furthermore, the aforementioned tree structure can be converted into a semantically enhanced intermediate representation. This intermediate representation can solve problems existing in traditional Markdown processing, such as symbol pollution, i.e., the interference of original markup symbols on model input; loss of structural information, i.e., the weakening of hierarchical relationships during linear textification; and poor compatibility with downstream tasks, i.e., the lack of a unified and parsable structured input interface.

[0177] In some embodiments, structured data can be transformed into an intermediate representation by stripping redundant syntactic symbols, preserving the hierarchical relationships of the logical structure, and labeling semantic types. When a language model receives a retrieval fragment based on this intermediate representation in a Retrieval Enhanced Generation (RAG) scenario, it can clearly identify the hierarchical structure and contextual dependencies, thereby generating response content with a clearer structure and more accurate citations, thus enhancing the accuracy of knowledge base retrieval and the interpretability of the generated results.

[0178] In some embodiments, the knowledge building layer receives structured data output from the data preprocessing layer and data to be processed as input. The structured data includes, but is not limited to, an intermediate representation generated after preprocessing that retains the original document's logical structure and semantic information.

[0179] The knowledge construction layer includes a graph construction module, an experience construction module, and a database construction module, which enables the extraction of explicit and implicit knowledge from unstructured or semi-structured documents and their transformation into a structured form that can be understood and reasoned about by machines.

[0180] The graph construction module is used to convert structured data into entities, attributes, and relationships, and store them in a graph database, such as Neo4j. Specifically, for tree-structured data, it parses the node types and hierarchical relationships, identifies first-level headings as topic entities, treats their subordinate second-level headings as concept entities, and establishes semantic relationships of "contains" or "belongs to". For tabular data in documents, it parses each row into a set of "entity-attribute-value" triples. For example, it converts "parameter name: sampling rate, value: 48kHz" in a row into the form (sampling rate, value, 48kHz) and stores this triple as a node or relationship in the graph database.

[0181] The experience building module is used to process unstructured content such as interview recordings and technical reports. Specifically, it guides the Large Language Model (LLM) to complete multi-stage, task-based knowledge extraction through prompt engineering, thereby improving the accuracy and efficiency of LLM processing.

[0182] Specifically, the experience building module can execute the following four-stage processing flow: In the first stage, input commands are used to instruct the large language model to generate summaries. For technical report documents, the model first generates an overall summary that summarizes the main idea of ​​the entire document and then extracts the core points layer by layer according to the chapter structure to form a hierarchical summary. For interview-type audio transcripts, the model generates a meeting minutes-style summary using the structured question-and-answer pairs identified in the preprocessing stage, summarizing the main viewpoints of each expert, the discussion process, and the final conclusions reached.

[0183] In the second stage, question-answer pairs are processed. Using the coarse-grained structured question-answer pairs identified in the preprocessing layer (e.g., question-answer pairs via XML tags and symbols), the large language model is prompted to optimize them. Specifically, the optimization process includes: correcting incomplete or semantically ambiguous question statements and supplementing context to make them semantically complete; merging questions and answers scattered in different locations but with the same topic to output a set of high-quality question-answer pairs; and generating a corresponding title for each question-answer pair to facilitate subsequent retrieval, thereby improving the quality and usability of the question-answer pairs.

[0184] In the third stage, context-enhanced knowledge triple extraction is performed. The large language model is prompted to extract triples of the form (entity, relation, entity) or (entity, attribute, value) from text fragments. Specifically, the prompts provide the language model with corresponding contextual information, such as indicating that the currently processed knowledge is located in a specific section of a document (e.g., "The following content comes from the 'Noise Filtering' section of Chapter 3 'Core Technologies' in 'XX System Design'"), enabling the language model to perform semantic understanding based on the contextual information and output triples for updating.

[0185] In the fourth stage, dynamic updates and source management of the knowledge graph are implemented. New entities and relationships identified from summaries, question-answer pairs, and triples are injected into the graph database to expand the coverage of the existing knowledge graph. Simultaneously, associations are established, recording source information (such as document identifiers) in the nodes and relationships of the knowledge graph. Document identifiers can include the identifier of the source document, fragment location, or timestamp range from an audio file.

[0186] The database construction module supports subsequent semantic retrieval tasks. It segments the preprocessed and knowledge-extracted data into document fragments of reasonable size based on semantic integrity principles, such as paragraphs, sections, or question-answer pairs. Then, it calls an embedding model, such as the BGE-M3 model, to convert each document fragment into a feature vector. This feature vector, along with its corresponding metadata (such as the identifier of the source data, fragment location, or timestamps from audio files), is stored in a vector database, such as FAISS. This vector database supports efficient approximate nearest neighbor search, quickly matching semantically similar text fragments during user queries, providing high-quality contextual input for Retrieval Augmentation (RAG).

[0187] Ultimately, the output of the knowledge construction layer includes: a structured knowledge graph for storing semantic relationships and attribute information between entities; a vector database for supporting semantic similarity retrieval; triples for updating, including question-answer pairs with titles; and a summary. The knowledge graph and the vector database are linked through a mapping relationship that records the identifier of the feature vector corresponding to each entity, thus establishing a connection between graph-structured knowledge and vector query results.

[0188] In summary, the embodiments of this application achieve deep semantic mining and structuring of standardized document content by constructing a knowledge graph, structuring expert experience, and generating a vector database, thereby improving the completeness, accuracy, and usability of the knowledge system.

[0189] In some embodiments, the retrieval and inference layer receives query information input by the user as input. The query information may include keywords entered by the user, images uploaded by the user, voice commands from the user, etc. For example, "Common faults and handling steps of 10kV circuit breakers". In some embodiments, the query information may include natural language, code language, etc. For example, Chinese, English, Japanese, Basic, Pascal, ObjectPascal, C, etc.

[0190] In some embodiments, the retrieval and inference layer includes a KG query agent, a workflow agent, a hybrid retrieval agent, and an LLM management agent. Each module has a clear division of labor and works together to complete the task from query parsing to answer generation.

[0191] The KG Query Agent is used to execute structured query tasks based on knowledge graphs. This agent first performs semantic parsing of the user's query information, including named entity recognition and intent analysis. For example, it can call a pre-trained domain-specific named entity recognition (NER) model to identify key terms in the query information, such as "10kV," "circuit breaker," and "trip," which are proprietary entities in the power industry, and output the type of each entity (e.g., voltage level, equipment name, fault phenomenon) and its confidence score. It can also use a trained classification model to predict the semantic intent of the query information, outputting the confidence distribution of each type of intent, such as [fault diagnosis: 0.8, operation steps: 0.6, technical principle: 0.2], to guide the selection of subsequent retrieval paths.

[0192] Based on the results of named entity recognition and intent analysis, the KG query agent generates targeted graph query information, such as Cypher language commands, to retrieve relevant subgraphs in knowledge graphs (e.g., Neo4j). It can also adjust the priority of query relation types based on the confidence distribution of various sub-inference intents. For example, when the confidence of the "fault diagnosis" intent is high, the query prioritizes the relational path (fault phenomenon) - [:cause] -> (cause); when the confidence of the "operation steps" intent is high, the query prioritizes the relational path (device) - [:steps] -> (steps). After the retrieval is complete, a subgraph, along with its structural features (e.g., the number of nodes and edges) and correlation, is returned as a basis for evaluating the quality of the graph query results.

[0193] The workflow agent coordinates the execution logic of the entire retrieval process, enabling dynamic scheduling and control of the retrieval strategy. This workflow agent receives the subgraphs and their relevance scores returned by the KG query agent and uses these scores to assess the confidence level of the KG query agent's retrieval. If the subgraph has significant structural features and its relevance score is higher than a preset threshold, the confidence level of the KG query agent's retrieval is considered high, and the weight of vector retrieval is reduced or this step is skipped. Conversely, if the subgraph information has limited structural features or the query results are empty, a broader semantic retrieval task is triggered to supplement potentially relevant information.

[0194] For complex queries, such as those involving "why," "how to compare," or "advantages and disadvantages analysis" that require multi-step reasoning, the workflow agent performs query decomposition processing, breaking it down into multiple logical sub-queries, and scheduling the KG query agent and vector retrieval module to execute them sequentially. For example, for the query "Why did the circuit breaker trip?", the KG query agent first retrieves a list of possible causes, and then retrieves each cause from the vector database.

[0195] A hybrid search engine is used to perform fusion search, enabling the synergistic use of graph and semantic information. This engine receives a subgraph returned by the KG query agent and converts it into a natural language summary text. For example, it can use templated or lightweight text generation models to transform triple relationships in the graph into coherent descriptive statements. Subsequently, this summary text is combined with the original user's query to form an enhanced query vector, and a semantic similarity search is performed in a vector database (such as FAISS). Based on relevance scores, the most relevant text fragments are retrieved.

[0196] In some embodiments, when ranking the results of semantic similarity retrieval, the hybrid retrieval system employs a dynamic weighted fusion strategy to calculate the final relevance score. The formula for calculating the relevance score is: Relevance Score = 0.6 × Similarity + 0.3 × Relevance + 0.1 × Business Weight. Here, vector similarity represents the semantic matching degree between the feature vector and the enhanced query vector; relevance reflects the connection strength and path relevance of the corresponding entity in the knowledge graph; and business weight can be a dynamically generated, normalized adjustment factor that comprehensively considers the entity's importance rating in the power sector, the timeliness index of the source document's publication, historical user feedback data, etc., used to improve the ranking priority of high-value, high-timeliness content. This weighted approach ensures that the query results are not only semantically relevant but also meet the business priority requirements of a specific domain.

[0197] The LLM management agent integrates all query results and generates the final natural language answer. This agent receives the Top-K most relevant document fragments (from a vector library) from the hybrid retrieval system and a subgraph (such as triples and relationships between nodes in the subgraph) from the KG query agent. It then organizes this information into structured prompts, which are input into the Large Language Model (LLM). The prompts include instructions, contextual information, citation tags, and output format requirements to ensure the generated content is accurate and well-organized.

[0198] Large language models can generate complete and logically rigorous natural language answers based on structured prompts in the input. For example, they can list fault types and their handling steps in bullet points and embed citation tags in the answer to indicate the source of each piece of information. The final output can include the generated answer content, as well as the relationships of the corresponding subgraphs (e.g., (circuit breaker) - [:common faults] -> (tripping) - [:handling methods] -> (check contact wear)) and the citation information of the corresponding original document fragments (e.g., document ID, chapter title, or timestamp range), thereby enhancing the credibility, traceability, and interpretability of the results.

[0199] In this embodiment, the security monitoring layer can be integrated into the output and operation and maintenance of the entire knowledge processing system, spanning the entire lifecycle from answer generation to knowledge base maintenance. This security monitoring layer not only achieves real-time protection of sensitive entities and full recording of operational behaviors, but also supports efficient updates of knowledge content and closed-loop optimization based on user feedback.

[0200] In some embodiments, the security monitoring layer is used to perform a security review of the generated answer content before the answer is output, preventing the disclosure of sensitive entities. Specifically, before the final answer is returned to the user, de-identification processing is performed, such as real-time detection and masking of any sensitive entities that may remain in the answer. Sensitive entities include, but are not limited to, medium- to high-risk entities such as phone numbers, email addresses, and employee IDs. For example, an evaluation based on a combination of preset rules and models can be used. If a sensitive entity is found, it is replaced with a mask symbol (such as ***) to ensure that the sensitive entity is not visible.

[0201] Furthermore, to avoid accidental desensitization of non-sensitive contextual information, a context-based risk assessor is configured. This risk assessor analyzes the semantic type and thematic background of the text segment containing the information to be processed, determining whether its severity level is high-risk. For example, when a phone number appears in the "Contact Information" section, it is determined that desensitization processing is required; however, when the same number only appears as a technical parameter example in the "Test Case" document and is not associated with personal identification information, the original content can be retained adaptively. Through this contextual information, data security can be ensured while minimizing interference and accidental damage to normal business information.

[0202] In some embodiments, the security monitoring layer is also used for auditing and tracing, recording all critical operational events during system operation and generating tamper-proof audit logs. Audit logs include user identity, query time, input query content, data source for system calls, retrieval path, knowledge graph access records, vector query results, generated answer content, and final output format. The audit log data is timestamped and stored in a secure log repository, and digital signatures or blockchain technology can be used to prevent tampering, facilitating subsequent accountability and security audits.

[0203] In some embodiments, the security monitoring layer is also used for incremental updates to maintain the timeliness and consistency of the knowledge base, enabling real-time responses to document changes. A file system listener continuously monitors document addition, deletion, and modification operations in the knowledge source directory; once a document is updated, the corresponding index update process is automatically triggered. Unlike traditional full reprocessing, incremental updates are achieved using a tree-structured data and knowledge graph in Intermediate Representation (IR).

[0204] Specifically, when a document (e.g., a Word document) is detected to have been modified, the modified document is parsed through a preprocessing layer to generate a new version of the tree structure data. Subsequently, the new version of the tree structure data is compared with the old version stored in the system using a difference analysis, comparing features including heading levels, paragraph text hash values, and node semantic fingerprints, thereby accurately locating the knowledge units that have changed and identifying newly added, deleted, or modified nodes.

[0205] Based on the discrepancy analysis results, updates are performed only on the changed text fragments. For example, for vector databases, only the affected text fragments have their embedding vectors regenerated, and the corresponding entries in the vector database are updated, avoiding the computational overhead of re-indexing the entire database. For knowledge graphs, only the new triples extracted from the text fragments are updated with the corresponding graph operations, including adding nodes, deleting invalid nodes, and correcting or supplementing relationships between entities. This reduces resource consumption and indexing latency, improving the real-time responsiveness of the knowledge base.

[0206] In some embodiments, the security monitoring layer is also used for feedback learning to achieve closed-loop optimization of the knowledge system. Specifically, optimization suggestions are generated based on user feedback. When negative feedback from users on the generated answers (such as "the answer is inaccurate" or "information is missing") is received, the feedback information is recorded, and a source tracing and diagnostic process is executed to analyze the root cause of the problem.

[0207] Specifically, the process begins by checking the retrieval process and retrieving intermediate results from the current query, including the source, content, similarity score, relevance score, and final weighted score of each document fragment from the mixed retrieval phase. Then, the reasons for retrieval failures are analyzed, including possible causes such as missing key entity relationships in the knowledge graph, failure to match semantically similar fragments in vector similarity calculations, or the complete absence of relevant knowledge in the knowledge base.

[0208] If the analysis results show that no relevant knowledge exists in the knowledge base, a knowledge gap alert is generated. This alert includes a clear problem description, such as "Regarding 'fault of the magnetic control mechanism of a 10kV vacuum circuit breaker,' this search found 3 relevant results, but user feedback was invalid, suggesting that the knowledge base may lack a solution for this type of fault," and includes a complete search log, context snippets, and the original record of the user's query. This alert can be pushed to the terminals of system administrators or knowledge maintenance personnel via a predetermined interface, prompting them to supplement relevant technical documents or update structured data, thus forming a closed-loop optimization path from user feedback to the knowledge source.

[0209] The following explanation illustrates the secure processing of power documents containing sensitive entities using a sensitivity-aware dual-mode preprocessing engine. This dual-mode adaptive processing module can include a sensitivity detection module and a complexity evaluation module.

[0210] This embodiment relates to a secure preprocessing method for internal power system documents containing sensitive personal information. The original document uploaded by the user is "XX Substation Personnel Information and Equipment Configuration Manual.docx", which contains both equipment technical parameters and maintenance personnel identification information.

[0211] The system first invokes the sensitivity detection module to perform named entity recognition and classification assessment on the entire original document. This module pre-stores a dictionary of sensitive entities in the power industry, capable of recognizing fields such as "employee ID number," "mobile phone number," and "address," and determining the corresponding sensitivity level based on preset rules. For example, "ID number" is marked as a High-level sensitive entity, and "mobile phone number" is marked as Medium-level. The system can also calculate a comprehensive sensitivity score based on the number of sensitive entities, the weight of sensitivity types, and the contextual distribution. If the comprehensive sensitivity score exceeds a preset threshold of 5.0, the overall sensitivity rating of the original document is determined to be "High."

[0212] At the same time, the complexity assessment module analyzes the document structure features and identifies that the original document includes a multi-level heading system (such as "Chapter 1 General Provisions" and "3.2 Personnel Configuration Table") as well as multiple complex tables with intersecting row and column relationships (such as a personnel-position-contact information comparison table). Based on this, the document structure complexity is rated as "High".

[0213] During the decision-making phase, the system, based on a preset priority strategy, forcibly activates the security-first TXT mode when the sensitivity rating is "High," regardless of complexity. This means converting the original document into a TXT format document. At this point, the dual-mode adaptive processing module calls the `strip_formatting` function to strip away all formatting information, including but not limited to Markdown symbols, table borders, font styles, color marks, and headers and footers, retaining only continuous plain text. While this process loses the original table's row and column structure, it effectively prevents the risk of sensitive information being fully extracted during structured processing and subsequently leaked.

[0214] The output data is an unformatted plain text file, in which highly sensitive fields such as ID card numbers can be automatically replaced with masks (e.g., "110***********1234") by the downstream de-identification module. This plain text file is then sent to the knowledge building layer to generate features and store them in a vector database for subsequent secure retrieval.

[0215] This embodiment achieves usability transformation of unstructured documents while ensuring data security by introducing sensitivity-driven selective format conversion. It is particularly suitable for heavily regulated industries such as power and energy, automatically balancing information utilization and privacy protection without manual intervention, ensuring compliance requirements are met when processing documents containing sensitive content.

[0216] The following explanation illustrates how a hybrid retrieval mechanism based on multi-agent collaboration responds to complex fault diagnosis queries. The process of processing query information in complex natural language with geographical limitations, equipment hierarchies, and multiple intents is described as an example. For instance, a user might input the query: "What are the common causes of main transformer tripping in a 220kV substation in Xinjiang? How should it be handled?"

[0217] The system first uses a KG query agent to perform knowledge retrieval. This agent invokes a named entity recognition model to extract three core entities: "a 220kV substation in Xinjiang" (site entity), "main transformer" (equipment entity), and "tripping" (fault phenomenon entity). It then uses an intent classifier to determine the user's target query intent as [relational reasoning intent, fault cause: 0.88, handling measures: 0.82]. Based on this, the KG query agent dynamically generates a Cypher query statement, traversing the fault subgraph related to the substation's main transformer in the Neo4j knowledge graph to obtain multiple associated triples, such as (main transformer tripping) - [:possible cause] -> (insulation aging), (main transformer tripping) - [:possible cause] -> (lightning overvoltage), (main transformer tripping) - [:standard handling] -> (isolate the fault point), etc.

[0218] The workflow agent receives the subgraph returned by the KG query agent, parses out the entity list of causes and the entity list of handling measures, and semantically enhances the original query statement accordingly. It enhances the original natural language query statement into a strengthened query sequence containing key entities: "Main transformer tripping, insulation aging, lightning strike, overvoltage isolation, and standby switch closing at a 220kV substation in Xinjiang," thereby improving the domain specificity and keyword coverage of the query.

[0219] The hybrid search engine then performs a fusion search: the enhanced query sequence is input into the FAISS vector database for semantic similarity matching to obtain Top-K relevant text fragments, including detailed records of lightning strikes in the "2019 Fault Report of XX Substation" and specific clauses regarding emergency operations for main transformer tripping in the "Power Grid Dispatch Regulations V3.2"; on the other hand, relevant structured factual information in the subgraphs is retained as hard constraints. Finally, the hybrid search engine ranks the candidate results according to a weighted formula (final score = 0.6 × vector similarity + 0.3 × KG correlation + 0.1 × business weight) to ensure that document fragments with high authority and high timeliness are returned first.

[0220] The LLM management agent integrates the list provided by the KG query agent and the document fragments of detailed procedures returned by the vector database into structured prompts, which are then input into the large language model. Based on this, the LLM generates natural language answers, including "common causes" and "handling steps," with specific document sources appended after each operation. For example: "Handling steps: 1. Immediately isolate the fault point; for details, see Article 5.4.2 of the 'Power Grid Dispatch Regulations V3.2'; 2. Check the status of the standby switch and close it; see page 3 of the 'XX Substation 2019 Fault Report'."

[0221] In this embodiment, the KG query agent provides causal logic, the vector database retrieval supplements operational details, and multiple agents collaborate to ensure the accuracy and reliability of the retrieved content. The generated answers are not only highly professional and operable, but also enhance credibility and auditability by explicitly referencing the original documents, effectively solving the problems of low accuracy and missing content in answers in traditional question-answering systems in complex industrial scenarios.

[0222] One or more embodiments of this specification also provide a schematic diagram of a data processing apparatus. The data processing apparatus may include: The acquisition module is used to acquire multiple types of data to be processed; the multiple types of data to be processed include at least two of the following: document data, audio data, and image data. The processing module is used to process the data to be processed using the corresponding data processing methods to obtain the corresponding structured data, which includes information associated with entities. A knowledge graph and a vector database are constructed based on structured data. The knowledge graph includes multiple nodes and edges between nodes. Nodes correspond to entities, and edges are used to indicate the relationships between the entities corresponding to the nodes. The vector database includes feature vectors obtained by feature extraction from structured data. There is a mapping relationship between the node corresponding to the same entity in the knowledge graph and the corresponding feature vector in the vector database. Based on the obtained query information, determine the target query intent of the target entity in the query information, and determine the target query strategy based on the target query intent; the target query intent includes one of relational reasoning intent, content retrieval intent, and fuzzy reasoning intent; If the target query intent includes relational reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph; if the target query intent includes content retrieval intent, the target query strategy includes querying the candidate feature vectors associated with the target entity in the vector database; if the target query intent includes fuzzy reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph and querying the candidate feature vectors associated with the target entity in the vector database. The query results are obtained by querying knowledge graphs and / or vector databases based on the target query strategy.

[0223] The acquisition module and the processing module can be used to execute the corresponding embodiments of the above data processing methods. For the specific implementation methods of these modules and more details, please refer to the corresponding method section, which will not be elaborated here.

[0224] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0225] Figure 5 This is a schematic diagram of the structure of an electronic device according to some embodiments of this application. For example... Figure 5 As shown, the electronic device 500 may include a processor 501 and a memory 502. The electronic device 500 may also include one or more of a multimedia component 503, an input / output (I / O) component 504, and a communication component 505. In this embodiment, the electronic device 500 may be a device that implements the data processing method provided in this embodiment.

[0226] The processor 501 controls the overall operation of the electronic device 500 to complete all or part of the steps in the data processing method described above. The memory 502 stores various types of data to support the operation of the electronic device 500. This data may include, for example, instructions for any application or method operating on the electronic device 500, and application-related data such as images, audio, video, etc. The memory 502 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 503 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used for outputting and / or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 502 or transmitted via communication component 505. The audio component also includes at least one speaker for outputting audio signals. I / O component 504 provides an interface between processor 501 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 505 is used for wired or wireless communication between the electronic device 500 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, Narrowband Internet of Things (NB-IoT), Enhanced Machine Type Communication (eMTC), or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 505 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0227] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the data processing method described above.

[0228] In another exemplary embodiment, a computer-readable storage medium is also provided, having program instructions stored thereon, which, when executed by a processor, implement the steps of the data processing method described above. For example, the computer-readable storage medium may be the memory 502 containing the program instructions, which may be executed by the processor 501 of the electronic device 500 to implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application. Alternatively, when the instructions are executed by a computer, they may be used to implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application.

[0229] In another exemplary embodiment, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the data processing method described above. For example, the computer program product may be the memory 502 containing the computer program, which may be executed by the processor 501 of the electronic device 500 to implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application. Alternatively, when the instructions are executed by a computer, they may be used to implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application.

[0230] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0231] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.

[0232] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although the descriptions of each embodiment in this application have different focuses, and the parts not described in detail in a certain embodiment can be referred to the relevant embodiments of other embodiments, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of this application without departing from the content of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A data processing method, characterized in that, The method includes: Acquire multiple types of data to be processed; the multiple types of data to be processed include at least two of document data, audio data, and image data. The data to be processed is processed using the corresponding data processing method to obtain the corresponding structured data, which includes information associated with entities; A knowledge graph and a vector database are constructed based on the structured data. The knowledge graph includes multiple nodes and edges between nodes. The nodes correspond to the entities, and the edges are used to indicate the relationships between the entities corresponding to the nodes. The vector database includes feature vectors obtained by feature extraction from the structured data. There is a mapping relationship between the node corresponding to the same entity in the knowledge graph and the feature vector corresponding to it in the vector database. Based on the obtained query information, the target query intent of the target entity in the query information is determined, and based on the target query intent, a target query strategy is determined; the target query intent includes one of relational reasoning intent, content retrieval intent, and fuzzy reasoning intent; If the target query intent includes relational reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph; if the target query intent includes content retrieval intent, the target query strategy includes querying the candidate feature vector associated with the target entity in the vector database; if the target query intent includes fuzzy reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph and querying the candidate feature vector associated with the target entity in the vector database. Based on the target query strategy, queries are performed from the knowledge graph and / or the vector database to obtain query results.

2. The method according to claim 1, characterized in that, At least one node in the knowledge graph corresponds to multiple edges, and one edge corresponds to one relation type; if the target query intent includes fuzzy reasoning intent, the target query strategy further includes querying the knowledge graph first and then querying the vector database; The step of querying the knowledge graph and / or vector database based on the target query strategy to obtain query results includes: Based on the query information, determine the target relationship type corresponding to the target entity; Based on the target relationship type and the query information, a subgraph associated with the target entity is obtained by querying the knowledge graph; based on the nodes contained in the subgraph, a candidate feature vector with a mapping relationship with the nodes contained in the subgraph is obtained by querying the vector database. or, Based on the target relationship type and the query information, a subgraph associated with the target entity is obtained by querying the knowledge graph; an enhanced query vector is generated based on the subgraph and the query information, and a candidate feature vector matching the enhanced query vector is obtained by querying the vector database based on the enhanced query vector. The query result is determined based on the subgraph and the candidate feature vector.

3. The method according to claim 2, characterized in that, Determining the query result based on the subgraph and the candidate feature vector includes: Determine the correlation between the subgraph and the target entity, and the similarity between each candidate feature vector and the target entity; Based on the structural features of the subgraph, a first weight corresponding to the subgraph and a second weight corresponding to the candidate feature vector are determined. For each candidate feature vector, a weighted average is performed based on the correlation degree of the subgraph and the first weight, as well as the similarity of the candidate feature vector and the second weight, to obtain the matching score value of the candidate feature vector; Based on the matching score of each of the candidate feature vectors, the target feature vector is determined from the plurality of candidate feature vectors; The query result is obtained based on the target feature vector and the subgraph.

4. The method according to any one of claims 1 to 3, characterized in that, The process of processing the data to be processed using a corresponding data processing method to obtain the corresponding structured data includes: The data to be processed is processed in a manner corresponding to the type of the data to be processed to obtain standardized data; The standardized data is evaluated to obtain at least one of sensitivity information and complexity information of the standardized data; wherein, the sensitivity information includes at least one of sensitivity score, sensitive entity, sensitivity type and severity level corresponding to the sensitive entity, and sensitivity level label of the data to be processed to which the standardized data belongs; the complexity information includes complexity. If the sensitivity information meets preset conditions, and / or the complexity is greater than a preset complexity threshold, the standardized data is format-converted to obtain converted standardized data, and the converted standardized data is structured to obtain corresponding structured data. The preset conditions include: the sensitivity score exceeds a preset score threshold, and / or the number of sensitive entities with a severity level greater than a preset level threshold exceeds a preset quantity threshold, and / or the standardized data contains sensitive entities with a target severity level and / or a target sensitivity type, and / or the sensitivity level is marked as a preset sensitivity level mark.

5. The method according to claim 4, characterized in that, The data to be processed is document data. The process of processing the data to be processed using a method corresponding to the type of data to be processed to obtain standardized data includes: The document data is parsed and extracted to obtain at least one first content block; Based on the at least one first content block, standardized data corresponding to the document data is obtained, wherein each first content block has at least one attribute information for describing the first content block, the attribute information of the first content block includes the type information of the first content block and the hierarchy information, and the type information of the first content block includes at least one of title, list, table or paragraph.

6. The method according to claim 4, characterized in that, The data to be processed is audio data. The data to be processed is then processed in a manner corresponding to the type of data to obtain standardized data, including: The transcribed text obtained from the audio data is segmented to obtain at least one speech unit; The at least one discourse unit is identified to obtain the question text and the answer text; By associating the question text and the answer text, at least one question-answer pair can be obtained; Based on the at least one question-answer pair, standardized data corresponding to the audio data is obtained.

7. The method according to claim 4, characterized in that, The data to be processed is image data. The data to be processed is processed in a manner corresponding to the type of data to be processed to obtain standardized data, including: The image data is processed to obtain text description information that describes the content of the image data; The text description information is parsed and extracted to obtain at least one second content block; Based on the at least one second content block, standardized data corresponding to the image data is obtained, wherein each second content block has at least one attribute information for describing the second content block, and the attribute information of the second content block includes the type information of the second content block, which includes at least one of title, list, table or paragraph.

8. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire multiple types of data to be processed; the multiple types of data to be processed include at least two of document data, audio data, and image data. The processing module is used to process the data to be processed using a corresponding data processing method to obtain corresponding structured data, wherein the structured data includes information associated with entities; A knowledge graph and a vector database are constructed based on the structured data. The knowledge graph includes multiple nodes and edges between nodes. The nodes correspond to the entities, and the edges are used to indicate the relationships between the entities corresponding to the nodes. The vector database includes feature vectors obtained by feature extraction from the structured data. There is a mapping relationship between the node corresponding to the same entity in the knowledge graph and the feature vector corresponding to it in the vector database. Based on the obtained query information, the target query intent of the target entity in the query information is determined, and based on the target query intent, a target query strategy is determined; the target query intent includes one of relational reasoning intent, content retrieval intent, and fuzzy reasoning intent; If the target query intent includes relational reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph; if the target query intent includes content retrieval intent, the target query strategy includes querying the candidate feature vector associated with the target entity in the vector database; if the target query intent includes fuzzy reasoning intent, the target query strategy includes querying the subgraph associated with the target entity in the knowledge graph and querying the candidate feature vector associated with the target entity in the vector database. Based on the target query strategy, queries are performed from the knowledge graph and / or the vector database to obtain query results.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images