Signaling analysis method and system, electronic equipment and storage medium

Abstract

The invention discloses a signaling analysis method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring signaling text information; loading a retrieval enhancement generation database, fusing a multi-layer structure knowledge graph, and carrying out retrieval analysis on the signaling text information to obtain a cue word and a retrieval result; the multi-layer structure knowledge graph is obtained by performing multi-layer data set construction on domain knowledge through a large language model; and inputting the cue word and the retrieval result into a large language model to obtain an analysis conclusion. According to the embodiment of the invention, the generation and the large language model are enhanced through retrieval, so that the quality and efficiency of signaling analysis are improved. The method can be widely applied to the technical field of Internet of Things.

Description

Technical Field

[0001] The present application relates to the technical field of Internet of Things, and in particular to a signaling analysis method, system, electronic device and storage medium. Background Art

[0002] Signaling analysis is the basis of communication network operation and maintenance. Most operation and maintenance personnel use tools such as Wireshark to perform in-depth analysis of bit streams to diagnose network problems. This method relies heavily on the professional level of technicians and is not very efficient. The main difficulties include the following:

[0003] Difficulty in screening massive amounts of data: When capturing large amounts of network traffic, it is difficult to accurately find specific signaling flows from the huge amount of data. It takes a lot of time and effort to screen and filter, and it is easy to miss key signaling.

[0004] Complex protocol analysis: To effectively use Wireshark for signaling analysis, you need to have a deep understanding of various network protocols. This includes the structure, working principle, and common signaling process of the protocol. Even for professional engineers, end-to-end signaling analysis from wireless, transmission, core network to user MEC in a 5G private network environment requires correlation analysis across multiple disciplines. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to propose an efficient signaling analysis method, system, electronic device and storage medium.

[0006] To achieve the above purpose, one aspect of an embodiment of the present application proposes a signaling analysis method, the method comprising: obtaining signaling text information; loading a retrieval enhancement generation database, fusing a multi-layer structure knowledge graph, performing retrieval analysis on the signaling text information, and obtaining prompt words and retrieval results; the multi-layer structure knowledge graph is obtained by constructing a multi-layer data set of domain knowledge by a large language model; the prompt words and the retrieval results are input into the large language model to obtain an analysis conclusion. The embodiment of the present application is conducive to improving the quality and efficiency of signaling analysis through retrieval enhancement generation and a large language model.

[0007] In some embodiments, the method provided by the embodiments of the present application, the multi-layer structure knowledge graph is established by the following steps:

[0008] Acquire first text information from a signaling domain knowledge base;

[0009] Segmenting and extracting elements from the first text information to obtain vector information;

[0010] According to the hierarchical relationship constructed by the knowledge base base, entities corresponding to each level in the vector information are linked to obtain graph information;

[0011] The graph information is merged to obtain the multi-layer structure knowledge graph.

[0012] In some embodiments, the method provided in the embodiments of the present application, the hierarchical relationship constructed according to the knowledge base base, linking entities corresponding to each level in the vector information to obtain graph information, includes:

[0013] Using directly captured information in the vector information as the first entity of the bottom layer;

[0014] Determining the relevance of the vector information through the large language model, taking the complex interpretation information in the vector information as a second entity of the second layer, and linking the second entity with the first entity;

[0015] The clear interpretation information in the vector information is used as a third entity of the third layer, and the third entity is linked with the second entity to obtain the graph information.

[0016] In some embodiments, the method provided in the embodiments of the present application, segmenting and extracting elements from the first text information to obtain vector information includes:

[0017] Performing text segmentation on the first text information to obtain text blocks;

[0018] Extract elements from the text block using the BERT model to obtain a text vector;

[0019] All text vectors are processed by mean pooling and text similarity to obtain vector information.

[0020] In some embodiments, the method provided in the embodiments of the present application, wherein the loading of the search enhancement generation database, the integration of the multi-layer structure knowledge graph, the search analysis of the signaling text information, and the acquisition of the prompt words and the search results include:

[0021] Based on the multi-layer structure knowledge graph, the signaling text information is subjected to association analysis to determine keywords;

[0022] Based on the keyword, a hierarchical search is performed through a matching process in the multi-layer structure knowledge graph, and entity activation and knowledge collection are performed based on the search results to obtain primary information;

[0023] The primary information is combined with the signaling text information, and retrieved by a hybrid retriever to obtain secondary information;

[0024] The secondary information is filtered through the clear interpretation information in the knowledge base base to obtain the tertiary information;

[0025] The primary information, the secondary information, and the tertiary information are integrated to obtain prompt words and search results.

[0026] In some embodiments, in the method provided by the embodiments of the present application, the hybrid retriever determines the secondary information by the following steps:

[0027] Determine the scoring function by ranking the documents to be retrieved;

[0028] Determine a relevance function according to the score function and the relevance score of the document to be retrieved;

[0029] Determining a search score according to a correlation function of the primary information and a correlation function of the signaling text information;

[0030] Secondary information is determined based on the search score.

[0031] In some embodiments, the method provided by the embodiments of the present application further includes:

[0032] A signaling analysis architecture is constructed, the architecture comprising:

[0033] The computing network resource layer is used to provide resources for building the knowledge base base;

[0034] A technical layer, used to establish the large language model;

[0035] The agent layer is used to implement signaling analysis based on expert orchestration and task scheduling;

[0036] Small model layer, used for signaling analysis in specific fields.

[0037] To achieve the above object, another aspect of an embodiment of the present application provides a signaling analysis system, the system comprising:

[0038] The first module is used to obtain signaling text information;

[0039] The second module is used to load the retrieval enhancement generation database, integrate the multi-layer structure knowledge graph, perform retrieval analysis on the signaling text information, and obtain prompt words and retrieval results; the multi-layer structure knowledge graph is obtained by constructing a multi-layer data set of domain knowledge with a large language model;

[0040] The third module is used to input the prompt word and the search result into a large language model to obtain an analysis conclusion.

[0041] To achieve the above objective, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0042] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0043] The embodiments of the present application include at least the following beneficial effects: The method provided by the embodiments of the present application includes: obtaining signaling text information; loading a search enhancement generation database, integrating a multi-layer structure knowledge graph, performing search analysis on the signaling text information, and obtaining prompt words and search results; the multi-layer structure knowledge graph is obtained by constructing a multi-layer data set of domain knowledge by a large language model; the prompt words and the search results are input into the large language model to obtain an analysis conclusion. The embodiments of the present application are conducive to improving the quality and efficiency of signaling analysis through search enhancement generation and a large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is an interface diagram of the analysis process based on wireshark in the related technology;

[0045] Figure 2 It is an application scenario diagram of the DPI-based analysis process in the related technology;

[0046] Figure 3 is a flowchart of an embodiment of the signaling analysis method provided by the present application;

[0047] Figure 4 is a flowchart of another embodiment of the signaling analysis method provided by the present application;

[0048] Figure 5 is a flowchart of an embodiment of a retrieval process based on retrieval enhancement provided by the present application;

[0049] Figure 6 is a flowchart of an embodiment of the signaling analysis architecture provided by the present application;

[0050] Figure 7 This is a schematic diagram of an interface of an embodiment of a signaling analysis process provided by the present application;

[0051] Figure 8 It is a schematic diagram of an interface of another embodiment of the signaling analysis process provided by the present application;

[0052] Fig. 9 It is a structural diagram of a signaling analysis system provided in an embodiment of the present application;

[0053] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0055] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0056] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0059] Signaling analysis: is a technical process of capturing, parsing, monitoring and analyzing the control signaling messages within the communication system. Signaling usually refers to the control information transmitted between network nodes (such as base stations, switches, servers, etc.), which is responsible for managing and coordinating the establishment, maintenance, termination of calls, and other management and configuration tasks of the network. Analyzing signaling can help understand the operation of the communication system, discover potential problems, evaluate service quality, support troubleshooting, and optimize network configuration.

[0060] Deep Packet Inspection (DPI) is an application-layer based traffic detection and control technology. It not only checks the header information of the data packet, but also can identify different application layer protocols, user behaviors, traffic types, etc. It can be used to monitor network traffic, track user behaviors, identify and prevent network attacks, etc.

[0061] The Evolved Packet Core (EPC), also known as the 4G core network, includes the Mobility Management Entity (MME), the Serving Gateway (S-GW), the Packet Data Network Gateway (P-GW) and the Home Subscriber Server (HSS).

[0062] 5G Core Network (5GC), including network elements for control plane functions (AMF, SMF, PCF), user plane (UPF) and network elements for data storage and management functions (UDM).

[0063] IP Multimedia Subsystem (IMS) is an IP-based network architecture used to provide multimedia communication services.

[0064] Large Language Model (LLM) is an artificial intelligence algorithm trained using deep learning technology and massive data. It is mainly used to process and understand human language.

[0065] Knowledge Graph / Vault (KG), also known as scientific knowledge graph, uses various graphics and other visualization technologies to describe knowledge resources and their carriers, and to mine, analyze, construct, draw and display knowledge and their interrelationships.

[0066] Resource Description Framework (RDF), RDF is a data model expressed in XML syntax. The function of RDF is to describe the characteristics of resources and the relationship between resources in the form of triples, a way to store triple data line by line in the form of text. Use relational databases to store knowledge graphs. Graph data is represented by triples, and each triple is recorded as a row in the table.

[0067] Retrieval-Augmented Generation (RAG) is an artificial intelligence technology that combines information retrieval and language generation. Its main purpose is to use information retrieved from external sources to improve the accuracy, reliability and information richness of the output results of the generative artificial intelligence model.

[0068] Signaling analysis is the basis of communication network operation and maintenance. Most operation and maintenance personnel use tools such as Wireshark to perform in-depth analysis of bit streams to diagnose network problems. This method relies heavily on the professional level of technicians and is not very efficient. The main difficulties include the following:

[0069] Difficulty in screening massive amounts of data: When capturing large amounts of network traffic, it is difficult to accurately find specific signaling flows from the huge amount of data. It takes a lot of time and effort to screen and filter, and it is easy to miss key signaling.

[0070] Complex protocol analysis: To effectively use Wireshark for signaling analysis, you need to have a deep understanding of various network protocols. This includes the structure, working principle, and common signaling process of the protocol. Even for professional engineers, end-to-end signaling analysis from wireless, transmission, core network to user MEC in a 5G private network environment requires correlation analysis across multiple disciplines.

[0071] Ambiguity in analytical reports: Although large models can improve the knowledge content of analytical conclusions, they are prone to hallucinations and provide erroneous judgments and inferences.

[0072] Lack of visualization and ease of use: Wireshark's interface is relatively complex, including function options and parameter settings, which makes it difficult to use. At the same time, although Wireshark can display the detailed content of signaling data, it is still lacking in visualization. For complex signaling processes, the lack of intuitive graphical display makes it more difficult to understand and analyze.

[0073] like Figure 1 As shown in the figure, the typical wireshark analysis process requires analyzing the signaling interaction flow and content line by line and frame by frame to analyze and confirm the problems in the interaction process.

[0074] In the application of 5G private network, signaling analysis software mainly exists in the form of DPI perception probes. In the operation and maintenance management of private networks, there is also a lack of language and interface presentation that terminal users can understand, so as to explain the professional analysis results of DPI perception probes to customers and support customers to carry out in-depth self-operation and self-diagnosis of private networks. Figure 2 As shown, DPI mainly completes the acquisition around the edge UPF. The acquisition interfaces include N3 / N4 / N9 and other interfaces to generate XDR code stream files.

[0075] In view of this, a signaling analysis method is provided in an embodiment of the present application, aiming to improve the quality and efficiency of signaling analysis. The present invention can be used in the fields of software technology and network communication.

[0076] The signaling analysis method provided in the embodiment of the present application relates to the field of Internet of Things technology. The signaling analysis method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms and other basic cloud computing services. The cloud server, the server can also be a node server in the blockchain network; the software can be an application that implements the signaling analysis method, etc., but is not limited to the above forms.

[0077] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0078] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0079] Figure 3 This is an optional flowchart of the signaling analysis method provided in the embodiment of the present application; Figure 3The method may include but is not limited to steps S100 to S300.

[0080] Step S100, obtaining signaling text information;

[0081] Step S200, loading the search enhancement generation database, integrating the multi-layer structure knowledge graph, performing search analysis on the signaling text information, and obtaining prompt words and search results; the multi-layer structure knowledge graph is obtained by constructing a multi-layer data set of domain knowledge by a large language model;

[0082] Step S300: input the prompt words and search results into the large language model to obtain analysis conclusions.

[0083] The embodiment of this application proposes a method for constructing a signaling analysis intelligent body based on an AI big model, which combines the signaling analysis workflow to intelligently upgrade traditional signaling analysis measures, integrates wireless / core network signaling data analysis and processing capabilities and big model language understanding and content generation capabilities, and realizes automatic understanding, analysis, reasoning, and signaling knowledge retrieval and question-answering of network signaling, greatly improving the quality and efficiency of complaint analysis and network optimization analysis. Figure 4 As shown, this application realizes high-order automated and intelligent signaling analysis based on a large model, and enhances the reliability of signaling analysis through RAG+knowledge graph fusion.

[0084] In some embodiments, the method provided by the embodiments of the present application, the multi-layer structure knowledge graph is established by the following steps:

[0085] Acquire first text information from a signaling domain knowledge base;

[0086] Segmenting and extracting elements of the first text information to obtain vector information;

[0087] According to the hierarchical relationship constructed by the knowledge base base, the entities corresponding to each level in the vector information are linked to obtain graph information;

[0088] Merge the graph information to obtain a multi-layer structured knowledge graph.

[0089] In some possible implementations, the signaling domain entity relationships are extracted through LLM to realize the construction of a three-layer data set of domain knowledge, and LLM is used for segmentation and block division to extract label information to realize automatic graph construction. In some embodiments, the signaling domain entity relationships are extracted through LLM to construct a hierarchical data set of domain knowledge, and the labels are extracted in segments and blocks to realize graph construction.

[0090] In some embodiments, the method provided in the embodiments of the present application links entities corresponding to each level in the vector information according to the hierarchical relationship constructed by the knowledge base base to obtain graph information, including:

[0091] Take the directly captured information in the vector information as the first entity of the bottom layer;

[0092] Determine the relevance of the vector information through a large language model, use the complex interpretation information in the vector information as the second entity of the second layer, and link the second entity with the first entity;

[0093] The clear interpretation information in the vector information is used as the third entity of the third layer, and the third entity is linked to the second entity to obtain the graph information.

[0094] In some possible implementations, the present application provides hierarchical links: communication signaling is a professional field that uses a precise terminology system and is based on many clear definitions and relationships. The present application also provides graph construction: using LLM, specific entity objects and their relationships are identified from the text of the signaling domain knowledge base, converted into structured knowledge representations, and presented through the structure of a graph. In some embodiments, the present application constructs a three-layer linked knowledge database for communication signaling to form a comprehensive signaling analysis knowledge graph. The bottom layer is the basic information entity, the second layer is a simple knowledge graph layer constructed after LLM links the bottom-level entities, and the third layer is the signaling display analysis layer after linking the second-layer entities.

[0095] In some embodiments, the method provided in the embodiment of the present application segments and extracts elements of the first text information to obtain vector information, including:

[0096] Performing text segmentation on the first text information to obtain text blocks;

[0097] Use the BERT model to extract elements from text blocks and obtain text vectors;

[0098] All text vectors are processed by mean pooling and text similarity to obtain vector information.

[0099] In some possible implementations, there are many ways to extract elements, and those skilled in the art can also extract elements in other ways, which are not specifically limited in this application. It can be understood that this application uses semantic segmentation before and after signaling, uses a sliding window to process a fixed number of paragraphs each time and continuously adjusts the window to maintain attention to the consistency of signaling information. LLM is used to complete the identification and extraction of graph node examples from source text blocks, and example guidance is provided for LLM training by using the text vectorization method of the BERT model.

[0100] In some embodiments, reference Figure 5 As shown, the method provided in the embodiment of the present application loads the search enhancement generation database, integrates the multi-layer structure knowledge graph, performs search analysis on the signaling text information, and obtains prompt words and search results, including:

[0101] Step 210, based on the multi-layer structure knowledge graph, performing association analysis on the signaling text information to determine keywords;

[0102] Step 220, based on the keywords, hierarchical retrieval is performed through a matching process in a multi-layer structure knowledge graph, and entity activation and knowledge collection are performed based on the retrieval results to obtain primary information;

[0103] Step 230, combining the primary information with the signaling text information, and performing a search through a hybrid searcher to obtain secondary information;

[0104] Step 240, filtering the secondary information through the clear interpretation information in the knowledge base base to obtain tertiary information;

[0105] Step 250, the primary information, the secondary information, and the tertiary information are integrated to obtain prompt words and search results.

[0106] In some possible implementations, the input signaling is automatically analyzed based on the retrieval enhancement generation technology (RAG). It can be understood that the present application adopts an efficient strategy retrieval method based on a primary STM-R as the processing process of the RAG retrieval enhancement generation technology, outputs the entity content and related content within the scope, and provides it as the input of the secondary LLM enhanced retrieval.

[0107] In some embodiments, in the method provided by the embodiments of the present application, the hybrid retriever determines the secondary information by the following steps:

[0108] Determine the scoring function by ranking the documents to be retrieved;

[0109] Determine a relevance function according to the score function and the relevance score of the document to be retrieved;

[0110] Determine a search score according to a correlation function of the primary information and a correlation function of the signaling text information;

[0111] Based on the search score, secondary information is determined.

[0112] In some possible implementations, the secondary LLM enhanced retrieval is combined with LLM to associate the second-level sentence meaning of the dataset with the prompt words, thereby enhancing the model's understanding of specific issues. This application combines the original query with the additional prompt words generated after the additional context information is merged and processed, and uses a hybrid retrieval method to fuse the document retrieval results to enhance the model's understanding of specific issues.

[0113] In some embodiments, the method provided by the embodiments of the present application further includes:

[0114] Build a signaling analysis architecture, which includes:

[0115] The computing network resource layer is used to provide resources for building the knowledge base base;

[0116] The technical layer is used to build large language models;

[0117] The agent layer is used to implement signaling analysis based on expert orchestration and task scheduling;

[0118] Small model layer, used for signaling analysis in specific fields.

[0119] The embodiment of the present application provides a technical architecture, specifically a technical framework of a signaling analysis agent based on an AI big model. Figure 6 As shown in the figure, the technical architecture specifically includes:

[0120] Computing network resources: Provide the operating base resources for large AI models to meet the purpose of computing resource allocation and network configuration optimization.

[0121] Technical layer: professional large model base and core algorithms.

[0122] AI Agent: AI agent that realizes intelligent computing through expert orchestration, task scheduling and AI smart executors.

[0123] Small model: that is, domain model, through enhanced intelligent computing, it provides prediction and warning, perception evaluation, failure code positioning, network element health assessment, fault location, impact analysis and cross-domain correlation analysis in the signaling field.

[0124] Instructions: Based on the agent evaluation results, make decisions on the underlying core control instructions.

[0125] Public components: Provides basic domain knowledge vector knowledge base, constructs teleprompters, evaluation tools, and AI embedding technology, etc.

[0126] Application scenarios: Provides core functions such as signaling analysis report services, signaling tracing, signaling interpretation, and knowledge Q&A for front-line complaint personnel and network optimization personnel.

[0127] Input learning: The input includes alarms, knowledge questions and answers, and data queries, and the AI ​​Agent is trained based on the input.

[0128] In the following, the scheme of the embodiment of the present invention is described in detail and explained in conjunction with a specific application example. The signaling analysis process provided by the present application includes the following steps:

[0129] Step 0: Build the knowledge base to form an initial signaling knowledge database based on signaling protocol standards (including signaling message / cell parameter interpretation, etc.), classic cases, technical documents and expert experience for multi-domain business scenarios such as EPC / 5GC / IMS. Here, the data information is divided into three layers:

[0130] At the lowest level, developers can directly capture files, context associations, and error data through signaling packet capture.

[0131] The second layer includes credible signaling-related manuals, technical documents, textbooks, and signaling analysis papers in the communication professional field, which have complex and related interpretations.

[0132] The third layer is the protocol source file that defines the meaning of each signaling. Direct interpretation.

[0133] Step 1: Extract the signaling domain entity relationship through LLM to realize the three-layer data set construction of domain knowledge (of course, technical personnel in this field can adjust the number of structural layers of the data set and knowledge graph according to actual needs). Use LLM to segment and block, extract label information, and realize automated graph construction (that is, the process of establishing a multi-layer structure knowledge graph in this application).

[0134] Sub-step 1.1: Graph construction: Using LLM, specific entity objects and their relationships are identified from the text of the signaling domain knowledge base, converted into structured knowledge representations, and presented through a graph structure.

[0135] Sub-step 1.1.1: Original text segmentation: First, the data text is segmented. The segmentation uses a hybrid method based on signaling words and sentences and segmentation based on structural units. Specifically, line breaks are used to segment each paragraph in the signaling document. After that, the signaling pre- and post-delimiters are applied to perform semantic block segmentation. The methods include proposition transfer and sub-block derivation. Using the sliding window technology, five paragraphs are processed each time. By continuously adjusting the window, the previous signaling block is removed and the next signaling block is added, keeping an eye on the consistency of the signaling information.

[0136] Sub-step 1.1.2: Element Extraction: Identify and extract graph node examples from each source text block by using an LLM designed to identify all relevant entities in the text.

[0137] Specifically, LLM uses a pre-trained language model (Bidirectional EncoderRepresentations from Transform, BERT) and a neural network with a transformer mechanism during entity discovery. For each entity, LLM is prompted to output the name, type, and description. The name can be the exact text in the document or a paraphrase statement in the signaling technology document, which is carefully selected to reflect the precise paraphrase information suitable for subsequent processing. The type is selected by LLM from a predefined table, and the description is the entity interpretation generated by LLM, combined with the context in the document. To ensure the effectiveness of the model, some examples are prepared here to guide LLM to generate the desired output.

[0138] Here, in order to better retrieve valid entities in the database, the BERT model used outputs entities through steps such as vectorization, generating triplets, and using a three-fold repeated network with adaptive boundaries to map vectors to a lower-dimensional field.

[0139] In specific use, this project uses an advanced text vectorization method in this field, using the BERT model mentioned above as the text encoder, for any text sequence x containing N words i After the BERT model is read into the text, the objective function is maximized by optimizing the training embedding matrix M and the neural network weight size, as shown in the following formula:

[0140]

[0141] x represents all the words in the vocabulary. In order to represent all the texts as vectors, we use mean pooling to derive them, as shown below:

[0142]

[0143] Among them, V i,j represents the jth word vector in the i-th text, J i Represents the total number of words in the entity text. After calculating the value of the text vector, the cosine similarity method is used to calculate the entity text similarity, as shown below:

[0144]

[0145] Where D represents the vector dimension.

[0146] The output of the final hidden layer is x i The initial representation h i :

[0147] [h1,h2,…,h n ]=BERT φ ([x1,x2,…,x n ]),

[0148] in represents the parameters of BERT. Then for each predefined entity category c j , by averaging the values ​​labeled c j The word embedding representation is used to construct its initial prototype vector hc j .

[0149] Sub-step 1.1.3: Hierarchical linking (i.e., the process of determining graph information in this application): Communication signaling is a professional field that uses a precise terminology system and is based on many clear definitions and relationships. Such as the meaning of a certain signaling or the meaning of an error value. In this field, LLM cannot distort, modify or add creative or random elements. Therefore, a special hierarchical linking model is prepared here for communication signaling to form a comprehensive communication signaling analysis knowledge graph. Previously, a three-layer knowledge database has been prepared. Here, we preset the underlying information according to the above steps to form entities. The second-layer information first uses LLM with a simple knowledge graph construction method, combined with LLM to build a graph. The entities of the second layer are linked to the entities of the first layer according to the correlation detected by LLMs. Then the third-layer signaling is clearly defined and linked to the entities of the second layer. For each entity, the text embedding of its name is compared with the vocabulary in the communication protocol.

[0150] Sub-step 1.1.4: Label generation and graph merging: After the meta-graph is constructed, each data block is scanned to develop a global graph that connects all meta-graphs together. The merged meta-graph nodes are linked to each other using the above hierarchical linking rules.

[0151] After completing the above steps, a three-layer knowledge graph is formed, in which nodes represent entities and edges represent the relationships between entities. By constructing a knowledge graph, the extracted entity relationships are presented in a visual way.

[0152] Step 2: Using the retrieval-enhanced generation technology (RAG), the input signaling is automatically analyzed (i.e., steps S210 to S250 in this application). In the process, LLM uses a strategy called STM (Signaling Transfer Matching)-Retrieve in this patent to efficiently retrieve information in response. Output sequence text H n have:

[0153] H n =STM-R φ ([x1,x2,…,x n ,Y])

[0154] Among them, x i is the input signaling text, Y is the database, and φ is the preset parameters of the retrieval strategy, including the priority and the initial amount of the hierarchical retrieval iteration range.

[0155] Then, based on the analysis scenario, key information is selected as retrieval keywords, and the signaling entity relationship, RAG retrieval information and prompt words are combined to realize logical reasoning of the signaling process, thereby improving the accuracy and reliability of signaling analysis.

[0156] Sub-step 2.1: Input: Input the code stream file (ie, the signaling text information in this application).

[0157] Sub-step 2.2: Parsing: Parse the original bitstream according to the protocol.

[0158] Sub-step 2.3: Database loading: Load the RAG knowledge base, including analysis cases, domain knowledge, field explanations, and failure code rules, etc. Use text embedding and vector search techniques to convert the original query into a vector representation.

[0159] Sub-step 2.4: An STM (Signaling Transfer Matching)-retrieve strategy retrieval:

[0160] Sub-step 2.4.1: Extract keywords: First, use the three-layer structure knowledge graph that has been formed to perform association analysis on the vector form of the original code stream, generate a summary label description, extract the feature information of the signaling plaintext, and generate a summary label description in the form of keywords.

[0161] Sub-step 2.4.2: Hierarchical retrieval: Next, the extracted keyword information is used to identify the most relevant meta-graph units through a top-down matching process in the three-layer knowledge graph. This process starts with a relatively large graph, using keywords to define the scope, and then gradually indexing the small graphs it contains, and repeating it.

[0162] Sub-step 2.4.3: Entity activation: Then, after hierarchical retrieval and gradually narrowing the scope, the search will eventually reach the meta-layer and lock some entities, collect these retrieved entities, and complete the activation of these entities.

[0163] Sub-step 2.4.4: Collect knowledge: Finally, based on these entities, collect the entity content, and all related content of related entities within its scope, including related protocol content knowledge, technical document knowledge, correlation and relationship with other entities, content of any linked entities, etc. The retrieval is complete.

[0164] Sub-step 2.5: Secondary LLM enhanced retrieval: The retrieved information is used as additional context input and merged with the original query to generate additional prompt words. Here, an advanced hybrid retriever is used for document retrieval. This method fuses the retrieval results of the sparse retrieval model and the dense retrieval model through a convex linear combination fusion retriever (Convex-fusion), as shown in the formula. Among them, Sconvex (q, d) represents the final relevance score of the document; q represents the query, d represents the document; the value range of α is (0,1]; 'S is the standardized formula for the relevance score; Smax is the highest relevance score of the current candidate document, S min is the lowest relevance score of the current candidate document, rank represents the document ranking; k is an adjustable parameter.

[0165] S conver (q,d)=(1-α)S′ sparse (q,d)+αS′ dense (q,d)

[0166]

[0167] Based on the above retrieval method combined with LLM, the second-level sentence meaning of the dataset is associated with prompt words, which enhances the model's understanding of specific problems.

[0168] Sub-step 2.6: Three-fold accurate knowledge screening: The knowledge generated in the secondary enhanced retrieval is screened according to the underlying serious signaling definition information in the three-layer data structure to provide more rigorous information for the generated knowledge, delete obviously erroneous information, reduce large model illusions, and reduce speculation and uncertainty.

[0169] Sub-step 2.7: Knowledge fusion: Use a common embedding model to convert all the knowledge vectors obtained above and the entities and relationships in the graph into a unified vector representation, and perform calculations and analysis in the same vector space. This facilitates the integration of knowledge vector information into the final conclusion, and can also use the structural information of the graph to optimize the representation and calculation of knowledge vectors.

[0170] Step 3: Through the fusion of the signaling knowledge vector knowledge base + graph data in the previous step, combined with the prompt words and search results, input the large model again for logical reasoning to generate analysis conclusions and reports.

[0171] Step 4: Update the domain knowledge base, including cases and expert experience, and repeat steps 0 to 3.

[0172] In a specific example, the application effect is described as follows:

[0173] Based on the content analysis of the bitstream, the analysis scheme is automatically recommended. Users can select the specific data type of signaling analysis according to their needs. Figure 7 At the same time, this application can realize multi-domain code stream analysis, output analysis reports, and assist analysts to quickly locate problems. Figure 8 shown.

[0174] This application proposes the concept of signaling analysis agent based on AI big model and the composition of technical framework. The core business process and specific execution steps of signaling analysis agent based on big model, especially the processing steps of enhancing the reliability of signaling analysis through RAG+knowledge graph fusion, including the three-layer knowledge structure model and hybrid retrieval strategy.

[0175] This application innovatively introduces a new BERT-based text vectorization method in the field of signaling analysis, including adaptive boundary rules and multiple mapping mechanisms. By overlapping the data with adaptive boundaries in three layers and mapping them to low dimensions, it is possible to better obtain reliable entities in the database.

[0176] This application innovatively uses a three-layer data base model and performs hierarchical linking with preset rules, and pre-classifies the analysis database to be entered into the large model according to different credibility levels. The rules for hierarchical linking are based on the credibility rating standards for data from different sources within the database.

[0177] In view of the characteristics of the signaling analysis problem, this application combines the above two points and innovatively proposes a special strategy for signaling analysis based on a three-layer knowledge structure model and hybrid retrieval. Through directed iteration during the retrieval process, meta-layer entity activation, and secondary enhanced retrieval priority rule setting, the hallucination problem that occurs during the general large model analysis operation is solved, thereby reducing errors.

[0178] Compared with the prior art, the present invention has the following beneficial effects:

[0179] First, a visualization tool for signaling analysis based on large model interaction classes is implemented, which displays signaling data in a graphical manner supplemented by text questions and answers, so that the complex signaling interaction process can be understood more intuitively.

[0180] Second, build a large model + knowledge base, create a knowledge brain in the signaling field, complete multi-format corpus parsing and semantic vector index construction. The process uses structured databases and special retrieval strategies to reduce hallucinations that may occur during large model deduction. It can complete work order status queries, knowledge Q&A in the operation and maintenance field, and impromptu document Q&A, output conclusions, promote knowledge sharing, enable intelligent operation and maintenance, and improve the work efficiency of operation and maintenance personnel.

[0181] Third, by combining domain knowledge to quickly expand upper-level applications, the upper-level application of the core network configuration data audit agent can be realized based on the platform, integrating natural language processing technology and large model capabilities, cleaning, retrieving and structuring the original document data, and realizing grammar and standard auditing; using entity recognition to extract key instructions and their attributes, converting them into code, and realizing automated auditing.

[0182] Possible future application scenarios of this application include:

[0183] By combining domain knowledge to quickly expand upper-level applications, the upper-level application of the core network configuration data auditing intelligent body can be realized based on the platform. Natural language processing technology and large model capabilities can be integrated to clean, retrieve and structure the original document data to achieve grammar and standard auditing. Entity recognition can be used to extract key instructions and their attributes, convert them into codes, and realize automated auditing.

[0184] Based on the performance work order data, alarms are classified and key information of the work order is extracted. Agent intelligent body technology is used to call the corresponding capability API interface to coordinate the model to achieve fault demarcation and location, thereby improving the efficiency of alarm handling.

[0185] This application is conducive to improving the efficiency of telecommunications professionals in the network operation and maintenance process; improving the depth and breadth of business perception probe applications in the 5G private network platform, creating a fool-proof professional service capability for customers, and further enhancing product competitiveness.

[0186] See also Fig. 9 The embodiment of the present application further provides a signaling analysis system, which can implement the above signaling analysis method, and the system includes:

[0187] The first module 810 is used to obtain signaling text information;

[0188] The second module 820 is used to load the search enhancement generation database, integrate the multi-layer structure knowledge graph, perform search analysis on the signaling text information, and obtain prompt words and search results; the multi-layer structure knowledge graph is obtained by constructing a multi-layer data set of domain knowledge by a large language model;

[0189] The third module 830 is used to input the prompt words and the search results into the large language model to obtain the analysis conclusion.

[0190] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0191] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above signaling analysis method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0192] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0193] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0194] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0195] The memory 902 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 may store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the signaling analysis method of the embodiment of this application;

[0196] Input / output interface 903, used to implement information input and output;

[0197] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0198] A bus 905 that transmits information between the various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0199] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0200] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned signaling analysis method is implemented.

[0201] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0202] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0203] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0204] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0205] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0207] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0208] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0209] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0210] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0211] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0212] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0213] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A signaling analysis method, characterized in that: The method comprises: Get signaling text information; Loading the search enhancement generation database, integrating the multi-layer structure knowledge graph, performing search analysis on the signaling text information, and obtaining prompt words and search results; the multi-layer structure knowledge graph is obtained by constructing a multi-layer data set of domain knowledge by a large language model; The prompt words and the search results are input into a large language model to obtain an analysis conclusion.

2. The method according to claim 1, characterized in that The multi-layer structure knowledge graph is established by the following steps: Acquire first text information from a signaling domain knowledge base; Segmenting and extracting elements from the first text information to obtain vector information; According to the hierarchical relationship constructed by the knowledge base base, entities corresponding to each level in the vector information are linked to obtain graph information; The graph information is merged to obtain the multi-layer structure knowledge graph.

3. The method according to claim 2, characterized in that The hierarchical relationship constructed according to the knowledge base base links the entities corresponding to each level in the vector information to obtain graph information, including: Using directly captured information in the vector information as the first entity of the bottom layer; Determining the relevance of the vector information through the large language model, taking the complex interpretation information in the vector information as a second entity of the second layer, and linking the second entity with the first entity; The clear interpretation information in the vector information is used as a third entity of the third layer, and the third entity is linked with the second entity to obtain the graph information.

4. The method according to claim 2, characterized in that: The segmenting and element extraction of the first text information to obtain vector information includes: Performing text segmentation on the first text information to obtain text blocks; Extract elements from the text block using the BERT model to obtain a text vector; All text vectors are processed by mean pooling and text similarity to obtain vector information.

5. The method according to claim 1, characterized in that The loading and retrieval enhancement generates a database, integrates a multi-layer structure knowledge graph, performs retrieval analysis on the signaling text information, and obtains prompt words and retrieval results, including: Based on the multi-layer structure knowledge graph, the signaling text information is subjected to association analysis to determine keywords; Based on the keyword, a hierarchical search is performed through a matching process in the multi-layer structure knowledge graph, and entity activation and knowledge collection are performed based on the search results to obtain primary information; The primary information is combined with the signaling text information, and retrieved by a hybrid retriever to obtain secondary information; The secondary information is filtered through the clear interpretation information in the knowledge base base to obtain the tertiary information; The primary information, the secondary information, and the tertiary information are integrated to obtain prompt words and search results.

6. The method according to claim 5, characterized in that The hybrid retriever determines the secondary information by the following steps: Determine the scoring function by ranking the documents to be retrieved; Determine a relevance function according to the score function and the relevance score of the document to be retrieved; Determining a search score according to a correlation function of the primary information and a correlation function of the signaling text information; Secondary information is determined based on the search score.

7. The method according to claim 1, characterized in that The method further comprises: A signaling analysis architecture is constructed, the architecture comprising: The computing network resource layer is used to provide resources for building the knowledge base base; A technical layer, used to establish the large language model; The agent layer is used to implement signaling analysis based on expert orchestration and task scheduling; Small model layer, used for signaling analysis in specific fields.

8. A signaling analysis system, characterized in that: The system comprises: The first module is used to obtain signaling text information; The second module is used to load the retrieval enhancement generation database, integrate the multi-layer structure knowledge graph, perform retrieval analysis on the signaling text information, and obtain prompt words and retrieval results; the multi-layer structure knowledge graph is obtained by constructing a multi-layer data set of domain knowledge by a large language model; The third module is used to input the prompt word and the search result into a large language model to obtain an analysis conclusion.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images