Signaling analysis method, system, electronic device, and storage medium

Abstract

The application discloses a signaling analysis method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring signaling text information; loading a retrieval enhancement generation database, fusing a multi-layer structure knowledge graph, and performing retrieval analysis on the signaling text information to obtain a prompt word and a retrieval result; the multi-layer structure knowledge graph is obtained by a large language model constructing a multi-layer data set on domain knowledge; and the prompt word and the retrieval result are input into the large language model to obtain an analysis conclusion. The embodiments of the application are beneficial to improving the quality and efficiency of signaling analysis through retrieval enhancement generation and a large language model. The application can be widely applied in the technical field of Internet of Things.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and in particular to a signaling analysis method, system, electronic device, and storage medium. Background Technology

[0002] Signaling analysis is fundamental to communication network operation and maintenance. Most maintenance personnel use tools like Wireshark for in-depth bitstream analysis to diagnose network problems. However, this method is highly dependent on the expertise of technical personnel and is not very efficient. The main difficulties include the following:

[0003] The difficulty of filtering massive amounts of data: When capturing a large amount of network traffic, it is difficult to accurately find specific signaling flows from the huge amount of data. It requires a lot of time and effort to filter and screen, and it is easy to miss key signaling.

[0004] Complex Protocol Analysis: Effective signaling analysis using Wireshark requires a deep understanding of various network protocols. This includes the protocol structure, working principles, and common signaling flows. Even for professional engineers, performing end-to-end signaling analysis from wireless, transport, core network to user MEC in a 5G private network environment requires cross-disciplinary analysis. Summary of the Invention

[0005] The main objective of this application is to provide an efficient signaling analysis method, system, electronic device, and storage medium.

[0006] To achieve the above objectives, one aspect of this application proposes a signaling analysis method. The method includes: acquiring signaling text information; loading a retrieval enhancement generation database, fusing a multi-layered knowledge graph, and performing retrieval analysis on the signaling text information to obtain prompt words and retrieval results; the multi-layered knowledge graph is constructed from domain knowledge using a large language model through a multi-layered dataset; and inputting the prompt words and retrieval results into the large language model to obtain analysis conclusions. This application embodiment, through retrieval enhancement generation and a large language model, helps to improve the quality and efficiency of signaling analysis.

[0007] In some embodiments, the method provided in this application establishes a multi-layered knowledge graph through the following steps:

[0008] Obtain the first text information from the signaling domain knowledge base;

[0009] The first text information is segmented and elements are extracted to obtain vector information;

[0010] Based on the hierarchical relationship constructed by the knowledge base foundation, the entities corresponding to each level in the vector information are linked to obtain the graph information;

[0011] The graph information is merged to obtain the multi-layered knowledge graph.

[0012] In some embodiments, the method provided in this application, wherein linking entities corresponding to each level in the vector information according to the hierarchical relationship constructed by the knowledge base foundation to obtain graph information includes:

[0013] The directly extracted information from the vector information is used as the first entity at the bottom layer;

[0014] The relevance of the vector information is determined by the large language model, the complex semantic information in the vector information is taken as the second entity of the second layer, and the second entity is linked with the first entity.

[0015] The explicit interpretation information in the vector information is used as the third entity of the third layer, and the third entity is linked with the second entity to obtain the graph information.

[0016] In some embodiments, the method provided in this application, wherein segmenting and extracting elements from the first text information to obtain vector information includes:

[0017] The first text information is segmented to obtain text blocks;

[0018] The text block is element-wise extracted using the BERT model to obtain a text vector;

[0019] Mean pooling and text similarity processing are performed on all text vectors to obtain vector information.

[0020] In some embodiments, the method provided in this application, wherein loading the retrieval enhancement generation database, fusing a multi-layered knowledge graph, and performing retrieval analysis on the signaling text information to obtain prompt words and retrieval results includes:

[0021] Based on the aforementioned multi-layered knowledge graph, the signaling text information is analyzed for correlation to determine keywords;

[0022] Based on the keywords, a hierarchical retrieval is performed through the matching process in the multi-layered knowledge graph, and entity activation and knowledge collection are carried out based on the retrieval results to obtain primary information;

[0023] The primary information is combined with the signaling text information and retrieved using a hybrid retrieval tool to obtain secondary information.

[0024] The secondary information is filtered by explicitly defining the information in the knowledge base to obtain tertiary information;

[0025] The primary, secondary, and tertiary information are fused to obtain prompts and search results.

[0026] In some embodiments, the hybrid retrieval device determines secondary information through the following steps:

[0027] The scoring function is determined by the ranking of the documents to be retrieved;

[0028] The relevance function is determined based on the scoring function and the relevance score of the document to be retrieved;

[0029] The retrieval score is determined based on the correlation function of the primary information and the correlation function of the signaling text information;

[0030] Secondary information is determined based on the search score.

[0031] In some embodiments, the method provided in this application further includes:

[0032] An architecture for constructing signaling analysis is provided, the architecture comprising:

[0033] The computing network resource layer is used to provide resources for building the knowledge base foundation;

[0034] The technical layer is used to build the large language model.

[0035] The intelligent agent layer is used to perform signaling analysis based on expert orchestration and task scheduling.

[0036] Small model layers are used for signaling analysis in specific domains.

[0037] To achieve the above objectives, another aspect of this application proposes a signaling analysis system, the system comprising:

[0038] The first module is used to acquire signaling text information;

[0039] The second module is used to load the retrieval enhancement generation database, integrate a multi-layered knowledge graph, and perform retrieval analysis on the signaling text information to obtain prompt words and retrieval results; the multi-layered knowledge graph is constructed by a large language model using multi-layered datasets of domain knowledge.

[0040] The third module is used to input the prompt words and the search results into the large language model to obtain analysis conclusions.

[0041] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0042] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0043] The embodiments of this application include at least the following beneficial effects: The method provided in the embodiments of this application includes: acquiring signaling text information; loading a retrieval enhancement generation database, fusing a multi-layered knowledge graph, performing retrieval analysis on the signaling text information, and obtaining prompt words and retrieval results; the multi-layered knowledge graph is constructed by a large language model using a multi-layered dataset of domain knowledge; inputting the prompt words and the retrieval results into the large language model to obtain analysis conclusions. The embodiments of this application, through retrieval enhancement generation and a large language model, are beneficial to improving the quality and efficiency of signaling analysis. Attached Figure Description

[0044] Figure 1 This is a diagram illustrating the interface of the Wireshark-based analysis process in related technologies.

[0045] Figure 2 This diagram illustrates application scenarios of DPI-based analysis processes in related technologies.

[0046] Figure 3 This is a flowchart of an embodiment of the signaling analysis method provided in this application;

[0047] Figure 4 This is a flowchart of another embodiment of the signaling analysis method provided in this application;

[0048] Figure 5 This is a flowchart of an embodiment of the retrieval process based on retrieval enhancement provided in this application;

[0049] Figure 6 This is a flowchart of an embodiment of the signaling analysis architecture provided in this application;

[0050] Figure 7 This is a schematic diagram of an embodiment of the signaling analysis process provided in this application;

[0051] Figure 8 This is a schematic diagram of the interface of another embodiment of the signaling analysis process provided in this application;

[0052] Figure 9 This is a schematic diagram of the signaling analysis system provided in the embodiments of this application;

[0053] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0055] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0056] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0058] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0059] Signaling analysis is a technical process for capturing, parsing, monitoring, and analyzing control signaling messages within a communication system. Signaling typically refers to control information transmitted between network nodes (such as base stations, switches, and servers). It manages and coordinates call establishment, maintenance, termination, and other network management and configuration tasks. Analyzing signaling can help understand the operation of the communication system, identify potential problems, assess service quality, support troubleshooting, and optimize network configuration.

[0060] Deep Packet Inspection (DPI) is an application-layer traffic inspection and control technology. It does more than just examine packet headers; it can identify different application-layer protocols, user behaviors, traffic types, etc. It can be used to monitor network traffic, track user behavior, and identify and prevent network attacks.

[0061] The Evolved Packet Core (EPC), also known as the 4G core network, includes the Mobility Management Entity (MME), Serving Gateway (S-GW), Packet Data Network Gateway (P-GW), and Home Subscriber Server (HSS).

[0062] The 5G Core Network (5GC) includes network elements with control plane functions (AMF, SMF, PCF), user plane (UPF), and network elements with data storage and management functions (UDM).

[0063] IP Multimedia Subsystem (IMS) is an IP-based network architecture used to provide multimedia communication services.

[0064] Large Language Models (LLMs) are artificial intelligence algorithms trained using deep learning techniques and massive amounts of data, primarily used to process and understand human language.

[0065] Knowledge graphs (KG), also known as scientific knowledge graphs, use various graphical and other visualization techniques to describe knowledge resources and their carriers, and to mine, analyze, construct, draw, and display knowledge and the relationships between them.

[0066] The Resource Description Framework (RDF) is a data model that uses XML syntax. RDF's function is to describe the characteristics of resources and the relationships between them using triples. It stores triple data line by line in text format. Relational databases are used to store knowledge graphs. Graph data is represented using triples, with each triple representing a row in the table.

[0067] Retrieval-Augmented Generation (RAG) is an artificial intelligence technique that combines information retrieval and language generation. Its main purpose is to improve the accuracy, reliability, and information richness of the output of generative artificial intelligence models by utilizing information retrieved from external sources.

[0068] Signaling analysis is fundamental to communication network operation and maintenance. Most maintenance personnel use tools like Wireshark for in-depth bitstream analysis to diagnose network problems. However, this method is highly dependent on the expertise of technical personnel and is not very efficient. The main difficulties include the following:

[0069] The difficulty of filtering massive amounts of data: When capturing a large amount of network traffic, it is difficult to accurately find specific signaling flows from the huge amount of data. It requires a lot of time and effort to filter and screen, and it is easy to miss key signaling.

[0070] Complex Protocol Analysis: Effective signaling analysis using Wireshark requires a deep understanding of various network protocols. This includes the protocol structure, working principles, and common signaling flows. Even for professional engineers, performing end-to-end signaling analysis from wireless, transport, core network to user MEC in a 5G private network environment requires cross-disciplinary analysis.

[0071] Analysis of report ambiguity: While large models can enhance the knowledge content of analytical conclusions, they are prone to creating illusions and providing incorrect judgments and inferences.

[0072] Lack of visualization and ease of use: Wireshark's interface is relatively complex, containing function options and parameter settings, making it difficult to use. While Wireshark can display detailed signaling data, it lacks visualization capabilities. The lack of intuitive graphical representations for complex signaling flows makes understanding and analysis even more difficult.

[0073] like Figure 1 As shown, a typical Wireshark analysis process requires analyzing the signaling interaction flow and content line by line and frame by frame to identify problems in the interaction process.

[0074] In customer-deployed 5G private network applications, signaling analysis software primarily exists in the form of DPI (Distributed Point of Interest) sensing probes. However, in private network operation and maintenance management, there is a lack of user-friendly language and interfaces to explain the professional analysis results of the DPI sensing probes to customers, thus hindering their in-depth self-operation and self-diagnosis of the private network. For example... Figure 2 As shown, DPI mainly collects data around the edge UPF, and the acquisition interfaces include N3 / N4 / N9 interfaces, generating XDR bitstream files.

[0075] In view of this, this application provides a signaling analysis method to improve the quality and efficiency of signaling analysis. This invention can be applied to the fields of software technology and network communication.

[0076] The signaling analysis method provided in this application relates to the field of Internet of Things (IoT) technology. The signaling analysis method provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the signaling analysis method, but is not limited to the above forms.

[0077] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0078] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0079] Figure 3 This is an optional flowchart of the signaling analysis method provided in the embodiments of this application; Figure 3The method may include, but is not limited to, steps S100 to S300.

[0080] Step S100: Obtain signaling text information;

[0081] Step S200: Load the retrieval enhancement generation database, integrate the multi-layer structured knowledge graph, and perform retrieval analysis on the signaling text information to obtain prompt words and retrieval results; the multi-layer structured knowledge graph is constructed by a large language model using multi-layered datasets of domain knowledge;

[0082] Step S300: Input the prompt words and search results into the large language model to obtain the analysis conclusions.

[0083] This application proposes a method for constructing a signaling analysis intelligent agent based on an AI large-scale model. Combining the signaling analysis workflow, it intelligently upgrades traditional signaling analysis measures, integrating wireless / core network signaling data analysis and processing capabilities with large-scale model language understanding and content generation capabilities. This enables automatic understanding, analysis, reasoning, and signaling knowledge retrieval and question answering of network signaling, significantly improving the quality and efficiency of complaint analysis and network optimization analysis. (Refer to...) Figure 4 As shown, this application achieves high-order automated and intelligent signaling analysis based on a large model, and enhances the reliability of signaling analysis through RAG+ knowledge graph fusion.

[0084] In some embodiments, the method provided in this application establishes a multi-layered knowledge graph through the following steps:

[0085] Obtain the first text information from the signaling domain knowledge base;

[0086] The first text information is segmented and its elements are extracted to obtain vector information;

[0087] Based on the hierarchical relationship constructed by the knowledge base, the entities corresponding to each level in the vector information are linked to obtain the graph information;

[0088] By merging the graph information, a multi-layered knowledge graph is obtained.

[0089] In some possible implementations, a three-layer dataset of domain knowledge is constructed by extracting entity relationships in the signaling domain using LLM. LLM is then used for segmentation and block extraction to extract label information, achieving automated graph construction. In some embodiments, a hierarchical dataset of domain knowledge is constructed by extracting entity relationships in the signaling domain using LLM, and the graph is constructed by extracting labels in segments and blocks.

[0090] In some embodiments, the method provided in this application links entities corresponding to each level in the vector information according to the hierarchical relationship constructed by the knowledge base foundation to obtain graph information, including:

[0091] The directly extracted information from the vector information is used as the first entity at the bottom layer;

[0092] The relevance of vector information is determined by a large language model, and the complex semantic information in the vector information is used as the second entity of the second layer. The second entity is then linked to the first entity.

[0093] The explicit interpretation information in the vector information is used as the third entity of the third layer. The third entity is linked with the second entity to obtain the graph information.

[0094] In some possible implementations, this application provides hierarchical linking: communication signaling is a specialized field that uses a precise terminology system and is built upon many well-defined definitions and relationships. This application also provides graph construction: utilizing LLM, specific entity objects and their relationships are identified from the text of the signaling domain knowledge base, transformed into a structured knowledge representation, and presented through a graph structure. In some embodiments, this application constructs a three-layer linked knowledge database for communication signaling to form a comprehensive signaling analysis knowledge graph. The bottom layer consists of basic information entities, the second layer is a simple knowledge graph layer constructed by linking the bottom-layer entities using LLM, and the third layer is a signaling display and analysis layer that links the entities in the second layer.

[0095] In some embodiments, the method provided in this application segmentes and extracts elements from first text information to obtain vector information, including:

[0096] The first text information is segmented to obtain text blocks;

[0097] The text vector is obtained by extracting elements from the text block using the BERT model.

[0098] Mean pooling and text similarity processing are performed on all text vectors to obtain vector information.

[0099] In some possible implementations, there are many ways to extract elements, and those skilled in the art can also extract elements in other ways; this application does not impose any specific limitations. It is understood that this application uses semantic segmentation before and after signaling, employs a sliding window to process a fixed number of segments each time and continuously adjusts the window to maintain consistency in signaling information. LLM is used to identify and extract graph node examples from source text blocks, and the text vectorization method of the BERT model is used to provide example guidance for LLM training.

[0100] In some embodiments, refer to Figure 5 As shown in the embodiments of this application, the method loads a retrieval enhancement generation database, integrates a multi-layered knowledge graph, performs retrieval analysis on signaling text information, and obtains prompt words and retrieval results, including:

[0101] Step 210: Based on a multi-layered knowledge graph, perform association analysis on the signaling text information to determine keywords;

[0102] Step 220: Based on keywords, perform hierarchical retrieval through the matching process in the multi-layered knowledge graph, and perform entity activation and knowledge collection based on the retrieval results to obtain primary information;

[0103] Step 230: Combine the primary information with the signaling text information, and retrieve the secondary information using a hybrid retrieval tool;

[0104] Step 240: Filter the secondary information using the explicit definition information in the knowledge base base to obtain the tertiary information;

[0105] Step 250: Merge the primary, secondary, and tertiary information to obtain prompts and search results.

[0106] In some possible implementations, a Retrieval Enhancement Generation (RAG) technique is employed, with automatic analysis of the input signaling. It is understood that this application uses a high-efficiency strategy retrieval method based on primary STM-R as the processing step for the RAG retrieval enhancement generation technique, outputting entity content and related content within its scope, providing input for secondary LLM enhanced retrieval.

[0107] In some embodiments, the method provided in this application provides that the hybrid retrieval system determines secondary information through the following steps:

[0108] The scoring function is determined by the ranking of the documents to be retrieved;

[0109] The relevance function is determined based on the scoring function and the relevance score of the document to be retrieved;

[0110] The retrieval score is determined based on the correlation function of the primary information and the correlation function of the signaling text information;

[0111] Secondary information is determined based on the search score.

[0112] In some possible implementations, secondary LLM is used to enhance retrieval. By combining LLM with the second-level semantic meaning of the dataset through prompt words, the model's understanding of specific questions is enhanced. This application combines the original query with additional prompt words generated after merging and processing additional contextual information, and uses a hybrid retrieval approach to fuse document retrieval results, thereby enhancing the model's understanding of specific questions.

[0113] In some embodiments, the method provided in this application further includes:

[0114] The architecture for signaling analysis is constructed, and the architecture includes:

[0115] The computing network resource layer is used to provide resources for building the knowledge base foundation;

[0116] The technical layer is used to build large language models;

[0117] The intelligent agent layer is used to perform signaling analysis based on expert orchestration and task scheduling.

[0118] Small model layers are used for signaling analysis in specific domains.

[0119] This application provides a technical architecture, specifically a technical framework for a signaling analysis intelligent agent based on an AI large model. (Refer to...) Figure 6 As shown, the technical architecture specifically includes:

[0120] Computing network resources: Provides the foundational resources for running large AI models, satisfying the purpose of computing resource allocation and network configuration optimization.

[0121] Technical layer: Professional large-scale model foundation and core algorithms.

[0122] AI Agent: An AI intelligent agent that achieves intelligent computing through expert orchestration, task scheduling, and AI intelligent actuators.

[0123] Small models: also known as domain models, provide predictive warnings, perception assessments, failure code localization, network element health assessments, fault localization, impact analysis, and cross-domain correlation analysis in the signaling domain through enhanced intelligent computing.

[0124] Instructions: Based on the agent's evaluation results, make decisions on the underlying core control instructions.

[0125] Public components: Provide basic domain knowledge vector knowledge base, construct teleprompter, evaluation tools and AI embedding technology, etc.

[0126] Application scenarios: It provides core functions such as signaling analysis and reporting services, signaling source tracing, signaling interpretation, and Q&A for frontline complaint handlers and network optimization personnel.

[0127] Input learning: Inputs include alerts, knowledge Q&A, and data queries, and the AI ​​Agent is trained based on the inputs.

[0128] The following is a detailed description and explanation of the solutions in the embodiments of the present invention, using specific application examples. The signaling analysis process provided in this application includes the following steps:

[0129] Step 0: Knowledge Base Construction, forming an initial signaling knowledge database based on signaling protocol standards (including signaling message / element parameter explanations), classic cases, technical documents, and expert experience across multiple business scenarios such as EPC / 5GC / IMS. Here, the data information is divided into three layers:

[0130] At the lowest level, the developer directly captures files from signaling packets, including contextual relationships and error data connections.

[0131] The second layer consists of reliable signaling-related manuals, technical documents, textbooks, and signaling analysis papers in the field of communications, which contain complex and related interpretations.

[0132] The third layer consists of protocol source files that define the meaning of each signaling message. This is a direct interpretation.

[0133] Step 1: Extract entity relationships in the signaling domain using LLM to construct a three-layer dataset of domain knowledge (of course, those skilled in the art can adjust the number of structural layers of the dataset and knowledge graph according to actual needs). Use LLM to segment and block, extract label information, and realize automated graph construction (i.e., the process of establishing a multi-layer structured knowledge graph in this application).

[0134] Sub-step 1.1: Graph construction: Using LLM, specific entity objects and their relationships are identified from the text of the signaling domain knowledge base, transformed into a structured knowledge representation, and presented through the structure of a graph.

[0135] Sub-step 1.1.1: Raw Text Segmentation: First, the data text is segmented using a hybrid method combining signaling phrases and structural unit-based segmentation. Specifically, line breaks are used to segment paragraphs within the signaling document. Then, semantic blocks are created using pre- and post-signaling delimiters, employing methods including propositional shifting and sub-block derivation. A sliding window technique is used, processing five paragraphs at a time. By continuously adjusting the window, removing the previous signaling block, and adding the next, the consistency of the signaling information is maintained.

[0136] Sub-step 1.1.2: Element extraction: Identify and extract graph node examples from each source text block, using an LLM designed to identify all relevant entities in the text.

[0137] Specifically, LLM uses a pre-trained language model (Bidirectional Encoder Representations from Transform, BERT). During entity discovery, a neural network using a transformer mechanism is employed. For each entity, LLM is prompted to output a name, type, and description. The name can be the exact text from a document or a paraphrase from a signaling technical document, carefully chosen to reflect precise semantic information suitable for subsequent processing. The type is selected by LLM from a predefined table, and the description is an entity explanation generated by LLM, combined with the context from the document. To ensure the model's effectiveness, some examples are provided to guide LLM in generating the required output.

[0138] Here, in order to better retrieve valid entities from the database, the BERT model used outputs entities through steps such as vectorization, generating triples, and using a triple-repeating network with adaptive boundaries to map the vectors to a lower-dimensional domain.

[0139] In practical application, this project uses a state-of-the-art text vectorization method, employing the BERT model described earlier as the text encoder, to process any text sequence x containing N words. i After reading the text into the BERT model, the objective function is maximized by optimizing the training embedding matrix M and the weights of the neural network, as shown in the following equation:

[0140]

[0141] x represents all words in the vocabulary. To derive the vector representation of the entire text, mean pooling is used, as shown below:

[0142]

[0143] Among them, V i,j Let J represent the vector of the j-th word in the i-th text. i This represents the total number of words in the entity text. After calculating the text vector values, the cosine similarity method is used to calculate the entity text similarity, as shown below:

[0144]

[0145] Where D represents the vector dimension.

[0146] The final output of the hidden layer is x i The initial representation h i :

[0147] [h1,h2,…,h n ] = BERT φ ([x1,x2,…,x n ]),

[0148] in This represents the parameters of BERT. Then, for each predefined entity category c... j , by average labeling c j The word embedding representation is used to construct its initial prototype vector hc j .

[0149] Sub-step 1.1.3: Hierarchical Linking (i.e., the process of determining graph information in this application): Communication signaling is a specialized field that uses a precise terminology system and is built upon many well-defined definitions and relationships, such as the meaning of a signal or the meaning of an error value. In this field, LLM cannot distort, modify, or add creative or random elements; therefore, a special hierarchical linking model is prepared here for communication signaling to form a comprehensive communication signaling analysis knowledge graph. Previously, a three-layer knowledge database has been prepared. Here, we pre-set the bottom-level information according to the steps described above to form entities. The second-level information is first constructed using a simple knowledge graph construction method with LLM, and the graph is constructed by combining LLM. The entities in the second layer are linked to the entities in the first layer based on the relevance detected by LLMs. Then, the signaling of the third layer is explicitly defined and linked to the entities in the second layer. For each entity, the text embedding of its name is compared with the vocabulary in the communication protocol.

[0150] Sub-step 1.1.4: Label Generation and Graph Merging: After constructing the metagraph, scan each data block to develop a global graph that connects all metagraphs. The merged metagraph nodes are linked together using the hierarchical linking rules described above.

[0151] After completing the above steps, a three-layer knowledge graph is formed, where nodes represent entities and edges represent relationships between entities. By constructing the knowledge graph, the extracted entity relationships are presented in a visual way.

[0152] Step 2: Using Retrieval Enhancement Generation (RAG) technology, the input signaling is automatically analyzed (i.e., steps S210 to S250 in this application). During this process, information will be efficiently retrieved in response using a strategy referred to in this patent as STM (Signaling Transfer Matching)-Retrieve through LLM. The output sequence text H is... n have:

[0153] H n =STM-R φ ([x1,x2,…,x n ,Y])

[0154] Where, x i Y is the input signaling text, φ is the database, and φ is the preset parameters of the retrieval strategy, including priority, hierarchical retrieval iteration range, and initial value.

[0155] Next, key information is selected as search keywords based on the analysis scenario. By combining signaling entity relationships, RAG search information, and prompts, logical reasoning of the signaling process is achieved, thereby improving the accuracy and reliability of signaling analysis.

[0156] Sub-step 2.1: Input: Input the bitstream file (i.e., the signaling text information in this application).

[0157] Sub-step 2.2: Parsing: Parse the raw bitstream according to the protocol.

[0158] Sub-step 2.3: Database Loading: Load the RAG knowledge base, including analysis cases, domain knowledge, field explanations, and failure code rules. Use text embedding and vector search techniques to convert the original query into a vector representation.

[0159] Sub-step 2.4: One STM (Signaling Transfer Matching)-retrieve strategy retrieval:

[0160] Sub-step 2.4.1: Keyword extraction: First, using the existing three-layer knowledge graph, perform association analysis on the vector form of the original code stream to generate summary tag descriptions. Then, extract the feature information of the signaling plaintext to generate summary tag descriptions in the form of keywords.

[0161] Sub-step 2.4.2: Hierarchical Retrieval: Next, using the extracted keyword information, the most relevant meta-graph units are identified through a top-down matching process across the three-layer knowledge graph. This process begins with larger graphs, using keywords to define the scope, and then gradually indexes the smaller graphs they contain, repeating the process.

[0162] Sub-step 2.4.3: Entity activation: Then, after hierarchical retrieval and gradually narrowing the scope, the retrieval will eventually reach the meta layer and lock some entities. These retrieved entities will be collected to complete the activation of these entities.

[0163] Sub-step 2.4.4: Knowledge Collection: Finally, based on these entities, collect the entity content and all related content of related entities within their scope, including knowledge of relevant protocol content, technical documentation, relevance and relationships with other entities, and content of any linked entities, etc. Retrieval complete.

[0164] Sub-step 2.5: Secondary LLM Enhanced Retrieval: The retrieved information is used as additional contextual input and merged with the original query to generate additional prompts. Here, an advanced hybrid retrieval engine is used for document retrieval. This method fuses the retrieval results of sparse and dense retrieval models through Convex linear combination fusionretriever (Convex-fusion), as shown in the equation. Where Sconvex(q,d) represents the final relevance score of the document; q represents the query, d represents the document; α ranges from (0,1]; S' is the standardized formula for the relevance score; Smax It is the candidate document with the highest relevance score, S min It is the lowest relevance score of the current candidate document, rank represents the document ranking; k is an adjustable parameter.

[0165] S conver (q,d)=(1-α)S′ sparse (q,d)+αS′ dense (q,d)

[0166]

[0167] Based on the above retrieval method, combined with LLM to associate the dataset with two-level sentence meaning through prompt words, the model's understanding of specific questions is enhanced.

[0168] Sub-step 2.6: Three-stage accurate knowledge screening: The knowledge generated in the secondary enhanced retrieval is screened according to the underlying serious signaling definition information in the three-layer data structure to provide more rigorous information for the generated knowledge, delete obvious erroneous information, reduce large model illusions, and reduce guessing and uncertainty.

[0169] Sub-step 2.7: Knowledge Fusion: Using a common embedding model, all the knowledge vectors and entities and relationships in the graph obtained above are transformed into a unified vector representation, and computation and analysis are performed in the same vector space. This facilitates the integration of knowledge vector information into the final conclusion and also allows the use of the graph's structural information to optimize the representation and computation of knowledge vectors.

[0170] Step 3: By fusing the signaling knowledge vector knowledge base and graph data from the previous step, and combining the prompt words and search results, input the data into the large model again for logical reasoning to generate analysis conclusions and reports.

[0171] Step 4: Update the domain knowledge base, including case studies and expert experience, and repeat steps 0 to 3.

[0172] In a specific example, its application effect is described below:

[0173] Based on bitstream content analysis, the system automatically recommends analysis schemes. Users can select the specific data type of signaling analysis according to their needs. Figure 7 As shown. This application can also perform multi-domain bitstream analysis, output analysis reports, and assist analysts in quickly locating problems. (Refer to...) Figure 8 As shown.

[0174] This application proposes the concept and technical framework of a signaling analysis agent based on an AI large-scale model. It details the core business processes and specific execution steps of the signaling analysis agent implemented based on the large-scale model, particularly the processing steps that enhance the reliability of signaling analysis through RAG+knowledge graph fusion, including a three-layer knowledge structure model and a hybrid retrieval strategy.

[0175] This application innovatively introduces a new BERT-based text vectorization method in the field of signaling analysis, including adaptive boundary rules and multiple mapping mechanisms. By overlaying data with adaptive boundaries in three layers and mapping it to a lower dimension, it is possible to obtain reliable entities in the database more effectively.

[0176] This application innovatively uses a three-layer data foundation model and performs hierarchical linking with pre-defined rules. The analysis database into which the large model will be entered is pre-classified according to different levels of credibility, and the rules for hierarchical linking are based on the credibility rating standards of different data sources within the database.

[0177] This application addresses the unique characteristics of signaling analysis by combining the two points mentioned above. It innovatively proposes a dedicated signaling analysis strategy based on a three-layer knowledge structure model and hybrid retrieval. Through targeted iteration during the retrieval process, activation of meta-layer entities, and secondary enhancement of retrieval priority rule settings, it solves the illusion problem that often occurs during the analysis of large models, thereby reducing errors.

[0178] Compared with the prior art, the present invention has the following beneficial effects:

[0179] First, we will develop a visualization tool for signaling analysis based on a large-scale interactive model. This tool will present signaling data using text-based Q&A and graphical methods, allowing for a more intuitive understanding of complex signaling interaction processes.

[0180] Second, we build a large model and knowledge base to create a knowledge brain for the signaling field. We complete the parsing of multi-format corpora and the construction of semantic vector indexes. The process uses structured databases and special retrieval strategies to reduce the illusions that may occur during the large model inference process. It can complete work order status queries, Q&A in the operation and maintenance field, and ad-hoc Q&A in documents, output conclusions, promote knowledge sharing, empower intelligent operation and maintenance, and improve the work efficiency of operation and maintenance personnel.

[0181] Third, by combining domain knowledge, upper-layer applications can be rapidly expanded. The platform can be used to implement upper-layer applications of the core network configuration data auditing agent, integrate natural language processing technology and large model capabilities to clean, retrieve and structure raw document data, and achieve grammar and standardization auditing; and use entity recognition to extract key instructions and their attributes, convert them into code, and achieve automated auditing.

[0182] Potential future applications of this application include:

[0183] By combining domain knowledge to rapidly expand upper-layer applications, the platform can be used to implement upper-layer applications of the core network configuration data auditing agent. It integrates natural language processing technology and large model capabilities to clean, retrieve, and structure raw document data, achieving grammar and standardization auditing; and uses entity recognition to extract key instructions and their attributes, converting them into code to achieve automated auditing.

[0184] Alarms are classified based on performance work order data, key information of the work orders is extracted, and agent intelligent technology is used to call the corresponding capability API interface to achieve fault delimitation and location through model collaboration, thereby improving the efficiency of alarm handling.

[0185] This application is conducive to improving the efficiency of telecommunications professionals in network operation and maintenance; increasing the depth and breadth of service awareness probe application in 5G private network platforms; creating customer-friendly professional service capabilities; and further enhancing product competitiveness.

[0186] Please see Figure 9 This application also provides a signaling analysis system that can implement the above-described signaling analysis method. The system includes:

[0187] The first module 810 is used to acquire signaling text information;

[0188] The second module 820 is used to load the retrieval enhancement generation database, integrate a multi-layered knowledge graph, perform retrieval analysis on signaling text information, and obtain prompt words and retrieval results; the multi-layered knowledge graph is constructed by a large language model using multi-layered datasets of domain knowledge.

[0189] The third module, 830, is used to input the prompt words and search results into the large language model to obtain analysis conclusions.

[0190] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0191] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described signaling analysis method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0192] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0193] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0194] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0195] The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the signaling analysis method of the embodiments of this application.

[0196] The input / output interface 903 is used to implement information input and output;

[0197] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0198] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0199] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0200] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described signaling analysis method.

[0201] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0202] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0203] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0204] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0205] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0206] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0207] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0208] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0209] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0210] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0211] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0212] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0213] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A signaling analysis method, characterized in that, The method includes: Obtain signaling text information; The system loads an enhanced retrieval database, integrates a multi-layered knowledge graph, and performs retrieval analysis on the signaling text information to obtain prompts and retrieval results. The multi-layered knowledge graph is constructed from a large language model using a multi-layered dataset of domain knowledge. Input the suggested words and the search results into the large language model to obtain the analysis conclusions; The multi-layered knowledge graph is established through the following steps: Obtain the first text information from the signaling domain knowledge base; The first text information is segmented and elements are extracted to obtain vector information; Based on the hierarchical relationship constructed by the knowledge base foundation, the entities corresponding to each level in the vector information are linked to obtain the graph information; The graph information is merged to obtain the multi-layered knowledge graph; The step of linking entities at each level in the vector information according to the hierarchical relationship constructed based on the knowledge base foundation to obtain graph information includes: The directly extracted information from the vector information is used as the first entity at the bottom layer; The relevance of the vector information is determined by the large language model, the complex semantic information in the vector information is taken as the second entity of the second layer, and the second entity is linked with the first entity. The explicit interpretation information in the vector information is used as the third entity of the third layer, and the third entity is linked with the second entity to obtain the graph information.

2. The method according to claim 1, characterized in that, The step of segmenting and extracting elements from the first text information to obtain vector information includes: The first text information is segmented to obtain text blocks; The text block is element-wise extracted using the BERT model to obtain a text vector; Mean pooling and text similarity processing are performed on all text vectors to obtain vector information.

3. The method according to claim 1, characterized in that, The loading and retrieval enhancement database is integrated with a multi-layered knowledge graph to perform retrieval analysis on the signaling text information, obtaining prompts and retrieval results, including: Based on the aforementioned multi-layered knowledge graph, the signaling text information is analyzed for correlation to determine keywords; Based on the keywords, a hierarchical retrieval is performed through the matching process in the multi-layered knowledge graph, and entity activation and knowledge collection are carried out based on the retrieval results to obtain primary information; The primary information is combined with the signaling text information and retrieved using a hybrid retrieval tool to obtain secondary information. The secondary information is filtered by explicitly defining the information in the knowledge base to obtain tertiary information; The primary, secondary, and tertiary information are fused to obtain prompts and search results.

4. The method according to claim 3, characterized in that, The hybrid retrieval system determines secondary information through the following steps: The scoring function is determined by the ranking of the documents to be retrieved; The relevance function is determined based on the scoring function and the relevance score of the document to be retrieved; The retrieval score is determined based on the correlation function of the primary information and the correlation function of the signaling text information; Secondary information is determined based on the search score.

5. The method according to claim 1, characterized in that, The method further includes: An architecture for constructing signaling analysis is provided, the architecture comprising: The computing network resource layer is used to provide resources for building the knowledge base foundation; The technical layer is used to build the large language model. The intelligent agent layer is used to perform signaling analysis based on expert orchestration and task scheduling. Small model layers are used for signaling analysis in specific domains.

6. A system for implementing the signaling analysis method as described in any one of claims 1-5, characterized in that, The system includes: The first module is used to acquire signaling text information; The second module is used to load the retrieval enhancement generation database, integrate a multi-layered knowledge graph, and perform retrieval analysis on the signaling text information to obtain prompt words and retrieval results; the multi-layered knowledge graph is constructed by a large language model using multi-layered datasets of domain knowledge. The third module is used to input the prompt words and the search results into the large language model to obtain analysis conclusions.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images