Network topology image analysis method, device and equipment and readable storage medium

Abstract

The invention discloses a network topology image analysis method, device and equipment and a readable storage medium, and is applied to the technical field of network communication, and the method comprises the steps: carrying out the recognition of a to-be-analyzed network topology image, and obtaining a target node list and a text information list; obtaining associated description information according to the target node list and the text information list; according to the association description information and the target problem description information, analyzing the to-be-analyzed network topology image by using a network topology analysis large model to obtain a network topology image analysis result corresponding to the target problem description information; the target problem description information is an analysis demand related to the to-be-analyzed network topology image. Compared with current manual analysis, the network topology analysis method has the advantages that the to-be-analyzed network topology image is analyzed by using the network topology analysis large model to obtain the network topology image analysis result, intelligent and accurate analysis of the network topology image is realized, and the network topology image analysis result corresponding to the target problem description information is obtained.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a network topology image parsing method, device, equipment and readable storage medium. Background Art

[0002] Network topology usually exists in the form of images in various technical solutions, user manuals, configuration documents and other carriers, providing users with intuitive and clear topology information. However, with the complexity of the topology structure, manually looking up the network topology image and understanding the complex connection relationship therein is inefficient and error-prone. However, since the network topology exists in the form of an image, it is difficult to obtain accurate topology information through automated means without supporting formatted description information.

[0003] It can be seen that how to improve the accuracy of network topology image analysis is a technical problem that people in this field need to solve urgently. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a network topology image parsing method, device, equipment and readable storage medium, which solves the technical problem in the prior art that the network topology image cannot be accurately parsed.

[0005] In order to solve the above technical problems, the present invention provides a network topology image parsing method, comprising:

[0006] Identify the network topology image to be parsed and obtain the target node list and text information list;

[0007] According to the target node list and the text information list, associated description information is obtained; wherein the associated description information is information indicating the relationship between the node and the text information;

[0008] According to the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using a large network topology analysis model to obtain a network topology image analysis result corresponding to the target problem description information; wherein the target problem description information is a analysis requirement related to the network topology image to be analyzed.

[0009] Optionally, the network topology image to be parsed is identified to obtain a target node list and a text information list, including:

[0010] The network topology image to be parsed is identified using a target recognition model to obtain the target node list and the text information list; wherein the target node list is a list including network node icons, and the text information list is a list including node location information and node text information.

[0011] Optional object recognition model training process, including:

[0012] Constructing a target recognition data set; wherein the target recognition data set is a data set including a training network topology image and its corresponding training node information and training text information;

[0013] The training network topology image in the target recognition data set is used as input, and the training node information and training text information are used as output, and the machine learning model is trained to obtain the target recognition model.

[0014] Optionally, according to the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using a network topology analysis large model to obtain a network topology image analysis result corresponding to the target problem description information, including:

[0015] generating identification auxiliary information corresponding to the associated description information;

[0016] Determine target recognition auxiliary prompt information based on the associated description information and the recognition auxiliary information;

[0017] Based on the target identification auxiliary prompt information and the target problem description information, the network topology image to be analyzed is analyzed using the network topology analysis large model to obtain the network topology image analysis result.

[0018] Optionally, according to the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using a network topology analysis large model to obtain a network topology image analysis result corresponding to the target problem description information, including:

[0019] Determine multi-level problem description information based on the associated description information; wherein the multi-level problem description information is problem description information determined based on at least two types of information among nodes, edges and attributes;

[0020] The network topology parsing large model is iteratively called according to the multi-level problem description information and the target problem description information, and a bottom-up multi-level image parsing is performed on the network topology image to be parsed to obtain the network topology image parsing result.

[0021] Optionally, the multi-level problem description information is three-level problem description information determined based on three dimensions of nodes, edges and attributes, the three levels including an atomic layer, a combination layer and an overall layer, the atomic layer sets problems from a single dimension, the combination layer sets combined problems from any two dimensions, and the overall layer sets overall problems from three dimensions;

[0022] Correspondingly, the network topology parsing large model is iteratively called according to the multi-level problem description information and the target problem description information, and a bottom-up multi-level image parsing is performed on the network topology image to be parsed to obtain the network topology image parsing result, including:

[0023] Input the network topology image to be analyzed and the atomic-level problem description information into the network topology analysis large model to obtain the atomic-level network topology image analysis result;

[0024] Inputting the problem description information of the atomic layer, the network topology image analysis result of the atomic layer, and the problem description information of the combined layer into the network topology analysis large model to obtain the network topology image analysis result of the combined layer;

[0025] Inputting the problem description information of the combined layer, the network topology image analysis result of the combined layer, and the problem description information of the overall layer into the network topology analysis large model to obtain the network topology image analysis result of the overall layer;

[0026] The problem description information of the overall layer, the network topology image analysis result of the overall layer, the target problem description information, and the network topology image to be analyzed are input into the network topology analysis large model to obtain the network topology image analysis result.

[0027] Optional, the training process of the large network topology parsing model includes:

[0028] Acquire network topology question-and-answer training data; wherein the network topology question-and-answer data is data including a network topology training image and a question-and-answer training result corresponding to the network topology training image, wherein the question-and-answer training result includes multi-level questions and answers corresponding to the questions at each level;

[0029] The network topology training images and questions at each level in the network topology question-answering training data are taken as input, and the answers corresponding to the questions at each level are taken as output, and the multimodal large model is trained to obtain the network topology parsing large model.

[0030] The present invention also provides a network topology image parsing device, comprising:

[0031] The recognition module is used to recognize the network topology image to be parsed and obtain the target node list and text information list;

[0032] An associated description information determination module, used to obtain associated description information according to the target node list and the text information list; wherein the associated description information is information indicating the relationship between the node and the text information;

[0033] A parsing module is used to parse the network topology image to be parsed using a large network topology parsing model according to the associated description information and the target problem description information, and obtain a network topology image parsing result corresponding to the target problem description information; wherein the target problem description information is a parsing requirement related to the network topology image to be parsed.

[0034] The present invention also provides a network topology image parsing device, comprising:

[0035] Memory for storing computer programs;

[0036] A processor is used to execute the computer program to implement the steps of the above-mentioned network topology image parsing method.

[0037] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned network topology image parsing method are implemented.

[0038] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned network topology image parsing method when executed by a processor.

[0039] It can be seen that the present invention obtains a target node list and a text information list by identifying the network topology image to be parsed; obtains associated description information based on the target node list and the text information list; wherein the associated description information is information representing the relationship between the node and the text information; and parses the network topology image to be parsed using a large network topology parsing model based on the associated description information and the target problem description information, and obtains a network topology image parsing result corresponding to the target problem description information; wherein the target problem description information is the parsing requirement related to the network topology image to be parsed.

[0040] The beneficial effect of the embodiments of the present invention is that compared with the current manual analysis of network topology images, the present application uses a large network topology analysis model to parse the network topology image to be analyzed based on the associated description information and the target problem description information, and obtains the network topology image analysis result, thereby realizing the analysis of the network topology image to be analyzed based on the associated description information, and obtaining the network topology image analysis result corresponding to the target problem description information, and the analysis result is accurate.

[0041] In addition, the present invention also provides a network topology image analysis device, equipment and readable storage medium, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 A flowchart of a network topology image parsing method provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a structural framework for building and applying a target recognition model provided by an embodiment of the present invention;

[0045] Figure 3 A flowchart of a network topology image analysis method provided by an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of the structural framework of a network topology image parsing method provided by an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of constructing a multimodal network topology training data set provided by an embodiment of the present invention;

[0048] Figure 6 A schematic diagram of the structure of a network topology image analysis device provided by an embodiment of the present invention;

[0049] Figure 7 A schematic diagram of the structure of a network topology image parsing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Please refer to Figure 1 , Figure 1 A flowchart of a network topology image parsing method provided by an embodiment of the present invention. The method may include:

[0052] S101, identifying the network topology image to be parsed, and obtaining a target node list and a text information list.

[0053] The execution subject of this embodiment is an electronic device. The electronic device in this embodiment can be a computer; or the electronic device in this embodiment can be a mobile phone, etc. In this embodiment, the network topology image to be parsed is a network topology image with a simple background, and the nodes and connection relationships (edges) of the network topology, as well as the attributes of the nodes or edges that may exist, can be clearly found in the image; this embodiment does not limit the specific method for identifying the network topology image to be parsed. For example, this embodiment can use a target recognition model to identify the parsed network topology image to obtain a target node list and a text information list. Or this embodiment can use image processing technology to extract node features in the image. For example, an edge detection algorithm can be used to identify the outline of the node; this embodiment can use OCR (optical character recognition) technology to detect the text area in the image. This embodiment does not limit the specific target node list and the specific text information list. In this embodiment, the target node list is a list composed of node icons, or the target node list in this embodiment is a table composed of node icons and node icon positions; the text information list in this embodiment is a list of information including text descriptions, for example, the text information in the text information list in this embodiment is the relationship between network nodes.

[0054] It should be further explained that, based on any of the above embodiments, in order to improve the accuracy of obtaining the target node list and the text information list, the above-mentioned identification of the network topology image to be parsed to obtain the target node list and the text information list may include: using the target recognition model to identify the network topology image to be parsed to obtain the target node list and the text information list; wherein the target node list is a list including network node icons, and the text information list is a list including node position information and node text information. This embodiment does not limit the specific target recognition model. For example, the target recognition model in this embodiment may be a CNN (convolutional neural network); or the target recognition model in this embodiment may also be a Transformer (deep learning model). The target node list in this embodiment is a list including node icons, and the target node list in this embodiment can determine the node. The text information list in this embodiment may include node position information, and the node position information can determine the position of each node; the node text information in this embodiment can determine the association parameters between nodes. The target recognition model in this embodiment may be a deep learning target recognition model, which takes the network node icon and text description as the recognition target, constructs the corresponding target recognition data set (the input is an image, and the output is a node target list and a text information list), and obtains the target recognition model through training. After the network topology image is input into the auxiliary model, a network node target list and a text list can be output. The list contains the category (network node icon, text description) and location information (pixel position in the image) of each target.

[0055] This embodiment can directly identify the network topology image based on the target recognition model to obtain a marked node list and a text information list, thereby improving the accuracy and efficiency of determining the marked node list and the text information list.

[0056] It should be further explained that, based on any of the above embodiments, in order to improve the accuracy of the target recognition model, the training process of the target recognition model may include:

[0057] S1: Construct a target recognition data set; wherein the target recognition data set is a data set including a training network topology image and its corresponding training node information and training text information.

[0058] The construction of the target recognition data set in this embodiment includes the construction of training samples for node target and text target recognition.

[0059] S2: Take the training network topology image in the target recognition dataset as input, and the training node information and training text information as output, train the machine learning model, and obtain the target recognition model.

[0060] In this embodiment, based on open source target detection algorithm models (for example: yolov5 (the fifth generation model of the YOLO series), CenterNet (a target detection algorithm that determines the target by detecting the center point of an object), etc.), a target recognition model for two types of targets, nodes and text, is trained. This model can detect topological node icon targets and text targets such as node names in network topology images; secondly, the text targets are further recognized through a common OCR (Optical Character Recognition) model to obtain specific text information. For ease of understanding, please refer to Figure 2 , Figure 2 A schematic diagram of the structural framework for building and applying a target recognition model provided in an embodiment of the present invention. The application of the target recognition model can be divided into two stages: training and reasoning. In the training stage, the output results of the target recognition model are used to identify nodes in the original topology image, including node positions and text information, and then these information are used to perform node update operations to generate a network topology image; in the reasoning stage, the output results of the target recognition model can be used to generate target recognition auxiliary prompts, thereby assisting in the understanding of the network topology image. This embodiment provides a specific training process for the target recognition model, which improves the accuracy of the target recognition model training.

[0061] S102, obtaining associated description information according to the target node list and the text information list; wherein the associated description information is information indicating the relationship between the node and the text information.

[0062] The association description information in this embodiment is text information combined by nodes and text. This embodiment does not limit the specific association description information. For example, the association description information in this embodiment can be that the weight between node A and node B is 3.

[0063] S103, according to the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using the network topology analysis model to obtain a network topology image analysis result corresponding to the target problem description information; wherein the target problem description information is the analysis requirements related to the network topology image to be analyzed.

[0064] This embodiment does not limit the specific process of obtaining the target problem description information. For example, the target problem description information in this embodiment may be a custom parsing problem; or the target problem description information in this embodiment may be a set default problem description information. This embodiment does not limit the specific target problem description information. For example, the target problem description information in this embodiment may be all the information in the output network topology image; or the target problem description information in this embodiment may be part of the information in the output network topology image.

[0065] It should be further explained that, based on any of the above embodiments, the above-mentioned network topology image to be parsed is parsed using a network topology parsing large model according to the associated description information and the target problem description information to obtain a network topology image parsing result corresponding to the target problem description information, which may include: generating identification auxiliary information corresponding to the associated description information; determining target identification auxiliary prompt information based on the associated description information and the identification auxiliary information; parsing the network topology image to be parsed using a network topology parsing large model based on the target identification auxiliary prompt information and the target problem description information to obtain a network topology image parsing result. This embodiment does not limit the specific identification auxiliary information. For example, the identification auxiliary information in this embodiment may be: in the current network topology image, we have identified the following key elements: 2 core switch nodes, located at the upper and lower sides of the center area of ​​the image respectively; 4 access layer router nodes, and then we need to determine the connection relationship between the 2 core switch nodes; or the identification auxiliary information in this embodiment may also be: the currently identified nodes and edges are the nodes and edges in the network image to be identified, and the subsequently identified information needs to be included in the nodes and edges. In this embodiment, the target recognition auxiliary prompt information prompt is prompt information for the network topology analysis large model, which is used to provide auxiliary information and constrain the generated content of the large model. The target recognition auxiliary prompt information is to describe the target recognition result list output by the target recognition model in the form of natural language text, and add prompts to inform the network topology analysis large model that the relevant recognition results are known information in the network topology large model.

[0066] It should be further explained that, based on any of the above embodiments, in order to improve the accuracy of the analysis, the above-mentioned network topology image to be analyzed is analyzed using a network topology analysis large model according to the associated description information and the target problem description information to obtain a network topology image analysis result corresponding to the target problem description information, which may include: determining multi-level problem description information based on the associated description information; wherein the multi-level problem description information is the problem description information determined according to at least two types of information among nodes, edges and attributes; iteratively calling the network topology analysis large model according to the multi-level problem description information and the target problem description information, performing a bottom-up multi-level image analysis on the network topology image to be analyzed, and obtaining a network topology image analysis result. The specific multi-level problem description information is not limited in this embodiment. For example, the multi-level problem description information in this embodiment may be two levels; or the multi-level problem description information in this embodiment may be three levels. This embodiment does not limit the specific levels. For example, the multi-level image parsing in this embodiment is respectively setting questions from the node perspective and setting questions from the edge perspective; or the multi-level image parsing in this embodiment is respectively parsing from a single perspective and parsing from multiple perspectives, wherein the single perspective parsing is parsing from at least one single perspective of nodes, edges and attributes; the multi-angle parsing in this embodiment is parsing from a combination of at least two perspectives of nodes, edges and attributes. This embodiment provides a specific method of multi-level parsing, which improves the accuracy of parsing.

[0067] It should be further explained that, in order to improve the accuracy of image parsing, the above multi-level problem description information is based on the three dimensions of nodes, edges and attributes, and is determined to be a three-level problem description information. The three levels include atomic layer, combination layer and overall layer. The atomic layer sets the problem from a single dimension, the combination layer sets the combination problem from any two dimensions, and the overall layer sets the overall problem from three dimensions.

[0068] Accordingly, the network topology parsing large model is iteratively called according to the multi-level problem description information and the target problem description information, and the network topology image to be parsed is subjected to bottom-up multi-level image parsing to obtain the network topology image parsing result, which may include:

[0069] S1031, inputting the network topology image to be analyzed and the problem description information at the atomic level into the network topology analysis large model to obtain the network topology image analysis result at the atomic level;

[0070] S1032, inputting the problem description information of the atomic layer, the network topology image analysis result of the atomic layer, and the problem description information of the combination layer into the network topology analysis large model to obtain the network topology image analysis result of the combination layer;

[0071] S1033, inputting the problem description information of the combination layer, the network topology image analysis result of the combination layer, and the problem description information of the overall layer into the network topology analysis large model to obtain the network topology image analysis result of the overall layer;

[0072] S1034, inputting the overall layer problem description information, the overall layer network topology image analysis result, the target problem description information, and the network topology image to be analyzed into the network topology analysis large model to obtain the network topology image analysis result.

[0073] The method in this embodiment is to adopt a bottom-up multi-dimensional image understanding. Specifically, 1) construct a bottom-up topological image understanding question list, including three levels (atomic level, combination level, overall level) and three dimensions (nodes, edges, attributes), wherein each level sets relevant questions for three dimensions, the atomic level only sets questions from a single dimension, the combination level sets combination questions from any two dimensions, and the overall level includes all information of the three dimensions. 2) In the order of atomic level, combination level and overall level, the network topology image (image mode) and the questions (text mode) of this level are input into the network topology model in turn to obtain the answers corresponding to the questions; the previous questions and answers are used as historical dialogues, spliced ​​before the next question, and continue to be input into the network topology model to obtain the corresponding answers; after multiple iterations, all questions and answers of the three levels are completed (here, the three dimensions of the atomic level correspond to three questions asked at a time, and the combination level randomly selects two dimensions for questions). 3) The questions and answers at the overall level are taken as historical dialogues, spliced ​​before the questions raised by the user, and then input into the network topology model together with the network topology image as text modality to obtain the final result.

[0074] It should be further explained that, based on any of the above embodiments, the training process of the above-mentioned network topology parsing large model may include: obtaining network topology question-answering training data; wherein the network topology question-answering data is data including network topology training images and question-answering training results corresponding to the network topology training images, and the question-answering training results include multi-level questions and answers corresponding to the questions at each level; the network topology training images and questions at each level in the network topology question-answering training data are used as input, and the answers corresponding to the questions at each level are used as output to train the multimodal large model to obtain the network topology parsing large model. The network topology parsing large model in this embodiment, i.e., the multimodal network topology large model, is a multimodal network topology data set (network topology question-answering training data) constructed on the basis of an open source multimodal large model (for example: Qwen2_VL (visual language model), GLM4V (visual understanding ability), etc.), and a dedicated multimodal large model for network topology understanding is obtained by fine-tuning instructions. The instruction fine-tuning method is a general large language model fine-tuning method, which can be one of the fine-tuning methods such as full parameter fine-tuning, Lora (low-rank adaptation), P-Tuning (prompt adjustment), P-Tuningv2, etc. Among them, the multimodal network topology dataset is a dataset specially designed and constructed for training large network topology parsing models. The construction method is as follows: 1) construct a random generation algorithm for topological structure, which can randomly generate a network topological structure according to the set conditions. The main data of the structure includes node information (name, attributes) and edge information (starting point, end point, weight, direction, attributes, etc.). The network topological structure data is structured data; 2) construct topological question and answer data based on structured topological data, that is, generate bottom-up (the definition method is the same as above, including three levels (nodes, edges, combined overall layer) and three dimensions (nodes, edges and attributes)) questions and answers for the topological structure; 3) use the topological visualization method to visualize the generated structured data and generate the original topological image; 4) use the target recognition method to find the nodes in the original topological image, and replace the nodes with pre-prepared network node icons, that is, generate a network topological image. The network topological image is combined with the topological question and answer data generated in the second step to form a multimodal network topological data; 5) iteratively use the above method to construct a data set of a certain scale to form a multimodal network topological data set. During training, since each piece of training data contains multiple sets of question-answer combinations, a certain number of question-answer combinations are randomly selected as text information for training, and a large network topology model is formed through appropriate training fine-tuning parameters. This embodiment provides a specific method for training a large network topology model, which improves the accuracy of obtaining a large network topology model.

[0075] The network topology image parsing method provided by the embodiment of the present invention may include: S101, identifying the network topology image to be parsed, and obtaining a target node list and a text information list; S102, obtaining associated description information according to the target node list and the text information list; wherein the associated description information is information indicating the relationship between the node and the text information; S103, parsing the network topology image to be parsed using a network topology parsing large model according to the associated description information and the target problem description information, and obtaining a network topology image parsing result corresponding to the target problem description information; wherein the target problem description information is a parsing requirement related to the network topology image to be parsed. Compared with the current manual parsing of network topology images, the present application parses the network topology image to be parsed using a network topology parsing large model according to the associated description information and the target problem description information, and obtains a network topology image parsing result, thereby realizing accurate parsing of the network topology image to be parsed based on the associated description information, and obtaining a network topology image parsing result corresponding to the target problem description information.

[0076] In order to make the present invention easier to understand, please refer to Figure 3 , Figure 3 A flowchart of a network topology image parsing method provided by an embodiment of the present invention may specifically include:

[0077] S201. Obtain a network topology image to be analyzed and a natural language problem description.

[0078] The natural language question description in this embodiment is the content of the network topology map that the user wants to know (for example, how many nodes and edges there are in the topology map). The user can define the question by himself. If the user does not enter a specific question, the default question is to output the formatted representation of the complete information of the network topology map. For easier understanding, please refer to Figure 4 , Figure 4 A schematic diagram of the structural framework of a network topology image parsing method provided by an embodiment of the present invention, Figure 4 When displaying the network topology analysis, from bottom to top, there are atomic layer questions and answers, combination layer questions and answers, and overall layer questions and answers. In addition, the question and answer content of each layer will be input to the next layer as historical questions and answers. The natural language input question description in this embodiment is optional and may or may not be present.

[0079] S202: Use the target recognition model to recognize the network topology image to be analyzed, and obtain a network node target list and a text information list.

[0080] The target recognition model (target recognition auxiliary model) in this embodiment is a deep learning model that uses network node icons and text descriptions as recognition targets.

[0081] S203: Convert the network node target list and the text information list into text information, and add prompt information to obtain target identification auxiliary prompt information.

[0082] The target recognition auxiliary prompt information in this embodiment is used to prompt auxiliary information and constrain the information of the content generated by the large model.

[0083] S204, taking the target recognition auxiliary prompt information and the natural language problem description as text modal information, taking the network topology image as image modal information, and using the multimodal network topology parsing large model to generate a network topology image parsing result.

[0084] The multimodal network topology parsing large model in this embodiment corresponds to the network topology parsing large model mentioned above. This embodiment can train the multimodal large model based on the generated multimodal network topology training data set (corresponding to the network topology question and answer training data mentioned above) to form a multimodal network topology parsing large model. Based on this model, topology image understanding (understanding means parsing) can improve the accuracy of topology image understanding, and further adopts a bottom-up multi-dimensional image understanding method to finally generate complete structured network topology information. The natural language problem description in this embodiment corresponds to the multi-level problem description information mentioned above. For ease of understanding, please refer to Figure 5 , Figure 5A schematic diagram of constructing a multimodal network topology training data set provided in an embodiment of the present invention, the main steps may include: 1. Developing a random generation algorithm for a topology structure, including a node generation strategy, an edge generation strategy, and an attribute generation strategy, and randomly generating a network topology structure according to set strategy conditions. The main data of the structure includes node information (name, attributes) and edge information (starting point, end point, weight, direction, attributes, etc.), and the network topology structure data is structured data; 2. Generating topology question-answering data based on the structured topology data, that is, generating bottom-up (defined in the same way as above, including three levels and three dimensions) questions and answers for the topology structure, wherein The atomic layer contains three types of question and answer data: nodes, edges, and attributes. The combination layer contains questions and answers of three combinations of nodes, edges, and attributes (nodes + edges, nodes + attributes, and edges + attributes). The overall layer contains all the information of nodes, edges, and attributes in the question and answer. 3. Use the topological visualization method (for example: based on the networkx library of Python) to visualize the generated structured network topological structure data and generate the original topological image. 4. Use the target recognition method (using the target recognition auxiliary model) to find the nodes in the original topological image and replace the nodes with the pre-prepared network node icons, that is, generate the network topological image. Since the topological question and answer data associated with each network topological image contains multiple sets of question and answer combinations, according to the rules (you can set a specific number of question and answer combinations or a random number of question and answer combinations, or specify that a certain level of question and answer combinations must be included), a certain number of question and answer combinations are randomly selected as the text information corresponding to the network topological image. The network topological image and the topological question and answer data generated in step 2 are combined to form a multimodal network topological data. 5. Use the above method iteratively to construct a data set of a certain scale to form a multimodal network topological data set. During training, a large network topology model is formed through appropriate training and fine-tuning of parameters.

[0085] The present invention combines the big model with network topology image analysis, trains a special big model for network topology analysis based on the open source multimodal big model, and adds a target recognition auxiliary model to improve the ability to understand network topology images. Specifically, it solves the following technical problems:

[0086] (1) Design a method for generating multimodal network topology training data;

[0087] In order to solve the problems of large workload and high cost in network topology image collection and annotation, a simulated data generation method is adopted. First, formatted topological data (nodes, edges, attributes and other information) are randomly generated, and the topological image corresponding to the topological data is generated using a topological structure visualization tool; secondly, a network node icon set is constructed, and the ordinary nodes in the generated topological image are replaced with network node icons by target positioning and replacement to form the corresponding network topological image; finally, based on the network topological image and the corresponding topological data, multiple rounds of question-and-answer data for the topological image are constructed to understand all the available information in the network topological image. Through this process, a multimodal network topology dataset is produced.

[0088] (2) Propose a multimodal network topology analysis method;

[0089] Based on the generated multimodal network topology training data set, the multimodal large model is trained to form a multimodal network topology analysis large model. Based on this model, topology image analysis is performed. In order to improve the accuracy of topology image understanding, a bottom-up multi-dimensional image understanding method is further adopted to finally generate complete structured network topology information.

[0090] (3) A method for auxiliary knowledge generation based on target recognition model is proposed;

[0091] In order to reduce the problems of hallucinations and incomplete information acquisition that may occur in the image parsing process of the large network topology model, a network topology image target recognition model was constructed. This model is a deep learning target recognition model that can recognize node icons, description text, characters and other information in the topology image, and provide it to the large network topology model as the associated knowledge of the network topology image to enhance its understanding ability.

[0092] The beneficial effects of the technical solution of the present invention are as follows: (1) Improving the integrity of topological information extraction in network topological images. The proposed auxiliary knowledge generation method based on the target recognition model can provide prior knowledge for the multimodal large model, prompt and constrain the large model to obtain relevant information, and reduce the risk of missing information extracted by the large model. (2) Improving the accuracy of understanding the topological information in the network topological image

[0093] The designed network topology training data generation method can guide the large model to learn to parse the network topology image from different dimensions and levels, thereby improving the model's understanding ability. The proposed bottom-up multi-dimensional network topology image parsing method can further guide the large model to accurately capture various types of information, greatly improving the accuracy of topology information parsing.

[0094] The network topology image parsing device provided by an embodiment of the present invention is introduced below. The network topology image parsing device described below and the network topology image parsing method described above can be referred to each other.

[0095] Please refer to Figure 6 , Figure 6 A schematic diagram of the structure of a network topology image parsing device provided by an embodiment of the present invention may include:

[0096] The identification module 100 is used to identify the network topology image to be analyzed and obtain a target node list and a text information list;

[0097] The associated description information determination module 200 is used to obtain the associated description information according to the target node list and the text information list; wherein the associated description information is information indicating the relationship between the node and the text information;

[0098] The parsing module 300 is used to parse the network topology image to be parsed using a large network topology parsing model according to the associated description information and the target problem description information, and obtain a network topology image parsing result corresponding to the target problem description information; wherein the target problem description information is a parsing requirement related to the network topology image to be parsed.

[0099] Further, based on the above embodiment, the identification module 100 may include:

[0100] A list determination unit is used to use a target recognition model to identify the network topology image to be parsed to obtain the target node list and the text information list; wherein the target node list is a list including network node icons, and the text information list is a list including node location information and node text information.

[0101] Further, based on any of the above embodiments, the above network topology image analysis device may further include:

[0102] A target recognition data set construction module is used to construct a target recognition data set; wherein the target recognition data set is a data set including a training network topology image and its corresponding training node information and training text information;

[0103] The target recognition model training module is used to take the training network topology image in the target recognition data set as input, and the training node information and training text information as output, to train the machine learning model to obtain the target recognition model.

[0104] Further, based on any of the above embodiments, the above parsing module 300 may include:

[0105] an identification auxiliary information determination unit, configured to generate identification auxiliary information corresponding to the associated description information;

[0106] a target recognition auxiliary prompt information determining unit, configured to determine target recognition auxiliary prompt information based on the associated description information and the recognition auxiliary information;

[0107] The parsing unit is used to parse the network topology image to be parsed based on the target identification auxiliary prompt information and the target problem description information using the network topology parsing large model to obtain the network topology image parsing result.

[0108] Further, based on any of the above embodiments, the above parsing module 300 may include:

[0109] A multi-level problem description information determination unit, configured to determine multi-level problem description information based on the associated description information; wherein the multi-level problem description information is problem description information determined based on at least two types of information among nodes, edges and attributes;

[0110] A bottom-up parsing unit is used to iteratively call the network topology parsing large model according to the multi-level problem description information and the target problem description information, perform bottom-up multi-level image parsing on the network topology image to be parsed, and obtain the network topology image parsing result.

[0111] Further, based on the above embodiment, the above multi-level problem description information is three-level problem description information determined based on the three dimensions of nodes, edges and attributes, and the three levels include an atomic layer, a combination layer and an overall layer, the atomic layer sets problems from a single dimension, the combination layer sets combined problems from any two dimensions, and the overall layer sets overall problems from three dimensions;

[0112] Correspondingly, the bottom-up parsing unit may include:

[0113] The atomic layer parsing subunit is used to input the network topology image to be parsed and the atomic layer problem description information into the network topology parsing large model to obtain the atomic layer network topology image parsing result;

[0114] A combined layer parsing subunit, used for inputting the problem description information of the atomic layer and the network topology image parsing result of the atomic layer, as well as the problem description information of the combined layer, into the network topology parsing large model to obtain the network topology image parsing result of the combined layer;

[0115] The overall layer parsing subunit is used to input the problem description information of the combined layer, the network topology image parsing result of the combined layer, and the problem description information of the overall layer into the network topology parsing large model to obtain the network topology image parsing result of the overall layer;

[0116] The final parsing sub-unit is used to input the problem description information of the overall layer, the network topology image parsing result of the overall layer, the target problem description information, and the network topology image to be parsed into the network topology parsing large model to obtain the network topology image parsing result.

[0117] Further, based on any of the above embodiments, the above network topology image analysis device may further include:

[0118] A network topology question-answering training data acquisition module, used to acquire network topology question-answering training data; wherein the network topology question-answering data is data including a network topology training image and a question-answering training result corresponding to the network topology training image, wherein the question-answering training result includes multi-level questions and answers corresponding to the questions at each level;

[0119] The network topology image parsing model training module is used to take the network topology training image and each level of questions in the network topology question-answering training data as input, and the answers corresponding to each level of questions as output, to train the multimodal large model and obtain the network topology parsing large model.

[0120] It should be noted that the order of the modules and units in the above network topology image analysis device can be changed without affecting the logic.

[0121] The network topology image parsing device provided by the embodiment of the present invention may include: an identification module 100, which is used to identify the network topology image to be parsed and obtain a target node list and a text information list; an associated description information determination module 200, which is used to obtain associated description information according to the target node list and the text information list; wherein the associated description information is information indicating the relationship between the node and the text information; and a parsing module 300, which is used to parse the network topology image to be parsed using a network topology parsing large model according to the associated description information and the target problem description information, and obtain a network topology image parsing result corresponding to the target problem description information; wherein the target problem description information is a parsing requirement related to the network topology image to be parsed. Compared with the current manual parsing of network topology images, the present application parses the network topology image to be parsed using a network topology parsing large model according to the associated description information and the target problem description information, and obtains a network topology image parsing result, thereby realizing accurate parsing of the network topology image to be parsed based on the associated description information, and obtaining a network topology image parsing result corresponding to the target problem description information.

[0122] A network topology image parsing device provided by an embodiment of the present invention is introduced below. The network topology image parsing device described below and the network topology image parsing method described above can be referred to each other.

[0123] Please refer to Figure 7 , Figure 7 A schematic diagram of the structure of a network topology image parsing device provided by an embodiment of the present invention may include:

[0124] A memory 10, used for storing computer programs;

[0125] The processor 20 is used to execute a computer program to implement the above-mentioned network topology image analysis method.

[0126] The memory 10 , the processor 20 , and the communication interface 30 all communicate with each other via a communication bus 40 .

[0127] In the embodiment of the present invention, the memory 10 is used to store one or more programs, and the program may include program code, and the program code includes computer operation instructions. In the embodiment of the present invention, the memory 10 may store programs for implementing the following functions:

[0128] Identify the network topology image to be parsed and obtain the target node list and text information list;

[0129] According to the target node list and the text information list, the associated description information is obtained; wherein the associated description information is information indicating the relationship between the node and the text information;

[0130] According to the associated description information and the target problem description information, the network topology image to be parsed is parsed using the network topology parsing model to obtain a network topology image parsing result corresponding to the target problem description information; wherein the target problem description information is the parsing requirement related to the network topology image to be parsed.

[0131] In a possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0132] In addition, the memory 10 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include an NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0133] The processor 20 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, a microprocessor or any conventional processor, etc. The processor 20 may call a program stored in the memory 10 .

[0134] The communication interface 30 may be an interface of a communication module, and is used to connect to other devices or systems.

[0135] Of course, it should be noted that Figure 7 The structure shown does not constitute a limitation on the network topology image parsing device in the embodiment of the present invention. In actual applications, the network topology image parsing device may include Figure 7 More or fewer components than shown, or combinations of certain components.

[0136] The following is an introduction to a readable storage medium provided in an embodiment of the present invention. The readable storage medium described below and the network topology image parsing method described above can be referenced to each other.

[0137] The present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned network topology image parsing method are implemented.

[0138] The readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0139] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0140] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0141] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0142] The above is a detailed introduction to a network topology image analysis method, device, equipment and readable storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A network topology image analysis method, characterized in that: include: Identify the network topology image to be parsed and obtain the target node list and text information list; Obtaining association description information according to the target node list and the text information list; wherein the association description information is information indicating the relationship between the node and the text information; According to the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using a large network topology analysis model to obtain a network topology image analysis result corresponding to the target problem description information; wherein the target problem description information is a analysis requirement related to the network topology image to be analyzed.

2. The network topology image analysis method according to claim 1, characterized in that: The network topology image to be parsed is identified to obtain a target node list and a text information list, including: The network topology image to be parsed is identified using a target recognition model to obtain the target node list and the text information list; wherein the target node list is a list including network node icons, and the text information list is a list including node location information and node text information.

3. The network topology image analysis method according to claim 2, characterized in that: The training process of the target recognition model includes: Constructing a target recognition data set; wherein the target recognition data set is a data set including a training network topology image and its corresponding training node information and training text information; The training network topology image in the target recognition data set is used as input, and the training node information and training text information are used as output, and the machine learning model is trained to obtain the target recognition model.

4. The network topology image analysis method according to claim 1, characterized in that: According to the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using a network topology analysis large model to obtain a network topology image analysis result corresponding to the target problem description information, including: generating identification auxiliary information corresponding to the associated description information; Determine target recognition auxiliary prompt information based on the associated description information and the recognition auxiliary information; Based on the target identification auxiliary prompt information and the target problem description information, the network topology image to be analyzed is analyzed using the network topology analysis large model to obtain the network topology image analysis result.

5. The network topology image analysis method according to any one of claims 1 to 4, characterized in that: According to the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using a network topology analysis model to obtain a network topology image analysis result corresponding to the target problem description information, including: Determine multi-level problem description information based on the associated description information; wherein the multi-level problem description information is problem description information determined based on at least two types of information among nodes, edges and attributes; The network topology parsing large model is iteratively called according to the multi-level problem description information and the target problem description information, and a bottom-up multi-level image parsing is performed on the network topology image to be parsed to obtain the network topology image parsing result.

6. The network topology image analysis method according to claim 5, characterized in that: The multi-level problem description information is a three-level problem description information determined based on the three dimensions of nodes, edges and attributes. The three levels include an atomic level, a combination level and an overall level. The atomic level sets problems from a single dimension, the combination level sets combined problems from any two dimensions, and the overall level sets overall problems from three dimensions. Correspondingly, the network topology parsing large model is iteratively called according to the multi-level problem description information and the target problem description information, and a bottom-up multi-level image parsing is performed on the network topology image to be parsed to obtain the network topology image parsing result, including: Input the network topology image to be analyzed and the atomic-level problem description information into the network topology analysis large model to obtain the atomic-level network topology image analysis result; Inputting the problem description information of the atomic layer, the network topology image analysis result of the atomic layer, and the problem description information of the combined layer into the network topology analysis large model to obtain the network topology image analysis result of the combined layer; Inputting the problem description information of the combined layer, the network topology image analysis result of the combined layer, and the problem description information of the overall layer into the network topology analysis large model to obtain the network topology image analysis result of the overall layer; The problem description information of the overall layer, the network topology image analysis result of the overall layer, the target problem description information, and the network topology image to be analyzed are input into the network topology analysis large model to obtain the network topology image analysis result.

7. The network topology image analysis method according to claim 1, characterized in that: The training process of the large network topology analysis model includes: Acquire network topology question-and-answer training data; wherein the network topology question-and-answer data is data including a network topology training image and a question-and-answer training result corresponding to the network topology training image, wherein the question-and-answer training result includes multi-level questions and answers corresponding to the questions at each level; The network topology training images and questions at each level in the network topology question-answering training data are taken as input, and the answers corresponding to the questions at each level are taken as output, and the multimodal large model is trained to obtain the network topology parsing large model.

8. A network topology image analysis device, characterized in that: include: The recognition module is used to recognize the network topology image to be parsed and obtain the target node list and text information list; An associated description information determination module, used to obtain associated description information according to the target node list and the text information list; wherein the associated description information is information indicating the relationship between the node and the text information; A parsing module is used to parse the network topology image to be parsed using a large network topology parsing model according to the associated description information and the target problem description information, and obtain a network topology image parsing result corresponding to the target problem description information; wherein the target problem description information is a parsing requirement related to the network topology image to be parsed.

9. A network topology image analysis device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the network topology image parsing method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the network topology image parsing method according to any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that It comprises a computer program / instruction, which, when executed by a processor, implements the steps of the network topology image parsing method as claimed in any one of claims 1 to 7.

Images