A network topology image analysis method, device, equipment and readable storage medium

Abstract

The application discloses a network topology image analysis method and device, equipment and a readable storage medium, applied to the network communication technical field, comprising: identifying the network topology image to be analyzed, obtaining a target node list and a text information list; obtaining associated description information according to the target node list and the text information list; according to the associated description information and the target problem description information, the network topology analysis large model is used to analyze the network topology image to be analyzed, and a network topology image analysis result corresponding to the target problem description information is obtained; the target problem description information is an analysis requirement related to the network topology image to be analyzed. Compared with the current manual analysis, the network topology analysis large model is used to analyze the network topology image to be analyzed, and the network topology image analysis result is obtained, so that the network topology image is intelligently and accurately analyzed, and the network topology image analysis result corresponding to the target problem description information is obtained.

Description

Technical Field

[0001] This invention relates to the field of network communication technology, and in particular to a method, apparatus, device, and readable storage medium for network topology image parsing. Background Technology

[0002] Network topology is typically presented as an image in various technical solutions, user manuals, configuration documents, and other media, providing users with intuitive and clear topology information. However, as topologies become more complex, manually consulting network topology images to understand their intricate connections becomes inefficient and prone to errors. Furthermore, since network topology exists in image form, without accompanying formatted descriptive information, it is difficult to obtain accurate topology information automatically.

[0003] It is evident that improving the accuracy of network topology image parsing is a technical problem that urgently needs to be solved by those in this field. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, apparatus, device and readable storage medium for network topology image parsing, which solves the technical problem that the prior art cannot accurately parse network topology images.

[0005] To address the aforementioned technical problems, this invention provides a network topology image parsing method, comprising:

[0006] The network topology image to be parsed is identified to obtain a list of target nodes and a list of text information.

[0007] Based on the target node list and the text information list, associated description information is obtained; wherein, the associated description information is information representing the relationship between nodes and text information;

[0008] Based on the associated description information and the target problem description information, the network topology image to be parsed is parsed using a large network topology parsing model to obtain the network topology image parsing result corresponding to the target problem description information; wherein, the target problem description information is the parsing requirement related to the network topology image to be parsed.

[0009] Optionally, the process of identifying the network topology image to be parsed to obtain a list of target nodes and a list of text information includes:

[0010] The target recognition model is used to identify the network topology image to be parsed, thereby obtaining the target node list and the text information list; wherein, the target node list is a list including network node icons, and the text information list is a list including node location information and node text information.

[0011] Optional, the training process for the target recognition model includes:

[0012] Construct a target recognition dataset; wherein, the target recognition dataset is a dataset including a training network topology image and its corresponding training node information and training text information;

[0013] The target recognition model is obtained by using the training network topology image in the target recognition dataset as input, and the training node information and training text information as output to train the machine learning model.

[0014] Optionally, based on the associated description information and the target problem description information, the network topology image to be parsed is analyzed using a large network topology parsing model to obtain a network topology image parsing result corresponding to the target problem description information, including:

[0015] Generate identification assistance information corresponding to the associated description information;

[0016] Target identification assistance prompts are determined based on the associated description information and the identification assistance information;

[0017] Based on the target recognition auxiliary prompt information and the target problem description information, the network topology image to be parsed is parsed using the network topology parsing large model to obtain the network topology image parsing result.

[0018] Optionally, based on the associated description information and the target problem description information, the network topology image to be parsed is analyzed using a large network topology parsing model to obtain a network topology image parsing result corresponding to the target problem description information, including:

[0019] Multi-level problem description information is determined based on the associated description information; wherein, the multi-level problem description information is problem description information determined based on at least two types of information among nodes, edges, and attributes;

[0020] Based on the multi-level problem description information and the target problem description information, the network topology parsing big model is iteratively invoked to perform bottom-up multi-level image parsing on the network topology image to be parsed, so as to obtain the network topology image parsing result.

[0021] Optionally, the multi-level problem description information is a problem description information with three levels determined based on three dimensions: nodes, edges, and attributes. The three levels include an atomic layer, a combination layer, and a global layer. The atomic layer sets the problem from a single dimension, the combination layer sets the combined problem from any two dimensions, and the global layer sets the overall problem from all three dimensions.

[0022] Accordingly, the iterative invocation of the network topology parsing model based on the multi-level problem description information and the target problem description information to perform bottom-up multi-level image parsing on the network topology image to be parsed, and to obtain the network topology image parsing result, includes:

[0023] The network topology image to be parsed and the problem description information of the atomic layer are input into the large network topology parsing model to obtain the network topology image parsing result of the atomic layer;

[0024] The problem description information of the atomic layer, the network topology image parsing result of the atomic layer, and the problem description information of the combined layer are input into the large network topology parsing model to obtain the network topology image parsing result of the combined layer.

[0025] The problem description information of the combined layer, the network topology image parsing result of the combined layer, and the problem description information of the overall layer are input into the network topology parsing large model to obtain the network topology image parsing result of the overall layer.

[0026] The problem description information of the overall layer, the network topology image parsing result of the overall layer, the target problem description information, and the network topology image to be parsed are input into the large network topology parsing model to obtain the network topology image parsing result.

[0027] Optional, the training process for a large network topology analysis model includes:

[0028] Obtain network topology question-answering training data; wherein, the network topology question-answering data includes network topology training images and question-answering training results corresponding to the network topology training images, and the question-answering training results include multi-level questions and answers corresponding to each level of questions;

[0029] The network topology training image and the questions at each level in the network topology question-answering training data are used as inputs, and the answers corresponding to the questions at each level are used as outputs. The multimodal large model is trained to obtain the network topology analytical large model.

[0030] The present invention also provides a network topology image parsing device, comprising:

[0031] The recognition module is used to identify the network topology image to be parsed, and obtain a list of target nodes and a list of text information;

[0032] The association description information determination module is used to obtain association description information based on the target node list and the text information list; wherein, the association description information is information representing the relationship between nodes and text information;

[0033] The parsing module is used to parse the network topology image to be parsed using a large network topology parsing model based on the associated description information and the target problem description information, to obtain the network topology image parsing result corresponding to the target problem description information; wherein, the target problem description information is the parsing requirement related to the network topology image to be parsed.

[0034] The present invention also provides a network topology image parsing device, comprising:

[0035] Memory, used to store computer programs;

[0036] A processor is used to execute the computer program to implement the steps of the network topology image parsing method described above.

[0037] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the network topology image parsing method described above.

[0038] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the network topology image parsing method described above.

[0039] As can be seen, this invention identifies the network topology image to be parsed to obtain a list of target nodes and a list of text information; based on the list of target nodes and the list of text information, it obtains associated description information; wherein, the associated description information is information representing the relationship between nodes and text information; based on the associated description information and the target problem description information, it uses a large network topology parsing model to parse the network topology image to be parsed to obtain the network topology image parsing result corresponding to the target problem description information; wherein, the target problem description information is the parsing requirement related to the network topology image to be parsed.

[0040] The beneficial effects of this invention are as follows: Compared with the current method of manually analyzing network topology images, this application uses a large network topology analysis model to analyze the network topology image to be analyzed based on the associated description information and the target problem description information, thereby obtaining the network topology image analysis result based on the associated description information, and thus realizing the analysis of the network topology image to be analyzed based on the associated description information, and obtaining the network topology image analysis result corresponding to the target problem description information, with accurate analysis results.

[0041] In addition, the present invention also provides a network topology image parsing apparatus, device and readable storage medium, which also have the above-mentioned beneficial effects. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0043] Figure 1 A flowchart illustrating a network topology image parsing method provided in an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of a structural framework for constructing and applying a target recognition model, provided by an embodiment of the present invention.

[0045] Figure 3 A flowchart illustrating a network topology image parsing method provided in an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram of the structural framework of a network topology image parsing method provided in an embodiment of the present invention;

[0047] Figure 5 A schematic diagram illustrating the construction of a multimodal network topology training dataset according to an embodiment of the present invention;

[0048] Figure 6 This is a schematic diagram of the structure of a network topology image parsing device provided in an embodiment of the present invention;

[0049] Figure 7 This is a schematic diagram of the structure of a network topology image parsing device provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Please refer to Figure 1 , Figure 1 A flowchart illustrating a network topology image parsing method provided in an embodiment of the present invention. The method may include:

[0052] S101, identify the network topology image to be parsed to obtain a list of target nodes and a list of text information.

[0053] The execution subject of this embodiment is an electronic device. The electronic device in this embodiment can be a computer; or the electronic device in this embodiment can be a mobile phone, etc. The network topology image to be parsed in this embodiment is a network topology image with a simple background, in which the nodes and connections (edges) of the network topology can be clearly identified, as well as the attributes of any possible nodes or edges. This embodiment does not limit the specific method for recognizing the network topology image to be parsed. For example, this embodiment can use a target recognition model to recognize the parsed network topology image, obtaining a list of target nodes and a list of text information. Alternatively, this embodiment can use image processing technology to extract node features from the image. For example, an edge detection algorithm can be used to identify the outline of nodes; this embodiment can use OCR (Optical Character Recognition) technology to detect text regions in the image. This embodiment does not limit the specific list of target nodes and the specific list of text information. In this embodiment, the list of target nodes is a list composed of node icons, or the list of target nodes in this embodiment is a table composed of node icons and node icon positions; the list of text information in this embodiment is a list including textual descriptions, for example, the text information in the text information list in this embodiment is the relationship between network nodes.

[0054] It should be further explained that, based on any of the above embodiments, in order to improve the accuracy of obtaining the target node list and text information list, the above-mentioned identification of the network topology image to be analyzed to obtain the target node list and text information list may include: using a target recognition model to identify the network topology image to be analyzed to obtain the target node list and text information list; wherein, the target node list is a list including network node icons, and the text information list is a list including node location information and node text information. This embodiment does not limit the specific target recognition model. For example, the target recognition model in this embodiment can be a CNN (Convolutional Neural Network); or the target recognition model in this embodiment can also be a Transformer (Deep Learning Model). The target node list in this embodiment is a list including node icons, and the target node list in this embodiment can identify nodes. The text information list in this embodiment may include node location information, and the node location information can determine the position of each node; the node text information in this embodiment can determine the association parameters between nodes. The target recognition model in this embodiment can be a deep learning target recognition model, which uses network node icons and text descriptions as recognition targets, constructs a corresponding target recognition dataset (the input is an image, and the output is a node target list and a text information list), and trains the target recognition model. When a network topology image is input into this auxiliary model, it can output a list of network node targets and a list of text. The list contains the category (network node icon, text description) and location information (pixel position in the image) of each target.

[0055] This embodiment can directly identify network topology images based on target recognition models to obtain a list of labeled nodes and a list of text information, thereby improving the accuracy and efficiency of determining the list of labeled nodes and the list of text information.

[0056] It should be further noted that, based on any of the above embodiments, in order to improve the accuracy of the target recognition model, the training process of the target recognition model may include:

[0057] S1: Construct the target recognition dataset; wherein, the target recognition dataset is a dataset that includes the training network topology image and its corresponding training node information and training text information.

[0058] The construction of the target recognition dataset in this embodiment includes the construction of training samples for node target and text target recognition.

[0059] S2: Using the training network topology image in the target recognition dataset as input, and the training node information and training text information as output, train the machine learning model to obtain the target recognition model.

[0060] This embodiment is based on open-source object detection algorithm models (such as YOLOv5 (the fifth generation of the YOLO series), CenterNet (an object detection algorithm that determines the target by detecting the center point of the object), etc.), trained to form an object recognition model for two types of targets: nodes and text. This model can detect topological node icons and text targets such as node names in network topology images. Secondly, the text targets are further recognized using a general OCR (Optical Character Recognition) model to obtain specific text information. For easier understanding, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This embodiment of the invention provides a structural framework for constructing and applying a target recognition model. The application of the target recognition model can be divided into two stages: training and inference. In the training stage, the output of the target recognition model is used to identify nodes in the original topology image, including node positions and text information. This information is then used to perform node update operations to generate a network topology image. In the inference stage, the output of the target recognition model can be used to generate a target recognition prompt, thereby aiding in the understanding of the network topology image. This embodiment provides a specific training process for the target recognition model, improving the accuracy of the target recognition model training.

[0061] S102, based on the target node list and the text information list, obtain the association description information; wherein, the association description information is information representing the relationship between the nodes and the text information.

[0062] In this embodiment, the association description information is text information combining nodes and text. This embodiment does not limit the specific association description information. For example, the association description information in this embodiment could be that the weight between node A and node B is 3.

[0063] S103, Based on the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using the large network topology analysis model to obtain the network topology image analysis result corresponding to the target problem description information; wherein, the target problem description information is the analysis requirement related to the network topology image to be analyzed.

[0064] This embodiment does not limit the specific process of obtaining the target problem description information. For example, the target problem description information in this embodiment can be a custom parsing problem; or the target problem description information in this embodiment can be a set default problem description information. This embodiment does not limit the specific target problem description information. For example, the target problem description information in this embodiment can be all the information in the output network topology image; or the target problem description information in this embodiment can be part of the information in the output network topology image.

[0065] It should be further explained that, based on any of the above embodiments, the above-described method of parsing the network topology image to be parsed using a large network topology parsing model based on the associated description information and the target problem description information to obtain the network topology image parsing result corresponding to the target problem description information may include: generating identification auxiliary information corresponding to the associated description information; determining target identification auxiliary prompt information based on the associated description information and the identification auxiliary information; and parsing the network topology image to be parsed using a large network topology parsing model based on the target identification auxiliary prompt information and the target problem description information to obtain the network topology image parsing result. This embodiment does not limit the specific identification auxiliary information. For example, the identification auxiliary information in this embodiment may be: in the current network topology image, we have identified the following key elements: 2 core switch nodes, located on the upper and lower sides of the image center area respectively; 4 access layer router nodes, and next we need to determine the connection relationship between the 2 core switch nodes; or the identification auxiliary information in this embodiment may also be: the currently identified nodes and edges are the nodes and edges in the network image to be identified, and the information subsequently identified needs to be included in these nodes and edges. In this embodiment, the target recognition auxiliary prompt is a prompt message for the large-scale network topology analysis model. It provides auxiliary information and constrains the content generated by the large-scale model. The target recognition auxiliary prompt message describes the list of target recognition results output by the target recognition model in natural language text, and adds prompts to inform the large-scale network topology analysis model that the relevant recognition results are known information in the large-scale network topology model.

[0066] It should be further explained that, based on any of the above embodiments, in order to improve the accuracy of the analysis, the above-mentioned method of using a large network topology analysis model to analyze the network topology image to be analyzed, based on the associated description information and the target problem description information, to obtain the network topology image analysis result corresponding to the target problem description information, may include: determining multi-level problem description information based on the associated description information; wherein, the multi-level problem description information is the problem description information determined based on at least two types of information among nodes, edges, and attributes; iteratively calling the large network topology analysis model based on the multi-level problem description information and the target problem description information to perform bottom-up multi-level image analysis on the network topology image to be analyzed, and obtaining the network topology image analysis result. This embodiment does not limit the specific multi-level problem description information. For example, the multi-level problem description information in this embodiment may be two levels; or the multi-level problem description information in this embodiment may be three levels. This embodiment does not limit the specific levels. For example, in this embodiment, the multi-level image parsing is configured to set the problem from the perspective of nodes and from the perspective of edges; or, in this embodiment, the multi-level image parsing is configured to parse from a single angle and from multiple angles, wherein single-angle parsing is parsing from a single angle of at least one node, edge, and attribute; and multi-angle parsing is parsing from a combination of at least two angles of nodes, edges, and attributes. This embodiment provides a specific method for multi-level parsing, improving the accuracy of the parsing.

[0067] It should be further explained that, in order to improve the accuracy of image parsing, the above-mentioned multi-level problem description information is based on three dimensions: nodes, edges, and attributes. The three levels include the atomic layer, the combination layer, and the overall layer. The atomic layer sets the problem from a single dimension, the combination layer sets the combined problem from any two dimensions, and the overall layer sets the overall problem from all three dimensions.

[0068] Accordingly, based on the multi-level problem description information and the target problem description information, the network topology parsing model is iteratively invoked to perform bottom-up multi-level image parsing on the network topology image to be parsed, obtaining the network topology image parsing result, which may include:

[0069] S1031, Input the network topology image to be parsed and the problem description information of the atomic layer into the large network topology parsing model to obtain the network topology image parsing result of the atomic layer;

[0070] S1032, Input the problem description information of the atomic layer and the network topology image analysis result of the atomic layer, as well as the problem description information of the combined layer, into the large network topology analysis model to obtain the network topology image analysis result of the combined layer;

[0071] S1033, input the problem description information of the combined layer, the network topology image analysis result of the combined layer, and the problem description information of the overall layer into the large network topology analysis model to obtain the network topology image analysis result of the overall layer;

[0072] S1034. Input the overall layer problem description information, the overall layer network topology image parsing results, the target problem description information, and the network topology image to be parsed into the large network topology parsing model to obtain the network topology image parsing results.

[0073] The method in this embodiment employs a bottom-up, multi-dimensional image understanding approach. Specifically, 1) a bottom-up topological image understanding question list is constructed, including three levels (atomic layer, composite layer, and global layer) and three dimensions (node, edge, and attribute). Each level sets relevant questions for the three dimensions. The atomic layer sets questions from only a single dimension, the composite layer sets combined questions from any two dimensions, and the global layer includes all information from all three dimensions. 2) Following the order of the atomic layer, composite layer, and global layer, the network topology image (image modality) and the question (text modality) of that level are sequentially input into the large network topology model to obtain the answer corresponding to the question. The previous question and answer are used as historical dialogues, spliced ​​before the next question, and input into the large network topology model to obtain the corresponding answer. After multiple iterations, all questions and answers at the three levels are completed (here, the three dimensions of the atomic layer correspond to three questions asked at a time, while the composite layer randomly selects two dimensions to ask questions). 3) The questions and answers of the overall layer are treated as historical dialogues and spliced ​​before the questions raised by the user. They are then input into the large network topology model as a text modality along with the network topology image to obtain the final result.

[0074] It should be further explained that, based on any of the above embodiments, the training process of the aforementioned large-scale network topology parsing model may include: acquiring network topology question-and-answer training data; wherein, the network topology question-and-answer data includes network topology training images and question-and-answer training results corresponding to the network topology training images, the question-and-answer training results including multi-level questions and answers corresponding to each level of questions; using the network topology training images and questions at each level in the network topology question-and-answer training data as input, and the answers corresponding to each level of questions as output, the multimodal large-scale model is trained to obtain the large-scale network topology parsing model. The large-scale network topology parsing model in this embodiment is a multimodal network topology large-scale model, which is a dedicated multimodal large-scale model for network topology understanding obtained by constructing a multimodal network topology dataset (network topology question-and-answer training data) based on open-source multimodal large-scale models (e.g., Qwen2_VL (visual language model), GLM4V (visual understanding ability), etc.) and fine-tuning it through instructions. Instruction fine-tuning methods are general fine-tuning methods for large language models, and can be one of the following: full parameter fine-tuning, LoRa (low-rank adaptation), P-Tuning (cue-based tuning), P-Tuningv2, etc. Among them, the multimodal network topology dataset is a dataset specifically designed and constructed for training large-scale analytical models of network topology. The construction method is as follows: 1) Construct a topology random generation algorithm that can randomly generate network topology structures according to set conditions. The main data of this structure includes node information (name, attributes) and edge information (start point, end point, weight, direction, attributes, etc.). This network topology data is structured data; 2) Construct topology question-and-answer data based on the structured topology data, that is, generate bottom-up questions and answers (defined in the same way as above, including three levels (nodes, edges, and overall combined layer) and three dimensions (nodes, edges, and attributes)) for this topology structure; 3) Visualize the generated structured data using a topology visualization method to generate an original topology image; 4) Use a target recognition method to find the nodes in the original topology image and replace the nodes with pre-prepared network node icons to generate a network topology image. This network topology image is combined with the topology question-and-answer data generated in step 2 to form a multimodal network topology data; 5) Iteratively use the above methods to construct a dataset of a certain scale to form a multimodal network topology dataset. During training, since each training data set contains multiple question-and-answer combinations, a certain number of these combinations are randomly selected as text information for training. The network topology model is then trained by fine-tuning appropriate parameters. This embodiment provides a specific method for training a large-scale network topology model, improving the accuracy of the obtained model.

[0075] The network topology image parsing method provided in this invention embodiment may include: S101, identifying the network topology image to be parsed to obtain a target node list and a text information list; S102, obtaining associated description information based on the target node list and the text information list; wherein, the associated description information is information representing the relationship between nodes and text information; S103, parsing the network topology image to be parsed using a large-scale network topology parsing model based on the associated description information and the target problem description information to obtain a network topology image parsing result corresponding to the target problem description information; wherein, the target problem description information is the parsing requirement related to the network topology image to be parsed. Compared with the current method where network topology images can only be parsed manually, this application uses a large-scale network topology parsing model to parse the network topology image to be parsed based on the associated description information and the target problem description information to obtain a network topology image parsing result corresponding to the target problem description information.

[0076] For a clearer understanding of this invention, please refer to the following details. Figure 3 , Figure 3 A flowchart illustrating a network topology image parsing method provided in this embodiment of the invention may specifically include:

[0077] S201. Obtain the network topology image and natural language problem description to be parsed.

[0078] In this embodiment, the natural language question description is what the user wants to know about the network topology graph (e.g., how many nodes and edges are in the topology graph). The user can define their own question. If the user does not input a specific question, the default question is a formatted representation of the complete information of the network topology graph. For easier understanding, please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram of the structural framework of a network topology image parsing method provided in an embodiment of the present invention. Figure 4 When displaying network topology resolution, the question-and-answer format is arranged from bottom to top as follows: atomic layer question-and-answer, composite layer question-and-answer, and overall layer question-and-answer. Furthermore, the question-and-answer content of each layer is input as historical question-and-answer data to the next layer. In this embodiment, inputting the question description in natural language is optional and can be omitted.

[0079] S202. Use the target recognition model to identify the network topology image to be parsed, and obtain a list of network node targets and a list of text information.

[0080] The target recognition model (target recognition auxiliary model) in this embodiment is a deep learning model that uses network node icons and text descriptions as targets for recognition.

[0081] S203. Convert the network node target list and text information list into text information and add prompt information to obtain target recognition auxiliary prompt information.

[0082] In this embodiment, the target recognition auxiliary prompt information is used to provide auxiliary information and constrain the information of the content generated by the large model.

[0083] S204: The target recognition auxiliary prompts and natural language problem descriptions are used as text modal information, and the network topology image is used as image modal information. The network topology image analysis result is generated by using a multimodal network topology analysis model.

[0084] The multimodal network topology parsing model in this embodiment corresponds to the network topology parsing model mentioned above. This embodiment can train the multimodal large model based on the generated multimodal network topology training dataset (corresponding to the network topology question-answering training data mentioned above), forming a multimodal network topology parsing model. Topology image understanding (understanding refers to parsing) based on this model can improve the accuracy of topology image understanding. Furthermore, a bottom-up multi-dimensional image understanding method is adopted, ultimately generating complete structured network topology information. The natural language question description in this embodiment corresponds to the multi-level question description information mentioned above. For easier understanding, please refer to... Figure 5 , Figure 5This is a schematic diagram illustrating the construction of a multimodal network topology training dataset according to an embodiment of the present invention. The main steps may include: 1. Developing a random topology generation algorithm, including node generation strategies, edge generation strategies, and attribute generation strategies. The algorithm randomly generates a network topology structure according to set strategy conditions. The main data of this structure includes node information (name, attributes) and edge information (start point, end point, weight, direction, attributes, etc.). This network topology data is structured data; 2. Generating topology question-and-answer data based on the structured topology data, i.e., generating bottom-up (defined as above, including three levels and three dimensions) questions and answers for this topology structure. The intermediate atomic layer contains three types of question-and-answer data: nodes, edges, and attributes. The combination layer contains three combinations of nodes, edges, and attributes (node ​​+ edge, node + attribute, edge + attribute) in the question-and-answer format. The overall layer contains all information about nodes, edges, and attributes in the question-and-answer format. 3. The generated structured network topology data is visualized using a topology visualization method (e.g., using the Python-based networkx library) to generate the original topology image. 4. Nodes in the original topology image are found using a target recognition method (using a target recognition auxiliary model), and the nodes are replaced with pre-prepared network node icons, thus generating the network topology image. Since each network topology image is associated with multiple sets of question-and-answer combinations in its topology question-and-answer data, a certain number of question-and-answer combinations are randomly selected as the text information corresponding to the network topology image according to rules (a specific number of question-and-answer combinations can be selected, or a random number of question-and-answer combinations can be selected, or a certain level of question-and-answer combinations must be included). This network topology image, combined with the topology question-and-answer data generated in step 2, constitutes a multimodal network topology data set. 5. The above methods are used iteratively to construct a dataset of a certain size, forming a multimodal network topology dataset. During training, a large network topology model is formed by fine-tuning appropriate training parameters.

[0085] This invention combines a large-scale model with network topology image analysis. It enhances the understanding of network topology images by training a dedicated large-scale model for network topology analysis based on an open-source multimodal large-scale model and adding an object recognition auxiliary model. Specifically, it addresses the following technical problems:

[0086] (1) Design a method for generating training data for multimodal network topology;

[0087] To address the challenges of high workload and cost in network topology image acquisition and annotation, a simulated data generation approach was adopted. First, formatted topology data (nodes, edges, attributes, etc.) was randomly generated, and a topology visualization tool was used to generate the corresponding topology image. Second, a set of network node icons was constructed, and ordinary nodes in the generated topology image were replaced with network node icons using target localization and replacement methods to form the corresponding network topology image. Finally, based on the network topology image and the corresponding topology data, multi-round question-and-answer data was constructed to understand all usable information within the network topology image. This process resulted in the creation of a multimodal network topology dataset.

[0088] (2) A method for analyzing the topology of multimodal networks is proposed;

[0089] Based on the generated multimodal network topology training dataset, a large multimodal model is trained to form a large multimodal network topology parsing model. Topology image parsing is then performed based on this model. To improve the accuracy of topology image understanding, a bottom-up multi-dimensional image understanding method is further adopted, ultimately generating complete structured network topology information.

[0090] (3) An auxiliary knowledge generation method based on a target recognition model is proposed;

[0091] To reduce potential issues such as illusions and incomplete information acquisition during image parsing of large network topology models, a network topology image target recognition model was constructed. This model is a deep learning target recognition model that can identify node icons, descriptive text, characters, and other information in the topology image, providing them as associated knowledge of the network topology image to the large network topology model and enhancing its understanding capabilities.

[0092] The beneficial effects of the technical solution of this invention are as follows: (1) Improve the completeness of topological information extraction from network topology images. The proposed auxiliary knowledge generation method based on target recognition model can provide prior knowledge for multimodal large models, prompt and constrain the large models to acquire relevant information, and reduce the risk of missing information extracted by the large models. (2) Improve the accuracy of understanding topological information in network topology images.

[0093] The designed network topology training data generation method can guide large models to learn how to parse network topology images from different dimensions and levels, thereby improving the model's understanding ability. The proposed bottom-up multi-dimensional network topology image parsing method further guides large models to accurately capture various types of information, significantly improving the accuracy of topology information parsing.

[0094] The network topology image parsing device provided in the embodiments of the present invention will be described below. The network topology image parsing device described below can be referred to in correspondence with the network topology image parsing method described above.

[0095] Please refer to the details. Figure 6 , Figure 6 A schematic diagram of a network topology image parsing device provided in an embodiment of the present invention may include:

[0096] The recognition module 100 is used to recognize the network topology image to be parsed and obtain a list of target nodes and a list of text information.

[0097] The association description information determination module 200 is used to obtain association description information based on the target node list and the text information list; wherein, the association description information is information representing the relationship between nodes and text information;

[0098] The parsing module 300 is used to parse the network topology image to be parsed using a large network topology parsing model based on the associated description information and the target problem description information, to obtain the network topology image parsing result corresponding to the target problem description information; wherein, the target problem description information is the parsing requirement related to the network topology image to be parsed.

[0099] Furthermore, based on the above embodiments, the identification module 100 may include:

[0100] The list determination unit is used to identify the network topology image to be parsed using a target recognition model to obtain the target node list and the text information list; wherein, the target node list is a list including network node icons, and the text information list is a list including node location information and node text information.

[0101] Furthermore, based on any of the above embodiments, the network topology image parsing apparatus may further include:

[0102] The target recognition dataset construction module is used to construct the target recognition dataset; wherein, the target recognition dataset is a dataset including a training network topology image and its corresponding training node information and training text information;

[0103] The target recognition model training module is used to train the machine learning model by taking the training network topology image in the target recognition dataset as input and the training node information and training text information as output, so as to obtain the target recognition model.

[0104] Furthermore, based on any of the above embodiments, the parsing module 300 may include:

[0105] A recognition auxiliary information determination unit is used to generate recognition auxiliary information corresponding to the associated description information;

[0106] A target recognition auxiliary prompt information determination unit is used to determine target recognition auxiliary prompt information based on the associated description information and the recognition auxiliary information;

[0107] The parsing unit is used to parse the network topology image to be parsed based on the target recognition auxiliary prompt information and the target problem description information, using the network topology parsing big model, to obtain the network topology image parsing result.

[0108] Furthermore, based on any of the above embodiments, the parsing module 300 may include:

[0109] A multi-level problem description information determination unit is used to determine multi-level problem description information based on the associated description information; wherein, the multi-level problem description information is problem description information determined based on at least two types of information among nodes, edges, and attributes;

[0110] The bottom-up parsing unit is used to iteratively call the network topology parsing big model according to the multi-level problem description information and the target problem description information to perform bottom-up multi-level image parsing on the network topology image to be parsed, and obtain the network topology image parsing result.

[0111] Furthermore, based on the above embodiments, the multi-level problem description information is a problem description information with three levels determined based on three dimensions: nodes, edges, and attributes. The three levels include an atomic layer, a combination layer, and an overall layer. The atomic layer sets the problem from a single dimension, the combination layer sets the combined problem from any two dimensions, and the overall layer sets the overall problem from all three dimensions.

[0112] Correspondingly, a bottom-up parsing unit may include:

[0113] The atomic layer parsing subunit is used to input the network topology image to be parsed and the problem description information of the atomic layer into the large network topology parsing model to obtain the network topology image parsing result of the atomic layer;

[0114] The combined layer parsing subunit is used to input the problem description information of the atomic layer, the network topology image parsing result of the atomic layer, and the problem description information of the combined layer into the large network topology parsing model to obtain the network topology image parsing result of the combined layer.

[0115] The overall layer parsing subunit is used to input the problem description information of the combined layer, the network topology image parsing result of the combined layer, and the problem description information of the overall layer into the network topology parsing large model to obtain the network topology image parsing result of the overall layer.

[0116] The final parsing subunit is used to input the problem description information of the overall layer, the network topology image parsing result of the overall layer, the target problem description information, and the network topology image to be parsed into the large network topology parsing model to obtain the network topology image parsing result.

[0117] Furthermore, based on any of the above embodiments, the network topology image parsing apparatus may further include:

[0118] A network topology question-answering training data acquisition module is used to acquire network topology question-answering training data; wherein, the network topology question-answering data includes a network topology training image and question-answering training results corresponding to the network topology training image, and the question-answering training results include multi-level questions and answers corresponding to each level of questions;

[0119] The network topology image parsing model training module is used to train the multimodal large model by taking the network topology training image and the questions at each level in the network topology question-answering training data as input and the answers corresponding to the questions at each level as output, so as to obtain the network topology parsing large model.

[0120] It should be noted that the order of the modules and units in the above-mentioned network topology image analysis device can be changed without affecting the logic.

[0121] The network topology image parsing apparatus provided in this embodiment of the invention may include: an identification module 100, used to identify the network topology image to be parsed, and obtain a target node list and a text information list; an association description information determination module 200, used to obtain association description information based on the target node list and the text information list; wherein the association description information is information representing the relationship between nodes and text information; and a parsing module 300, used to parse the network topology image to be parsed using a large network topology parsing model based on the association description information and the target problem description information, and obtain a network topology image parsing result corresponding to the target problem description information; wherein the target problem description information is a parsing requirement related to the network topology image to be parsed. Compared with the current method of manually parsing network topology images, this application uses a large network topology parsing model based on the association description information and the target problem description information to parse the network topology image to be parsed, and obtains a network topology image parsing result, thereby achieving accurate parsing of the network topology image to be parsed based on the association description information, and obtaining a network topology image parsing result corresponding to the target problem description information.

[0122] The following describes a network topology image parsing device provided by an embodiment of the present invention. The network topology image parsing device described below can be referred to in correspondence with the network topology image parsing method described above.

[0123] Please refer to Figure 7 , Figure 7 A schematic diagram of a network topology image parsing device provided in an embodiment of the present invention may include:

[0124] Memory 10 is used to store computer programs;

[0125] The processor 20 is used to execute computer programs to implement the network topology image parsing method described above.

[0126] The memory 10, processor 20, and communication interface 30 all communicate with each other through the communication bus 40.

[0127] In this embodiment of the invention, the memory 10 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 10 may store programs for implementing the following functions:

[0128] The network topology image to be parsed is identified to obtain a list of target nodes and a list of text information.

[0129] Based on the target node list and the text information list, the associated description information is obtained; whereby the associated description information is information that represents the relationship between the nodes and the text information.

[0130] Based on the associated description information and the target problem description information, the network topology image to be analyzed is analyzed using the large network topology analysis model to obtain the network topology image analysis result corresponding to the target problem description information; where the target problem description information is the analysis requirement related to the network topology image to be analyzed.

[0131] In one possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0132] Furthermore, memory 10 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores operating systems and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and handling hardware-based tasks.

[0133] Processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic device. Processor 20 can be a microprocessor or any conventional processor. Processor 20 can call programs stored in memory 10.

[0134] The communication interface 30 can be an interface for the communication module, used to connect with other devices or systems.

[0135] Of course, it should be noted that, Figure 7 The structure shown does not constitute a limitation on the network topology image parsing device in the embodiments of the present invention. In practical applications, the network topology image parsing device may include devices such as... Figure 7 More or fewer components as shown, or combinations of certain components.

[0136] The readable storage medium provided in the embodiments of the present invention is described below. The readable storage medium described below can be referred to in correspondence with the network topology image parsing method described above.

[0137] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the network topology image parsing method described above.

[0138] The readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0140] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0141] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0142] The foregoing has provided a detailed description of a network topology image parsing method, apparatus, device, and readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for parsing network topology images, characterized in that, include: A target recognition model is used to identify the network topology image to be parsed, thereby obtaining a list of target nodes and a list of text information. The target recognition model is a model obtained by constructing a corresponding target recognition dataset and training a deep learning model, using network node icons and text descriptions as recognition targets. Based on the target node list and the text information list, associated description information is obtained; wherein, the associated description information is information representing the relationship between nodes and text information; Based on the associated description information and the target problem description information, the network topology image to be parsed is parsed using a large network topology parsing model to obtain the network topology image parsing result corresponding to the target problem description information; wherein, the target problem description information is the parsing requirement related to the network topology image to be parsed. Specifically, based on the associated description information and the target problem description information, the network topology image to be parsed is analyzed using a large network topology parsing model to obtain the network topology image parsing result corresponding to the target problem description information, including: Multi-level problem description information is determined based on the associated description information; wherein, the multi-level problem description information is a problem description information with three levels determined based on three dimensions: nodes, edges and attributes. The three levels include atomic layer, combination layer and global layer. The atomic layer sets the problem from a single dimension, the combination layer sets the combined problem from any two dimensions, and the global layer sets the global problem from all three dimensions. Based on the multi-level question description information and the target question description information, the network topology parsing model is iteratively invoked to perform bottom-up multi-level image parsing on the network topology image to be parsed, obtaining the network topology image parsing result; wherein, the question and answer content of each layer is used as historical question and answer input to the next layer; Specifically, based on the multi-level problem description information and the target problem description information, the network topology parsing model is iteratively invoked to perform bottom-up multi-level image parsing on the network topology image to be parsed, obtaining the network topology image parsing result, including: The network topology image to be parsed and the problem description information of the atomic layer are input into the large network topology parsing model to obtain the network topology image parsing result of the atomic layer; The problem description information of the atomic layer, the network topology image parsing result of the atomic layer, and the problem description information of the combined layer are input into the large network topology parsing model to obtain the network topology image parsing result of the combined layer. The problem description information of the combined layer, the network topology image parsing result of the combined layer, and the problem description information of the overall layer are input into the network topology parsing large model to obtain the network topology image parsing result of the overall layer. The problem description information of the overall layer, the network topology image parsing result of the overall layer, the target problem description information, and the network topology image to be parsed are input into the large network topology parsing model to obtain the network topology image parsing result.

2. The network topology image parsing method according to claim 1, characterized in that, The process involves identifying the network topology image to be parsed, resulting in a list of target nodes and a list of text information, including: The target recognition model is used to identify the network topology image to be parsed, thereby obtaining the target node list and the text information list; wherein, the target node list is a list including network node icons, and the text information list is a list including node location information and node text information.

3. The network topology image parsing method according to claim 2, characterized in that, The training process of the target recognition model includes: Construct a target recognition dataset; wherein, the target recognition dataset is a dataset including a training network topology image and its corresponding training node information and training text information; The target recognition model is obtained by using the training network topology image in the target recognition dataset as input, and the training node information and training text information as output to train the machine learning model.

4. The network topology image parsing method according to claim 1, characterized in that, Based on the associated description information and the target problem description information, the network topology image to be parsed is analyzed using a large network topology parsing model to obtain the network topology image parsing result corresponding to the target problem description information, including: Generate identification assistance information corresponding to the associated description information; Target identification assistance prompts are determined based on the associated description information and the identification assistance information; Based on the target recognition auxiliary prompt information and the target problem description information, the network topology image to be parsed is parsed using the network topology parsing large model to obtain the network topology image parsing result.

5. The network topology image parsing method according to claim 1, characterized in that, The training process of a large-scale network topology analysis model includes: Obtain network topology question-answering training data; wherein, the network topology question-answering data includes network topology training images and question-answering training results corresponding to the network topology training images, and the question-answering training results include multi-level questions and answers corresponding to each level of questions; The network topology training image and the questions at each level in the network topology question-answering training data are used as inputs, and the answers corresponding to the questions at each level are used as outputs. The multimodal large model is trained to obtain the network topology analytical large model.

6. A network topology image parsing device, characterized in that, The network topology image parsing method according to any one of claims 1 to 5 includes: The recognition module is used to recognize the network topology image to be parsed using a target recognition model, and obtain a list of target nodes and a list of text information; wherein, the target recognition model is a model obtained by constructing a corresponding target recognition dataset and training a deep learning model using network node icons and text descriptions as recognition targets; The association description information determination module is used to obtain association description information based on the target node list and the text information list; wherein, the association description information is information representing the relationship between nodes and text information; The parsing module is used to parse the network topology image to be parsed using a large network topology parsing model based on the associated description information and the target problem description information, to obtain the network topology image parsing result corresponding to the target problem description information; wherein, the target problem description information is the parsing requirement related to the network topology image to be parsed. The aforementioned parsing module includes: A multi-level problem description information determination unit is used to determine multi-level problem description information based on the associated description information; wherein, the multi-level problem description information is problem description information determined based on three dimensions: nodes, edges, and attributes, and the three levels include an atomic layer, a combination layer, and an overall layer. The atomic layer sets the problem from a single dimension, the combination layer sets a combined problem from any two dimensions, and the overall layer sets an overall problem from all three dimensions. The bottom-up parsing unit is used to iteratively call the network topology parsing big model according to the multi-level question description information and the target question description information to perform bottom-up multi-level image parsing on the network topology image to be parsed, and obtain the network topology image parsing result; wherein, the question and answer content of each layer is used as historical question and answer input to the next layer.

7. A network topology image parsing device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the network topology image parsing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the network topology image parsing method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, It includes a computer program / instructions that, when executed by a processor, implement the steps of the network topology image parsing method as described in any one of claims 1 to 5.

Images