UML class diagram information extraction method and system, medium and equipment

Abstract

The invention discloses a UML (Unified Modeling Language) class diagram information extraction method, system and equipment, and aims to solve the problems that the existing UML class diagram information extraction depends on manpower, is low in efficiency and is easy to make mistakes. The method comprises the following steps: collecting and marking a large number of UML class diagram images with different styles and complexities, and dividing the UML class diagram images into training, verification and test sets; constructing a YOLO model taking a convolutional neural network as a basic structure, setting proper parameters, training by using a training set, optimizing according to a loss function and a back propagation algorithm, and monitoring by using a verification set to prevent overfitting; and inputting a to-be-processed UML class diagram image into the trained model, identifying a quasi-rectangular contour and a relation type symbol, determining a position, extracting text information through an OCR technology, and further extracting UML class diagram information. According to the method, the information extraction efficiency and accuracy are remarkably improved, the method is high in expandability and suitable for UML class diagram processing in different fields and styles, and powerful support is provided for software engineering related tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and software engineering, and in particular relates to a method, system, medium and equipment for extracting UML class diagram information. Background Art

[0002] In the field of software engineering, UML class diagram is an important tool for software design and analysis. It intuitively displays the structure, properties, methods and relationships between classes in the software system. As the scale of software systems continues to expand, the demand for processing UML class diagram information is growing.

[0003] Traditional methods for extracting information from UML class diagrams are mainly divided into three categories: (1) manual analysis, (2) text processing based on xmi, and (3) based on traditional image processing algorithms. Manual analysis methods are inefficient and prone to errors when processing UML class diagrams, especially when faced with large-scale and complex class diagrams. The workload is huge and it is difficult to ensure accuracy. Although text processing based on xmi has improved efficiency to a certain extent, it requires the formulation of complex rule sets for different UML styles and specifications, and is difficult to adapt to some irregular representations or changes in class diagrams. It has poor flexibility and cannot effectively process UML class diagrams in image format, which limits its widespread use in practical applications. The recognition rate based on traditional image processing algorithms is greatly affected by image resolution and the accuracy is not high enough. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of low efficiency, poor accuracy, insufficient flexibility and difficulty in processing image format class diagrams in traditional UML class diagram information extraction methods, and to provide a UML class diagram information extraction method, system, medium and device. The method of the present invention can improve the efficiency and accuracy of information extraction and enhance the adaptability to UML class diagrams of different types and styles.

[0005] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for extracting UML class diagram information, which comprises the following steps:

[0007] S1. Obtain a UML class diagram image dataset containing multiple UML class diagram elements, and annotate the UML class diagram image dataset;

[0008] S2. Use k-fold cross validation to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal-sized parts using non-repeated sampling to form k sub-datasets. In each round of training, (k-2) sub-datasets are used for model training, 1 data set is used for model testing, and 1 data set is used for model verification. The YOLO model is trained for k rounds in total, and finally a trained YOLO model is obtained;

[0009] S3, inputting the UML class diagram image of the information to be extracted into the trained YOLO model, extracting and analyzing features by the trained YOLO model, identifying various types of UML class diagram elements, and outputting category information and location information of the UML class diagram elements;

[0010] S4. According to the category information and position information of the UML class diagram elements, the optical character recognition technology is used to identify the text information in the rectangular area representing the class, and the extracted text information is divided into multiple strings according to the line break character, and the first string is used as the class name. For the remaining strings, the strings containing parentheses are used as class member functions and the function signature information is extracted, and the strings without parentheses are used as class member variables and the variable names and variable types are extracted;

[0011] S5. According to the category information and position information of the UML class diagram elements, all boundary boxes representing relationship types are traversed, and each boundary box representing a relationship type is used as a relationship type boundary box. A new relationship is constructed according to the relationship type of the relationship type boundary box and a corresponding related class list is formed. According to the category information and position information of the UML class diagram elements, all boundary boxes representing classes are traversed, and each boundary box representing a class is used as a class boundary box. It is determined whether a relationship type boundary box is adjacent to each class boundary box: if the relationship type boundary box is adjacent to a class boundary box, the class of the class boundary box is added to the related class list corresponding to the new relationship; otherwise, it is continued to determine whether the relationship type boundary box is adjacent to the next class boundary box, until all relationship type boundary boxes are determined, and a mapping list of corresponding relationships between all relationship types and new relationships is obtained;

[0012] S6. Using an image processing library, remove the content in the bounding box corresponding to the position information, calculate the contour of the remaining line segments, and use the returned contour as a reference bounding box;

[0013] S7. For a reference boundary box, if the reference boundary box is adjacent to a relationship type boundary box, the relationship type represented by the relationship type boundary box is used as the reference relationship type. According to the reference relationship type, the relationship corresponding to the reference relationship type is queried in the mapping list and used as the reference relationship. The reference relationship is used as the relationship corresponding to the reference boundary box, and it is continued to be determined whether the reference boundary box is adjacent to each class boundary box. If the reference boundary box is adjacent to a class boundary box, the class represented by the class boundary box is added to the list of related classes of the relationship corresponding to the reference boundary box. If the reference boundary box is not adjacent to a class boundary box, it is skipped and it is continued to be determined whether the reference boundary box is adjacent to the next class boundary box until all class boundary boxes are traversed. If the reference boundary box is not adjacent to a relationship type boundary box, it is skipped and it is continued to be determined whether the reference boundary box is adjacent to the next relationship type boundary box until all reference boundary boxes are traversed. The class structure information and all new constructed relationships are output to complete the information extraction based on the UML class diagram.

[0014] As a preferred embodiment of the above-mentioned first aspect, in step S1, when annotating the UML class diagram image dataset, use an annotation tool to annotate the UML class diagram image, use the target rectangular box to select the target, generate annotation text with the same name as the UML class diagram image, and then convert the annotation text into YOLO annotation format and save it.

[0015] As a preferred embodiment of the above-mentioned first aspect, in step S1, when annotating the UML class diagram image data set, the annotation text corresponding to each UML class diagram image contains a total of 5 data, and two adjacent data are separated by spaces, namely the class name, the ratio of the horizontal coordinate of the center of the target rectangular box to the width of the UML class diagram image, the ratio of the vertical coordinate of the center of the target rectangular box to the height of the UML class diagram image, the ratio of the width of the target rectangular box to the width of the UML class diagram image, and the ratio of the height of the target rectangular box to the height of the UML class diagram image.

[0016] As a preferred embodiment of the first aspect, in step S2, the UML class diagram image dataset and its corresponding annotation files are randomly divided into five equal-sized portions to form five sub-datasets.

[0017] As a preferred embodiment of the first aspect, in step S6, the contour enclosing the remaining line segments is calculated by the cv2.findContours function in the OpenCV library.

[0018] As a preferred embodiment of the first aspect above, in step S7, the class structure information is the class name, class member variables and class member functions extracted in S4.

[0019] In a second aspect, the present invention provides a UML class diagram information extraction system, comprising:

[0020] A data processing module is used to obtain a UML class diagram image data set containing multiple UML class diagram elements, and to annotate the UML class diagram image data set;

[0021] The model training module is used to adopt k-fold cross validation and use non-repeated sampling to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal-sized parts to form k sub-datasets. In each round of training, (k-2) sub-datasets are used for model training, 1 data set is used for model testing, and 1 data set is used for model verification. The YOLO model is trained for k rounds in total to finally obtain a trained YOLO model.

[0022] The feature extraction module is used to input the UML class diagram image of the information to be extracted into the trained YOLO model, and the trained YOLO model extracts and analyzes the features, identifies various types of UML class diagram elements, and outputs the category information and location information of the UML class diagram elements;

[0023] A text extraction module is used to identify text information in a rectangular area representing a class using optical character recognition technology according to category information and position information of UML class diagram elements, divide the extracted text information into multiple strings according to line breaks, use the first string as the class name, and for the remaining strings, use the string containing parentheses as a class member function and extract function signature information, and use the string without parentheses as a class member variable and extract the variable name and variable type;

[0024] A mapping list acquisition module is used to traverse all boundary boxes representing relationship types according to the category information and position information of the UML class diagram elements, take each boundary box representing the relationship type as a relationship type boundary box, build a new relationship according to the relationship type of the relationship type boundary box and form a corresponding related class list, traverse all boundary boxes representing classes according to the category information and position information of the UML class diagram elements, take each boundary box representing a class as a class boundary box, and determine whether a relationship type boundary box is adjacent to each class boundary box: if the relationship type boundary box is adjacent to a class boundary box, then add the class of the class boundary box to the related class list corresponding to the new relationship, otherwise continue to determine whether the relationship type boundary box is adjacent to the next class boundary box, until all relationship type boundary boxes are determined, and obtain a mapping list of corresponding relationships between all relationship types and new relationships;

[0025] A contour extraction module, used to use an image processing library to remove the content in the bounding box corresponding to the position information and calculate the contour that wraps the remaining line segments and use the returned contour as a reference bounding box;

[0026] The information extraction module is used for a reference boundary box. If the reference boundary box is adjacent to a relationship type boundary box, the relationship type represented by the relationship type boundary box is used as the reference relationship type. According to the reference relationship type, the relationship corresponding to the reference relationship type is queried in the mapping list and used as the reference relationship. The reference relationship is used as the relationship corresponding to the reference boundary box, and the reference boundary box is continuously judged whether the reference boundary box is adjacent to each class boundary box. If the reference boundary box is adjacent to a class boundary box, the class represented by the class boundary box is added to the related class list of the relationship corresponding to the reference boundary box. If the reference boundary box is not adjacent to a class boundary box, the module is skipped and the judgment is continuously made on whether the reference boundary box is adjacent to the next class boundary box until all class boundary boxes are traversed. If the reference boundary box is not adjacent to a relationship type boundary box, the module is skipped and the judgment is continuously made on whether the reference boundary box is adjacent to the next relationship type boundary box until all reference boundary boxes are traversed. The module outputs the class structure information and all the newly constructed relationships to complete the information extraction based on the UML class diagram.

[0027] In a third aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the UML class diagram information extraction method as described in any one of the solutions of the first aspect above.

[0028] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for extracting UML class diagram information as described in any one of the schemes of the first aspect is implemented.

[0029] In a fifth aspect, the present invention provides a computer electronic device comprising a memory and a processor;

[0030] The memory is used to store computer programs;

[0031] The processor is used to implement the UML class diagram information extraction method as described in any solution of the first aspect when executing the computer program.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] This paper applies the YOLO model to the field of UML class diagram information extraction for the first time, opening up new application scenarios for computer vision technology in software engineering. A model training and information extraction process specifically for UML class diagram elements is designed, including targeted processing of multiple class diagram elements and structured information organization methods.

[0034] The advantages of the present invention are: 1) high efficiency. Compared with manual analysis, it can quickly process a large number of UML class diagram images, greatly shortening the information extraction time. 2) high accuracy. Through model training and optimization, the accuracy of UML class diagram element recognition and information extraction is significantly higher than that of traditional rule-based text processing technology. 3) strong flexibility. It can adapt to UML class diagram images of different styles and different representation forms, without the need to formulate complex rules for each situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of the steps of the method of the present invention;

[0036] Figure 2 This is a schematic diagram of a UML class diagram image of information to be extracted in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of a class diagram image with category information and location information of UML class diagram elements marked after the UML class diagram image of the information to be extracted in an embodiment of the present invention is processed by a YOLO model;

[0038] Figure 4 This is a schematic diagram of the remaining portion after erasing the boundary box of the UML class diagram image of the information to be extracted in an embodiment of the present invention;

[0039] Figure 5 A schematic diagram of a boundary box including relationship lines identified by an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of a confusion matrix between the actual result and the predicted result of an embodiment of the present invention;

[0041] Figure 7 Schematic diagram of the relationship between accuracy and confidence in an embodiment of the present invention;

[0042] Figure 8 Schematic diagram of the relationship curve between recall rate (retrieval rate) and confidence in an embodiment of the present invention;

[0043] Fig. 9 Schematic diagram of the relationship between precision and recall in an embodiment of the present invention;

[0044] Fig.10 Schematic diagram of the relationship curve between the F1 score and the confidence level in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0046] like Figure 1 As shown, in a preferred implementation of the present invention, the above-mentioned UML class diagram information extraction method includes the following steps S1 to S7. The method uses YOLO target detection, OCR text recognition and OpenCV image processing to extract information from the UML class diagram. The specific implementation process is described below.

[0047] S1. Obtain a UML class diagram image dataset containing multiple UML class diagram elements, and annotate the UML class diagram image dataset.

[0048] It should be noted that in step S1, when annotating the UML class diagram image dataset, the UML class diagram image is annotated using an annotation tool, the target is selected using a target rectangular box, and an annotation text with the same name as the UML class diagram image is generated, and then the annotation text is converted into the YOLO annotation format and saved.

[0049] It should be noted that in step S1, when the UML class diagram image data set is annotated, the annotation text corresponding to each UML class diagram image contains a total of 5 data, and two adjacent data are separated by spaces, namely the class name, the ratio of the horizontal coordinate of the center of the target rectangular box to the width of the UML class diagram image, the ratio of the vertical coordinate of the center of the target rectangular box to the height of the UML class diagram image, the ratio of the width of the target rectangular box to the width of the UML class diagram image, and the ratio of the height of the target rectangular box to the height of the UML class diagram image.

[0050] In step S1 of the embodiment of the present invention, a UML class diagram image data set containing various UML class diagram elements such as classes, interfaces, attributes, methods, association relationships, aggregation relationships, combination relationships, and inheritance relationships is collected from the Internet. Then, the UML class diagram image is annotated using an annotation tool, the target is selected with a target rectangle, and an annotation text with the same name as the UML class diagram image is generated, and it is converted into a YOLO annotation format and saved in a .txt file. There are a total of 5 data separated by spaces, namely, the class name, the ratio of the horizontal coordinate of the center of the target rectangular box to the width of the UML class diagram image, the ratio of the vertical coordinate of the center of the target rectangular box to the height of the UML class diagram image, the ratio of the width of the target rectangular box to the width of the UML class diagram image, and the ratio of the height of the target rectangular box to the height of the UML class diagram image, as shown in Table 1. Finally, the UML class diagram image and the annotation file are placed in the images / and labels / folders of the root directory, respectively, as shown below:

[0051]

[0052]

[0053] Table 1. Examples of data in annotated text

[0054] 4 405.0 222.0 437.0 254.0 4 188.0 377.0 220.0 409.0

[0055] S2. Use k-fold cross validation and use non-repeated sampling to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal-sized parts to form k sub-datasets. In each round of training, (k-2) sub-datasets are used for model training, 1 data set is used for model testing, and 1 data set is used for model verification. The YOLO model is trained for k rounds in total to finally obtain a trained YOLO model.

[0056] It should be noted that in step S2, k-fold cross validation is adopted, and the UML class diagram image dataset and its corresponding annotation files are randomly divided into k equal-sized portions using non-repeated sampling. In each round of training, (k-2) portions of data are used for model training, 1 portion of data is used for testing, and 1 portion of data is used for verification, and a total of k rounds of training are performed. The divided sub-datasets are shown below, with k=5 for illustration. The UML class diagram image dataset and its corresponding annotation files are divided into 5 portions, namely set1 to set5, and each portion of data contains UML class diagram images (images) and annotation files (labels).

[0057]

[0058]

[0059] Then, the present invention writes the yaml file required for YOLO model training and trains the YOLO model. In this embodiment, the yaml file saves the necessary composition information of the YOLO model, including the training set path, the test set path, the validation set path and the detection category. In addition, in each round of training of the YOLO model, different yaml files need to be written. In this embodiment, a total of 5 yaml files need to be written, and the content of a single yaml file is as follows:

[0060] path: / dataset

[0061] train:[set3 / images,set4 / images,set5 / images]

[0062] val:set1 / images

[0063] test:set2 / images

[0064] names:

[0065] 0:assosiation

[0066] 1: inheritance

[0067] 2: Realization

[0068] 3:dependency

[0069] 4: aggregation

[0070] 5: composition

[0071] 6:class

[0072] S3. Input the UML class diagram image to be extracted into the trained YOLO model, extract and analyze features through the trained YOLO model, identify various types of UML class diagram elements, and output category information and location information of the UML class diagram elements.

[0073] It should be noted that in step S3, the UML class diagram image (such as Figure 2 The trained YOLO model is input into the model, which outputs the category information and location information (such as bounding box coordinates) of the UML class diagram elements. At this time, each class and relationship symbol corresponds to a rectangular area at runtime, and has the two-dimensional coordinate information of the upper left corner and the lower right corner, which will be used for subsequent adjacency judgment.

[0074] S4. According to the category information and position information of the UML class diagram elements, use optical character recognition technology to identify the text information in the rectangular area representing the class, divide the extracted text information into multiple strings according to line breaks, use the first string as the class name, and for the remaining strings, use the strings containing parentheses as class member functions, extract the function signature information therein, and use the strings without parentheses as class member variables, extract the variable names and variable types therein.

[0075] It should be noted that in step S4, the element representing the class can be found according to the category information of the UML class diagram element, and then the position information of the element representing the class can be found, and then the optical character recognition (OCR) technology is used to recognize the text information such as the class name and class members in the rectangular area corresponding to the position information. Figure 3 As shown in the figure, the red box represents the class, the yellow box represents the aggregation relationship, and the purple box represents the association relationship. Therefore, the text in the red rectangular area is recognized and the extracted text information is divided into multiple strings according to the line break. At this time, the first string is the class name. For the following strings, whether there are parentheses is used as the basis for distinguishing variables and functions. For class member variables, the variable name and type are extracted; for class member functions, their function signature information is extracted.

[0076] S5. According to the category information and position information of the UML class diagram elements, all the boundary boxes representing the relationship type are traversed, and each boundary box representing the relationship type is used as a relationship type boundary box. According to the relationship type of the relationship type boundary box, a new relationship is constructed (the relationship type in the new relationship is an adjacent relationship type, and the related classes in the new relationship are adjacent classes) and a corresponding related class list is formed. According to the category information and position information of the UML class diagram elements, all the boundary boxes representing the classes are traversed, and each boundary box representing the class is used as a class boundary box. It is determined whether a relationship type boundary box is adjacent to each class boundary box: if the relationship type boundary box is adjacent to a class boundary box, the class of the class boundary box is added to the related class list corresponding to the new relationship; otherwise, it is continued to determine whether the relationship type boundary box is adjacent to the next class boundary box, until all the relationship type boundary boxes are determined, and a mapping list of corresponding relationships between all relationship types and new relationships is obtained.

[0077] It should be noted that in step S5, to determine whether two bounding boxes are adjacent, it is only necessary to determine whether there is an intersection in the horizontal and vertical directions respectively, and allow an error in a certain pixel range. The judgment of the adjacency relationship in the present invention can refer to this standard. Of course, the corresponding adjacency relationship judgment conditions can also be set by technical personnel in this field, so it will not be repeated here.

[0078] S6. Use an image processing library to remove the content in the bounding box corresponding to the position information, calculate the contour that wraps the remaining line segments, and use the returned contour as a reference bounding box.

[0079] It should be noted that in step S6, the image processing library is used to erase the above boundary box content, leaving only the relationship connection lines and some unerased line segments, such as Figure 4 Then, the cv2.findContours function in the OpenCV library calculates the contour that wraps the remaining line segments and uses it as the reference bounding box, as shown in Figure 5 shown.

[0080] S7. For a reference boundary box, if the reference boundary box is adjacent to a relationship type boundary box, the relationship type represented by the relationship type boundary box is used as the reference relationship type. According to the reference relationship type, the relationship corresponding to the reference relationship type is queried in the mapping list and used as the reference relationship. The reference relationship is used as the relationship corresponding to the reference boundary box, and it is continued to be determined whether the reference boundary box is adjacent to each class boundary box. If the reference boundary box is adjacent to a class boundary box, the class represented by the class boundary box is added to the list of related classes of the relationship corresponding to the reference boundary box. If the reference boundary box is not adjacent to a class boundary box, it is skipped and it is continued to be determined whether the reference boundary box is adjacent to the next class boundary box until all class boundary boxes are traversed. If the reference boundary box is not adjacent to a relationship type boundary box, it is skipped and it is continued to be determined whether the reference boundary box is adjacent to the next relationship type boundary box until all reference boundary boxes are traversed. The class structure information and all newly constructed relationships are output to complete the information extraction based on the UML class diagram; wherein the class structure information is the class name, class member variables and class member functions extracted in S4.

[0081] It should be noted that in step S7, the complete UML class diagram information, including all classes and their member variables and the relationships between all classes, is finally extracted and output as shown below.

[0082] Class Account:

[0083] accountNum:integer

[0084] balance:float

[0085] getBalance()

[0086] 'return()'

[0087] Class remittance account:

[0088] chargePerCheck:float

[0089] numCheck: integer

[0090] minBalance:float

[0091] getBalance()

[0092] class Savings Account:

[0093] interestRate:float

[0094] getBalance()

[0095] Customer class:

[0096] custname:string

[0097] address:string

[0098] Class Bank:

[0099] name:string

[0100] routingNum:integer

[0101] createAccount()

[0102] Remittance Account-(Link)->Account

[0103] Savings Account-(Link)->Account

[0104] Account-(Aggregation)->Bank

[0105] Account-(Relationship)-Customer

[0106] Figure 6 It is a confusion matrix between the true result and the predicted result, which is used to show the comparison between the predicted result of the classification model and the true result. Each column of the matrix represents the instance prediction of a class, and each row represents an actual instance of the class. Figure 7 is the relationship curve between accuracy and confidence in this embodiment, Figure 8 is the relationship curve between recall rate (recall rate) and confidence in this embodiment, Fig. 9 is the relationship curve between precision and recall in this embodiment, Fig.10 It is the relationship curve between F1 score and confidence in this embodiment.

[0107] It should also be noted that the UML class diagram information extraction method in the above embodiment can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a UML class diagram information extraction system corresponding to the UML class diagram information extraction method provided in the above embodiment, which includes:

[0108] A data processing module is used to obtain a UML class diagram image data set containing multiple UML class diagram elements, and to annotate the UML class diagram image data set;

[0109] The model training module is used to adopt k-fold cross validation and use non-repeated sampling to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal-sized parts to form k sub-datasets. In each round of training, (k-2) sub-datasets are used for model training, 1 data set is used for model testing, and 1 data set is used for model verification. The YOLO model is trained for k rounds in total to finally obtain a trained YOLO model.

[0110] The feature extraction module is used to input the UML class diagram image of the information to be extracted into the trained YOLO model, and the trained YOLO model extracts and analyzes the features, identifies various types of UML class diagram elements, and outputs the category information and location information of the UML class diagram elements;

[0111] A text extraction module is used to identify text information in a rectangular area representing a class using optical character recognition technology according to category information and position information of UML class diagram elements, divide the extracted text information into multiple strings according to line breaks, use the first string as the class name, and for the remaining strings, use the string containing parentheses as a class member function and extract function signature information, and use the string without parentheses as a class member variable and extract the variable name and variable type;

[0112] A mapping list acquisition module is used to traverse all boundary boxes representing relationship types according to the category information and position information of the UML class diagram elements, take each boundary box representing the relationship type as a relationship type boundary box, build a new relationship according to the relationship type of the relationship type boundary box and form a corresponding related class list, traverse all boundary boxes representing classes according to the category information and position information of the UML class diagram elements, take each boundary box representing a class as a class boundary box, and determine whether a relationship type boundary box is adjacent to each class boundary box: if the relationship type boundary box is adjacent to a class boundary box, then add the class of the class boundary box to the related class list corresponding to the new relationship, otherwise continue to determine whether the relationship type boundary box is adjacent to the next class boundary box, until all relationship type boundary boxes are determined, and obtain a mapping list of corresponding relationships between all relationship types and new relationships;

[0113] A contour extraction module, used to use an image processing library to remove the content in the bounding box corresponding to the position information and calculate the contour that wraps the remaining line segments and use the returned contour as a reference bounding box;

[0114] The information extraction module is used for a reference boundary box. If the reference boundary box is adjacent to a relationship type boundary box, the relationship type represented by the relationship type boundary box is used as the reference relationship type. According to the reference relationship type, the relationship corresponding to the reference relationship type is queried in the mapping list and used as the reference relationship. The reference relationship is used as the relationship corresponding to the reference boundary box, and the reference boundary box is continuously judged whether the reference boundary box is adjacent to each class boundary box. If the reference boundary box is adjacent to a class boundary box, the class represented by the class boundary box is added to the related class list of the relationship corresponding to the reference boundary box. If the reference boundary box is not adjacent to a class boundary box, the module is skipped and the judgment is continuously made on whether the reference boundary box is adjacent to the next class boundary box until all class boundary boxes are traversed. If the reference boundary box is not adjacent to a relationship type boundary box, the module is skipped and the judgment is continuously made on whether the reference boundary box is adjacent to the next relationship type boundary box until all reference boundary boxes are traversed. The module outputs the class structure information and all the newly constructed relationships to complete the information extraction based on the UML class diagram.

[0115] It can be understood that the UML class diagram information extraction method described in S1 to S7 above can be substantially implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the UML class diagram information extraction method provided in the above embodiment, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, the UML class diagram information extraction method described in the above embodiment can be implemented.

[0116] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the UML class diagram information extraction method provided in the above embodiment, which includes a memory and a processor;

[0117] The memory is used to store computer programs;

[0118] The processor is used to implement the UML class diagram information extraction method in the above embodiment when executing the computer program.

[0119] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0120] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the UML class diagram information extraction method provided in the above embodiment, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the UML class diagram information extraction method in the above embodiment can be implemented.

[0121] It is understandable that the above storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. The storage medium may also be a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc., which can store program codes.

[0122] It is understandable that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0123] It should also be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0124] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A method for extracting UML class diagram information, characterized in that: The following steps are involved: S1. Obtain a UML class diagram image dataset containing multiple UML class diagram elements, and annotate the UML class diagram image dataset; S2. Use k-fold cross validation to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal-sized parts using non-repeated sampling to form k sub-datasets. In each round of training, (k-2) sub-datasets are used for model training, 1 data set is used for model testing, and 1 data set is used for model verification. The YOLO model is trained for k rounds in total, and finally a trained YOLO model is obtained; S3, inputting the UML class diagram image of the information to be extracted into the trained YOLO model, extracting and analyzing features by the trained YOLO model, identifying various types of UML class diagram elements, and outputting category information and location information of the UML class diagram elements; S4. According to the category information and position information of the UML class diagram elements, the optical character recognition technology is used to identify the text information in the rectangular area representing the class, and the extracted text information is divided into multiple strings according to the line break character, and the first string is used as the class name. For the remaining strings, the strings containing parentheses are used as class member functions and the function signature information is extracted, and the strings without parentheses are used as class member variables and the variable names and variable types are extracted; S5. According to the category information and position information of the UML class diagram elements, all boundary boxes representing relationship types are traversed, and each boundary box representing a relationship type is used as a relationship type boundary box. A new relationship is constructed according to the relationship type of the relationship type boundary box and a corresponding related class list is formed. According to the category information and position information of the UML class diagram elements, all boundary boxes representing classes are traversed, and each boundary box representing a class is used as a class boundary box. It is determined whether a relationship type boundary box is adjacent to each class boundary box: if the relationship type boundary box is adjacent to a class boundary box, the class of the class boundary box is added to the related class list corresponding to the new relationship; otherwise, it is continued to determine whether the relationship type boundary box is adjacent to the next class boundary box, until all relationship type boundary boxes are determined, and a mapping list of corresponding relationships between all relationship types and new relationships is obtained; S6, using an image processing library to remove the content in the bounding box corresponding to the position information and calculate the contour of the remaining line segments and use the returned contour as a reference bounding box; S7. For a reference boundary box, if the reference boundary box is adjacent to a relationship type boundary box, the relationship type represented by the relationship type boundary box is used as the reference relationship type. According to the reference relationship type, the relationship corresponding to the reference relationship type is queried in the mapping list and used as the reference relationship. The reference relationship is used as the relationship corresponding to the reference boundary box, and it is continued to be determined whether the reference boundary box is adjacent to each class boundary box. If the reference boundary box is adjacent to a class boundary box, the class represented by the class boundary box is added to the list of related classes of the relationship corresponding to the reference boundary box. If the reference boundary box is not adjacent to a class boundary box, it is skipped and it is continued to be determined whether the reference boundary box is adjacent to the next class boundary box until all class boundary boxes are traversed. If the reference boundary box is not adjacent to a relationship type boundary box, it is skipped and it is continued to be determined whether the reference boundary box is adjacent to the next relationship type boundary box until all reference boundary boxes are traversed. The class structure information and all new constructed relationships are output to complete the information extraction based on the UML class diagram.

2. A UML class diagram information extraction method as claimed in claim 1, characterized in that: In step S1, when annotating the UML class diagram image dataset, annotate the UML class diagram image using an annotation tool, select the target using a target rectangular box, generate annotation text with the same name as the UML class diagram image, and then convert the annotation text into the YOLO annotation format and save it.

3. A UML class diagram information extraction method as claimed in claim 2, characterized in that: In step S1, when annotating the UML class diagram image data set, the annotation text corresponding to each UML class diagram image contains a total of 5 data, and two adjacent data are separated by spaces, namely the class name, the ratio of the horizontal coordinate of the center of the target rectangular box to the width of the UML class diagram image, the ratio of the vertical coordinate of the center of the target rectangular box to the height of the UML class diagram image, the ratio of the width of the target rectangular box to the width of the UML class diagram image, and the ratio of the height of the target rectangular box to the height of the UML class diagram image.

4. A UML class diagram information extraction method as claimed in claim 1, characterized in that: In step S2, the UML class diagram image dataset and its corresponding annotation files are randomly divided into five equal-sized portions to form five sub-datasets.

5. A UML class diagram information extraction method as claimed in claim 1, characterized in that: In step S6, the cv2.findContours function in the OpenCV library calculates the contour that wraps the remaining line segments.

6. A UML class diagram information extraction method as claimed in claim 1, characterized in that: In step S7, the class structure information is the class name, class member variables and class member functions extracted in S4.

7. A UML class diagram information extraction system, characterized in that: include: A data processing module is used to obtain a UML class diagram image data set containing multiple UML class diagram elements, and to annotate the UML class diagram image data set; The model training module is used to adopt k-fold cross validation and use non-repeated sampling to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal-sized parts to form k sub-datasets. In each round of training, (k-2) sub-datasets are used for model training, 1 data set is used for model testing, and 1 data set is used for model verification. The YOLO model is trained for k rounds in total to finally obtain a trained YOLO model. The feature extraction module is used to input the UML class diagram image of the information to be extracted into the trained YOLO model, and the trained YOLO model extracts and analyzes the features, identifies various types of UML class diagram elements, and outputs the category information and location information of the UML class diagram elements; A text extraction module is used to identify text information in a rectangular area representing a class using optical character recognition technology according to category information and position information of UML class diagram elements, divide the extracted text information into multiple strings according to line breaks, use the first string as the class name, and for the remaining strings, use the string containing parentheses as a class member function and extract function signature information, and use the string without parentheses as a class member variable and extract the variable name and variable type; A mapping list acquisition module is used to traverse all boundary boxes representing relationship types according to the category information and position information of the UML class diagram elements, take each boundary box representing the relationship type as a relationship type boundary box, build a new relationship according to the relationship type of the relationship type boundary box and form a corresponding related class list, traverse all boundary boxes representing classes according to the category information and position information of the UML class diagram elements, take each boundary box representing a class as a class boundary box, and determine whether a relationship type boundary box is adjacent to each class boundary box: if the relationship type boundary box is adjacent to a class boundary box, then add the class of the class boundary box to the related class list corresponding to the new relationship, otherwise continue to determine whether the relationship type boundary box is adjacent to the next class boundary box, until all relationship type boundary boxes are determined, and obtain a mapping list of corresponding relationships between all relationship types and new relationships; A contour extraction module, used to use an image processing library to remove the content in the bounding box corresponding to the position information and calculate the contour that wraps the remaining line segments and use the returned contour as a reference bounding box; The information extraction module is used for a reference boundary box. If the reference boundary box is adjacent to a relationship type boundary box, the relationship type represented by the relationship type boundary box is used as the reference relationship type. According to the reference relationship type, the relationship corresponding to the reference relationship type is queried in the mapping list and used as the reference relationship. The reference relationship is used as the relationship corresponding to the reference boundary box, and the reference boundary box is continuously judged whether the reference boundary box is adjacent to each class boundary box. If the reference boundary box is adjacent to a class boundary box, the class represented by the class boundary box is added to the related class list of the relationship corresponding to the reference boundary box. If the reference boundary box is not adjacent to a class boundary box, the module is skipped and the judgment is continuously made on whether the reference boundary box is adjacent to the next class boundary box until all class boundary boxes are traversed. If the reference boundary box is not adjacent to a relationship type boundary box, the module is skipped and the judgment is continuously made on whether the reference boundary box is adjacent to the next relationship type boundary box until all reference boundary boxes are traversed. The module outputs the class structure information and all the newly constructed relationships to complete the information extraction based on the UML class diagram.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the UML class diagram information extraction method according to any one of claims 1 to 6 can be implemented.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the UML class diagram information extraction method according to any one of claims 1 to 6 is implemented.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the UML class diagram information extraction method according to any one of claims 1 to 6 when executing the computer program.

Images