A method, system, medium, and device for extracting UML class diagram information.

Abstract

This invention discloses a method, system, and device for extracting UML class diagram information, aiming to solve the problems of low efficiency and error-proneness in existing UML class diagram information extraction methods that rely on manual labor. The method includes: collecting and labeling a large number of UML class diagram images of different styles and complexities, dividing them into training, validation, and test sets; constructing a YOLO model based on a convolutional neural network, setting appropriate parameters and training it using the training set, optimizing it using a loss function and backpropagation algorithm, while monitoring with a validation set to prevent overfitting; inputting the UML class diagram image to be processed into the trained model, identifying the rectangular outline and relation type symbols and determining their positions, extracting text information using OCR technology, and thus extracting the UML class diagram information. This invention significantly improves the efficiency and accuracy of information extraction, and the method is highly scalable, applicable to UML class diagram processing in different fields and styles, providing strong support for software engineering-related tasks.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and software engineering technology, and particularly relates to a method, system, medium and device for extracting UML class diagram information. Background Technology

[0002] In the field of software engineering, UML class diagrams are an important tool for software design and analysis. They visually represent the structure, attributes, methods, and relationships between classes in a software system. As software systems continue to grow in scale, the demand for processing UML class diagram information is increasing.

[0003] Traditional methods for extracting information from UML class diagrams can be categorized into three main types: (1) manual analysis, (2) text processing based on XMI, and (3) traditional image processing algorithms. Manual analysis is inefficient and prone to errors when processing UML class diagrams, especially when dealing with large-scale, complex diagrams, resulting in a huge workload and difficulty in ensuring accuracy. While text processing based on XMI improves efficiency to some extent, it requires complex rule sets for different UML styles and specifications, and it struggles to adapt to irregular representations or changes in class diagrams, exhibiting poor flexibility. Furthermore, it cannot effectively process UML class diagrams in image format, limiting its widespread use in practical applications. Traditional image processing algorithms are highly sensitive to image resolution and lack sufficient accuracy. Summary of the Invention

[0004] The purpose of this invention is to solve the problems of low efficiency, poor accuracy, insufficient flexibility, and difficulty in handling image-formatted class diagrams in traditional UML class diagram information extraction methods, and to provide a UML class diagram information extraction method, system, medium, and device. The method of this invention can improve the efficiency and accuracy of information extraction and enhance its adaptability to different types and styles of UML class diagrams.

[0005] To achieve the above-mentioned objectives, the present invention specifically adopts the following technical solution:

[0006] In a first aspect, the present invention provides a method for extracting UML class diagram information, which includes the following steps:

[0007] S1. Obtain a UML class diagram image dataset containing various UML class diagram elements, and annotate the UML class diagram image dataset;

[0008] S2. Using k-fold cross-validation, the UML class diagram image dataset and its corresponding annotation files are randomly divided into k equal parts using non-repeating sampling, forming k subsets. In each training round, (k-2) subsets are used for model training, 1 dataset is used for model testing, and 1 dataset is used for model validation. The YOLO model is trained for a total of k rounds, and finally a trained YOLO model is obtained.

[0009] S3. Input the UML class diagram image of the information to be extracted into the trained YOLO model. The trained YOLO model extracts and analyzes features, identifies various UML class diagram elements, and outputs the category information and position information of the UML class diagram elements.

[0010] S4. Based on the category and position information of the UML class diagram elements, use optical character recognition technology to identify the text information within the rectangular area representing the class. Divide the extracted text information into multiple strings according to the newline character. Use the first string as the class name. For the remaining strings, use the strings containing parentheses as class member functions and extract the function signature information. Use the strings without parentheses as class member variables and extract the variable name and variable type.

[0011] S5. Based on the category and position information of the UML class diagram elements, traverse all bounding boxes representing relation types, and treat each bounding box representing a relation type as a relation type bounding box. Construct new relations based on the relation types of the relation type bounding boxes and form a list of related classes corresponding to them. Based on the category and position information of the UML class diagram elements, traverse all bounding boxes representing classes, and treat each bounding box representing a class as a class bounding box. Determine whether a relation type bounding box is adjacent to each class bounding box: if the relation type bounding box is adjacent to a class bounding box, add the class of the class bounding box to the list of related classes corresponding to the new relation; otherwise, continue to determine whether the relation type bounding box is adjacent to the next class bounding box until all relation type bounding boxes have been determined, and obtain a mapping list of all relation types and corresponding relations between the new relations.

[0012] S6. Use an image processing library to remove the contents of the bounding box corresponding to the location information, calculate the contour that wraps the remaining line segments, and use the returned contour as a reference bounding box.

[0013] S7. For a reference bounding box, if it is adjacent to a relation type bounding box, the relation type represented by the relation type bounding box is used as the reference relation type. The corresponding relation is then queried in the mapping list based on the reference relation type and used as the reference relation. This reference relation is then used as the relation corresponding to the reference bounding box. The process continues to determine if the reference bounding box is adjacent to any class bounding box. If it is, the class represented by the class bounding box is added to the related class list of the relation corresponding to the reference bounding box. If it is not, the process skips the class bounding box and continues to determine if it is adjacent to the next class bounding box, until all class bounding boxes have been traversed. Similarly, if it is not, the process continues to determine if it is adjacent to the next relation type bounding box, until all reference bounding boxes have been traversed. Finally, the class structure information and all newly constructed relations are output, completing the information extraction based on the UML class diagram.

[0014] As a preferred embodiment of the first aspect above, in step S1, when annotating the UML class diagram image dataset, the annotation tool is used to annotate the UML class diagram image, the target is selected using the target rectangle, annotation text with the same name as the UML class diagram image is generated, and then the annotation text is converted into YOLO annotation format and saved.

[0015] As a preferred embodiment of the first aspect mentioned above, in step S1, when annotating the UML class diagram image dataset, the annotation text corresponding to each UML class diagram image contains a total of 5 data points, with adjacent data points separated by spaces. These data points are: category name, ratio of the horizontal coordinate of the center of the target rectangle to the width of the UML class diagram image, ratio of the vertical coordinate of the center of the target rectangle to the height of the UML class diagram image, ratio of the width of the target rectangle to the width of the UML class diagram image, and ratio of the height of the target rectangle to the height of the UML class diagram image.

[0016] As a preferred embodiment of the first aspect mentioned above, in step S2, the UML class diagram image dataset and its corresponding annotation file are randomly divided into 5 equal parts to form 5 sub-datasets.

[0017] As a preferred embodiment of the first aspect above, in step S6, the contour of the remaining line segment is calculated by the cv2.findContours function in the OpenCV library.

[0018] As a preferred embodiment of the first aspect above, in step S7, the class structure information is the class name, class member variables, and class member functions extracted in S4.

[0019] Secondly, the present invention provides a UML class diagram information extraction system, comprising:

[0020] The data processing module is used to acquire a UML class diagram image dataset containing various UML class diagram elements and to annotate the UML class diagram image dataset.

[0021] The model training module is used to employ k-fold cross-validation. It uses non-repeating sampling to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal parts, forming k subsets. In each training round, (k-2) subsets are used for model training, 1 dataset is used for model testing, and 1 dataset is used for model validation. The YOLO model is trained for a total of k rounds, and finally a trained YOLO model is obtained.

[0022] The feature extraction module is used to input the UML class diagram image of the information to be extracted into the trained YOLO model. The trained YOLO model extracts and analyzes features, identifies various UML class diagram elements, and outputs the category information and location information of the UML class diagram elements.

[0023] The text extraction module is used to identify the text information within the rectangular area representing the class based on the category and position information of the UML class diagram elements using optical character recognition technology. The extracted text information is divided into multiple strings according to the newline character. The first string is used as the class name. For the remaining strings, strings containing parentheses are used as class member functions and their function signature information is extracted. Strings without parentheses are used as class member variables and their variable names and types are extracted.

[0024] The mapping list acquisition module is used to traverse all bounding boxes representing relation types based on the category and position information of UML class diagram elements, treat each bounding box representing a relation type as a relation type bounding box, construct new relations based on the relation types of the bounding boxes, and form a list of related classes corresponding to them. It also traverses all bounding boxes representing classes based on the category and position information of UML class diagram elements, treating each bounding box representing a class as a class bounding box, and determines whether a relation type bounding box is adjacent to any class bounding box: if the relation type bounding box is adjacent to a class bounding box, the class of that class bounding box is added to the list of related classes corresponding to the new relation; otherwise, it continues to determine whether the relation type bounding box is adjacent to the next class bounding box, until all relation type bounding boxes have been determined, resulting in a mapping list of all relation types and corresponding new relations.

[0025] The contour extraction module is used to remove the contents of the bounding box corresponding to the location information using an image processing library, calculate the contour that wraps the remaining line segments, and use the returned contour as a reference bounding box.

[0026] The information extraction module is used to process a reference bounding box. If the reference bounding box is adjacent to a relation type bounding box, the relation type represented by the relation type bounding box is used as the reference relation type. Based on the reference relation type, the module queries the mapping list for the corresponding relation and uses it as the reference relation. This reference relation is then used as the relation corresponding to the reference bounding box. The module continues to determine whether the reference bounding box is adjacent to each class bounding box. If the reference bounding box is adjacent to a class bounding box, the class represented by that class bounding box is added to the related class list of the relation corresponding to the reference bounding box. If the reference bounding box is not adjacent to a class bounding box, the module skips this step and continues to determine whether the reference bounding box is adjacent to the next class bounding box, until all class bounding boxes have been traversed. If the reference bounding box is not adjacent to a relation type bounding box, the module skips this step and continues to determine whether the reference bounding box is adjacent to the next relation type bounding box, until all reference bounding boxes have been traversed. Finally, the module outputs the class structure information and all newly constructed relations, completing the information extraction based on the UML class diagram.

[0027] Thirdly, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the UML class diagram information extraction method as described in any of the solutions of the first aspect above.

[0028] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the UML class diagram information extraction method as described in any of the solutions of the first aspect above.

[0029] Fifthly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0030] The memory is used to store computer programs;

[0031] The processor is configured to, when executing the computer program, implement the UML class diagram information extraction method as described in any of the solutions of the first aspect above.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] This invention is the first to apply the YOLO model to the field of UML class diagram information extraction, opening up new application scenarios for computer vision technology in software engineering. A model training and information extraction process specifically designed for UML class diagram elements was developed, including targeted processing of various class diagram elements and methods for structured information organization.

[0034] The advantages of this invention are: 1) High efficiency. Compared to manual analysis, it can quickly process a large number of UML class diagram images, greatly shortening the information extraction time. 2) High accuracy. Through model training and optimization, the accuracy of UML class diagram element recognition and information extraction is significantly higher than that of traditional rule-based text processing techniques. 3) High flexibility. It can adapt to UML class diagram images of different styles and representations without the need to formulate complex rules for each case. Attached Figure Description

[0035] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0036] Figure 2 This is a schematic diagram of a UML class diagram image from which information to be extracted is shown in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of a UML class diagram image, after being processed by the YOLO model, to output a class diagram image with labeled category and position information of UML class diagram elements in an embodiment of the present invention.

[0038] Figure 4 This is a schematic diagram of the remaining part of the UML class diagram image after erasing the bounding box of the information to be extracted in an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the bounding box containing the relationship lines identified in an embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of the confusion matrix between the actual results and the predicted results in an embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of the relationship between accuracy and confidence level in an embodiment of the present invention;

[0042] Figure 8 This is a schematic diagram illustrating the relationship between recall and confidence in an embodiment of the present invention.

[0043] Figure 9 This is a schematic diagram illustrating the relationship between accuracy and recall in an embodiment of the present invention.

[0044] Figure 10 This is a schematic diagram of the relationship between F1 score and confidence level in an embodiment of the present invention. Detailed Implementation

[0045] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0046] like Figure 1 As shown, in a preferred embodiment of the present invention, the above-mentioned UML class diagram information extraction method includes the following steps S1 to S7. This method utilizes YOLO object detection, OCR text recognition, and OpenCV image processing to extract information from the UML class diagram. The specific implementation process is described in detail below.

[0047] S1. Obtain a UML class diagram image dataset containing various UML class diagram elements, and annotate the UML class diagram image dataset.

[0048] It should be noted that in step S1, when annotating the UML class diagram image dataset, the annotation tool is used to annotate the UML class diagram image, the target is selected using the target rectangle, the annotation text with the same name as the UML class diagram image is generated, and then the annotation text is converted into YOLO annotation format and saved.

[0049] It should be noted that in step S1, when annotating the UML class diagram image dataset, the annotation text corresponding to each UML class diagram image contains a total of 5 data, with adjacent data separated by spaces. These data are: category name, ratio of the horizontal coordinate of the center of the target rectangle to the width of the UML class diagram image, ratio of the vertical coordinate of the center of the target rectangle to the height of the UML class diagram image, ratio of the width of the target rectangle to the width of the UML class diagram image, and ratio of the height of the target rectangle to the height of the UML class diagram image.

[0050] In step S1 of this embodiment of the invention, a dataset of UML class diagram images containing various UML class diagram elements such as classes, interfaces, attributes, methods, associations, aggregations, compositions, and inheritances is collected from the internet. Then, the UML class diagram images are annotated using a labeling tool. Targets are selected using a target rectangle, and annotation text with the same name as the UML class diagram image is generated and converted to YOLO annotation format, saved in a .txt file. The .txt file contains five data points separated by spaces: category name, ratio of the x-coordinate of the target rectangle center to the width of the UML class diagram image, ratio of the y-coordinate of the target rectangle center to the height of the UML class diagram image, ratio of the width of the target rectangle to the width of the UML class diagram image, and ratio of the height of the target rectangle to the height of the UML class diagram image, as shown in Table 1. Finally, the UML class diagram images and annotation files are placed in the images / and labels / folders in the root directory, respectively, as shown below:

[0051]

[0052]

[0053] Table 1. Examples of data in the labeled text

[0054] 4 405.0 222.0 437.0 254.0 4 188.0 377.0 220.0 409.0

[0055] S2. Using k-fold cross-validation, the UML class diagram image dataset and its corresponding annotation files are randomly divided into k equal parts using non-repeating sampling, forming k subsets. In each training round, (k-2) subsets are used for model training, 1 dataset is used for model testing, and 1 dataset is used for model validation. The YOLO model is trained for a total of k rounds, and finally a trained YOLO model is obtained.

[0056] It should be noted that in step S2, k-fold cross-validation is used. The UML class diagram image dataset and its corresponding annotation files are randomly divided into k equal parts using non-repeating sampling. In each training round, (k-2) parts of the data are used for model training, 1 part for testing, and 1 part for validation, for a total of k training rounds. The divided datasets are shown below. Taking k=5 as an example, the UML class diagram image dataset and its corresponding annotation files are divided into 5 parts, namely set1 to set5. Each part contains UML class diagram images and annotation files.

[0057]

[0058]

[0059] Then, this invention writes the YAML files required for YOLO model training and trains the YOLO model. In this embodiment, the YAML files store the necessary structural information of the YOLO model, including the training set path, test set path, validation set path, and detection categories. Furthermore, a different YAML file needs to be written for each round of YOLO model training. In this embodiment, a total of 5 YAML files need to be written, and the content of a single YAML file is as follows:

[0060] path: / dataset

[0061] train:[set3 / images,set4 / images,set5 / images]

[0062] val:set1 / images

[0063] test:set2 / images

[0064] names:

[0065] 0:assosiation

[0066] 1: inheritance

[0067] 2:realization

[0068] 3: dependency

[0069] 4: aggregation

[0070] 5: composition

[0071] 6:class

[0072] S3. Input the UML class diagram image from which the information to be extracted into the trained YOLO model. The trained YOLO model extracts and analyzes features, identifies various UML class diagram elements, and outputs the category and location information of the UML class diagram elements.

[0073] It should be noted that in step S3, the UML class diagram image from which the information to be extracted (such as...) Figure 2 As shown, the trained YOLO model is input, and the model outputs the category information and location information (e.g., bounding box coordinates) of the UML class diagram elements. At this time, each class and relation symbol corresponds to a rectangular area at runtime, with two-dimensional coordinate information of the top left and bottom right corners. This information will be used for subsequent adjacency determination.

[0074] S4. Based on the category and position information of the UML class diagram elements, use optical character recognition technology to identify the text information within the rectangular area representing the class. Divide the extracted text information into multiple strings according to the newline character. Use the first string as the class name. For the remaining strings, use the strings containing parentheses as class member functions and extract the function signature information. Use the strings without parentheses as class member variables and extract the variable name and variable type.

[0075] It should be noted that in step S4, the element representing the class can be found based on the category information of the UML class diagram elements. Then, the position information of the element representing the class is found, and then Optical Character Recognition (OCR) technology is used to recognize the text information such as the class name and class members within the rectangular area corresponding to that position information. For example... Figure 3 As shown, the red box represents a class, the yellow box represents an aggregation relationship, and the purple box represents an association relationship. Therefore, the text within the red rectangle is identified, and the extracted text information is divided into multiple strings based on newline characters. The first string is the class name. For subsequent strings, the presence or absence of parentheses is used to distinguish between variables and functions. For class member variables, the variable name and type are extracted; for class member functions, their function signature information is extracted.

[0076] S5. Based on the category and position information of the UML class diagram elements, traverse all bounding boxes representing relation types. Treat each bounding box representing a relation type as a relation type bounding box. Construct new relations based on the relation types of the relation type bounding boxes (the relation types in the new relations are adjacent relation types, and the related classes in the new relations are adjacent classes) and form a list of related classes corresponding to them. Based on the category and position information of the UML class diagram elements, traverse all bounding boxes representing classes. Treat each bounding box representing a class as a class bounding box. Determine whether a relation type bounding box is adjacent to each class bounding box: if the relation type bounding box is adjacent to a class bounding box, add the class of the class bounding box to the list of related classes corresponding to the new relation. Otherwise, continue to determine whether the relation type bounding box is adjacent to the next class bounding box until all relation type bounding boxes have been determined, and obtain a mapping list of the corresponding relations between all relation types and the new relations.

[0077] It should be noted that in step S5, determining whether two bounding boxes are adjacent only requires checking whether there is an intersection in the horizontal and vertical directions, and allowing for a certain range of pixel error. This standard can be used to determine the adjacency relationship in this invention. Of course, those skilled in the art can also set the corresponding adjacency relationship judgment conditions, so they will not be elaborated further.

[0078] S6. Use an image processing library to remove the contents of the bounding box corresponding to the location information, calculate the contour that wraps the remaining line segments, and use the returned contour as a reference bounding box.

[0079] It should be noted that in step S6, an image processing library is used to erase the bounding box content, leaving only the relationship connecting lines and some line segments that were not erased, such as... Figure 4 As shown. Then, the cv2.findContours function in the OpenCV library calculates the contour that wraps around the remaining line segment and uses it as a reference bounding box, as shown. Figure 5 As shown.

[0080] S7. For a reference bounding box, if it is adjacent to a relation type bounding box, the relation type represented by the relation type bounding box is used as the reference relation type. The corresponding relation is queried in the mapping list based on the reference relation type and used as the reference relation. This reference relation is then used as the relation corresponding to the reference bounding box. The process continues to determine whether the reference bounding box is adjacent to each class bounding box. If it is adjacent to a class bounding box, the class represented by that class bounding box is added to the related class list of the relation corresponding to the reference bounding box. If it is not adjacent to a class bounding box, the process is skipped, and the process continues to determine whether the reference bounding box is adjacent to the next class bounding box until all class bounding boxes have been traversed. If it is not adjacent to a relation type bounding box, the process is skipped, and the process continues to determine whether the reference bounding box is adjacent to the next relation type bounding box until all reference bounding boxes have been traversed. The class structure information and all newly constructed relations are output, completing the information extraction based on the UML class diagram. The class structure information refers to the class name, class member variables, and class member functions extracted in S4.

[0081] It should be noted that in step S7, the complete UML class diagram information is finally extracted, including all classes and their member variables, as well as the relationships between all classes, and then output, as shown below.

[0082] class account:

[0083] accountNum:integer

[0084] balance:float

[0085] getBalance()

[0086] withdraw()

[0087] Class remittance account:

[0088] chargePerCheck:float

[0089] numCheck:integer

[0090] minBalance:float

[0091] getBalance()

[0092] Class savings account:

[0093] interestRate:float

[0094] getBalance()

[0095] Class customers:

[0096] custname:string

[0097] address:string

[0098] Class Bank:

[0099] name:string

[0100] routingNum: integer

[0101] createAccount()

[0102] Remittance Account - (Linked) -> Account

[0103] Savings Account - (Linked) -> Account

[0104] Account - (Aggregator) -> Bank

[0105] Account - (Linked) - Customer

[0106] Figure 6 This is a confusion matrix between the actual and predicted results, used to show the comparison between the predictions and the actual results of the classification model. Each column of the matrix represents a predicted instance of a class, while each row represents an actual instance of the class. Figure 7 This is the relationship curve between accuracy and confidence level in this embodiment. Figure 8 This is the relationship curve between recall and confidence in this embodiment. Figure 9 This is the curve showing the relationship between precision and recall in this embodiment. Figure 10 This is the relationship curve between F1 score and confidence level in this embodiment.

[0107] It should also be noted that the UML class diagram information extraction method in the above embodiments can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a UML class diagram information extraction system corresponding to the UML class diagram information extraction method provided in the above embodiments, which includes:

[0108] The data processing module is used to acquire a UML class diagram image dataset containing various UML class diagram elements and to annotate the UML class diagram image dataset.

[0109] The model training module is used to employ k-fold cross-validation. It uses non-repeating sampling to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal parts, forming k subsets. In each training round, (k-2) subsets are used for model training, 1 dataset is used for model testing, and 1 dataset is used for model validation. The YOLO model is trained for a total of k rounds, and finally a trained YOLO model is obtained.

[0110] The feature extraction module is used to input the UML class diagram image of the information to be extracted into the trained YOLO model. The trained YOLO model extracts and analyzes features, identifies various UML class diagram elements, and outputs the category information and location information of the UML class diagram elements.

[0111] The text extraction module is used to identify the text information within the rectangular area representing the class based on the category and position information of the UML class diagram elements using optical character recognition technology. The extracted text information is divided into multiple strings according to the newline character. The first string is used as the class name. For the remaining strings, strings containing parentheses are used as class member functions and their function signature information is extracted. Strings without parentheses are used as class member variables and their variable names and types are extracted.

[0112] The mapping list acquisition module is used to traverse all bounding boxes representing relation types based on the category and position information of UML class diagram elements, treat each bounding box representing a relation type as a relation type bounding box, construct new relations based on the relation types of the bounding boxes, and form a list of related classes corresponding to them. It also traverses all bounding boxes representing classes based on the category and position information of UML class diagram elements, treating each bounding box representing a class as a class bounding box, and determines whether a relation type bounding box is adjacent to any class bounding box: if the relation type bounding box is adjacent to a class bounding box, the class of that class bounding box is added to the list of related classes corresponding to the new relation; otherwise, it continues to determine whether the relation type bounding box is adjacent to the next class bounding box, until all relation type bounding boxes have been determined, resulting in a mapping list of all relation types and corresponding new relations.

[0113] The contour extraction module is used to remove the contents of the bounding box corresponding to the location information using an image processing library, calculate the contour that wraps the remaining line segments, and use the returned contour as a reference bounding box.

[0114] The information extraction module is used to process a reference bounding box. If the reference bounding box is adjacent to a relation type bounding box, the relation type represented by the relation type bounding box is used as the reference relation type. Based on the reference relation type, the module queries the mapping list for the corresponding relation and uses it as the reference relation. This reference relation is then used as the relation corresponding to the reference bounding box. The module continues to determine whether the reference bounding box is adjacent to each class bounding box. If the reference bounding box is adjacent to a class bounding box, the class represented by that class bounding box is added to the related class list of the relation corresponding to the reference bounding box. If the reference bounding box is not adjacent to a class bounding box, the module skips this step and continues to determine whether the reference bounding box is adjacent to the next class bounding box, until all class bounding boxes have been traversed. If the reference bounding box is not adjacent to a relation type bounding box, the module skips this step and continues to determine whether the reference bounding box is adjacent to the next relation type bounding box, until all reference bounding boxes have been traversed. Finally, the module outputs the class structure information and all newly constructed relations, completing the information extraction based on the UML class diagram.

[0115] It is understood that the UML class diagram information extraction method described in S1 to S7 above can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the UML class diagram information extraction method provided in the above embodiments, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, they can implement the UML class diagram information extraction method as described in the above embodiments.

[0116] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the UML class diagram information extraction method provided in the above embodiments, which includes a memory and a processor;

[0117] The memory is used to store computer programs;

[0118] The processor is configured to implement the UML class diagram information extraction method in the above embodiments when executing the computer program.

[0119] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0120] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the UML class diagram information extraction method provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can implement the UML class diagram information extraction method in the above embodiments.

[0121] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0122] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0123] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0124] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for extracting information from UML class diagrams, characterized in that, Includes the following steps: S1. Obtain a UML class diagram image dataset containing various UML class diagram elements, and annotate the UML class diagram image dataset; S2. Using k-fold cross-validation, the UML class diagram image dataset and its corresponding annotation files are randomly divided into k equal parts using non-repeating sampling, forming k subsets. In each training round, k-2 subsets are used for model training, 1 dataset is used for model testing, and 1 dataset is used for model validation. The YOLO model is trained for a total of k rounds, and finally a trained YOLO model is obtained. S3. Input the UML class diagram image of the information to be extracted into the trained YOLO model. The trained YOLO model extracts and analyzes features, identifies various UML class diagram elements, and outputs the category information and position information of the UML class diagram elements. S4. Based on the category and position information of the UML class diagram elements, use optical character recognition technology to identify the text information within the rectangular area representing the class. Divide the extracted text information into multiple strings according to the newline character. Use the first string as the class name. For the remaining strings, use the strings containing parentheses as class member functions and extract the function signature information. Use the strings without parentheses as class member variables and extract the variable name and variable type. S5. Based on the category and position information of the UML class diagram elements, traverse all bounding boxes representing relation types, and treat each bounding box representing a relation type as a relation type bounding box. Construct new relations based on the relation types of the relation type bounding boxes and form a list of related classes corresponding to them. Based on the category and position information of the UML class diagram elements, traverse all bounding boxes representing classes, and treat each bounding box representing a class as a class bounding box. Determine whether a relation type bounding box is adjacent to each class bounding box: if the relation type bounding box is adjacent to a class bounding box, add the class of the class bounding box to the list of related classes corresponding to the new relation; otherwise, continue to determine whether the relation type bounding box is adjacent to the next class bounding box until all relation type bounding boxes have been determined, and obtain a mapping list of all relation types and corresponding relations between the new relations. S6. Use an image processing library to remove the contents of the bounding box corresponding to the location information, calculate the contour that wraps the remaining line segments, and use the returned contour as a reference bounding box. S7. For a reference bounding box, if it is adjacent to a relation type bounding box, the relation type represented by the relation type bounding box is used as the reference relation type. The corresponding relation is queried in the mapping list based on the reference relation type and used as the reference relation. This reference relation is then used as the relation corresponding to the reference bounding box. The process continues to determine if the reference bounding box is adjacent to any class bounding box. If it is adjacent to a class bounding box, the class represented by that class bounding box is added to the related class list of the relation corresponding to the reference bounding box. If it is not adjacent to a class bounding box, skip this step and continue to determine if it is adjacent to the next class bounding box, until all class bounding boxes have been traversed. If it is not adjacent to a relation type bounding box, skip this step and continue to determine if it is adjacent to the next relation type bounding box, until all reference bounding boxes have been traversed. Output the class structure information and all newly constructed relations, completing the information extraction based on the UML class diagram. In the new relations, the relation types are adjacent relation types, and the related classes are adjacent classes.

2. The method for extracting UML class diagram information as described in claim 1, characterized in that, In step S1, when annotating the UML class diagram image dataset, the annotation tool is used to annotate the UML class diagram image, the target is selected using the target rectangle, and annotation text with the same name as the UML class diagram image is generated. Then the annotation text is converted into YOLO annotation format and saved.

3. The method for extracting UML class diagram information as described in claim 2, characterized in that, In step S1, when annotating the UML class diagram image dataset, the annotation text corresponding to each UML class diagram image contains a total of 5 data points, with adjacent data points separated by spaces. These data points are: category name, ratio of the horizontal coordinate of the center of the target rectangle to the width of the UML class diagram image, ratio of the vertical coordinate of the center of the target rectangle to the height of the UML class diagram image, ratio of the width of the target rectangle to the width of the UML class diagram image, and ratio of the height of the target rectangle to the height of the UML class diagram image.

4. The method for extracting UML class diagram information as described in claim 1, characterized in that, In step S2, the UML class diagram image dataset and its corresponding annotation file are randomly divided into 5 equal parts to form 5 sub-datasets.

5. The method for extracting UML class diagram information as described in claim 1, characterized in that, In step S6, the cv2.findContours function in the OpenCV library is used to calculate the contour that encloses the remaining line segments.

6. The method for extracting UML class diagram information as described in claim 1, characterized in that, In step S7, the class structure information is the class name, class member variables, and class member functions extracted in S4.

7. A UML class diagram information extraction system, characterized in that, include: The data processing module is used to acquire a UML class diagram image dataset containing various UML class diagram elements and to annotate the UML class diagram image dataset. The model training module is used to employ k-fold cross-validation. It uses non-repeating sampling to randomly divide the UML class diagram image dataset and its corresponding annotation files into k equal parts, forming k subsets. In each training round, k-2 subsets are used for model training, 1 dataset is used for model testing, and 1 dataset is used for model validation. The YOLO model is trained for a total of k rounds, and finally a trained YOLO model is obtained. The feature extraction module is used to input the UML class diagram image of the information to be extracted into the trained YOLO model. The trained YOLO model extracts and analyzes features, identifies various UML class diagram elements, and outputs the category information and location information of the UML class diagram elements. The text extraction module is used to identify the text information within the rectangular area representing the class based on the category and position information of the UML class diagram elements using optical character recognition technology. The extracted text information is divided into multiple strings according to the newline character. The first string is used as the class name. For the remaining strings, strings containing parentheses are used as class member functions and their function signature information is extracted. Strings without parentheses are used as class member variables and their variable names and types are extracted. The mapping list acquisition module is used to traverse all bounding boxes representing relation types based on the category and position information of UML class diagram elements, treat each bounding box representing a relation type as a relation type bounding box, construct new relations based on the relation types of the bounding boxes, and form a list of related classes corresponding to them. It also traverses all bounding boxes representing classes based on the category and position information of UML class diagram elements, treating each bounding box representing a class as a class bounding box, and determines whether a relation type bounding box is adjacent to any class bounding box: if the relation type bounding box is adjacent to a class bounding box, the class of that class bounding box is added to the list of related classes corresponding to the new relation; otherwise, it continues to determine whether the relation type bounding box is adjacent to the next class bounding box, until all relation type bounding boxes have been determined, resulting in a mapping list of all relation types and corresponding new relations. The contour extraction module is used to remove the contents of the bounding box corresponding to the location information using an image processing library, calculate the contour that wraps the remaining line segments, and use the returned contour as a reference bounding box. The information extraction module is used to process a reference bounding box. If the reference bounding box is adjacent to a relation type bounding box, the relation type represented by the relation type bounding box is used as the reference relation type. Based on the reference relation type, the module queries the mapping list for the corresponding relation and uses it as the reference relation. This reference relation is then used as the relation corresponding to the reference bounding box. The module continues to determine whether the reference bounding box is adjacent to each class bounding box. If the reference bounding box is adjacent to a class bounding box, the class represented by that class bounding box is added to the related class list of the relation corresponding to the reference bounding box. If the reference bounding box is not adjacent to a class bounding box, the module skips this step and continues to determine whether the reference bounding box is adjacent to the next class bounding box, until all class bounding boxes have been traversed. If the reference bounding box is not adjacent to a relation type bounding box, the module skips this step and continues to determine whether the reference bounding box is adjacent to the next relation type bounding box, until all reference bounding boxes have been traversed. The module outputs the class structure information and all newly constructed relations, completing the information extraction based on the UML class diagram. In the new relations, the relation types are adjacent relation types, and the related classes are adjacent classes.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it can implement the UML class diagram information extraction method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the UML class diagram information extraction method as described in any one of claims 1 to 6.

10. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement the UML class diagram information extraction method as described in any one of claims 1 to 6 when executing the computer program.

Images