A method for identifying UML class diagram relationships and related equipment

Abstract

This application provides a method and related equipment for UML class diagram relationship recognition, belonging to the field of computer vision and pattern recognition technology. Addressing the problems of broken dashed lines in UML class diagram images, difficulty in connecting them, easy confusion of small endpoint symbols, and difficulty in establishing long-distance topological relationships, this application proposes the following solution: Morphological dilation and closing operations are performed on the obtained clean line layer image to connect discrete dashed line segments into complete simply connected domains; contours are extracted and collision detection is performed to determine valid relationship lines and their physical start and end points; local regions of interest are cropped from the original image centered on the start and end points; a lightweight convolutional neural network is used to perform fine-grained classification of endpoint symbols; and structured relationship data is generated based on the semantic types of the two endpoints combined with UML rules. This application achieves complete restoration of dashed line paths and high-precision recognition of small symbols, possesses robust global topological understanding capabilities, and can be used for automatic class diagram recognition in software reverse engineering and legacy system reconstruction.

Description

Technical Field

[0001] This application relates to the fields of computer vision and pattern recognition technology, and in particular to a UML class diagram relationship recognition method and related equipment. Background Technology

[0002] In the fields of software reverse engineering, legacy system refactoring, and automated code generation, restoring existing UML class diagram images into computer-processable structured data has significant industrial value. In real-world development scenarios, many software documents are saved only in bitmap formats such as PNG and JPEG, losing the original design model files. Therefore, accurately recovering class diagram models from these standard raster images has become a crucial step in achieving automated software understanding and refactoring.

[0003] UML class diagram recognition mainly comprises two subtasks: recognition of class diagram units (rectangles and their internal text) and recognition of inter-class relationships. While text extraction within class diagram units has become relatively mature with the development of Optical Character Recognition (OCR) technology, the recognition of inter-class relationships remains a significant challenge and a bottleneck restricting the overall recognition rate. This is mainly reflected in the following technical difficulties: 1) The connectivity problem between dashed lines and discrete pixels In standard UML specifications, dependencies and implementation relationships must be represented using dashed lines. In rasterized images, dashed lines appear at the pixel level as a series of unconnected short line segments or dots. Traditional line detection algorithms based on connected component analysis or Hough Transform often incorrectly identify a complete dashed line as dozens of independent noise points or short lines, making it impossible to construct a complete topological path from the source class to the target class. Furthermore, artifacts generated by image compression algorithms and anti-aliasing blur line edges, further interfering with traditional gradient-based edge detection algorithms and making line tracking easily interrupted.

[0004] 2) Visual confusion of subtle semantic symbols The semantic distinction between different relationships in UML class diagrams is highly dependent on the geometric shape of the line endpoints. For example, composition relationships use solid rhombuses, aggregation relationships use hollow rhombuses, generalization relationships use hollow triangles, and association relationships use ordinary arrows. In standard-resolution document images, these endpoint symbols often occupy only a very small pixel area (typically less than 40×40 pixels). Traditional methods based on template matching or geometric feature extraction are extremely sensitive to image resolution. Once the image is scaled or slightly compressed, solid areas may become noisy, and hollow areas may break due to thin lines, causing the algorithm to be unable to accurately distinguish between "solid" and "hollow," resulting in serious semantic misjudgments (such as misjudging "strong composition" as "weak aggregation").

[0005] 3) Topological gaps in existing deep learning solutions In recent years, solutions based on general object detection networks such as YOLO and Faster R-CNN have been applied to UML recognition. While these end-to-end methods can detect the locations of "arrows" or "diamonds" (bounding boxes) in images with high accuracy, they lack the ability to understand the global topology. In complex UML class diagrams, relationship lines often adopt an orthogonal routing layout, with numerous turns, intersections, and long spans. Object detection networks struggle to answer the core questions of "which polyline does this arrow belong to?" and "which two distant classes does this polyline connect?" This deficiency of "seeing only local symbols and not the overall connections" often leads to relationship mismatches when existing technologies process complex and dense standard class diagrams.

[0006] In summary, existing technologies struggle to simultaneously achieve complete restoration of dashed paths and accurate classification of tiny endpoint symbols when processing standard rasterized UML images. Summary of the Invention

[0007] The main objective of this application is to overcome the shortcomings of existing technologies in recognizing standard UML class diagrams, such as the difficulty in connecting broken dashed lines, the easy confusion of tiny endpoint symbols, and the difficulty in establishing long-distance topological relationships. This application provides a UML class diagram relationship recognition method, electronic device, storage medium, and program product based on deep learning and morphological processing. It can effectively solve the problem of the discreteness of dashed lines at the pixel level and use deep neural networks to perform high-precision classification of tiny connector symbols, thereby accurately extracting the topological structure and semantic relationships between classes.

[0008] To achieve the above objectives, one aspect of this application proposes a UML class diagram relationship identification method, the method comprising: Image preprocessing steps: Obtain the original image containing the UML class diagram, preprocess the original image to obtain a clean line layer image containing at least relation lines; Morphological connectivity repair step: Perform morphological processing on the clean line layer image, the morphological processing including at least dilation and / or closing operations, to connect line segments broken by dashed lines into complete simply connected regions; Topology connection analysis steps: Extract contours from the morphologically processed image, detect the collision between each contour and each type of rectangular region in the UML class diagram. If the same contour collides with both the first and second type of rectangular regions, then the contour is determined to be a valid relationship line connecting the first and second type of rectangular regions, and the physical starting point of the relationship line in the first type of rectangular region and the physical ending point in the second type of rectangular region are determined. Local region of interest extraction steps: Using the physical start point and the physical end point as centers, crop out the local regions of interest containing the endpoint symbols from the original image respectively; Endpoint semantic classification step: Use a pre-trained convolutional neural network to classify the endpoint symbols in each of the local regions of interest and identify the UML semantic type of the endpoint; Relationship generation steps: Based on the UML semantic types of the two endpoints of the same relationship line, and in conjunction with the UML semantic rule table, determine the UML relationship type and direction between the first type of rectangular region and the second type of rectangular region, and output structured relationship data.

[0009] In some embodiments, the morphological connectivity repair step includes the following morphological processing steps: first performing a closing operation, then performing a dilation operation; the kernel size used for the closing and dilation operations is dynamically set according to the resolution or degree of fragmentation of the original image, or a preset fixed size is used.

[0010] In some embodiments, for standard resolution images, the closing operation uses a 3×3 pixel rectangular kernel, the dilation operation uses a 3×3 pixel rectangular kernel, and the number of iterations is 1; for low resolution images or images with severe line breaks, the closing operation uses a 7×7 pixel rectangular kernel, the dilation operation uses a 5×5 pixel rectangular kernel, and the number of iterations is 2.

[0011] In some embodiments, the collision detection step of the topology connection analysis step specifically involves: calculating the shortest distance between the pixel point on the contour and the boundary of the identified rectangular region; if the distance is less than a preset collision threshold, a collision is determined to have occurred; the collision threshold is a fixed value, or is dynamically calculated based on the width of the original image.

[0012] In some embodiments, the local region of interest extraction step further includes: when the physical start point or physical end point is located at the edge of the original image, using a boundary filling strategy to expand the cropped region to ensure that the cropped local region of interest has complete size.

[0013] In some embodiments, the convolutional neural network is a lightweight convolutional neural network, whose backbone network adopts ResNet18, MobileNetV3-Small, or a custom shallow convolutional neural network; the input layer size of the network is 80×80 pixels, and the number of output layer nodes corresponds to the UML standard endpoint categories, which include at least: solid rhombus, hollow rhombus, hollow triangle, dashed hollow triangle, solid arrow, dashed arrow, and shapeless.

[0014] In some embodiments, the endpoint semantic classification step further includes: performing data augmentation preprocessing on the cropped local regions of interest, wherein the data augmentation includes random rotation, random horizontal flipping and / or adding Gaussian noise to enhance the robustness of the model to images of different styles.

[0015] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0017] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0018] Compared with the prior art, this application has the following advantages and beneficial effects: 1) Solved the connectivity problem of dashed path: Through an innovative morphological strategy of "powerful expansion + closing operation", this application can forcibly merge discrete dashed line segments in a standard UML diagram into a simply connected domain. This allows the algorithm to completely recover the topological path from the source class to the target class using simple contour extraction techniques without complex curve fitting or path search, greatly reducing the computational complexity of dashed line recognition.

[0019] 2) Achieved high-precision, fine-grained classification of minute symbols: Existing technologies often struggle to distinguish subtle differences such as "solid rhombuses" and "hollow rhombuses." This application employs a "localization-then-recognition" strategy, extracting endpoint symbols from the complex background and inputting them into a dedicated deep neural network. This local ROI classification method effectively eliminates interference from text and other lines, allowing the network to focus on minute geometric features and significantly improving the accuracy of recognizing hollow / solid and dashed / solid arrows.

[0020] 3) Robust global topology understanding capability: Unlike end-to-end solutions based on object detection, this application prioritizes establishing geometric topology connections. This means that even in cases where the class diagram layout is extremely dense and the lines intersect in a complex manner, the algorithm can clearly know that "this line connects A and B," without encountering logical errors such as "an arrow was detected but it is not known which line it belongs to," thus ensuring the logical correctness of the generated UML model. Attached Figure Description

[0021] Figure 1 This is a flowchart of a UML class diagram relationship identification method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the specific implementation process based on morphological processing and deep learning in the application embodiments; Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0024] To address the technical problems existing in the prior art, this application provides a UML class diagram relationship recognition method, electronic device, storage medium, and program product based on deep learning and morphological processing. The main technical contents of this solution include the following: 1) Relationship line extraction based on strongly connected morphology: For dashed lines (dependency / realization relationships) and broken lines in UML diagrams, edge detection combined with large-scale morphological closing and dilation operations is used to forcibly glue discrete dashed line paths into complete simply connected domains in space, achieving unified extraction of dashed and solid lines and ensuring the integrity of subsequent contour recognition.

[0025] 2) Topology construction based on boundary collision detection: Contour tracing is performed on the extracted relational line connected components. By detecting the overlap between contour pixels and the expanded boundary of the class graph rectangle, the logical connection between classes is determined. If the same contour simultaneously touches two class graph regions, the connection relationship is established, and the center of the contact area is calculated as the physical start and end point.

[0026] 3) Adaptive Extraction of Local ROIs at Endpoints: Centered on defined physical start and end points, regions of interest (ROIs) containing features such as arrows and rhombuses are cropped from the original image. A boundary padding strategy is introduced to ensure the integrity of endpoint symbol features located at image edges, generating standardized connector samples.

[0027] 4) Endpoint semantic classification based on lightweight CNN: A lightweight convolutional neural network is constructed to perform fine-grained morphological recognition on connector samples. The network directly outputs 7 UML standard endpoint categories, including solid / hollow rhombuses, triangles, and dashed / solid arrows, solving the problem of recognizing tiny symbols under rotation and ambiguity.

[0028] 5) Semantic mapping and directed relation generation: Based on the classification results of the endpoints at both ends of the connection, combined with the preset UML semantic rule table (such as solid diamonds corresponding to combination relationships, dashed arrows corresponding to dependency relationships, etc.), the relation type and direction (source class / target class) are automatically parsed and established, and structured UML relation description data is generated.

[0029] The UML class diagram relationship recognition method provided in this application relates to the field of computer vision and pattern recognition technology. This method can be applied to terminals, servers, or software running on either a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the UML class diagram relationship recognition method, but is not limited to the above forms.

[0030] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0031] like Figure 1 As shown, this embodiment provides a method for identifying relationships in a UML class diagram, specifically including the following steps: S1. Image preprocessing steps: Obtain the original image containing the UML class diagram, preprocess the original image to obtain a clean line layer image containing at least the relationship lines; S2. Morphological connection repair step: Perform morphological processing on the clean line layer image. The morphological processing includes at least dilation and / or closing operations to connect the broken line segments represented by dashed lines into complete simply connected regions. S3. Topology connection analysis steps: Extract contours from the morphologically processed image, detect the collision between each contour and each type of rectangular region in the UML class diagram. If the same contour collides with both the first and second type rectangular regions, then the contour is determined to be an effective relationship line connecting the first and second type rectangular regions, and the physical starting point of the relationship line in the first type rectangular region and the physical ending point in the second type rectangular region are determined. S4. Local Region of Interest Extraction Steps: Using the physical start point and physical end point as centers, crop out the local regions of interest containing the endpoint symbols from the original image respectively; S5. Endpoint semantic classification step: Use a pre-trained convolutional neural network to classify the endpoint symbols in each local region of interest and identify the UML semantic type of the endpoint; S6. Relationship generation steps: Based on the UML semantic types of the two ends of the same relationship line, and in conjunction with the UML semantic rule table, determine the UML relationship type and direction between the first type of rectangular region and the second type of rectangular region, and output the structured relationship data.

[0032] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific application examples.

[0033] Example 1 See Figure 2 This embodiment provides a UML class diagram relationship recognition method based on deep learning and morphological processing, suitable for processing standard, high-resolution UML class diagram images (e.g., PNG images directly exported from modeling software). The main process of this method is as follows: Step 1: Image input and preprocessing.

[0034] For example, input a grayscale UML class diagram image with a size of 1024×768 pixels. First, use the Otsu binarization algorithm to perform threshold segmentation on the image, setting the background to white (255) and the foreground lines and text to black (0). Use existing OCR technology to identify and remove all text regions in the image, and then use a contour detection algorithm to identify and remove all class rectangles, generating a "clean line layer image" containing only relational connection lines.

[0035] Step 2: Morphological connectivity repair.

[0036] As a specific implementation method, Canny edge detection is performed on the "clean line layer image," with a high threshold set to 100 and a low threshold set to 30. For any dashed lines (such as dependencies) that may exist in the UML diagram, morphological closing is used for initial connection, with a 3×3 pixel rectangular kernel. Subsequently, a morphological dilation operation is performed, with a 3×3 pixel kernel and one iteration. This step glues together the originally discontinuous dashed line segments into connected pixel paths.

[0037] Step 3: Topology connection analysis.

[0038] As a specific implementation, all outer contours are extracted from the morphologically processed image using `cv2.findContours`. Each contour is traversed, and the distance between points on it and the boundaries of the identified rectangles is calculated. A collision threshold of 15 pixels is set. If at least one point on a contour is less than 15 pixels from rectangle A, and another point is less than 15 pixels from rectangle B, then the contour is considered a valid connection line between rectangles A and B. The geometric centers of all contact points of the contour near the boundaries of rectangles A and B are calculated and marked as physical starting points. and physical endpoint .

[0039] Step 4: Build and train a lightweight convolutional neural network.

[0040] Specifically, a lightweight convolutional neural network based on the ResNet18 backbone is constructed, with the input layer size adjusted to 80×80. The training dataset is as follows: The training dataset is shown in Table 1: Table 1 Training Dataset

[0041] Step 5: Endpoint connector identification.

[0042] Specifically, with and Centered on the original image, 80×80 pixel local regions (ROIs) are cropped out. The two cropped ROI images are normalized and then input into the convolutional neural network trained in step 4 for classification. The network output layer contains 7 nodes, corresponding to: solid rhombus, hollow rhombus, hollow triangle, dashed hollow triangle, dashed arrow, solid arrow, and no shape. In this embodiment, if the model outputs "hollow rhombus" with the highest confidence (e.g., 0.96), then that endpoint is determined to be an aggregation relation symbol.

[0043] Step 6: Semantic generation.

[0044] Based on the recognition results, if the starting point is "shapeless" and the ending point is "hollow rhombus", then according to the UML semantic mapping table, it is determined that there is an "aggregation relationship" between rectangle A and rectangle B, and rectangle B is the whole (Target) and rectangle A is the part (Source).

[0045] The system outputs JSON data containing SourceID, TargetID, and RelationType.

[0046] Example 2 This embodiment optimizes and adjusts the morphological processing parameters and network model in Embodiment 1 for UML images with many dashed lines or severe line breaks (such as low-resolution scans).

[0047] Transformation point 1: Enhanced morphological processing parameters In step 2, to address more severe pixel fragmentation, the size of the morphological processing convolution kernel is significantly increased: a) Closed kernel: Adjusted to 7×7 pixels. A larger kernel can more effectively fill long gaps between dashed line segments.

[0048] b) Inflation kernel: Adjust to 5×5 pixels and increase the number of iterations to 2.

[0049] This powerful dilation strategy can force discrete points with a spacing of more than 5 pixels to merge into a single connected domain, ensuring that the topological path remains intact even if the brush breaks or the scan loses frames.

[0050] Transformation Point 2: Lightweight Adaptation of Network Structure In step 4, to improve inference speed, the backbone network is replaced with MobileNetV3-Small. The input image size remains unchanged at 80×80, but a stronger data augmentation strategy is introduced: during the training phase, the connector samples are randomly rotated (±30 degrees), randomly flipped horizontally, and Gaussian noise is added (standard deviation 0.01).

[0051] Example 3 This embodiment adjusts the collision detection logic and ROI cropping strategy for high-resolution and densely laid-out UML images.

[0052] Transformation point 1: Adaptive collision threshold In step 3, considering that the gaps between lines and rectangles may be magnified in high-resolution images, a fixed 15-pixel threshold is no longer used. Instead, it is based on the image width. W Dynamically set threshold T = W ×0.02 (for example, when the width is 2000 pixels, the threshold is 40 pixels).

[0053] Adaptive thresholding avoids "missed connections" caused by an excessively small threshold at high resolutions, while also preventing "false connections" caused by an excessively large threshold at low resolutions.

[0054] Transformation Point 2: ROI Pruning Based on Two-Way Validation In step 5, in addition to cropping the 80×80 region with the geometric center as the origin, an auxiliary ROI is cropped again by shifting inward by 10 pixels along the line extension direction. Both ROIs are then fed into the network simultaneously, and the maximum confidence score of their predictions is taken as the final result.

[0055] This two-way verification mechanism effectively solves the problem of recognition failure caused by the positioning deviation of the line endpoints (such as positioning them in the blank space outside the tip of the arrow).

[0056] Other alternative implementation methods In addition to the methods mentioned in the above embodiments, the embodiments of this application can also be modified as follows: 1) Morphological operator transformation: The morphological closing and dilation operations in step 2 can also be replaced by a combination of "Gaussian blur + binarization". Gaussian blur is used to fuse the broken lines at the gray level, and then binarization is used to extract the connected components.

[0057] 2) Network model transformation: The classification network in step 5 is not limited to ResNet or MobileNet. It can also use a custom shallow CNN (such as 3 convolutional layers + 2 fully connected layers), or use support vector machine (SVM) combined with histogram of oriented gradients (HOG) features for classification.

[0058] 3) Process sequence change: Step 3 (topology analysis) and Step 5 (endpoint identification) can be executed in parallel. That is, while extracting the contour, all candidate endpoint regions of the contour are batch-cropped and identified, and finally matched according to the topology.

[0059] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0060] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0061] Please see Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 301 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 302 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 302 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302 and is called and executed by the processor 301 using the methods described above in the embodiments of this application. Input / output interface 303 is used to implement information input and output; The communication interface 304 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 305 transmits information between various components of the device (e.g., processor 301, memory 302, input / output interface 303, and communication interface 304); The processor 301, memory 302, input / output interface 303, and communication interface 304 are connected to each other within the device via bus 305.

[0062] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0063] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0064] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0065] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0066] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0067] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0068] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0069] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0070] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0071] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0072] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0073] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0074] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0075] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0076] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0077] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for identifying UML class diagram relationships, characterized in that, The method includes the following steps: Image preprocessing steps: Obtain the original image containing the UML class diagram, preprocess the original image to obtain a clean line layer image containing at least relation lines; Morphological connectivity repair step: Perform morphological processing on the clean line layer image, the morphological processing including at least dilation and / or closing operations, to connect the broken line segments represented by dashed lines into complete simply connected regions; Topology connection analysis steps: Extract contours from the morphologically processed image, detect the collision between each contour and each type of rectangular region in the UML class diagram. If the same contour collides with both the first and second type of rectangular regions, then the contour is determined to be a valid relationship line connecting the first and second type of rectangular regions, and the physical starting point of the relationship line in the first type of rectangular region and the physical ending point in the second type of rectangular region are determined. Local region of interest extraction steps: Using the physical start point and the physical end point as centers, crop out the local regions of interest containing the endpoint symbols from the original image respectively; Endpoint semantic classification step: Use a pre-trained convolutional neural network to classify the endpoint symbols in each of the local regions of interest and identify the UML semantic type of the endpoint; Relationship generation steps: Based on the UML semantic types of the two endpoints of the same relationship line, and in conjunction with the UML semantic rule table, determine the UML relationship type and direction between the first type of rectangular region and the second type of rectangular region, and output structured relationship data.

2. The method of claim 1, wherein, In the morphological connectivity repair step, the morphological processing includes, in sequence: first performing a closing operation, then performing a dilation operation; the kernel size used for the closing and dilation operations is dynamically set according to the resolution or degree of fragmentation of the original image, or a preset fixed size is used.

3. The method of claim 2, wherein, For standard resolution images, the closing operation uses a 3×3 pixel rectangular kernel, the dilation operation uses a 3×3 pixel rectangular kernel, and the number of iterations is 1; for low resolution images or images with severe line breaks, the closing operation uses a 7×7 pixel rectangular kernel, the dilation operation uses a 5×5 pixel rectangular kernel, and the number of iterations is 2.

4. The method of claim 1, wherein, In the topology connectivity analysis step, detecting the collision between the contour and the rectangular region specifically involves: calculating the shortest distance between the pixels on the contour and the boundary of the identified rectangular region; if the distance is less than a preset collision threshold, a collision is determined to have occurred; the collision threshold is a fixed value, or it can be dynamically calculated based on the width of the original image.

5. The method according to claim 1, characterized in that, The local region of interest extraction step further includes: when the physical starting point or physical ending point is located at the edge of the original image, a boundary filling strategy is used to expand the cropped region to ensure that the cropped local region of interest has complete size.

6. The method according to claim 1, characterized in that, The convolutional neural network is a lightweight convolutional neural network, and its backbone network adopts ResNet18, MobileNetV3-Small or a custom shallow convolutional neural network; the input layer size of the network is 80×80 pixels, and the number of output layer nodes corresponds to the UML standard endpoint categories, which include at least: solid rhombus, hollow rhombus, hollow triangle, dashed hollow triangle, solid arrow, dashed arrow, and shapeless.

7. The method according to claim 1, characterized in that, The endpoint semantic classification step further includes: performing data augmentation preprocessing on the cropped local regions of interest, wherein the data augmentation includes random rotation, random horizontal flipping and / or adding Gaussian noise to enhance the robustness of the model to images of different styles.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images