A large model MLIR graph example visualization editing method and product

Abstract

The embodiment of the application provides a large model MLIR graph example visual editing method and product, the large model MLIR graph example visual editing method includes: obtaining the original SVG vector graph generated by the back end;Draw a labeling area on the original SVG vector graph, and obtain the position information of the labeling area and the attribute information corresponding to the labeling area, wherein the position information represents the geometric boundary of the labeling area in the global coordinate system, and the attribute information represents the to-be-injected compilation attribute, and the compilation attribute includes a type name and a value;Write the labeling layer data representing the labeling area, the position information and the attribute information into the SVG file corresponding to the original SVG vector graph to obtain a labeled SVG file;The labeled SVG file is sent to the back end, so that the back end matches the corresponding operator node according to the labeled SVG file and injects the compilation attribute into the matched operator node to obtain a target MLIR file.

Description

Technical Field

[0001] This application relates to the field of large model compilation technology, and more specifically, the embodiments of this application relate to a method and product for visual editing of large model MLIR legends. Background Technology

[0002] Large model inference, quantization, and custom operator development all rely on MLIR (Multi-Level Intermediate Representation) as the compilation infrastructure. During the optimization and tuning of large model operators, developers need to visually view the operator data flow graph and batch-mark the attribute information or target binding attributes of custom operators.

[0003] Currently, visualization schemes for MLIR operator graphs mainly include the following categories.

[0004] I. Native Graphviz Offline Visualization Solution

[0005] The LLVM-MLIR official documentation provides the `view-op-graph` tool, which converts MLIR files to dot text format and uses Graphviz to render PNG or SVG vector graphics. This solution only provides a static display of the operator graph. The accompanying `emit-op-map` tool outputs a JSON file mapping operator IDs to graph nodes to aid in operator location. The development workflow under this solution involves generating the SVG graph in a Linux environment, recording operator IDs via screenshots or manual recording, and then manually hardcoding compilation attributes such as `target` or `custom_op_name` in the code. In this workflow, viewing the graph and editing attributes are disconnected, lacking interactive visual annotation capabilities.

[0006] II. Web-based front-end visualization tools

[0007] MLIR operator graph visualization tools built on web front-end rendering engines (such as D3.js) support interactive operations such as mouse selection of nodes. However, these tools are heavily dependent on the browser environment, cannot run independently offline, and face difficulties in coordinating cross-Windows and Linux compilation backends. After editing, the annotation results need to be exported as a separate JSON script, which the backend needs to parse a second time before injecting into MLIR, increasing the file conversion process and system complexity.

[0008] III. Local Integrated Compilation and Visualization Tools

[0009] Some localization tools (such as Turbolizer and Netron) can provide local visualization of model structure or weight graphs, but they are not compatible with the annotation and editing of MLIR custom attributes, and only support single-node one-by-one modification, unable to achieve batch selection of operators and batch writing of compilation attributes. In scenarios where large model operator graphs often contain thousands of nodes, single-point operation is extremely inefficient.

[0010] In summary, existing MLIR legend visualization and editing tools mainly suffer from the following technical defects.

[0011] First, cross-platform collaboration capabilities are insufficient, with compilation and editing environments separated. The MLIR compilation backend is typically deployed only on Linux servers, while developers routinely use Windows terminals for development. Existing tools lack a unified cross-platform interaction link. After generating the graph on the Linux side, operator IDs need to be manually recorded and the MLIR source code manually modified. After modification, it needs to be recompiled and verified, resulting in a long iterative process that is prone to human error.

[0012] Secondly, it lacks native interactive legend annotation capabilities, resulting in low efficiency for batch operator annotation. The Graphviz solution only outputs static graphs without interactive bounding box annotation; front-end bounding box tools need to store annotation information as independent editing scripts, and the back-end needs to handle file format conversion and coordinate mapping during parsing. Furthermore, single-point modifications are time-consuming in large-scale operator graph scenarios.

[0013] Third, inconsistent file transfer and coordinate systems lead to operator mismatch. Multiple files, such as graphic files, operator mapping JSON, and editing scripts, are transferred separately. The front-end and back-end coordinate systems are independent, and the coordinates of selected annotations cannot be directly reused. This easily leads to misalignment between the annotation box and MLIR operator nodes, resulting in incorrect attribute binding.

[0014] Fourth, the environment is complex and the deployment threshold is high. To implement complete MLIR parsing and editing functions on Windows, the entire LLVM / MLIR dependency set needs to be compiled, which takes a huge amount of time; offline office environments cannot connect to the internet to install dependencies, making tool distribution and usage difficult.

[0015] Fifth, the editing results cannot be natively carried with the graphic carrier, making traceability difficult. Annotation information is stored independently in JSON or text files, separating the graphics from the editing script. If files are lost or versions are confused, historical annotation records cannot be traced. Multiple editing scripts need to be manually maintained during multiple iterations, resulting in chaotic version management. Summary of the Invention

[0016] The purpose of this application is to provide a method and product for visual editing of large-scale MLIR legends. Some embodiments of this application directly inject annotation information into the SVG file to form a single, transferable file, avoiding version confusion and file loss caused by the separation of graphics, operator mapping, and editing scripts in traditional solutions. At the same time, the backend can directly parse annotations using the same SVG file without the need for additional coordinate transformation scripts, improving the automation level of the entire process. Furthermore, the embodiments of this application can perform unified operations on multiple operators that need to batch modify attribute information by drawing annotation areas, effectively improving the problem of extremely low efficiency of single-point operations.

[0017] In a first aspect, embodiments of this application provide a method for visually editing a large-scale MLIR legend. The method includes: acquiring an original SVG vector image generated by a backend; drawing a labeled region on the original SVG vector image and acquiring the position information of the labeled region and attribute information corresponding to the labeled region, wherein the position information represents the geometric boundary of the labeled region in the global coordinate system, and the attribute information represents a compiled attribute to be injected, the compiled attribute including a type name and a value; writing the labeled layer data representing the labeled region, the position information, and the attribute information into an SVG file corresponding to the original SVG vector image to obtain a labeled SVG file; and sending the labeled SVG file to the backend, so that the backend can match corresponding operator nodes according to the labeled SVG file and inject the compiled attribute into the matched operator nodes to obtain a target MLIR file.

[0018] This application's embodiments directly inject annotation information into the SVG file to form a single, transferable file, avoiding version confusion and file loss issues caused by the separation of graphics, operator mapping, and editing scripts in traditional solutions. At the same time, the backend can directly parse annotations using the same SVG file without the need for additional coordinate transformation scripts, improving the automation level of the entire process. Furthermore, this application's embodiments can perform unified operations on operators that require batch modification of attribute information by drawing annotation areas, effectively improving the problem of extremely low efficiency in single-point operations.

[0019] In some embodiments, the location information includes the x-coordinate, y-coordinate, width, and height of the labeled area in the global coordinate system.

[0020] In the embodiments of this application, the location information is represented by the geometric boundary parameters of the labeled area, such as the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width of the rectangle, and the height of the rectangle. These four data items form the complete data basis for the backend to perform rectangle collision detection to match operator nodes. By explicitly defining the location information as a quadruple and defining the labeled area with standard geometric parameters, the frontend and backend have a unified mathematical basis for determining the range of the labeled box, ensuring the consistency of coverage discrimination.

[0021] In some embodiments, writing the annotation layer data representing the annotation region, the position information, and the attribute information into the SVG file corresponding to the original SVG vector image to obtain an annotated SVG file includes: parsing the SVG file corresponding to the original SVG vector image into an XML document tree; creating an independent layer node inside the root node of the XML document tree, wherein the independent layer node is identified using a preset method that distinguishes it from the original graphic layer of the original SVG vector image, so that the backend can recognize the independent layer node during parsing; serializing the position information and the attribute information corresponding to the annotation region into graphic tags, wherein the graphic tags carry compiled attribute type indication information using a preset method corresponding to the type of the attribute information, so that the backend can determine the type of the compiled attribute to be injected based on the indication information during parsing; and attaching the graphic tags as child nodes to the independent layer node to obtain the annotated SVG file.

[0022] The embodiments of this application use XML DOM tree node appending to write to the annotation layer, ensuring that the operator coordinates, styles and layouts in the original SVG are not modified or re-encoded, achieving lossless coexistence of annotation information and original graphics. At the same time, the backend can quickly locate annotation data through fixed layer identifiers, improving parsing efficiency.

[0023] In some embodiments, drawing the annotation area on the original SVG vector image includes: in response to a mouse drag operation, obtaining the pixel position of the mouse on the screen; reading the scaling factor and scroll bar offset of the current canvas; inversely mapping the screen pixel position to logical coordinates in the global coordinate system according to the scaling factor and the scroll bar offset; and determining whether to generate the annotation area on the original SVG vector image according to the logical coordinates.

[0024] The embodiments of this application convert screen pixel coordinates into SVG native logical coordinates through real-time inverse mapping, eliminating coordinate system deviations caused by view scaling and translation, and ensuring that the stored annotation area coordinates are always aligned with the original graph0 coordinate system output by Graphviz, regardless of how the user's current view changes, thus avoiding the problem of misalignment between the annotation box and the operator node.

[0025] In some embodiments, the scroll bar offset includes a horizontal scroll bar offset and a vertical scroll bar offset, wherein the step of inversely mapping the screen pixel position to logical coordinates in the global coordinate system includes: adding the horizontal coordinate of the screen pixel position to the horizontal scroll bar offset and dividing by the scaling factor to obtain the horizontal coordinate of the logical coordinate; adding the vertical coordinate of the screen pixel position to the vertical scroll bar offset and dividing by the scaling factor to obtain the vertical coordinate of the logical coordinate.

[0026] The inverse mapping formula of the embodiments of this application is: logical horizontal coordinate = (screen horizontal coordinate + horizontal scroll offset) / scaling factor, logical vertical coordinate = (screen vertical coordinate + vertical scroll offset) / scaling factor. This algorithm simplifies the complex inverse operation of affine transformation into addition and division, which has high computational efficiency and small accuracy loss.

[0027] In some embodiments, determining whether to generate the annotation region on the original SVG vector image based on the logical coordinates includes: comparing the logical coordinates with the boundary of the original SVG vector image; if the logical coordinates are outside the valid drawing area of ​​the original SVG vector image, then abandoning the determination of the logical coordinates as a vertex of the annotation region; if the logical coordinates are within the valid drawing area of ​​the original SVG vector image, then determining the logical coordinates as a vertex of the annotation region.

[0028] The embodiments of this application define the validity verification steps after inverse mapping. When a user drags the mouse outside the valid canvas area (e.g., excessive scrolling causes the field of view to exceed the SVG boundary), the logical coordinates obtained by inverse mapping may fall outside the valid drawing area. The embodiments of this application filter invalid coordinates through boundary comparison, avoiding the generation of invalid annotation areas that cannot match any operators. In other words, the embodiments of this application perform real-time boundary verification of coordinates during the drawing process, automatically filtering invalid annotations that exceed the valid SVG drawing area, preventing the writing of illegal coordinate data, and further improving the robustness of the file.

[0029] In some embodiments, the large model MLIR legend visualization editing method further includes: storing the labeled regions in a global state container in memory, wherein the global state container includes a list of currently effective labeled regions and an undo history stack; and in response to an undo command, reverting the list of currently effective labeled regions to the historical state stored in the undo history stack.

[0030] The embodiments of this application manage annotation history through a global state container in memory, support undo operations on drawn annotation boxes, allow users to gradually roll back erroneous box selections, and improve the error tolerance and editing efficiency in the process of large annotations.

[0031] In some embodiments, the global state container further includes a redo stack; the large model MLIR legend visualization editing method further includes: in response to a redo instruction, advancing the currently effective list of labeled regions to the historical state stored in the redo stack.

[0032] In the embodiments of this application, the redo stack and the undo stack are symmetrical, storing the undone historical state for recovery. Undo and redo constitute a complete backtrackable operation chain, allowing users to freely go back and forth in multiple trials and errors, greatly reducing the rework cost caused by misoperation.

[0033] In some embodiments, drawing a labeled area on the original SVG vector image and obtaining the position information of the labeled area and the attribute information corresponding to the labeled area includes: determining a target compiled attribute type in response to a selection operation among multiple visual styles; obtaining the pixel position of the mouse on the screen in response to a drag operation of the mouse on the original SVG vector image; inversely mapping the screen pixel position to logical coordinates in the global coordinate system according to the current canvas scaling factor and scroll bar offset; determining the position information of the labeled area according to the logical coordinates; drawing the labeled area on the original SVG vector image according to the visual style corresponding to the target compiled attribute type according to the drag operation of the mouse; using the target compiled attribute type as the type of the attribute information, and determining the value of the attribute information as a preset value or a user input value.

[0034] The embodiments of this application intuitively distinguish annotation types through visual styles. Users can identify the compilation attribute type bound to the current annotation by color or line type without additional operation, improving operational efficiency and visual recognition. Different compilation attribute types correspond to different visual styles, making the distribution of multiple annotation types on the same SVG image more intuitive, facilitating batch annotation and subsequent verification of large-scale operator graphs.

[0035] In some embodiments, drawing a labeled area on the original SVG vector image and obtaining the position information of the labeled area and the attribute information corresponding to the labeled area includes: obtaining the pixel position of the mouse on the screen in response to a drag operation of the mouse on the original SVG vector image; inversely mapping the screen pixel position to logical coordinates in the global coordinate system according to the current canvas scaling factor and scroll bar offset; determining the position information of the labeled area according to the logical coordinates; drawing the labeled area on the original SVG vector image with a unified visual style according to the drag operation of the mouse; and obtaining the compiled attribute type and compiled attribute value through interactive operation, which are respectively used as the type and value of the attribute information.

[0036] The embodiments of this application can eliminate the reliance on color as an information channel, with annotation information determined by semantic content actively input by the user rather than by visual style. This mode takes into account the accessibility needs of researchers with visual impairments such as color blindness and color weakness; it breaks through the limitation of the limited number of color types (the human eye can usually distinguish no more than seven colors), and theoretically can support the expansion of an infinite number of attribute categories. When displaying thousands of annotation boxes simultaneously on a large scale, a unified visual style avoids visual clutter caused by rainbow color blocks and reduces the misreading rate.

[0037] In some embodiments, obtaining the compilation attribute type and compilation attribute value through interactive operation includes: obtaining the compilation attribute type through a drop-down menu or radio button before drawing the annotation area; or, after drawing the annotation area, popping up an attribute editing dialog box to obtain the compilation attribute type and compilation attribute value in response to a specified operation on the annotation area.

[0038] In some embodiments, before obtaining the original SVG vector graphic generated by the backend, the large model MLIR legend visualization and editing method further includes: the backend reading the original MLIR file; the backend converting the operator data flow graph in the original MLIR file into DOT text; and the backend calling a graphics rendering tool to render the DOT text into the original SVG vector graphic.

[0039] The embodiments of this application provide a specific implementation method for backend to generate raw SVG vector graphics. The backend first reads the raw MLIR file, converts the operator data flow graph in it into DOT text (a graphical description language), and then calls a graphics rendering tool (such as Graphviz) to render the DOT text into an SVG vector graphic. During this process, the graphics rendering tool automatically assigns independent coordinates in the global coordinate system to each operator node.

[0040] In some embodiments, after the labeled SVG file is sent to the backend, the large model MLIR legend visualization and editing method further includes: the backend parsing the labeled SVG file to extract the position information and attribute information of the labeled area; the backend comparing the position information with the coordinates of all operators in the original SVG vector data one by one to obtain the target operator node covered by the labeled area; and the first processing end injecting compilation attributes corresponding to the attribute information into the target operator node to obtain the target MLIR file.

[0041] The embodiments of this application, upon receiving the annotated SVG file returned from the front end, can directly parse the single file, extract the position information of the annotated areas and the compilation attribute information to be injected, and compare the geometric boundaries of the annotated areas in the global coordinate system with the global coordinates of each operator node in the original SVG vector data one by one, thereby automatically and accurately determining all target operator nodes covered by the annotated areas. Based on this, the back end automatically injects the corresponding compilation attributes into the target operator nodes and generates the target MLIR file, thus achieving a complete closed-loop process from graphical batch annotation to automatic MLIR compilation attribute injection. This process eliminates the need for developers to manually locate operators one by one in the MLIR source code or edit attributes at single points, significantly improving the processing efficiency of large-scale batch configuration of operator attributes. Simultaneously, since the front end and back end interact and match coordinates based on the global coordinate system within the same SVG file, it effectively avoids the problems of incorrect or missing operator binding caused by coordinate system transformation, string matching, or improper manual selection in traditional methods, ensuring the accuracy and consistency of compilation attribute injection.

[0042] In some embodiments, the step of comparing the location information with the coordinates of all operators in the original SVG vector image data one by one by the backend to obtain the target operator covered by the labeled area includes: traversing all operator nodes in the data corresponding to the original SVG vector image; for the currently traversed i-th operator node, performing the following determination: if the left boundary of the labeled area is less than the x-coordinate of the i-th operator node and the x-coordinate of the i-th operator node is less than the right boundary of the labeled area, and the upper boundary of the labeled area is less than the y-coordinate of the i-th operator node and the y-coordinate of the i-th operator node is less than the lower boundary of the labeled area, then it is determined that the i-th operator node is covered by the labeled area; after traversal, all covered operator nodes are determined as the target operator.

[0043] The embodiments of this application specify a coordinate primary key matching mechanism for the backend matching operator nodes. The coordinates of the operator node in the global coordinate system are used as the unique matching identifier, rather than relying on the operator name or temporary ID. The location information (geometric boundary) of the labeled area is compared with the coordinates of each operator node for coverage determination (rectangular collision detection). Operator nodes falling within the coverage area of ​​the labeled area are determined as matching nodes. Operator names may change under different compilation optimization levels; the embodiments of this application use unchanging coordinates as the matching basis to eliminate attribute binding errors.

[0044] Secondly, some embodiments of this application provide a method for visually editing large-scale MLIR legends. The method includes: a first processing end generating an original SVG vector image based on an original MLIR file; a second processing end drawing a labeled region on the original SVG vector image data and obtaining the position information of the labeled region and attribute information corresponding to the labeled region, wherein the position information represents the geometric boundary of the labeled region in the global coordinate system, and the attribute information represents a compilation attribute to be injected, the compilation attribute including a type name and a value; the second processing end writing the labeled layer data representing the labeled region, the position information, and the attribute information into an SVG file corresponding to the original SVG vector image to obtain a labeled SVG file, and sending the labeled SVG file to the first processing end; the first processing end parsing the labeled SVG file and extracting the position information and attribute information of the labeled region; the first processing end comparing the position information with the coordinates of all operators in the original SVG vector image data one by one to obtain the target operator covered by the labeled region; and the first processing end injecting the attribute information into the target operator to obtain a target MLIR file.

[0045] Thirdly, some embodiments of this application provide an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the large model MLIR legend visualization and editing method as described in any of the embodiments included in the first aspect.

[0046] Fourthly, some embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large model MLIR legend visualization and editing method as described in any of the embodiments included in the first aspect. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is an architecture diagram of the large model MLIR legend visualization and editing system provided in the embodiments of this application.

[0049] Figure 2A flowchart illustrating the visualization and editing method for large-scale MLIR legends provided in this application embodiment.

[0050] Figure 3 A schematic diagram of the provided electronic equipment components. Detailed Implementation

[0051] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0052] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0053] Unlike related inventions, the technical concept of this application's embodiments lies in using SVG vector graphics as the sole data carrier, decoupling the MLIR compilation backend from the visual editing frontend. The frontend and backend share the SVG's built-in global coordinate system throughout the process. After completing visual annotations on the SVG image, the frontend directly embeds the annotation information into the same SVG file using XML tags. The backend parses the SVG file, automatically matches operator nodes, and injects compilation attributes, forming a complete chain of image generation, visual annotation, and automatic attribute injection. This invention enables cross-platform collaborative editing, eliminates coordinate transformation errors, and allows a single file to carry all image and annotation information without maintaining additional scripts, avoiding manual coding errors and significantly improving the batch annotation efficiency of large-scale operator images.

[0054] Please refer to Figure 1 ,Should Figure 1 The present application provides an overall architecture for a large-scale MLIR legend visualization and editing system, comprising two main components: a Linux backend 110 and a Windows frontend 200. The Linux backend 110 and the Windows frontend 200 exchange data bidirectionally via SVG vector graphics files. In other words, the system provided in this embodiment adopts a frontend-backend separation dual-end collaborative architecture, comprising two subsystems: a Linux backend and a Windows frontend. The backend and frontend interact via SVG vector graphics files and annotated SVG files.

[0055] Linux backend 110 includes the MLIR graphics generation module (including...) Figure 1 The system includes a graphical compilation tool 111, a graphics rendering engine 112, an SVG parsing module 120, a coordinate matching module 130, and an attribute injection module 140.

[0056] Windows front-end 200 includes SVG rendering canvas module 210, annotation drawing module 220, coordinate mapping module 230, global state container 240, and document object model tree writing module 250.

[0057] First, we will explain the composition and data flow of the Linux backend.

[0058] The MLIR graphics generation module is used to convert the original MLIR file into an SVG vector graphic file. In this embodiment, the input to the MLIR graphics generation module is the original MLIR file, which is an intermediate representation file describing the large model operator data flow graph in text form. It contains various operator nodes and their data dependencies, but does not yet contain any graphical location information or visual layout information, nor any hardware binding attributes (such as `target`) or custom operator attributes (such as `custom_op_name`). For example, MLIR (Multi-Level Intermediate Representation) is an intermediate representation format in the LLVM compiler infrastructure used to represent the semantic information of a program during compilation. In large model compilation scenarios, the MLIR file carries the computational graph structure of the model, with each operator corresponding to an operation instruction, and the tensor flow between operators forming a data dependency graph.

[0059] The compilation and graphical tool 111 first reads the original MLIR file and converts the MLIR operator data flow graph into graphical description text. This graphical description text only describes the logical connection relationships between operator nodes and does not contain any coordinate position information of the nodes. Then, the MLIR graphics generation module calls the graphics rendering engine 112, which automatically calculates the geometric position of each operator node based on the logical relationships in the graphical description text and writes these positions into the SVG file corresponding to the SVG vector graphic in coordinate form. The generated SVG vector graphic has a built-in global coordinate system. Each operator node exists as an independent layer group label within the SVG, and each layer group label has a coordinate position attribute, which is used to record the horizontal and vertical coordinates of the operator node in the global coordinate system.

[0060] The output of the MLIR graphics generation module is a raw SVG vector graphic. This file retains the original coordinate information of all operator nodes. This raw SVG vector graphic file is transferred to the Windows front-end 200 via file transfer as the data basis for subsequent annotation and editing operations.

[0061] In an optional embodiment of this application, the MLIR graphics generation module may also output an operator coordinate mapping file. This operator coordinate mapping file is used to record the one-to-one correspondence between operator identifiers and SVG node coordinates to obtain the operator coordinate mapping file, which is then used by the Windows front-end 200 to assist in displaying the operator number so that the operator can locate it.

[0062] The SVG parsing module 120 is used to extract annotation information from the annotated SVG file for subsequent processing by the coordinate matching module 130.

[0063] In this embodiment, the input of the SVG parsing module 120 is an annotated SVG file returned by the Windows front-end 200. The annotated SVG file is a data file with embedded annotation information on the basis of the original SVG vector image, and it contains an independent layer dedicated to carrying the annotation information.

[0064] The SVG parsing module 120 uses XML path query language to locate and extract all rectangular graphic labels (corresponding to each annotation area) under the independent annotation layer, and reads the four geometric parameters of each rectangular graphic label: x-coordinate, y-coordinate, width, and height, as well as all custom attribute information. The SVG parsing module 120 does not perform graphic rendering operations during the extraction process; it only performs plain text-level data scraping.

[0065] The output of the SVG parsing module 120 is a set of labeled regions in memory, where each labeled region contains its geometric boundary data in the global coordinate system and the corresponding compilation attribute indication information. This set of data is then transmitted to the coordinate matching module 130.

[0066] The coordinate matching module 130 is used to match the geometric boundaries of the labeled regions with the operator nodes in the original MLIR file to determine the operator nodes covered by each labeled region.

[0067] In this embodiment, the input of the coordinate matching module 130 is two types of data: the first type of data comes from the set of labeled regions output by the SVG parsing module 120, where each labeled region contains its geometric boundary data in the global coordinate system; the second type of data is the global coordinates of each operator node in the original MLIR file. The coordinate matching module 130 obtains this data by triggering the memory layout calculation of the graphics rendering engine consistent with the MLIR graphics generation module, without outputting any file.

[0068] The coordinate matching module 130 constructs a hash key using the global coordinate values ​​of each operator node, establishing a one-to-one mapping table from coordinates to operator pointers. For each labeled region in the labeled region set, the coordinate matching module 130 performs a geometric coverage determination between the geometric boundary of the labeled region and all coordinate keys in the mapping table. The determination condition is: the operator's x-coordinate is greater than the left boundary of the labeled region and less than the right boundary of the labeled region, while the operator's y-coordinate is greater than the upper boundary of the labeled region and less than the lower boundary of the labeled region. All operator nodes that meet the above conditions are determined to fall within the coverage area of ​​the labeled region.

[0069] The output of coordinate matching module 130 is a list of operator nodes covered by each labeled region, and the matching result is sent to attribute injection module 140.

[0070] The attribute injection module 140 is used to inject compiled attributes into the matched operator nodes to generate a target MLIR file carrying the compiled attributes.

[0071] In this embodiment, the input of the attribute injection module 140 is the matching result output by the coordinate matching module 130, namely the list of operator nodes covered by each labeled area and its corresponding compilation attribute indication information.

[0072] For each matching result, the attribute injection module 140 writes the compilation attribute key-value pairs corresponding to the compilation attribute indication information of the annotation region into the attribute dictionary of the matched operator node through the MLIR abstract syntax tree operation interface. This injection process operates directly on the MLIR abstract syntax tree node in memory. For special nodes such as return type nodes, the attribute injection module 140 has a built-in backup matching branch, which automatically switches to the backup matching logic when the current node type is detected as a return type node.

[0073] The output of the attribute injection module 140 is the target MLIR file, in which each operator node covered by the marked area carries compilation attributes and can be directly used in the subsequent compilation process.

[0074] Secondly, it provides the composition and data flow of the Windows front end.

[0075] The SVG rendering canvas module 210 is used to load and display the original SVG vector graphics, providing users with a visual operator graph display interface.

[0076] In this embodiment, the input to the SVG rendering canvas module 210 is the original SVG vector image transmitted by the Linux backend 110 and an optional operator coordinate mapping file.

[0077] The SVG rendering canvas module 210 loads the original SVG vector image through the built-in SVG rendering control of the front-end graphical interface framework, rendering and displaying the operator data flow graph in the center of the canvas for user visualization and operation. If an operator coordinate mapping file is also imported, the SVG rendering canvas module 210 parses the mapping file and displays the identification information of each operator node on the canvas to assist the user in locating the target operator.

[0078] The output of the SVG rendering canvas module 210 is an interactive operator graph displayed on the user interface, which provides the display basis for the annotation drawing module 220.

[0079] The annotation drawing module 220 is used to draw annotation areas on the original SVG vector graphic and obtain the geometric boundary information and compilation attribute information of the annotation areas. In this embodiment, the input of the annotation drawing module 220 is the operator graph displayed by the SVG rendering canvas module 210 and the user's mouse interaction on the canvas. The annotation drawing module 220 provides multiple annotation modes for the user to choose from. In one embodiment, the annotation drawing module 220 provides a visual differentiation mode, where the user distinguishes different types of compilation attributes by selecting different visual styles. In another embodiment, the annotation drawing module 220 provides a semantic differentiation mode, where the user assigns compilation attribute information to the annotation area through pre-selection via a drop-down menu before drawing or input via a right-click pop-up after drawing, and all annotation boxes adopt a uniform visual style.

[0080] It should be noted that, in the embodiments of this application, compilation attributes refer to compilation optimization information specified by the user through front-end visual annotation operations and injected into the MLIR operator node by the back-end. These attributes consist of key-value pairs of attribute types and values, used to guide the compiler to perform hardware binding, custom operator replacement, or specific optimization strategies for specific operators during the code generation phase. For example, when the attribute type is `target` and the value is `GPU`, it indicates that the operator is bound to the graphics processor for execution, represented in the MLIR code as `{target = "GPU"}`; when the attribute type is `custom_op_name` and the value is `MySuperAdd`, it indicates that the operator is replaced with a developer-defined addition implementation, represented in the MLIR code as `{custom_op_name = "MySuperAdd"}`; when the attribute type is `quantize_bit` and the value is 8, it indicates that the operator is quantized to INT8 precision for execution, represented in the MLIR code as `{quantize_bit = 8}`. The type and value of the compilation attributes are assigned on the front-end through visual style selection or interactive operations, and written into the operator attribute dictionary on the back-end through the MLIR abstract syntax tree operation interface, ultimately outputting with the target MLIR file. The types of compilation attributes can be flexibly added according to the actual extension needs of the backend compiler, and are not limited to the specific examples mentioned above. Equivalent extensions made to other optimization dimensions based on the core concept of this invention are all within the protection scope of this invention.

[0081] When a user drags the mouse to draw an annotation area on the SVG rendering canvas module 210, the annotation drawing module 220 forwards the mouse's pixel position on the screen, the current canvas scaling factor, and the scroll bar offset to the coordinate mapping module 230. The coordinate mapping module 230 calculates the logical coordinates in the global coordinate system based on the above parameters and returns them. The annotation drawing module 220 determines the geometric boundary of the annotation area based on the initial logical coordinates determined when the mouse is pressed and the current logical coordinates updated in real time during mouse movement, and draws the annotation area on the user interface canvas in real time according to the visual style corresponding to the selected mode. When the mouse is released, the annotation drawing module 220 determines the final annotation area and transmits the geometric boundary information of the annotation area and the corresponding compilation attribute information to the global state container 240 for storage management.

[0082] The coordinate mapping module 230 is used to inversely map the pixel position of the mouse on the screen to logical coordinates in the SVG global coordinate system in real time. In this embodiment, the input of the coordinate mapping module 230 is the screen pixel position of the mouse event, the scaling factor of the current canvas, and the scroll bar offset. The coordinate mapping module 230 first reads the horizontal scroll bar offset value, the vertical scroll bar offset value, and the view scaling factor of the current canvas, and then performs the inverse mapping operation. For example, the logical horizontal coordinate is obtained by adding the mouse screen horizontal coordinate to the horizontal scroll offset and dividing by the scaling factor; the logical vertical coordinate is obtained by adding the mouse screen vertical coordinate to the vertical scroll offset and dividing by the scaling factor. The coordinate mapping module 230 also compares the calculated logical coordinates with the boundary of the original SVG vector image. If the logical coordinate is outside the valid drawing area, the current mouse position is ignored and no annotation area is generated. If it is within the valid drawing area, the relevant annotation area is drawn according to the logical coordinate.

[0083] The output of the coordinate mapping module 230 is the logical coordinates of the labeled area in the SVG global coordinate system, including the horizontal coordinate, vertical coordinate, width and height. This coordinate data is transmitted to the annotation drawing module 220 to determine the position information of the labeled area.

[0084] The global state container 240 is used to store and manage all currently active annotation region data and operation history. In this embodiment, the input to the global state container 240 is the structured annotation region data transmitted by the annotation drawing module 220. Each annotation region includes its geometric boundaries in the global coordinate system, as well as its compiled attribute type and value. The global state container 240 maintains a list of currently active annotation regions and a history stack in memory to support undo and redo operations. Each time a user adds, modifies, or deletes an annotation region, the global state container 240 pushes a complete snapshot of the current annotation region list onto the undo history stack; when a user triggers an undo command, the global state container 240 rolls the annotation region list back to the historical state stored in the undo history stack; when a user triggers a clear command, the global state container 240 clears the current annotation region list and the corresponding history stack.

[0085] The output of the global state container 240 is a list of currently active annotation regions and their corresponding compilation attribute information. This data is transmitted to the document object model tree writing module 250 for persistent storage.

[0086] The Document Object Model Tree (DOM) writing module 250 is used to embed the annotation region data into the SVG file corresponding to the SVG vector image, generating an annotated SVG file. In this embodiment, the input to the DOM writing module 250 consists of two types of data: the first type is the complete text string of the original SVG vector image file, and the second type is the list of currently active annotation regions output by the global state container 240.

[0087] The Document Object Model (DOM) tree writing module 250 parses the text string of the original SVG vector file into a complete XML document tree using an embedded XML DOM parser. The root node of the document tree is the SVG element. Then, starting from the root node, it recursively traverses all child nodes under the SVG root node, locating the last child node under the root node. Next, a new layer group node is dynamically created before the closing tag of the SVG root node, and its identifier attribute is forcibly set as an independent layer to carry annotation information. If a layer with the same name already exists in the original SVG, the existing layer is deleted before creating a new layer.

[0088] Subsequently, the Document Object Model (DOM) tree writing module 250 traverses the list of labeled regions in the global state container 240. For each labeled region in the list, it extracts its x-coordinate, y-coordinate, width, height, and compiled attribute type and value, and concatenates them into a complete rectangular graphic tag according to SVG standard syntax. After traversing the entire list of labeled regions, all rectangular graphic tags are treated as a complete code block and attached to a newly created independent layer node. Finally, the modified complete XML document tree is overwritten using a file stream.

[0089] The output of the Document Object Model Tree writing module 250 is an annotated SVG file. This annotated SVG file adds an independent layer to the original SVG vector image to carry annotation information. In this embodiment, all graphic data and annotation information are carried in a single file. This file is transmitted to the SVG parsing module 120 of the Linux backend 110 via file return for use in the subsequent parsing and injection process.

[0090] The following is combined with Figure 1 The arrows in the diagram illustrate the complete data flow of this system.

[0091] First, in the forward data stream, the raw MLIR file is input to the MLIR graphics generation module on the Linux backend 110. The MLIR graphics generation module converts the raw MLIR file into graphic description text, and then calls the graphics rendering engine to render it into a raw SVG vector graphic. This raw SVG vector graphic is then transferred to the Windows frontend 200 via file transfer.

[0092] Then, the original SVG vector graphic enters the Windows front-end 200. The original SVG vector graphic is loaded by the SVG rendering canvas module 210 and displayed on the user interface. The user draws annotation areas by dragging the mouse in the annotation drawing module 220. The annotation drawing module 220 forwards the mouse operation to the coordinate mapping module 230 to obtain logical coordinates in the global coordinate system. The annotation drawing module 220 transmits the annotation area data to the global state container 240 for storage management. After the user completes all annotations, a save command is triggered. The document object model tree writing module 250 reads the annotation area list from the global state container 240, serializes the annotation data into graphic labels, embeds them into the original SVG vector graphic file, and outputs an annotated SVG file.

[0093] Finally, in the reverse data stream, the annotated SVG file is transmitted to the Linux backend 110 via file return. The annotated SVG file enters the SVG parsing module 120, which extracts the set of annotated regions and their corresponding compilation attribute indication information, and inputs the extraction results into the coordinate matching module 130. The coordinate matching module 130 uses operator coordinates as unique matching identifiers to perform geometric coverage determination between the annotated regions and operator nodes, and inputs the matching results into the attribute injection module 140. The attribute injection module 140 injects compilation attributes into the matched operator nodes through the MLIR abstract syntax tree operation interface, and outputs the target MLIR file carrying the compilation attributes.

[0094] Through the aforementioned data flow, this system completes a fully automated pipeline from raw MLIR to graphic generation, visual annotation, automatic attribute injection, and target MLIR output. The entire process involves data exchange between the two processing ends using only two files: the raw SVG vector image and the annotated SVG file. No additional intermediate files or network communication protocols are required.

[0095] The following is combined with Figure 2 This application provides exemplary methods for visualizing and editing large-scale MLIR legends, based on some embodiments.

[0096] like Figure 2 As shown, some embodiments of this application provide a method for visualizing and editing large model MLIR legends, including:

[0097] S110, retrieve the original SVG vector graphic generated by the backend.

[0098] S120, draw a labeled area on the original SVG vector image (e.g., draw a labeled area on the original SVG vector image by dragging the mouse), and obtain the position information of the labeled area and the attribute information corresponding to the labeled area, wherein the position information represents the geometric boundary of the labeled area in the global coordinate system, and the attribute information represents the compilation attribute to be injected, and the compilation attribute includes a type name and a value.

[0099] In some embodiments of this application, the annotation area is drawn by dragging the mouse on the original SVG vector graphic.

[0100] In some embodiments of this application, the type name is used to describe what kind of compilation attribute is injected, and the value determines the specific content of the attribute. This distinction allows the front-end to simply select the type and enter the value during annotation, and the back-end can then complete the injection of compilation attributes for the corresponding operator based on the tuple.

[0101] In some embodiments of this application, the location information is represented by the geometric boundary parameters of the labeled area, such as the x-coordinate of the upper left corner, the y-coordinate of the upper left corner, the width of the rectangle, and the height of the rectangle. These four data items form the complete data basis for the backend to perform rectangle collision detection to match operator nodes.

[0102] S130, the annotation layer data representing the annotation area, the position information, and the attribute information is written into the SVG file corresponding to the original SVG vector image to obtain an annotated SVG file.

[0103] S140, the labeled SVG file is sent to the backend, so that the backend can match the corresponding operator node according to the labeled SVG file and inject the compilation attribute into the matched operator node to obtain the target MLIR file.

[0104] This application's embodiments directly inject annotation information into the SVG file to form a single, transferable file, avoiding version confusion and file loss issues caused by the separation of graphics, operator mapping, and editing scripts in traditional solutions. At the same time, the backend can directly parse annotations using the same SVG file without the need for additional coordinate transformation scripts, improving the automation level of the entire process. Furthermore, this application's embodiments can perform unified operations on operators that require batch modification of attribute information by drawing annotation areas, effectively improving the problem of extremely low efficiency in single-point operations.

[0105] In some embodiments of this application, the location information includes the horizontal coordinate, vertical coordinate, width, and height of the labeled area in the global coordinate system.

[0106] In the embodiments of this application, the location information is represented by the geometric boundary parameters of the labeled area, such as the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the width of the rectangle, and the height of the rectangle. These four data items form the complete data basis for the backend to perform rectangle collision detection to match operator nodes. By explicitly defining the location information as a quadruple and defining the labeled area with standard geometric parameters, the frontend and backend have a unified mathematical basis for determining the range of the labeled box, ensuring the consistency of coverage discrimination.

[0107] In some embodiments of this application, step S130, writing the annotation layer data representing the annotation area, the position information, and the attribute information into the SVG file corresponding to the original SVG vector image to obtain an annotated SVG file, includes: parsing the SVG file corresponding to the original SVG vector image into an XML document tree; creating an independent layer node inside the root node of the XML document tree, wherein the independent layer node is identified using a preset method that distinguishes it from the original graphic layer of the original SVG vector image, so that the backend parser can recognize the independent layer node; serializing the position information and the attribute information corresponding to the annotation area into graphic tags, wherein the graphic tags carry compilation attribute type indication information using a preset method corresponding to the type of the attribute information, so that the backend parser can determine the compilation attribute type to be injected based on the indication information; and attaching the graphic tags as child nodes to the independent layer node to obtain the annotated SVG file.

[0108] The embodiments of this application use XML DOM tree node appending to write to the annotation layer, ensuring that the operator coordinates, styles and layouts in the original SVG are not modified or re-encoded, thus achieving lossless coexistence of annotation information and original graphics; at the same time, the backend can quickly locate annotation data through fixed layer identifiers, improving parsing efficiency.

[0109] In some embodiments of this application, S130, drawing the annotation area on the original SVG vector graphic, includes:

[0110] The first step is to respond to mouse drag operations and obtain the pixel position of the mouse on the screen.

[0111] The second step is to read the current canvas scaling factor and scroll bar offset.

[0112] The third step is to reverse map the screen pixel position to logical coordinates in the global coordinate system based on the scaling factor and the scroll bar offset.

[0113] For example, in some embodiments of this application, the scroll bar offset includes a horizontal scroll bar offset and a vertical scroll bar offset. Therefore, the third step of inversely mapping the screen pixel position to logical coordinates in the global coordinate system includes: adding the horizontal coordinate of the screen pixel position to the horizontal scroll bar offset and dividing by the scaling factor to obtain the horizontal coordinate of the logical coordinate; adding the vertical coordinate of the screen pixel position to the vertical scroll bar offset and dividing by the scaling factor to obtain the vertical coordinate of the logical coordinate.

[0114] The inverse mapping formula of the embodiments of this application is: logical horizontal coordinate = (screen horizontal coordinate + horizontal scroll offset) / scaling factor, logical vertical coordinate = (screen vertical coordinate + vertical scroll offset) / scaling factor. This algorithm simplifies the complex inverse operation of affine transformation into addition and division, which has high computational efficiency and small accuracy loss.

[0115] The fourth step is to determine whether to generate the labeled area on the original SVG vector image based on the logical coordinates.

[0116] For example, in some embodiments of this application, the fourth step includes: comparing the logical coordinates with the boundary of the original SVG vector image; if the logical coordinates are outside the valid drawing area of ​​the original SVG vector image, then abandoning the determination of the logical coordinates as a vertex of the annotation area; if the logical coordinates are within the valid drawing area of ​​the original SVG vector image, then determining the logical coordinates as a vertex of the annotation area.

[0117] The embodiments of this application convert screen pixel coordinates into SVG native logical coordinates through real-time inverse mapping, eliminating coordinate system deviations caused by view scaling and translation, and ensuring that the stored annotation area coordinates are always aligned with the original graph0 coordinate system output by Graphviz, regardless of how the user's current view changes, thus avoiding the problem of misalignment between the annotation box and the operator node.

[0118] The embodiments of this application define the validity verification steps after inverse mapping. When a user drags the mouse outside the valid canvas area (e.g., excessive scrolling causes the field of view to exceed the SVG boundary), the logical coordinates obtained by inverse mapping may fall outside the valid drawing area. The embodiments of this application filter invalid coordinates through boundary comparison, avoiding the generation of invalid annotation areas that cannot match any operators. In other words, the embodiments of this application perform real-time boundary verification of coordinates during the drawing process, automatically filtering invalid annotations that exceed the valid SVG drawing area, preventing the writing of illegal coordinate data, and further improving the robustness of the file.

[0119] In some embodiments of this application, the large model MLIR legend visualization editing method further includes: storing the labeled regions in a global state container in memory, wherein the global state container includes a list of currently effective labeled regions and an undo history stack; and in response to an undo command, reverting the list of currently effective labeled regions to the historical state stored in the undo history stack.

[0120] It should be noted that, in the embodiments of this application, the currently effective annotation region list is used to store all annotation region data that the user has drawn in the current editing session and has not yet been undone or deleted. Each entry in the list corresponds to an annotation region, recording the geometric boundaries (horizontal coordinate, vertical coordinate, width, height) of the region in the global coordinate system and the corresponding compilation attribute information (type and value). All annotation regions in the list are serialized and written to an SVG file. The undo history stack is used to store complete snapshots of the annotation region list at each moment in the user's historical operations. After each time the user performs drawing, modification, or deletion operations, the system will make a complete copy of the currently effective annotation region list and push it onto the undo history stack. This stack is a last-in-first-out structure to support undo operations. When undoing, the most recent snapshot is popped from the top of the stack, restoring the annotation state to that moment.

[0121] The embodiments of this application manage annotation history through a global state container in memory, support undo operations on drawn annotation boxes, allow users to gradually roll back erroneous box selections, and improve the error tolerance and editing efficiency in the process of large annotations.

[0122] In some embodiments of this application, the global state container further includes a redo stack; the large model MLIR legend visualization editing method further includes: in response to a redo instruction, advancing the currently effective list of labeled regions to the historical state stored in the redo stack.

[0123] It should be noted that, in the embodiments of this application, the redo stack is used to store the historical state reverted by the undone operation. Each time a user performs an undone operation, the system pushes a complete copy of the previously undoed annotated area list onto the redo stack. This stack also follows a last-in-first-out (LIFO) structure to support redo operations. During a redo, the undone state is popped from the top of the stack, restoring the annotated state before undone. The currently active annotated area list represents the current editing state. Each time a user performs a valid operation, a complete copy of the current list is pushed onto the undone history stack, while the redo stack is cleared (because the previous redo path becomes invalid after a new operation branch is generated). During undone operations, the current list is first pushed onto the redo stack, and then a snapshot is popped from the undone history stack to restore the current list. During redo operations, the current list is first pushed onto the undone history stack, and then a snapshot is popped from the redo stack to restore the current list.

[0124] In the embodiments of this application, the redo stack and the undo stack are symmetrical, storing the undone historical state for recovery. Undo and redo constitute a complete backtrackable operation chain, allowing users to freely go back and forth in multiple trials and errors, greatly reducing the rework cost caused by misoperation.

[0125] In some embodiments of this application, step S120, drawing a labeled area on the original SVG vector image and obtaining the position information of the labeled area and the attribute information corresponding to the labeled area, includes: determining the target compiled attribute type of the current annotation mode in response to a selection operation among multiple visual styles; obtaining the pixel position of the mouse on the screen in response to a drag operation of the mouse on the original SVG vector image; inversely mapping the screen pixel position to logical coordinates in the global coordinate system according to the current canvas scaling factor and scroll bar offset; determining the position information of the labeled area according to the logical coordinates; drawing the labeled area on the original SVG vector image according to the visual style corresponding to the target compiled attribute type according to the drag operation of the mouse; using the target compiled attribute type as the type of the attribute information, and determining the value of the attribute information as a preset value or a user input value.

[0126] The embodiments of this application intuitively distinguish annotation types through visual styles. Users can identify the compilation attribute type bound to the current annotation by color or line type without additional operation, improving operational efficiency and visual recognition. Different compilation attribute types correspond to different visual styles, making the distribution of multiple annotation types on the same SVG image more intuitive, facilitating batch annotation and subsequent verification of large-scale operator graphs.

[0127] In some embodiments of this application, the step of drawing a labeled area on the original SVG vector image and obtaining the position information of the labeled area and the attribute information corresponding to the labeled area includes: in response to a mouse drag operation on the original SVG vector image, obtaining the pixel position of the mouse on the screen; inversely mapping the screen pixel position to logical coordinates in the global coordinate system according to the current canvas scaling factor and scroll bar offset; determining the position information of the labeled area according to the logical coordinates; drawing the labeled area on the original SVG vector image with a unified visual style according to the mouse drag operation; and obtaining the compilation attribute type and compilation attribute value through interactive operation, which are respectively used as the type and value of the attribute information.

[0128] The embodiments of this application can eliminate the reliance on color as an information channel, with annotation information determined by semantic content actively input by the user rather than by visual style. This mode takes into account the accessibility needs of researchers with visual impairments such as color blindness and color weakness; it breaks through the limitation of the limited number of color types (the human eye can usually distinguish no more than seven colors), and theoretically can support the expansion of an infinite number of attribute categories. When displaying thousands of annotation boxes simultaneously on a large scale, a unified visual style avoids visual clutter caused by rainbow color blocks and reduces the misreading rate.

[0129] In some embodiments of this application, obtaining the compilation attribute type and compilation attribute value through interactive operation includes: obtaining the compilation attribute type through a drop-down menu or radio button before drawing the annotation area; or, after drawing the annotation area, popping up an attribute editing dialog box to obtain the compilation attribute type and compilation attribute value in response to a specified operation on the annotation area.

[0130] In some embodiments of this application, before obtaining the original SVG vector graphic generated by the backend, the large model MLIR legend visualization and editing method further includes: the backend reading the original MLIR file; the backend converting the operator data flow graph in the original MLIR file into DOT text; and the backend calling a graphics rendering tool to render the DOT text into the original SVG vector graphic.

[0131] The embodiments of this application provide a specific implementation method for backend to generate raw SVG vector graphics. The backend first reads the raw MLIR file, converts the operator data flow graph in it into DOT text (a graphical description language), and then calls a graphics rendering tool (such as Graphviz) to render the DOT text into an SVG vector graphic. During this process, the graphics rendering tool automatically assigns independent coordinates in the global coordinate system to each operator node.

[0132] In some embodiments of this application, after the labeled SVG file is sent to the backend, the large model MLIR legend visualization and editing method further includes: the backend parsing the labeled SVG file to extract the position information and attribute information of the labeled area; the backend comparing the position information with the coordinates of all operators in the original SVG vector data one by one to obtain the target operator covered by the labeled area; and the first processing end injecting compilation attributes corresponding to the attribute information into the target operator to obtain the target MLIR file.

[0133] The embodiments of this application, upon receiving the annotated SVG file returned from the front end, can directly parse the single file, extract the position information of the annotated areas and the compiled attribute information to be injected, and compare the geometric boundaries of the annotated areas in the global coordinate system with the global coordinates of each operator node in the original SVG vector data one by one, thereby automatically and accurately determining all target operator nodes covered by the annotated areas. Based on this, the back end automatically injects the corresponding compiled attributes into the target operator nodes and generates the target MLIR file, thus realizing a complete closed-loop process from graphical batch annotation to automatic MLIR compiled attribute injection. This process eliminates the need for developers to manually locate operators one by one in the MLIR source code or edit attributes at single points, significantly improving the processing efficiency of large-scale batch configuration of operator attributes. Simultaneously, since the front end and back end interact and match coordinates based on the global coordinate system within the same SVG file, it effectively avoids the problems of incorrect or missing operator binding caused by coordinate system transformation, string matching, or improper manual selection in traditional methods, ensuring the accuracy and consistency of compiled attribute injection.

[0134] In some embodiments of this application, the step of comparing the location information with the coordinates of all operators in the original SVG vector image data one by one to obtain the target operator covered by the labeled area by the backend includes: traversing all operator nodes in the data corresponding to the original SVG vector image; for the currently traversed i-th operator node, performing the following determination: if the left boundary of the labeled area is less than the x-coordinate of the i-th operator node and the x-coordinate of the i-th operator node is less than the right boundary of the labeled area, and the upper boundary of the labeled area is less than the y-coordinate of the i-th operator node and the y-coordinate of the i-th operator node is less than the lower boundary of the labeled area, then it is determined that the i-th operator node is covered by the labeled area; after traversal, all covered operator nodes are determined as the target operator. I is an integer greater than or equal to 1.

[0135] The embodiments of this application specify a coordinate primary key matching mechanism for the backend matching operator nodes. The coordinates of the operator node in the global coordinate system are used as the unique matching identifier, rather than relying on the operator name or temporary ID. The location information (geometric boundary) of the labeled area is compared with the coordinates of each operator node for coverage determination (rectangular collision detection). Operator nodes falling within the coverage area of ​​the labeled area are determined as matching nodes. Operator names may change under different compilation optimization levels; the embodiments of this application use unchanging coordinates as the matching basis to eliminate attribute binding errors.

[0136] Some embodiments of this application provide a method for visually editing large-scale MLIR legends. The method includes: a first processing end generating an original SVG vector image based on an original MLIR file; a second processing end drawing a labeled region on the original SVG vector image data and obtaining the position information of the labeled region and the attribute information corresponding to the labeled region, wherein the position information represents the geometric boundary of the labeled region in the global coordinate system, and the attribute information represents a compiled attribute to be injected, the compiled attribute including a type name and a value; the second processing end writing the labeled layer data representing the labeled region, the position information, and the attribute information into an SVG file corresponding to the original SVG vector image to obtain a labeled SVG file, and sending the labeled SVG file to the first processing end; the first processing end parsing the labeled SVG file and extracting the position information and attribute information of the labeled region; the first processing end comparing the position information with the coordinates of all operators in the original SVG vector image data one by one to obtain the target operator covered by the labeled region; and the first processing end injecting the attribute information into the target operator to obtain a target MLIR file.

[0137] Example 1

[0138] To address the shortcomings of existing MLIR legend visualization tools, such as cross-platform fragmentation, lack of interactive batch annotation, multiple file separation, heavy environment dependencies, mismatched operator coordinates, and difficulties in team distribution, this invention provides a novel method for visualizing and editing large-scale MLIR legends. It achieves a decoupled architecture combining a Linux compilation backend with a lightweight Windows UI frontend, embedding annotation editing information into the SVG vector image carrier, unifying the original SVG coordinate system, and eliminating the need for additional script editing. The frontend has zero compilation dependencies, is available offline, and can be packaged as a standalone executable. The backend directly parses the embedded annotations in the SVG to automatically generate MLIR files with custom attributes, significantly shortening the iteration chain for large-scale operator tuning and reducing cross-platform collaboration costs.

[0139] The large-scale MLIR legend visualization and editing method provided in this application includes three major processes: Linux backend graphics generation step, Windows frontend interactive legend annotation step, and Linux backend SVG parsing and regenerating MLIR step. The overall architecture is a bidirectional file flow between the Linux compilation layer and the Windows visualization editing layer.

[0140] (1) Linux backend graphics generation module

[0141] The input to this process is the original large model MLIR file, which is an intermediate representation file that describes the large model operator data flow graph in text form. It contains various operator nodes and their data dependencies, but does not yet contain any graphical location information or visual layout information.

[0142] Step S1: Read the original large model MLIR file and execute the --view-op-graph command using the lego-graph compilation tool to convert the MLIR operator data flow graph into standard dot text. This dot text is a plain text format graph description language that only describes the logical connection relationship between operator nodes and does not contain any node coordinate position information.

[0143] The input for step S1 is the original large model MLIR file, which records all operator nodes and their data dependencies in text form, but does not yet contain any graphical layout information.

[0144] Step S1 processing: Read the MLIR file and call the `--view-op-graph` command built into the lego-graph compiler. This command traverses all operators in the MLIR module through the LLVM / MLIR standard interface, serializing their types, names, and data flow edges into plain text descriptions according to the Graphviz DOT language specification. These descriptions contain only logical connections and do not include coordinate positions.

[0145] Output of step S1: Standard DOT graphic description text (.dot file).

[0146] Step S2: Use Graphviz's dot tool to render the dot text into an original SVG vector image. The SVG has a built-in global graph0 coordinate system, preserving the original coordinates of all operator nodes.

[0147] Input for step S2: The DOT text output in step S1.

[0148] Step S2 involves invoking the `dot -Tsvg` command from the open-source Graphviz toolset. Graphviz's internal layout engine automatically calculates the geometric position of each operator based on the logical relationships within the DOT text and writes these positions into an SVG file in absolute coordinates. This SVG file contains a global `graph0` coordinate system (defined by the `viewBox` attribute), and each operator is positioned independently... <g>The tag exists, and its transform="translate(X,Y)" attribute explicitly records the precise x and y coordinates of the operator in graph0. This step completely reuses Graphviz's native capabilities, performing only pure rendering operations.

[0149] The output of step S2 is the original SVG vector file (graph.svg) with the global graph0 uniform coordinate system.

[0150] Step S3: Optionally execute the --emit-op-map command to output the op_map.json mapping file, which maps operator IDs to SVG node coordinates one-to-one.

[0151] Input for step S3: the original MLIR file or the DOT intermediate data from step S1.

[0152] Step S3: Invoke the `--emit-op-map` command of `lego-graph`. This command associates the node numbers returned by Graphviz with temporary pointers to the MLIR operators in memory, outputting a JSON file in key-value pair format, recording the correspondence between each operator ID and its coordinates in the SVG.

[0153] The output of step S3 is the op_map.json mapping file, which is used by the front-end UI to display operator numbers and is not a necessary dependency for subsequent annotation parsing.

[0154] It should be noted that in steps S1-S3 above, the Graphviz engine automatically calculates the geometric position of each operator node on the canvas based on the logical relationships between nodes and writes these positions into the SVG file in coordinate form. The SVG vector graphic has a built-in global graph0 coordinate system, which uses the viewBox attribute to define the mathematical ruler of the global canvas. Each operator node exists within the SVG as an independent g tag, and each g tag has a transform attribute. The translate(X,Y) value of this attribute explicitly records the precise coordinate position of the operator node in the global graph0 coordinate system. Therefore, the original SVG vector graphic output after Graphviz rendering already retains the original coordinate information of all operator nodes. In addition, this process can optionally execute the emit-op-map command, which outputs an op_map.json mapping file. This file records the one-to-one correspondence between each operator ID and its corresponding node coordinates in the SVG. This file is an optional auxiliary output used to display the operator number on the front-end UI panel for the operator to locate, but it is not a necessary dependency for subsequent annotation and parsing processes.

[0155] Step S4: Transfer the original SVG and op_map.json to the Windows client.

[0156] Input for step S4: the original SVG file generated in step S2 and the op_map.json file optionally generated in step S3.

[0157] Step S4: Copy the above files from the Linux server to the Windows local working directory via local network sharing, removable storage, or a secure file transfer protocol.

[0158] Output of step S4: A copy of the original SVG and op_map.json files stored on the Windows client.

[0159] The following is a summary of the relevant content of steps S1-S4 above.

[0160] The output of this process (including steps S1-S4 above) consists of two files: the original SVG vector file and an optional op_map.json mapping file. These two files are then transferred to the Windows client for use by the front-end interactive editing module.

[0161] In this invention, the calls to the `dot -Tsvg` command of Graphviz and the `--view-op-graph` command of lego-graph in steps S1-S3 above are both well-known existing compiler tool instruction sets in the art. Those skilled in the art can reproduce the graph generation process without creative effort based on the above instruction names and the MLIR standard interface.

[0162] In the SVG source code generated by Graphviz, each operator is an independent <g>Tags (groups) have an explicit `transform="translate(X,Y)"` attribute. Existing technologies set independent coordinates for each operator to improve layout aesthetics, prevent node overlap, and make the generated image clearer and easier for humans to read. The patented technology in this application aims to use the position coordinates of each operator (i.e., the visual coordinates used by humans in existing technologies) as the primary key of a machine-readable database. Existing technologies treat SVG only as a display; this invention treats SVG as a database. Using this independent absolute coordinate, the front-end drawing frame and the back-end parsing frame can use the same mathematical ruler, eliminating the need for complex coordinate affine transformation algorithms—this is the true underlying foundation of this invention.

[0163] (2) Windows lightweight interactive legend visualization editing module

[0164] The inputs to this process are the original SVG vector file generated in the previous process and the optional op_map.json mapping file. This process is implemented based on the pure PySide6 graphical interface library, which is a lightweight cross-platform Python graphical interface framework. The entire front-end module does not require the installation of LLVM or MLIR compilation dependencies and can run completely with only PySide6.

[0165] Implementing a local UI without LLVM compilation dependencies using pure PySide6, the execution steps are as follows:

[0166] Step T1: Install PySide6 dependencies in an offline environment, or package it into a standalone exe program using PyInstaller, without needing to pre-install Python.

[0167] Input for step T1: PySide6 offline installer (.whl format) or the UI Python script to be packaged.

[0168] Step T1: One of two parallel methods is used to ensure the UI is functional. Method 1: Download PySide6 and its dependent whl files to an internet-connected machine, copy them to an offline Windows machine, and then execute `pip install --no-index --find-links` for local installation. No internet connection is required throughout the process. Method 2: Use the PyInstaller tool to package the UI script and all its dependencies into a single .exe executable file. This file can be run by double-clicking on any Windows device without needing a Python environment.

[0169] Output of step T1: A PySide6 UI environment or a standalone exe program that can start normally. Step T2: Load the original SVG vector image; optionally, import op_map.json to display the operator's unique ID in the UI panel.

[0170] Inputs to step T2: the original SVG file (graph.svg) transferred to local storage in step S4, and optional op_map.json.

[0171] Step T2 processing: Launch the UI program, load graph.svg using the built-in SVG rendering control, and display it in the center of the canvas. If op_map.json is also imported, parse the JSON and display its unique ID associated with each operator node in the UI panel (e.g., pop up when the mouse hovers over it) to help the user locate the target operator.

[0172] The output of step T2 is: the rendered interactive visualization canvas and the operator ID lookup table loaded in memory.

[0173] Once T2 (the environment is ready) is executed, the UI program is launched and the original SVG vector graphic is loaded and rendered on the UI canvas. If the user has also imported the op_map.json file, the UI panel can display the unique ID of the corresponding operator next to each operator node or in the hover tooltip to help the user identify the operator.

[0174] Step T3: Interactive Legend Annotation and Canvas Manipulation (merging the original T3 and T4). This step integrates drawing annotations, view zooming, undoing, and clearing into the same interactive context. Since these four functions share the same event loop, the same canvas coordinate system, and the same annotation history data, forcibly separating them would sever their inherent logical connection.

[0175] Input for step T3:

[0176] (1) Mouse drag events, scroll wheel events, and keyboard shortcut commands on the canvas;

[0177] (2) The current annotation mode selected by the user through the toolbar.

[0178] Processing in step T3:

[0179] The purpose of this step is to transform the user's intuitive operations into standardized memory-labeled objects in real time. Its processing logic includes the following three tightly coupled and collaborative sub-modules.

[0180] First, a real-time inverse mapping module from screen coordinates to native SVG coordinates.

[0181] In this invention, view scaling and panning operations are not isolated, but directly serve the accurate drawing of the annotation box. When the user drags the mouse to draw a rectangle, the system does not directly record the screen pixels (ScreenX / Y) as in existing technologies (such as ordinary screenshot tools or front-end drawing boards), but forcibly builds a set of coordinate inverse operation logic. This coordinate inverse operation logic includes: after the system captures the pixel position of the mouse on the screen, it combines the current canvas scaling factor (ZoomScale) and scroll bar offset (ScrollOffsetX / Y), and uses division and subtraction operations (LogicalX = ScreenX / ZoomScale - ScrollOffsetX) to back-calculate the logical coordinates under the native SVG viewBox. [l3.1] This real-time inverse mapping module ensures that no matter how many times the user zooms in or out before drawing, the coordinates of the finally generated rectangle are always strictly aligned with the original graph0 mathematical ruler output by Graphviz. Compared with existing technologies, the latter often performs coordinate compensation only during export, and the compensation algorithm is complex and error-prone. This invention puts the inverse mapping in the real-time response of each mouse event, fundamentally eliminating the coordinate drift problem. The technical effect of this method is "what you see is what you get, and the storage is accurate". The box drawn by the user in any scaling state can be accurately placed on the original target operator when restored, without the need for secondary calibration.

[0182] Second, the module for generating multi-mode annotation boxes and binding attributes.

[0183] This module is responsible for instantiating the user's drag-and-drop action into a specific rectangle object. This invention is not limited to the two modes of red (target binding) and blue (custom operator), but supports unlimited expansion. Specific implementation schemes include, but are not limited to, the following two types: First, a multi-color expansion mode, which dynamically adds a third or more color annotation mode according to actual optimization needs. For example, green binds quantization precision, yellow binds memory level, purple binds fusion grouping, etc. Each color corresponds to a data-* custom attribute name, and the backend injects different MLIR attribute key-value pairs accordingly. Second, a colorless differentiation mode, which abandons the use of color visual differences to distinguish annotation types. Annotation semantics can be assigned using either a "mode pre-selection dropdown menu combined with a unified color box" or "manual input of key-value pairs via a right-click pop-up after drawing." Regardless of the interaction modality used by the frontend, each final output annotation rectangle is standardized into a structured data object containing x, y, width, and height coordinate attributes, as well as one or more data-* custom attributes (such as data-target="tx" or data-custom-op-name="MyOp"). This design ensures that the diversity of interactions in the front-end UI layer does not affect the universality of back-end parsing.

[0184] Third, the input, processing, and output definitions of the canvas manipulation assistance module are as follows:

[0185] The input data definition for this module is as follows: (1) Zoom command issued by the user via the mouse wheel (carrying the Delta value of the scroll wheel rotation angle); (2) Cancel command issued by the user via the menu or shortcut key (Boolean trigger signal); (3) Clear command issued by the user via the button (Boolean trigger signal); (4) Global label state container in the current memory (see the output definition below for details. This container is also the read and write object of this module, that is, the module needs to read its current state before operation).

[0186] The data structure definition for this module (i.e., the input source for all subsequent modules):

[0187] This invention defines a unique global annotation state container, denoted as GlobalAnnotationState, in the memory space of the UI main program. This container consists of the following three parts:

[0188] The first part is the list of currently active marked rectangles, ActiveRects, which is a dynamic array (like std::vector in C++). <annotationrect>Or, in Python, a List[Dict], where each AnnotationRect structure strictly contains five fields: the logical x-coordinate of the top-left corner (double), the logical y-coordinate of the top-left corner (double), the width of the rectangle (double), the height of the rectangle (double), and a custom attribute dictionary CustomProps (used to store key-value pairs such as {"target": "tx"} or {"custom_op_name": "MyOp"}).

[0189] The second part is the UndoStack, which stores historical states. Each element in the stack is a complete deep copy (i.e., a complete std::vector) of the ActiveRects list at a given historical moment. <annotationrect>(Snapshot)

[0190] The third part is the RedoStack, which has the same structure as the UndoStack and is used to support forward recovery operations.

[0191] The specific processing flow and output determination method for control commands in this module are as follows:

[0192] Operation 1: View Zooming. When the user scrolls the mouse wheel, the system does not modify any values ​​in the GlobalAnnotationState, i.e., it does not modify the logical coordinates stored in ActiveRects. The processing flow is as follows: the system only reads the current scroll wheel Delta value, recalculates the scaling factor in the transformation matrix of the UI canvas control (QGraphicsView), and calls the setTransform() method to update the UI rendering layer. This operation only changes the conversion ratio between pixels and logical coordinates, which is stored as an external environment variable in the UI context object. The result of this processing is that although the output data is not changed, the divisor (ZoomFactor) in the division operation in the coordinate inverse mapping module (i.e., sub-steps A to E) when drawing new rectangles is changed. Therefore, the "output" of the zooming operation actually updates the global environment variable CurrentZoomFactor, which can be read by the first module in step T3 at any time.

[0193] Operation 2: Undo. When the user clicks "Undo," the system executes the following defined process: First, check if UndoStack is empty. If empty, ignore the instruction and return directly. Second, if the stack is not empty, the system first pushes a deep copy of the current ActiveRects list onto RedoStack (for recovery). Third, pop a historical snapshot (i.e., a complete std::vector) from the top of the UndoStack. <annotationrect>The fourth step is to assign the current ActiveRects list as a snapshot of the history, which involves clearing the existing list and copying all AnnotationRect structures from the history snapshot item by item. The fifth step is to call the canvas's update() method to trigger a redraw event. The redraw function iterates through all rectangles in the current ActiveRects and renders dashed outlines on the UI canvas based on their coordinates. Through these five sub-steps, the undo operation reverts the ActiveRects list in GlobalAnnotationState to the previous history state.

[0194] Operation 3: Clear. When the user clicks "Clear," the system executes the following defined process: First, a deep copy of the current ActiveRects list is pushed onto the UndoStack (ensuring the clear operation itself can be undone). Second, the ActiveRects.clear() method is called to completely clear the currently active list of marked rectangles, releasing the memory occupied by all rectangle objects. Third, the RedoStack is cleared (because the history chain is broken, the data in the redo stack is invalid). Fourth, the canvas's update() method is called to trigger a redraw event. At this time, the canvas renderer traverses the empty ActiveRects list, and all marked boxes on the screen are erased.

[0195] The standardized output definition of this module (i.e., the data structure that serves as input to the next module) is as follows: Regardless of the manipulation performed above, after each processing step, this module instantiates the GlobalAnnotationState container in memory into a specific output result. This output result is stored in memory and specifically contains two independent output data streams.

[0196] The first output stream is a list of currently active labeled rectangles, named ActiveRects, which is of type std::vector. <annotationrect>Each AnnotationRect's X, Y, Width, and Height are strictly recorded as logical floating-point values ​​relative to the SVG native viewBox coordinate system (which is exactly the value written by the coordinate inverse mapping module in improvement point two). CustomProps stores key-value pairs of strings corresponding to the color mode or drop-down menu selection.

[0197] The second output stream is the history snapshot stack UndoStack (and RedoStack), which records the operation history for the input of the next undo command (i.e., the subsequent operation of this module depends on the current state of this stack).

[0198] Step T3 will be explained below with examples.

[0199] This process provides an interactive legend annotation canvas. The interaction logic of this canvas is based on a general annotation box drawing and attribute binding mechanism. Its specific implementation is not limited to specific color types, nor does it rely on color as a necessary visual means to distinguish annotation types. This invention supports the coexistence of multiple annotation modes, and users can flexibly select or expand annotation types according to actual optimization needs. The following provides a detailed description of the diverse implementation methods of annotation modes.

[0200] The first implementation is a multi-color legend annotation extension mode. In this mode, the annotation types are visually distinguished by rectangles of different colors, and the annotation types are not limited to two, but can be infinitely expanded according to the actual attribute dimensions in the large model operator optimization.

[0201] Specifically, in addition to the red target binding annotation mode and the blue custom operator annotation mode, the UI interface can dynamically add a third or more annotation modes. For example, the green annotation mode is used for quantization precision binding. After the user drags and draws a green dashed rectangle, all operators within the selected area will be marked as needing to bind quantization precision attributes, such as the number of quantization bits. The yellow annotation mode is used for memory level binding. After the user drags and draws a yellow dashed rectangle, all operators within the selected area will be marked as needing to bind memory access level attributes, which guides the compiler to allocate high-bandwidth or regular memory for the operator. The purple annotation mode is used for operator fusion marking. After the user drags and draws a purple dashed rectangle, all operators within the selected area will be marked as needing to bind fusion group identifier attributes, which tells the backend compiler to merge these operators into a single fused operator for execution.

[0202] In addition, the orange annotation mode can be extended to bind sparse computation attributes, the gray annotation mode to bind parallelism attributes, and the cyan annotation mode to bind data precision conversion attributes, depending on the specific needs of the compiler backend.

[0203] The front-end implementation of the aforementioned multi-color annotation mode involves adding mode switching buttons for corresponding colors to the UI toolbar, with each button associated with a specific attribute type. When a user selects a color button and drags to draw a rectangle, the system writes the custom data attribute name corresponding to that color when generating the rect label. Examples include data quant bit, datamem tier, and data fuse group, instead of uniformly writing it as data target or data custom op name.

[0204] The coordinate capture logic, i.e., the screen coordinate inverse mapping mechanism, and the DOM tree appending logic all share the same underlying code as the two-color mode, requiring no additional development. During backend parsing, the parsing rules for the `apply svg edits` directive are correspondingly expanded to dynamically map the `data` attribute name carried by the `rect` tag to different attribute key-value pairs in MLIR, while the coordinate matching logic—rectangle collision detection based on the coordinate primary key—remains unchanged. The technical advantage of this multi-color extension implementation is that it supports simultaneous annotation of multiple dimensions in large model optimization. For example, if the same operator requires both hardware targets and quantization precision, users can achieve batch injection of multiple attributes for the same area by overlaying multiple annotation boxes of different colors, avoiding multiple rounds of iterative annotation and further improving batch tuning efficiency.

[0205] The second implementation is a colorless differentiation mode. This mode completely abandons the approach of distinguishing annotation types through color visual differences, and is suitable for colorblind users, black-and-white printing scenarios, or scenarios that require simplification of visual interference. The idea behind this approach is to decouple the visual style of the annotation box from the semantic content of the annotation attributes. The annotation box only serves as a location marker, and its attribute binding information is assigned through interactive behavior rather than color. This approach specifically includes the following two parallel sub-implementation methods.

[0206] Sub-implementation 1 uses a mode pre-selection switch combined with a unified color frame scheme. In this scheme, the UI interface eliminates the distinction between multiple colors such as red, blue, and green, retaining only a single-color rectangle drawing tool, such as a black solid line frame or a gray dashed line frame. A drop-down selector or radio button group is added to the toolbar. Before drawing the rectangle, the user must first select the attribute type to be bound in the drawing from this drop-down list. Options include, but are not limited to, target binding, custom operator, quantization precision, memory level, fusion grouping, sparse attributes, and parallelism. After selecting the attribute type, the rectangle drawn by the user uses a single preset color, without conveying any semantic information through the border color. After drawing, the system binds the currently selected attribute type to the rectangle object in an invisible metadata manner. When saving, the rect tag still carries the corresponding data custom attribute, but the style attribute does not set a colored border; it is only set to no fill and the border is a uniform color. The technical advantage of this solution lies in that it breaks free from the limitation of the limited number of colors. The human eye can usually distinguish no more than seven colors, while this solution can theoretically support an infinite number of attribute categories through drop-down menus. At the same time, the visual interface is more concise and unified, avoiding the visual clutter caused by rainbow color blocks when displaying a large number of thousands of annotation boxes at the same time.

[0207] Sub-implementation scheme two involves a right-click menu to pop up attribute assignment after drawing. In this scheme, the user drags and draws any rectangle. All rectangles use the same visual style by default, without distinguishing color, line type, or transparency. After the user finishes drawing, double-clicking or right-clicking the rectangle brings up an attribute editing dialog box. This dialog box contains a drop-down menu for attribute names and an attribute value input box. The user can select predefined attribute keys such as target or custom opname from the drop-down menu, or freely enter custom key-value pairs in the input box, such as entering custom opname equal to MySuperAdd or target equal to GPU. After the user confirms, the system serializes the key-value pair into a data attribute and writes it to the rect tag. If the user performs a right-click double-click operation on the same rectangle again, new attribute key-value pairs can be modified or added, achieving the effect of the same annotation box carrying multiple different attributes simultaneously. The technical effect of this scheme is that it completely eliminates the dependence of color as an information channel, making the annotation information entirely determined by the semantic content actively entered by the user rather than the visual style, while giving a single annotation box the ability to bind multiple different attributes, further enhancing the expressive power of a single selection operation. In addition, the solution naturally takes into account the accessibility needs of researchers with visual impairments such as color blindness and color weakness, and complies with software accessibility design specifications.

[0208] The aforementioned multi-color extension mode and colorless mode are equivalent alternatives or extended implementations of the core interaction logic of the annotation canvas of this invention. Their underlying technical foundations—the screen coordinate inverse mapping mechanism, the DOM tree node appending mechanism, and the backend matching mechanism based on coordinate primary keys—remain unchanged. Regardless of the color scheme or interaction modality used by the front-end UI layer to obtain attribute key-value pairs, the rect tag structure ultimately stored in the SVG's internal Lego Edits layer maintains a unified format, containing x, y, width, and height coordinate attributes, as well as one or more custom data attributes. Therefore, the backend parsing module, i.e., the `apply svg edits` command, does not need any structural adjustments for the differences in front-end interaction methods. It only relies on the coordinate values ​​of the rect tags and the data attribute names for matching and injection, completely independent of the color used by the front-end or whether any color is used at all. This layered design gives the annotation canvas interaction logic of this invention excellent scalability and adaptability, enabling flexible adaptation to different user groups, different display environments, and potential future extension needs for new attribute types. During the drawing process, this workflow also provides auxiliary interactive operations such as view zooming, undoing the last annotation, and clearing all annotation boxes. These auxiliary operations are independent of the annotation mode selection. Regardless of the color scheme or interaction method used by the user, the view zooming, undoing, and clearing functions maintain consistent behavior logic, making it convenient for users to perform fine-grained editing of large-scale operator graphs.

[0209] Example 1: Two-color legend labeling

[0210] Red target annotation mode: Drag and draw a red dashed rectangle, and bind the target="tx" attribute to all operators within the selected area; Custom operator annotation mode: Drag and draw a blue dashed rectangle, and bind the custom_op_name custom operator name to all operators within the selected area.

[0211] In other words, in some embodiments of this application, users can switch between two annotation modes. The first is a red target annotation mode, in which the user drags the mouse to draw a red dashed rectangle, and all operators within the selected area will be marked as needing to be bound to the target attribute. The second is a custom operator annotation mode, in which the user drags the mouse to draw a blue dashed rectangle, and all operators within the selected area will be marked as needing to be bound to the custom_op_name custom operator name.

[0212] During the drawing process, this workflow also provides auxiliary interactive operations such as view zooming, undoing the last annotation, and clearing all annotation boxes, making it convenient for users to perform fine-grained editing of large-scale operator graphs.

[0213] Example 2: Multicolor Legend Annotation Extended Mode

[0214] As a replacement or supplement to the aforementioned red / blue dual-color annotation mode, this invention provides the following extended annotation method in the Windows front-end UI. Its underlying coordinate storage and parsing logic (i.e., the DOM tree injection mechanism in step T5 and the coordinate primary key matching mechanism in step U3) remains completely unchanged; only adaptive adjustments are made at the user interaction and attribute field binding levels:

[0215] In addition to red (target binding) and blue (custom operators), the UI can dynamically add a third or more color annotation modes. For example: Green annotation mode (quantization precision binding): Drag and draw a green dashed rectangle. All operators within the selected area are automatically injected with quantization precision attributes (such as {quantize_bit = "8"} or {quantize_bit = "4"}), which is used to specify whether the operator uses the INT8 or INT4 quantization strategy during inference; Yellow annotation mode (memory level binding): Drag and draw a yellow dashed rectangle. All operators within the selected area are automatically injected with memory access attributes (such as {mem_tier = "HBM"} or {mem_tier = "DDR"}), which is used to guide the compiler to allocate high bandwidth or regular memory for the operator; Purple annotation mode (operator fusion marker): Drag and draw a purple dashed rectangle. All operators within the selected area are automatically injected with fusion group ID attributes (such as {fuse_group = "group_1"}), which is used to tell the backend compiler to merge these operators into a single fusion operator for execution.

[0216] Technical implementation differences: The front-end UI only needs to add a mode switching button of the corresponding color to the toolbar, and when drawing the rectangle... <rect>Tags can be written with different custom attribute names (e.g., data-quant-bit, data-mem-tier, data-fuse-group). Their coordinate capture logic (screen coordinate inverse mapping) and DOM tree appending logic share the same underlying code as the two-color mode, requiring no additional development. During backend parsing, only the parsing rules of the `--apply-svg-edits` directive need to be extended, according to... <rect>The data-* attribute names carried by the labels are dynamically mapped to different attribute key-value pairs in MLIR, while the remaining coordinate matching logic (rectangle collision detection) remains unchanged. Beneficial effects: Supports simultaneous annotation of multiple dimensions of attributes in large model optimization (e.g., the same operator needs to specify both hardware targets and quantization precision), avoiding multiple rounds of iterative annotation and further improving batch tuning efficiency.

[0217] Example 3: Color-neutral mode, attribute binding based on interactive operations

[0218] To address the needs of colorblind users, black-and-white printing scenarios, or the need to simplify visual interference, this invention provides an alternative interaction scheme that does not rely on color differentiation, specifically including the following two implementation schemes.

[0219] Example 4: Mode pre-selection switch and single color box

[0220] The UI interface eliminates the distinction between multiple colors such as red, blue, and green, retaining only a rectangle drawing tool with a fixed color (such as a black solid line frame).

[0221] Add a drop-down selector (ComboBox) or radio button group to the toolbar. Before drawing the rectangle, the user can select the type of attribute to be bound from the drop-down list (options include: "Target", "CustomOp", "Quantization Precision", "MemTier", etc.).

[0222] During drawing, the system binds the currently selected attribute type to the rectangle object as invisible metadata, rather than distinguishing it by color style.

[0223] When saving, <rect>The tag still carries the corresponding data-* custom attributes (such as data-target="tx"), but the style attribute does not set a colored border, but is only set to fill:none;stroke:black;stroke-width:1;, to achieve complete decoupling of annotation information and visual style.

[0224] Example 5: After drawing, right-click to bring up the attribute assignment menu.

[0225] After a user drags and draws any rectangle (with a uniform color by default), double-clicking or right-clicking the rectangle will bring up a property dialog box. The dialog box contains an input box and a drop-down menu, allowing the user to manually enter key-value pairs such as target="tx" or custom_op_name="MyOp".

[0226] After user confirmation, the system serializes the key-value pair into a data-* attribute and writes it. <rect>Label.

[0227] In this mode, different annotation boxes have the same appearance, but the content of the data-* attributes is different. When the backend is parsing, it only performs differential injection based on the data-* attribute name and does not depend on any color information.

[0228] Technical Implementation Differences: In the two sub-implementation schemes described above, the screen coordinate inverse mapping algorithm (step T4) and the XML DOM tree appending mechanism (step T5) remain completely unaffected. The only change is in the way the front-end UI layer obtains the "attribute key-value pairs" (from color mode mapping to obtaining them from dropdown menus or pop-up input). The back-end parsing module (steps U2-U4) requires no structural changes because the back-end only reads... <rect>The tag's data-* attributes are used instead of reading the stroke (border color) style, so the backend code can 100% reuse the parsing logic of the two-color mode.

[0229] Beneficial effects: ① It takes into account the accessibility needs of R&D personnel with visual impairments such as color blindness and color weakness, and complies with software accessibility design specifications; ② It overcomes the limitation of limited color types (the human eye can usually distinguish no more than 7 colors), and theoretically can support the expansion of an unlimited number of attribute categories, without being unable to add new annotation modes due to color exhaustion; ③ The annotation boxes are uniformly monochrome, making the visual interface simpler. When displaying thousands of annotation boxes at the same time, it avoids visual clutter caused by rainbow color blocks and reduces the misreading rate.

[0230] View zooming and panning operations do not directly use screen pixel coordinates to record the position of the annotation box. Instead, they use a built-in reverse mapping mechanism from screen coordinates to the native viewBox coordinates of SVG: when the mouse drags to draw a rectangle, the system obtains the pixel position of the mouse on the screen (Screen X / Y), combines it with the current canvas zoom scale and scroll offset, and uses division and subtraction operations to back-calculate the logical coordinates (Logical X / Y) in the SVG global coordinate system. This ensures that no matter how the interface zoom factor changes, the coordinates of the annotation rectangle stored inside the SVG are always consistent with the original graph0 coordinate system output by Graphviz.

[0231] The following exemplarily illustrates the reverse mapping process from screen coordinates to SVG native logical coordinates in an embodiment of this application.

[0232] When a user drags the mouse to draw a marker rectangle on the canvas, the system executes five processing steps in the following sequence to convert the screen pixel coordinates into logical coordinates in the native SVG coordinate system in real time.

[0233] The first step is to capture the viewport pixel coordinates of the mouse event. In the mouse drag event response function, the system calls the coordinate acquisition interface provided by the event object to read the horizontal and vertical offsets of the mouse pointer relative to the origin of the upper left corner of the current canvas control's client area, denoted as ViewX and ViewY respectively, with the unit being device-independent pixels.

[0234] The second step is to read the current transformation parameters of the canvas view. Within the same event response cycle, the system synchronously reads the current offset values ​​of the horizontal scroll bar (H_Scroll), the vertical scroll bar (V_Scroll), and the current view scaling factor (ZoomScale) of the canvas control. H_Scroll and V_Scroll represent the pixel offset of the top-left corner of the content viewport relative to the original top-left corner of the canvas, while ZoomScale represents the ratio of the current display size to the original size.

[0235] The third step is to calculate the absolute pixel coordinates of the mouse position relative to the top-left corner of the original SVG canvas. The system performs an addition operation, adding the ViewX obtained in the first step to the current horizontal scroll offset value H_Scroll to obtain AbsPixelX; and adding the ViewY to the current vertical scroll offset value V_Scroll to obtain AbsPixelY. This operation eliminates the coordinate translation effect caused by the scroll bar offset, restoring the absolute pixel position of the pixel pointed to by the mouse in the complete canvas coordinate system when it is not scaled.

[0236] The fourth step involves performing the inverse scaling operation to convert the absolute pixel coordinates to SVG logical coordinates. The system performs a division operation: dividing the AbsPixelX obtained in step three by the current view scaling factor ZoomScale to obtain LogicX; and dividing AbsPixelY by ZoomScale to obtain LogicY. This division operation is essentially the inverse operation of the scaling component in the affine transformation matrix, and the result is the precise floating-point coordinate value of the mouse pointer in the native SVG viewBox coordinate system output by Graphviz. All intermediate variables are stored as double-precision floating-point numbers, and integer truncation or rounding is prohibited during the calculation process to ensure that the coordinate precision is completely aligned with the original Graphviz output.

[0237] Step 5: Validation and Data Storage. The system compares the LogicX and LogicY calculated in Step 4 with the viewBox boundary attributes of the original SVG file. If the coordinates are within the valid drawing area, the system uses these coordinates as the logical starting point of the top-left corner of the currently drawn rectangle and writes them into the ActiveRects list of the global annotation state container in memory for use in subsequent saving steps. If the coordinates are outside the valid drawing area, the system ignores the current mouse position and does not generate any annotation rectangle objects.

[0238] By executing the above five steps sequentially, the system ensures that the coordinates of the rectangle generated by each mouse drag drawing operation remain strictly consistent with the original graph0 coordinate system output by Graphviz. Regardless of how many times the user zooms or scrolls on the front-end interface, the coordinate values ​​stored in memory and ultimately written to the SVG file do not require secondary conversion or correction. This processing logic is fundamentally different from existing technologies that only perform a one-time affine transformation compensation during the export stage. The inverse mapping of this invention occurs during the real-time response to each mouse event, avoiding accumulated errors and coordinate drift problems.

[0239] Step T4: Save the edited file and add a separate element within the original SVG. <g class="lego-edits">Layer, set all labeled rectangles to standard <rect>The tags are embedded in the SVG, with rectangular tags carrying custom attributes `data-target / data-custom-op-name`. The original coordinates of `graph0` in the SVG are reused, and the output is a `graph_edited.svg` file with embedded annotations. Notably, the save operation does not use the conventional `QImage.save()` bitmap overwrite method of graphics libraries. Instead, it reads the original SVG file using an embedded XML Document Object Model (XML DOM) parser and locates the desired coordinates. <svg> The position of the last child node under the root node, in< / svg> Dynamically created before closing tags <g id="lego-edits">Nodes and serialize all rectangular labels to <rect>The string is attached to this node, and finally, a file stream overwrite operation is performed. This method ensures that all vector nodes, coordinate data, and styles of the original image are not re-encoded or destroyed; only DOM tree nodes are appended.

[0240] The input for step T4 consists of two parts. The first part is the complete text string of the original SVG file, i.e., all the characters in graph.svg. This string was read and cached in the UI program's memory when the original SVG was loaded in step T2. This step directly references this cached copy and does not need to read it from disk again. The second part is the list of currently active annotation rectangles, ActiveRects, in the GlobalAnnotationState container defined in step T3. Each AnnotationRect structure in this list contains four double-precision floating-point numbers: X, Y, Width, and Height, as well as a custom attribute dictionary, CustomProps.

[0241] It is particularly important to emphasize that the UndoStack and RedoStack in step T3 are proprietary data within the step T3 module used to implement the undo / redo interactive function. Their lifecycle is limited to the interactive session of step T3, supporting repeated trial and error by the user. The save operation in step T4 only needs to solidify the user's currently confirmed final annotation result into the SVG file. Therefore, its input data source is strictly limited to the ActiveRects list. It does not read any historical snapshot data from the UndoStack and RedoStack, nor does it depend on the structure or content of this historical data. This design ensures that the result of the save operation always reflects the annotation state of the user's last confirmation, rather than a historical intermediate state, which meets the user's expectation of WYSIWYG (What You See Is What You Get).

[0242] The processing logic for step T4 is to employ a DOM tree node appending mechanism based on the XML Document Object Model. This mechanism differs from two commonly used methods in existing technologies. One existing method saves the annotation information as a separate edit.json file, which results in the graphics and annotations being separated into multiple files. This easily leads to file loss or version incompatibility issues during file transfer, version management, and team collaboration. Another existing method calls the graphics rendering API to save the canvas content as a PNG or JPG bitmap. This rasterizes the vector graphics, causing the original coordinate information to be permanently lost and preventing infinite scaling, completely negating the inherent advantages of the SVG vector format. To thoroughly resolve these shortcomings, this invention adopts the following specific processing flow.

[0243] The first step is to enable the embedded XML parser. This parser can be one of the built-in QXmlStreamReader and QXmlStreamWriter classes in the PySide6 framework, or it can be one of the xml.dom.minidom or xml.etree.ElementTree modules in the Python standard library. Those skilled in the art can choose any mature XML parsing solution based on their actual development environment. The system uses this parser to read the raw SVG string and parse it into a complete XML document tree. This document tree is represented in memory as a hierarchical node structure, with the root node being the SVG element.

[0244] The second step involves the system recursively traversing all child nodes under the SVG root node, starting from the root node of the document tree, and locating the position of the last child node under the root node. This location operation is implemented by calling the parser's `childNodes()` or equivalent interface to obtain a list of all child nodes of the root node, and then obtaining a reference to the last child node using the `list.lastChild` or `list.item(list.length-1)` method.

[0245] Third, the system dynamically creates a new `g` node before the closing tag of the SVG root node, i.e., after the last child node located in the second step. When creating this node, the system forcibly sets two attributes: its `class` attribute is set to `lego-edits`, and its `id` attribute is also set to `lego-edits`, thus creating a dedicated layer to hold annotation information. If a `lego-edits` layer with the same name already exists in the original SVG, the system first deletes the existing layer before creating the new one, ensuring that each save uses the current `ActiveRects` list and avoids old annotations remaining.

[0246] Fourth, the system starts a traversal loop, with the iterator pointing to the first element of the ActiveRects list, processing each element sequentially until the last. For each AnnotationRect structure in the list, the system extracts its four double-precision floating-point numbers (X, Y, Width, and Height), formats them into strings, and retains six decimal places to ensure consistency with the original coordinate precision of Graphviz. Simultaneously, it extracts all key-value pairs from the CustomProps dictionary, such as data-target equal to tx or data-custom-op-name equal to MyOp. Then, the system concatenates this data into a complete rect tag string according to SVG standard syntax. The x, y, width, and height attributes of this tag are filled with the corresponding coordinate values, the data attribute is filled with the key-value pairs extracted from CustomProps, and the style attribute is filled with the corresponding border color and line style based on the user's currently selected annotation mode, such as a red dashed line, a blue dashed line, or a solid line of a uniform color. After traversing the entire ActiveRects list, the system concatenates all generated rect tag strings into a complete XML code block.

[0247] Fifth, the system takes the complete XML code block generated in the fourth step as a collection of child nodes and attaches it to the lego-edits group node created in the third step, that is, sets the group node as its parent node.

[0248] The sixth step involves the system using the serialization interface provided by the XML parser to perform a file stream overwrite operation on the modified complete XML document tree. This re-outputs the document tree as a string and writes it to the original file path, overwriting the original file, or outputting it as a new file to a user-specified path. This technique ensures three key points: First, all vector nodes, coordinate data, and styles of the original image are completely preserved without being re-encoded or destroyed. Second, annotation information is embedded within the SVG's internal code, inseparable from the graphic, and no longer dependent on any external files. Third, the infinite scaling without distortion characteristic of the SVG vector format remains intact.

[0249] The output of step T4 is a graph_edited.svg file with an embedded annotation layer. This file contains a newly added layer with the gid "lego-edits". The coordinates of all rect tags under this layer directly reuse the original coordinate system of SVG's native graph0. From the point the coordinate inverse mapping module in step T3 writes the coordinates to the point of writing the DOM tree serialization in this step, no scaling, offsetting, or affine transformation processing is performed, ensuring the consistency of coordinates between the front-end and back-end. If the ActiveRects list is empty, the save module simply creates an empty layer with the gid "lego-edits" and does not write any rect tags.

[0250] (3) Linux backend SVG parsing and MLIR regeneration module

[0251] Step U1: Send the edited SVG file with embedded annotations back to the Linux server.

[0252] Input for step U1: the graph_edited.svg file generated by the Windows client.

[0253] Step U1 processing: Transfer graph_edited.svg from the local Windows machine to the Linux server project directory via file transfer protocol or shared storage.

[0254] The output of step U1 is a copy of the graph_edited.svg file stored on the Linux server.

[0255] Step U2: Call the lego-graph --apply-svg-edits command to read all rect annotation boxes under the lego-edits layer in the SVG.

[0256] Input for step U2: The graph_edited.svg file returned from step U1.

[0257] Step U2 processing: Invoke the `--apply-svg-edits` command of the `lego-graph` tool. This command integrates a lightweight XML parsing module. Instead of rendering the SVG, it uses XPath path lookup (` / / g[@id='lego-edits'] / / rect`) to specifically extract all elements under the `lego-edits` layer. <rect>Nodes. For each extracted rectangle, the tool reads its x, y, width, and height floating-point values, as well as all data-* custom attributes (such as data-target or data-custom-op-name), and temporarily stores them in a set of labeled rectangles in memory.

[0258] The output of step U2: a set of standardized labeled rectangles in memory (containing coordinate and attribute key-value pairs).

[0259] Step U3: Based on SVG native unified coordinate matching MLIR operator nodes, identify all operators covered by each annotation box.

[0260] Inputs to step U3: The set of labeled rectangles output from step U2; the original MLIR file.

[0261] Step U3 processing: This step abandons the existing technology's approach of relying on "operator name" or "temporary node ID" for string matching, because operator names may change under different compilation optimization levels, and temporary IDs are unstable in multiple runs. Existing technologies use a method of exporting JSON coordinates from the front end and performing affine transformation alignment on the back end, which is highly susceptible to errors in coordinate conversion leading to the injection of attributes into incorrect adjacent operators. This invention employs a "coordinate primary key matching algorithm," the specific processing logic of which is as follows: The back end first triggers a Graphviz memory layout calculation (no output file required) identical to steps S1 and S2, obtaining the absolute coordinates translate(X,Y) of each operator in the original MLIR in the graph0 coordinate system. The tool uses this (X,Y) value combination as a hash key to establish a one-to-one mapping table between coordinates and operator pointers. Subsequently, each labeled rectangle extracted in step U2 is traversed, and the rectangle boundary data is compared with all coordinate primary keys in the mapping table using pure numerical values ​​(rectangle collision detection): any rectangle whose operator center X is greater than the left boundary and less than the right boundary, and whose operator center Y is greater than the upper boundary and less than the lower boundary, is determined to fall within the coverage area of ​​the labeled box. This matching process only involves four arithmetic interval judgments of floating-point values, without any coordinate transformation, scaling correction, or affine transformation matrix operations.

[0262] Comparing the effects of existing technologies with those of this invention: Existing technologies, due to their use of separate storage of multiple files and secondary coordinate conversion, are not only time-consuming to process but also have a high error rate. In contrast, this invention simplifies the complex coordinate alignment problem to a pure numerical range comparison by using the same mathematical ruler from the native SVG graph0 throughout the front-end and back-end processes, resulting in extremely low computational load. The resulting technical effect is 100% operator matching accuracy, completely eliminating attribute binding errors caused by coordinate offset, and significantly reducing the time required for parsing and matching thousands of bounding boxes.

[0263] The output of step U3 is a list of pointers to the original MLIR operators covered by each labeled rectangle.

[0264] In other words, in the embodiments of this application, when matching MLIR operator nodes based on SVG native unified coordinates, the backend parser forces the use of each operator within the SVG. <g>The X and Y values ​​in the `transform="translate(X,Y)"` attribute of the tag serve as the unique primary key for this operator, rather than relying on operator names (such as `matmul`) or node IDs (such as `node1`) for fuzzy string matching. It identifies all operators covered by each bounding box: Rectangular collision detection is performed between the X, Y, Width, and Height of each bounding box and the center coordinates of all operators. Operators whose center X is greater than the left boundary and less than the right boundary of the bounding box, and whose center Y is greater than the top boundary and less than the bottom boundary of the bounding box, are considered to be covered by the bounding box. This coordinate primary key matching mechanism completely abandons the existing method of front-end outputting JSON coordinates and back-end performing a second affine transformation for alignment. Because the front-end and back-end share the same mathematical ruler as the native SVG graph0, no coordinate transformation or correction algorithm is required. (Corresponding to improvement point one: coordinates as unique primary key).

[0265] Step U4: Automatically inject the corresponding compilation attribute {target="tx"} or {custom_op_name="xxx"} into the matching operator.

[0266] The inputs to step U4 are: the mapping relationship between "marked rectangles → operator pointer list" output from step U3; and the data-* attribute key-value pairs extracted from each rectangle in step U2.

[0267] Step U4 processing: For each matching result, the value corresponding to data-target (e.g., "tx") is converted into the target attribute and written to the operator attribute dictionary using the LLVM / MLIR standard interface Operation::setAttr(StringRef name, Attribute value). Similarly, the value corresponding to data-custom-op-name (e.g., "MyOp") is converted into the custom_op_name attribute and written to the attribute dictionary. This injection process operates directly on the MLIR abstract syntax tree nodes in memory, rather than through text regular expression replacement, ensuring syntactic correctness and security. Furthermore, for special return nodes such as func.return (whose label structure differs from regular operators in the MLIR AST), the backend has a built-in special case fallback matching branch. When the current node type is detected as func.return, it automatically switches to the backup matching logic to avoid boundary node annotation failure.

[0268] The output of step U4 is an MLIR abstract syntax tree in memory that carries complete custom annotation attributes (operator nodes with new attributes).

[0269] Step U5: Output a brand-new optimized MLIR file with custom annotation attributes, completing the entire process from visual editing of large model operator legends to automatic injection of compilation attributes.

[0270] Input for step U5: The MLIR abstract syntax tree in memory after the modification in step U4.

[0271] Step U5 processing: The abstract syntax tree in memory is re-output as a plain text MLIR file through the MLIR standard serialization interface module->print(). During this process, all newly added attributes are written in a standardized manner before the closing parenthesis of the corresponding operator line.

[0272] The output of step U5 is a brand-new optimized MLIR file (output.mlir) carrying custom annotation attributes. This completes the fully automated pipeline from the original MLIR, graphic generation, visual annotation, and automatic attribute injection. No manual writing of any operator attribute code or maintenance of any intermediate JSON scripts is required. The entire editing and compilation information is carried through the flow of a single SVG file.

[0273] It should be noted that in some embodiments of this application, networked machines download the PySide6 whl offline package in batches, and offline Windows installs it through the local whl file without a network; the tool packaging and distribution scheme: PyInstaller packages the UI script into a single-file exe, which can be run directly without a Python environment; for scenarios where the node label returned by func.return does not match, the backend adds special case fallback matching logic to avoid label failure.

[0274] It is easy to understand that in the prior art, SVG coordinates are only used for visual rendering. In this invention, SVG coordinates serve as the primary key for data routing. To achieve this technical objective, the embodiments of this application need to introduce a screen logical coordinate inverse mapping algorithm and an XML DOM tree hierarchical injection mechanism. This is not a simple parameter configuration of existing drawing tools, but a fundamental functional reconstruction of the underlying data structure of the graphics file.

[0275] Compared with related technologies, the technical advantages of the embodiments of this application include:

[0276] 1) Dual-end decoupled architecture, eliminating barriers to cross-platform collaboration.

[0277] The Windows front-end only relies on the lightweight PySide6 graphics library, requiring no compilation of LLVM / MLIR, and can run on ordinary office computers; Linux focuses on compilation, graphics generation, and annotation parsing, with a clear division of labor between the front-end and back-end. Developers can complete visual annotations without logging into the Linux server, significantly reducing server resource consumption.

[0278] 2) The annotation information is embedded in the SVG carrier, enabling single-file circulation and easy version traceability.

[0279] All legend annotation rectangles are directly written into the internal layer of the SVG, eliminating the need for a separate JSON editing script. All editing information can be fully carried by transmitting a single SVG file, eliminating issues such as missing multiple files or version incompatibility. SVG vector graphics can be scaled infinitely without distortion, balancing both image viewing and annotation storage.

[0280] 3) Globally unified SVG original coordinate system, operator matching zero misalignment.

[0281] The front-end drawing of the annotation box and the back-end parsing of the rectangle both reuse the original coordinates of the SVG built-in graph0, without the need for coordinate transformation and affine correction. This completely solves the problem of operator binding errors caused by the inconsistency of the front-end and back-end coordinate systems, and the annotation accuracy is 100%.

[0282] 4) Batch selection of annotations significantly improves the efficiency of operator tuning for large models.

[0283] It supports drag-and-drop selection of batch operators for one-time attribute injection, replacing the existing single-point manual modification of MLIR source code; large model graphs with thousands of nodes can complete batch operator marking with only a few selection operations, reducing optimization iteration time by more than 70%.

[0284] 5) Lightweight deployment, available offline, and one-click distribution.

[0285] It supports offline dependency installation and packaging as a standalone .exe file, and can run on Windows devices without a network environment or Python environment; team distribution only requires sending a single .exe file, and newcomers can get started with zero configuration.

[0286] 6) Fully automated process, eliminating manual coding errors.

[0287] After visual annotation, the backend automatically parses and generates a new MLIR with attributes, eliminating the need for manual writing of operator attributes and avoiding compilation errors and inference anomalies caused by manual coding, omissions, or incorrect attribute writing.

[0288] 7) Compatible with the official native MLIR toolchain, without intrusive modifications.

[0289] It reuses the native Graphviz and lego-graph compilation tools of LLVM-MLIR, only adds SVG annotation parsing logic, does not modify the underlying MLIR syntax, is compatible with existing large model compilation pipelines, and has extremely low transformation and integration costs.

[0290] Example 2

[0291] This embodiment completes the legend visualization annotation based on the Falcon large model MLIR operator graph. The complete process is as follows:

[0292] Example Step 1: Linux generates the raw SVG and operator mapping file.

[0293] 1. Log in to the Linux server and navigate to the large model project directory: cd / home / work.

[0294] 2. Convert the input large model MLIR to a dot graph description file: install / bin / lego-graph --view-op-graph input.mlir > graph.dot.

[0295] 3. Graphviz renders the raw vector SVG: dot -Tsvg graph.dot -o graph.svg.

[0296] 4. Export operator ID mapping JSON (optional, used to display operator numbers in the UI panel): install / bin / lego-graph --emit-op-map input.mlir -o / dev / null 2> op_map.json.

[0297] 5. Copy graph.svg and op_map.json to your Windows local desktop.

[0298] Example Step 2: Interactive annotation of visual legends on the Windows front end.

[0299] 1. Environment preparation: For an online environment, run pip install PySide6; for an offline environment, use local whl packages for batch installation; for team distribution scenarios, simply double-click lego_graph_UI.exe to run.

[0300] 2. Launch the visual UI program:

[0301] Basic startup (loading graph only): python lego_graph_UI.py graph.svg.

[0302] Advanced startup (loading operator ID mapping): python lego_graph_UI.py graph.svg --op-mapop_map.json.

[0303] 3. Legend annotation operation:

[0304] Switch to Mode: Set Target (Red) mode, drag the mouse to draw a red dashed rectangle, select the matmul matrix multiplication operator, and automatically mark the target attribute.

[0305] Switch to Mode: Custom Op (Blue), drag the blue dotted box to select the add operator, and enter the custom operator name MyCustomOp.

[0306] Use the mouse wheel to zoom in on the large image; use Undo to undo incorrect selections; and Clear All to clear all annotations.

[0307] 4. Click "Save SVG" to generate the graph_edited.svg file with an embedded annotation layer. A new "lego-edits" group will be added inside the SVG. Example of a rectangular label:

[0308] xml

[0309] <g class="lego-edits" id="lego-edits">

[0310] <rect x="200" y="-250" width="100" height="80" class="lego-edit-target" data-target="tx" style="fill:none;stroke:red;stroke-width:2;stroke-dasharray:5,5" / >

[0311] <rect x="220" y="-150" width="80" height="60" class="lego-edit-custom" data-custom-op-name="MyCustomOp" style="fill:none;stroke:blue;stroke-width:2;stroke-dasharray:5,5" / >

[0312] < / g>

[0313] Example Step 3: Linux parses SVG annotations and generates a new MLIR with attributes.

[0314] 1. Transfer graph_edited.svg back to the Linux project directory.

[0315] 2. Execute the backend SVG parsing instructions:

[0316] bash

[0317] run

[0318] install / bin / lego-graph --apply-svg-edits --svg-file=graph_edited.svginput.mlir -o output.mlir

[0319] 3. The output file output.mlir automatically carries annotation attributes. Example operator code:

[0320] mlir

[0321] %0 = tosa.matmul %arg0, %arg1 {target = "tx"} : tensor<1024x1024xf32>, tensor<1024x512xf32> -> tensor<1024x512xf32>

[0322] %1 = tosa.add %0, %0 {custom_op_name = "MyCustomOp"} : tensor<1024x512xf32> -> tensor<1024x512xf32>

[0323] 4. For special nodes of func.return, the backend has built-in fallback matching logic to prevent boundary operator annotations from failing.

[0324] Example Step 4: Offline Deployment and Packaged Distribution

[0325] 1. Install PySide6 offline: On a machine connected to the internet, execute `pip download PySide6 -d C:\offline_packages`, and on an offline machine, execute `pip install *.whl --no-index --find-links .`.

[0326] 2. Package a standalone .exe file:

[0327] PowerShell

[0328] pip install PySide6 pyinstaller

[0329] pyinstaller --onefile --windowed --name lego_graph_UI lego_graph_UI.py

[0330] This generates a single-file .exe file in the dist directory, which can be run by double-clicking on any Windows system, without requiring a Python environment.

[0331] The innovative aspects of this invention include:

[0332] 1) Cross-platform decoupling dual-end architecture innovation. Some embodiments of this application place the heavy MLIR compilation logic entirely on the Linux backend, while the Windows frontend only implements a lightweight PySide6 graphical interface, completely abandoning the LLVM compilation dependency on the Windows side, thus solving the pain points of complex compilation environment and difficult deployment on the Windows side of existing tools.

[0333] 2) Innovative SVG Embedded Annotation Carrier. Some embodiments of this application abandon the independent JSON editing script scheme and embed the legend annotation rectangle as a native SVG sublayer into the vector image. A single file carries the graphic and all editing information, realizing integrated storage and transmission of graphics and annotations, and solving the problems of multiple file separation and version confusion.

[0334] 3) Globally unified SVG native coordinate matching mechanism. In some embodiments of this application, the front-end and back-end uniformly reuse the SVGgraph0 original coordinate system. The coordinates of the labeled rectangle do not need to be transformed twice. The back-end directly matches the operator nodes through coordinates, eliminating attribute binding errors caused by coordinate offset.

[0335] 4) Dual-mode visual batch legend selection and annotation. Some embodiments of this application distinguish between two types of annotation layers: target binding operators and custom operators, visually differentiated by red / blue boxes. It supports batch selection of operators and batch injection of compilation attributes, replacing the traditional single-point editing and manual source code modification mode, and greatly improving the efficiency of editing large-scale large model operator graphs.

[0336] 5) Independently packaged, zero-dependency front-end distribution solution. Some embodiments of this application use PyInstaller to package into a single-file EXE, independent of Python and third-party library dependencies, adapting to enterprise offline office and team rapid distribution scenarios. Existing MLIR visualization tools do not have integrated packaging and distribution capabilities.

[0337] 6) Fully automated SVG parsing and MLIR attribute injection pipeline. In some embodiments of this application, the backend-dedicated apply-svg-edits command directly parses SVG layer annotations and automatically generates MLIR files with custom attributes, creating a complete automated pipeline of "viewing images - annotation - generating new compiled files" without any manual intermediate operations.

[0338] It should be noted that the meanings of the relevant terms used in this application are as follows:

[0339] MLIR: Multi-level intermediate representation, a general-purpose compiled intermediate language for large model inference and operator optimization.

[0340] SVG: Scalable Vector Graphics, which this invention uses as an operator graph carrier and annotation storage carrier.

[0341] lego-graph: A dedicated compilation tool for Linux-based MLIR graph generation and SVG annotation parsing.

[0342] PySide6: A lightweight, cross-platform Python graphical interface library, the only dependency for Windows front-end UI.

[0343] op_map.json: A file mapping operator IDs to SVG node coordinates, used to help the UI locate operators.

[0344] The `target` attribute is an MLIR operator hardware binding flag used to specify the hardware on which the operator is deployed.

[0345] The `custom_op_name` attribute is a custom operator name flag used to bind custom operator implementations.

[0346] This invention is not limited to TOSA large model operator graphs; it can be extended to adapt to the visualization and editing of various MLIR dialect legends such as MHLO and Linalg. It can be extended to add legend annotation types such as multi-color annotation boxes, operator grouping, annotation text, and threshold markers. The backend can be extended to support PNG image embedded annotations and JSON script dual parsing modes, compatible with existing editing workflows. For ultra-large MLIR models with tens of thousands of nodes, SVG vector rendering has no resolution loss; the front-end UI supports canvas block loading to avoid lag when displaying large images; the backend SVG parsing uses a spatial rectangle interval fast matching algorithm, with parsing of thousands of annotation boxes taking less than 100ms, without affecting the efficiency of the compilation pipeline.

[0347] Some embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large model MLIR legend visualization and editing method as described in any of the embodiments included above.

[0348] like Figure 3 As shown, some embodiments of this application provide an electronic device 400, which includes a memory 410, a processor 420, and a program stored in the memory 410 and executable on the processor 420. When the processor 420 reads and executes the program via a bus 430, it implements the large model MLIR legend visualization and editing method as described in any of the above embodiments.

[0349] Processor 420 can process digital signals and may include various computing architectures. For example, it may be a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements multiple instruction set combinations. In some examples, processor 420 may be a microprocessor.

[0350] Memory 410 can be used to store instructions executed by processor 420 or data related to the execution of instructions. These instructions and / or data may include code used to implement some or all of the functions of one or more modules described in the embodiments of this application. The processor 420 of the embodiments of this disclosure can be used to execute the instructions in memory 410 to implement… Figure 2 The method shown. Memory 410 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memory well known to those skilled in the art.

[0351] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0352] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0353] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0354] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0355] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0356] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.< / g> < / rect> < / rect> < / g> < / rect> < / g> < / rect> < / rect> < / rect> < / rect> < / rect> < / annotationrect> < / annotationrect> < / annotationrect> < / annotationrect> < / g> < / g>

Claims

1. A method for visualizing and editing large-scale MLIR legends, characterized in that, The method for visualizing and editing the large-scale MLIR legend includes: Get the original SVG vector graphic generated by the backend; Annotated regions are drawn on the original SVG vector image, and the position information of the annotated regions and the attribute information corresponding to the annotated regions are obtained. The position information represents the geometric boundary of the annotated regions in the global coordinate system, and the attribute information represents the compiled attributes to be injected. The compiled attributes include type name and value. The annotation layer data, which represents the annotation area, the location information, and the attribute information, is written into the SVG file corresponding to the original SVG vector image to obtain an annotated SVG file; The labeled SVG file is sent to the backend, so that the backend can match the corresponding operator node according to the labeled SVG file and inject the compilation attribute into the matched operator node to obtain the target MLIR file.

2. The method for visualizing and editing large-scale MLIR legends as described in claim 1, characterized in that, The location information includes the horizontal coordinate, vertical coordinate, width, and height of the labeled area in the global coordinate system.

3. The method for visualizing and editing large-scale MLIR legends as described in claim 2, characterized in that, The step of writing the annotation layer data, which represents the annotation region, the location information, and the attribute information, into the SVG file corresponding to the original SVG vector image to obtain an annotated SVG file includes: The SVG file corresponding to the original SVG vector graphic is parsed into an XML document tree; An independent layer node is created inside the root node of the XML document tree. The independent layer node is identified using a preset method that distinguishes it from the original graphic layer of the original SVG vector image, so that the backend can recognize the independent layer node during parsing. The location information and attribute information corresponding to the labeled area are serialized into graphic tags. The graphic tags carry compilation attribute type indication information in a preset manner corresponding to the type of the attribute information, so that the backend can determine the type of the compilation attribute to be injected based on the indication information during parsing. The graphic tag is attached as a child node to the independent layer node to obtain the labeled SVG file.

4. The method for visualizing and editing large-scale MLIR legends as described in any one of claims 1-3, characterized in that, The step of drawing the annotation area on the original SVG vector image includes: In response to mouse drag operations, obtain the pixel position of the mouse on the screen; Read the current canvas scaling factor and scroll bar offset; Based on the scaling factor and the scroll bar offset, the screen pixel positions are inversely mapped to logical coordinates in the global coordinate system; Whether to generate the labeled area on the original SVG vector image is determined based on the logical coordinates.

5. The method for visualizing and editing large-scale MLIR legends as described in claim 4, characterized in that, The scroll bar offset includes a horizontal scroll bar offset and a vertical scroll bar offset, wherein, The step of inversely mapping the screen pixel position to logical coordinates in the global coordinate system includes: The horizontal coordinate of the logical coordinate is obtained by adding the horizontal coordinate of the screen pixel position to the horizontal scroll bar offset and then dividing by the scaling factor. The vertical coordinate of the logical coordinate is obtained by adding the vertical scroll bar offset to the vertical coordinate of the screen pixel position and then dividing by the scaling factor.

6. The method for visualizing and editing large-scale MLIR legends as described in claim 4, characterized in that, The step of determining whether to generate the labeled region on the original SVG vector image based on the logical coordinates includes: Compare the logical coordinates with the boundaries of the original SVG vector graphic; If the logical coordinates are outside the valid drawing area of ​​the original SVG vector image, then the determination of the logical coordinates as the vertex of the annotation area is abandoned; If the logical coordinates are located within the valid drawing area of ​​the original SVG vector image, then the logical coordinates are determined as a vertex of the labeled area.

7. The method for visualizing and editing large-scale MLIR legends as described in claim 1, characterized in that, The large-scale MLIR legend visualization and editing method also includes: The marked regions are stored in a global state container in memory, wherein the global state container contains a list of currently active marked regions and a revocation history stack; In response to the undo command, the list of currently active marked areas is rolled back to the historical state stored in the undo history stack.

8. The method for visualizing and editing large-scale MLIR legends as described in claim 7, characterized in that, The global state container also includes a redo stack; The large-scale MLIR legend visualization and editing method also includes: In response to a redo instruction, the list of currently active marked regions is advanced to the historical state stored in the redo stack.

9. The method as described in claim 1, characterized in that, The step of drawing a labeled region on the original SVG vector image and obtaining the position information of the labeled region and the attribute information corresponding to the labeled region includes: In response to a selection operation among multiple visual styles, determine the target compiled attribute type; In response to mouse dragging operations on the original SVG vector image, obtain the pixel position of the mouse on the screen; Based on the current canvas scaling factor and scroll bar offset, the screen pixel positions are inversely mapped to logical coordinates in the global coordinate system; The location information of the labeled area is determined based on the logical coordinates; Based on the mouse drag operation, the annotation area is drawn on the original SVG vector image according to the visual style corresponding to the target compiled attribute type; The target compiled attribute type is used as the type of the attribute information, and the value of the attribute information is determined to be a preset value or a user input value.

10. The method for visualizing and editing large-scale MLIR legends as described in claim 1, characterized in that, The step of drawing a labeled region on the original SVG vector image and obtaining the position information of the labeled region and the attribute information corresponding to the labeled region includes: In response to mouse dragging operations on the original SVG vector image, obtain the pixel position of the mouse on the screen; Based on the current canvas scaling factor and scroll bar offset, the screen pixel positions are inversely mapped to logical coordinates in the global coordinate system; The location information of the labeled area is determined based on the logical coordinates; Based on the mouse drag operation, the labeled area is drawn on the original SVG vector image with a uniform visual style; The compilation attribute type and compilation attribute value are obtained through interactive operations, and are used as the type and value of the attribute information, respectively.

11. The method for visualizing and editing large-scale MLIR legends as described in claim 10, characterized in that, The step of obtaining the compilation attribute type and compilation attribute value through interactive operation includes: Before drawing the marked area, obtain the compilation attribute type via drop-down menu or radio button; or, After the labeled area is drawn, in response to the specified operation on the labeled area, an attribute editing dialog box pops up to obtain the compilation attribute type and compilation attribute value.

12. The method for visualizing and editing large-scale MLIR legends as described in claim 1, characterized in that, Before obtaining the original SVG vector image generated by the backend, the large model MLIR legend visualization and editing method further includes: The backend reads the original MLIR file; The backend converts the operator data flow graph in the original MLIR file into DOT text; The backend calls a graphics rendering tool to render the DOT text into the original SVG vector graphic.

13. The method for visualizing and editing large-scale MLIR legends as described in claim 1, characterized in that, After sending the labeled SVG file to the backend, the large model MLIR legend visualization and editing method further includes: The backend parses the labeled SVG file and extracts the position information and attribute information of the labeled region; The backend compares the location information with the coordinates of all operators in the original SVG vector data one by one to obtain the target operator covered by the labeled area; The first processing unit injects compilation attributes corresponding to the attribute information into the target operator to obtain the target MLIR file.

14. The method for visualizing and editing large-scale MLIR legends as described in claim 13, characterized in that, The step of comparing the location information with the coordinates of all operators in the original SVG vector image data one by one by the backend to obtain the target operator covered by the labeled area includes: Iterate through all operator nodes in the data corresponding to the original SVG vector image; For the i-th operator node currently being traversed, the following judgment is performed: If the left boundary of the labeled area is less than the x-coordinate of the i-th operator node and the x-coordinate of the i-th operator node is less than the right boundary of the labeled area, and the upper boundary of the labeled area is less than the y-coordinate of the i-th operator node and the y-coordinate of the i-th operator node is less than the lower boundary of the labeled area, then it is determined that the i-th operator node is covered by the labeled area. After the traversal is complete, all covered operator nodes are identified as the target operator.

15. A method for visualizing and editing large-scale MLIR legends, characterized in that, The method for visualizing and editing the large-scale MLIR legend includes: The first processing unit generates the original SVG vector graphic based on the original MLIR file; The second processing end draws the annotation area on the original SVG vector data and obtains the position information of the annotation area and the attribute information corresponding to the annotation area. The position information represents the geometric boundary of the annotation area in the global coordinate system, and the attribute information represents the compilation attribute to be injected. The compilation attribute includes a type name and a value. The second processing end writes the annotation layer data representing the annotation area, the location information, and the attribute information into the SVG file corresponding to the original SVG vector image to obtain an annotated SVG file, and then sends the annotated SVG file to the first processing end; The first processing unit parses the labeled SVG file and extracts the position information and attribute information of the labeled region; The first processing unit compares the location information with the coordinates of all operators in the original SVG vector data one by one to obtain the target operator covered by the labeled area. The first processing unit injects the attribute information into the target operator to obtain the target MLIR file.

16. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the large model MLIR legend visualization and editing method as described in any one of claims 1 to 15.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the large model MLIR legend visualization and editing method as described in any one of claims 1 to 15.

Images