A real-time rendering and display system and method for UI interface layout information based on Hongmeng system

Abstract

The present application discloses a real-time rendering and display system and method for UI interface layout information based on the Hongmeng system, which relates to the field of human-computer interaction. The method includes obtaining the interface rendering data stream of the target application when the Hongmeng system is running, extracting the layout metadata set from the interface rendering data stream, determining a target element subset including percentage size attributes, nested scroll containers, and data binding relationships, and then generating a wireframe overlay layer with hierarchical transparency characteristics based on the target element subset. When a click operation of the tester on the wireframe overlay layer is detected, the hierarchical path information of the currently selected element, the pixel deviation rate between the actual rendering size and the design draft size, the text content overflow status mark, and the parent container constraint condition expression are displayed. The present invention provides a real-time rendering and display system and method for user interface layout information based on the Hongmeng system, which improves the efficiency of testers in repairing user interface problems.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a real-time rendering and display system and method for UI interface layout information based on the Hongmeng system. Background Art

[0002] With the diversification of smart terminal devices, application interfaces need to adapt to display devices with different screen sizes, resolutions, and aspect ratios. In existing technologies, due to the coupling of layout constraints and dynamic data binding, problems such as misaligned interface elements, truncated text content, and abnormal visual hierarchy overlap can easily occur when rendering on multiple devices, resulting in significant deviations from the original design.

[0003] Traditional adaptation and debugging methods rely on manual verification of each physical device, resulting in low visualization of layout structures and inefficient tracing of anomalies. This is particularly true for complex interfaces using percentage-based layouts and nested scrolling containers, where testers struggle to perceive the mapping between rendering data flows and layout constraints in real time. This results in delayed adaptation and optimization, making it impossible to effectively prevent display conflicts in multi-device environments. Summary of the Invention

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present application provides a real-time rendering and display system and method for UI interface layout information based on the Hongmeng system.

[0005] In the first aspect, the present application provides a method for real-time rendering and displaying UI interface layout information based on the Hongmeng system, the method comprising:

[0006] Obtain the interface rendering data stream of the target application when it is running on the Hongmeng system, and extract the layout metadata set including element type identification, layer depth value, absolute coordinate range, dynamic constraint expression and associated resource identifier from the interface rendering data stream through the underlying interface of the Hongmeng system;

[0007] Perform topological sorting and sensitive element tagging on the layout metadata collection to determine the target element subset including percentage size attributes, nested scroll containers, and data binding relationships;

[0008] Mapping the target element subset to the display interface in a spatial coordinate system to generate a wireframe overlay layer with hierarchical transparency. The color of the wireframe in the wireframe overlay layer changes gradually according to the hierarchical depth of the target elements in the target element subset according to a preset color scale rule, and the width of the wireframe is inversely proportional to the area of ​​the display area of ​​the target element.

[0009] An interactive layout analysis panel is generated on the side of the display interface. When the tester clicks on the wireframe overlay layer, it displays the hierarchical path information of the currently selected element, the pixel deviation rate between the actual rendering size and the design size, the text content overflow status mark, and the parent container constraint expression;

[0010] Based on the screen pixel density, resolution parameters and safe display area boundary information of the target device, the deviation between the actual rendering ratio of each target element in the target element subset and the preset standard ratio is calculated. When the deviation value exceeds the first preset threshold, an adaptation abnormality alarm message is generated and the potential area with display conflict is marked.

[0011] Preferably, the layer synthesis data of the frame buffer is intercepted through the graphics synthesis service interface of the Hongmeng system;

[0012] Parse the geometric vertex coordinates and texture mapping parameters of each drawing instruction in the layer synthesis data;

[0013] For elements containing dynamic constraints, extract the anchor connection relationship between them and their parent container and the relative weight distribution ratio;

[0014] Convert the parsing result into a structured data set containing a globally unique identifier, control type code, absolute coordinate range matrix, hierarchical depth index value, dynamic constraint condition logic expression and associated resource file feature hash value.

[0015] Preferably, a transparency decreasing gradient is set according to the layer depth value, so that each increase in layer depth corresponds to a decrease in layer transparency by a preset percentage;

[0016] Add periodic visual warning marks to target elements with text overflow. The display frequency of the warning mark is positively correlated with the amount of text overflow.

[0017] Draw a dotted warning boundary around the boundary of the target element with dynamic constraints. The interval between the dotted lines is dynamically adjusted according to the rate of change of the constraints.

[0018] The wireframe edges are smoothed using a multi-sampling anti-aliasing algorithm, and a text label containing the type abbreviation and unique identifier is superimposed and displayed at a preset position on each target element.

[0019] Preferably, when it is detected that the line height value of a text element is less than a preset multiple of the font size, it is marked as a line spacing compression abnormality;

[0020] For elements using absolute positioning, the offset distance between the center point coordinates and the ideal layout grid nodes is calculated. When the offset distance exceeds a second preset threshold, it is marked as a positioning deviation abnormality.

[0021] When the cumulative size of the sub-elements in the scroll container exceeds the preset multiple of the visible area size, a scroll performance degradation warning is triggered;

[0022] The scaling mode of the image element is detected, and when forced stretching is detected and the aspect ratio change exceeds a third preset threshold, it is marked as an image distortion abnormality.

[0023] Preferably, a wireless debugging channel is established with multiple Hongmeng test devices to obtain interface rendering data and layout analysis data of multiple Hongmeng test devices in real time;

[0024] Displaying comparison views of multiple HarmonyOS test devices side by side on the display interface, with the comparison views overlaying and displaying difference area marks of the wireframe layers of the multiple HarmonyOS test devices;

[0025] When a layout anomaly is detected on a specific device, the screen parameter set of the specific device is extracted, and historical optimization solutions are matched based on parameter similarity to generate layout adaptation suggestions.

[0026] Preferably, the visual focus coordinates of the test person are captured at preset sampling intervals by a visual acquisition device;

[0027] According to the mapping relationship between the visual focus coordinates and the wireframe, a first transparency adjustment strategy is implemented for the wireframe within a preset radius centered on the coordinates, and a second transparency adjustment strategy is implemented for the wireframe in other areas;

[0028] When it is detected that the tester's gaze time on the same element exceeds a first preset time threshold, the historical change record of the element's constraint conditions is displayed on the interactive layout analysis panel, and the corresponding layout source program position in the integrated development environment is located;

[0029] The interface layout complexity index is calculated based on the visual focus distribution data. When the index exceeds the preset complexity threshold, an optimization suggestion is generated to split the complex layout into atomic components.

[0030] Preferably, a version control database is established to record wireframe snapshot data, device adaptation reports, and corresponding source program change records after each layout modification;

[0031] When it is detected that the similarity between the current layout defect characteristics and historical cases exceeds the preset similarity threshold, the associated historical repair plan and verification test case set are pushed;

[0032] Build a layout defect prediction model based on graph neural network, input the current view tree structure data and device parameter set into the prediction model, and output the potential defect type identification and its confidence score;

[0033] For defect types whose confidence scores exceed the preset confidence threshold, a repair patch program containing constraint optimization rules and container logic improvement solutions is generated;

[0034] Verify the compatibility and performance improvement indicators of the repair patch in an isolated sandbox environment. After verification, deploy it to the target device through the hot update mechanism.

[0035] The second aspect is a real-time rendering and display system for UI interface layout information based on the Hongmeng system, including:

[0036] A metadata extraction unit is used to obtain the interface rendering data stream of the target application when running on the Hongmeng system, and extract a layout metadata set including element type identification, layer depth value, absolute coordinate range, dynamic constraint expression and associated resource identifier from the interface rendering data stream through the underlying interface of the Hongmeng system;

[0037] A target element subset determination unit is used to perform topological sorting and sensitive element tagging on the layout metadata set, and determine a target element subset including percentage size attributes, nested scroll containers, and data binding relationships;

[0038] A coordinate mapping unit is used to map the target element subset to the display interface in a spatial coordinate system, generating a wireframe overlay layer with hierarchical transparency characteristics, wherein the color of the wireframe in the wireframe overlay layer gradually changes according to the hierarchical depth of the target elements in the target element subset according to a preset color scale rule, and the width of the wireframe is inversely proportional to the area of ​​the display area of ​​the target element;

[0039] An analysis panel generation unit is used to generate an interactive layout analysis panel on the side of the display interface. When a click operation by the tester on the wireframe overlay layer is detected, the panel displays the hierarchical path information of the currently selected element, the pixel deviation rate between the actual rendering size and the design draft size, the text content overflow status mark, and the parent container constraint condition expression;

[0040] The warning unit is used to calculate the deviation between the actual rendering ratio of each target element in the target element subset and the preset standard ratio based on the screen pixel density, resolution parameters and safe display area boundary information of the target device. When the deviation value exceeds a first preset threshold, an adaptation abnormality alarm message is generated and a potential area with display conflict is marked.

[0041] Compared with the prior art, the present invention has the following characteristics and beneficial effects:

[0042] By extracting layout metadata collections from the interface rendering data stream in real time, the limitation of traditional debugging tools relying on manual program review is overcome, ensuring the acquisition of complete layout constraint relationships and hierarchical information, and providing a reliable data foundation for anomaly detection.

[0043] Based on topological sorting and sensitive element marking, a subset of target elements is screened, effectively focusing on key elements such as percentage sizes and nested containers with adaptation risks, avoiding resource waste of full analysis and significantly improving problem location efficiency.

[0044] The hierarchical wireframe overlay layer generated by spatial coordinate system mapping uses color gradients and dynamic line widths to intuitively present element distribution density and nesting depth, reducing the visual cognitive load of testers and enabling rapid understanding of layout structure.

[0045] Combined with the interactive analysis panel's real-time feedback mechanism, when a wireframe click is detected, the display element's deviation rate, constraints, and design draft comparison data are dynamically correlated, forming a closed-loop optimization process of "visual positioning - data diagnosis - program adjustment," significantly shortening the adaptation and debugging cycle. Furthermore, a dynamic proportional deviation calculation model based on device parameters can provide early warning of adaptation anomalies during the layout rendering phase. By marking potential conflicting areas and guiding targeted optimization, it prevents display issues such as interface deformation and content occlusion from the root, ensuring consistent interface rendering across multiple devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flowchart of a method for real-time rendering and displaying UI interface layout information based on the Hongmeng system, which is mainly reflected in this embodiment.

[0047] Figure 2 This is a structural block diagram of a real-time rendering and display system for UI interface layout information based on the Hongmeng system, which is mainly reflected in this embodiment. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below with reference to the following examples.

[0049] Reference Figure 1 A method for real-time rendering and displaying UI interface layout information based on the Hongmeng system includes the following steps:

[0050] S1. Obtain the interface rendering data stream of the target application when it is running on the HarmonyOS system, and extract a layout metadata set including element type identification, layer depth value, absolute coordinate range, dynamic constraint expression and associated resource identifier from the interface rendering data stream through the underlying interface of the HarmonyOS system.

[0051] S2. Perform topological sorting and sensitive element tagging on the layout metadata collection to determine a subset of target elements including percentage size attributes, nested scroll containers, and data binding relationships.

[0052] S3. Map the target element subset and the display interface to a spatial coordinate system to generate a wireframe overlay layer with hierarchical transparency characteristics, wherein the color of the wireframe in the wireframe overlay layer gradually changes according to the hierarchical depth of the target element in the target element subset according to a preset color scale rule, and the width of the wireframe is inversely proportional to the area of ​​the display area of ​​the target element.

[0053] S4. Generate an interactive layout analysis panel on the side of the display interface. When a click operation of the tester on the wireframe overlay layer is detected, the hierarchical path information of the currently selected element, the pixel deviation rate between the actual rendering size and the design draft size, the text content overflow status mark, and the parent container constraint condition expression are displayed.

[0054] S5. Based on the screen pixel density, resolution parameters and safe display area boundary information of the target device, calculate the deviation value between the actual rendering ratio of each target element in the target element subset and the preset standard ratio. When the deviation value exceeds the first preset threshold, an adaptation abnormality alarm message is generated and the potential area with display conflict is marked.

[0055] Specifically, first, by extracting layout metadata collections from the interface rendering data stream in real time, the limitations of traditional debugging tools that rely on manual program review are overcome, ensuring the acquisition of complete layout constraint relationships and hierarchical information, providing a reliable data foundation for anomaly detection. Secondly, based on topological sorting and sensitive element marking, a subset of target elements is screened, effectively focusing on key elements such as percentage sizes and nested containers that pose adaptation risks, avoiding the waste of resources in full analysis and significantly improving problem location efficiency. Furthermore, a hierarchical wireframe overlay layer generated by spatial coordinate system mapping intuitively presents element distribution density and nesting depth with color gradients and dynamic line widths, reducing the visual cognitive load of testers and enabling rapid understanding of layout structure. Combined with the real-time feedback mechanism of the interactive analysis panel, when a wireframe click operation is detected, the element deviation rate, constraint conditions, and design draft comparison data are dynamically associated and displayed, forming a closed-loop optimization process of "visual positioning-data diagnosis-program adjustment", which significantly shortens the adaptation and debugging cycle. In addition, the dynamic proportional deviation calculation model based on device parameters can provide early warning of adaptation anomalies during the layout rendering stage, guide targeted optimization by marking potential conflict areas, and prevent display problems such as interface deformation and content occlusion from the root, ensuring the consistency of interface rendering effects in multi-device environments.

[0056] The specific step S1 includes the following sub-steps:

[0057] Intercept the layer synthesis data of the frame buffer through the graphics synthesis service interface of the Hongmeng system;

[0058] Parse the geometric vertex coordinates and texture mapping parameters of each drawing instruction in the layer synthesis data;

[0059] For elements containing dynamic constraints, extract the anchor connection relationship between them and their parent container and the relative weight distribution ratio;

[0060] Convert the parsing result into a structured data set containing a globally unique identifier, control type code, absolute coordinate range matrix, hierarchical depth index value, dynamic constraint condition logic expression and associated resource file feature hash value.

[0061] Specifically, first, through the HarmonyOS graphics composition service interface, the framebuffer's layer composition data can be intercepted in real time, ensuring real-time information on interface rendering. Next, each drawing instruction in this layer composition data is parsed to extract geometric vertex coordinates and texture mapping parameters. These parameters help understand the drawing method and position of each element. For elements with dynamic constraints, the system can further extract the anchor connection relationship and relative weight distribution ratio between the element and its parent container. This information helps understand the dynamic adjustment behavior of the element in the interface. Finally, all these parsed results are converted into a structured data set containing a globally unique identifier, control type code, absolute coordinate range matrix, layer depth index value, dynamic constraint logical expression, and associated resource file feature hash values. This data set provides a solid foundation for subsequent layout analysis and anomaly detection. For example, in a real-world application scenario, the system intercepts the framebuffer's layer composition data, parses the geometric vertex coordinates and texture mapping parameters of a button element, and discovers an anomaly in the anchor connection relationship between the button element's dynamic constraint and its parent container. The system can then convert this information into a structured data set for subsequent analysis and processing.

[0062] The specific process of step S2 may be to topologically sort the layout metadata set and mark sensitive elements to determine a subset of target elements including percentage size attributes, nested scroll containers, and data binding relationships. Specifically, first, the system can topologically sort the layout metadata set to ensure that the hierarchical relationships and dependencies between elements are correctly handled. Then, the system can mark sensitive elements, which usually include elements with percentage size attributes, nested scroll containers, and data binding relationships. These elements have a higher adaptation risk in the interface layout. By screening out these subsets of target elements, the system can focus on key elements, avoid wasting resources for full analysis, and significantly improve the efficiency of problem locating. For example, in an actual application scenario, the system finds that the percentage size attribute of a nested scroll container is abnormal through topological sorting and sensitive element marking. The system can mark the container as part of the target element subset for further analysis and processing in subsequent steps.

[0063] The specific step S3 includes the following sub-steps:

[0064] Set the transparency decreasing gradient according to the layer depth value, so that each increase in layer depth corresponds to a decrease in the layer transparency by a preset percentage;

[0065] Add periodic visual warning marks to target elements with text overflow. The display frequency of the warning mark is positively correlated with the amount of text overflow.

[0066] Draw a dotted warning boundary around the boundary of the target element with dynamic constraints. The interval between the dotted lines is dynamically adjusted according to the rate of change of the constraints.

[0067] The wireframe edges are smoothed using a multi-sampling anti-aliasing algorithm, and a text label containing the type abbreviation and unique identifier is superimposed and displayed at a preset position on each target element.

[0068] Specifically, the system can set a decreasing transparency gradient based on the layer depth value, so that each increase in layer depth corresponds to a preset percentage decrease in layer transparency. This can intuitively display the nesting depth and distribution density of elements. Next, for target elements with text overflow, the system can add periodic visual warning markers. The frequency of the warning marker display is positively correlated with the amount of text overflow, helping testers quickly identify text overflow issues. For target elements with dynamic constraints, the system can draw a dashed warning border around their boundaries. The spacing of the dashed lines dynamically adjusts based on the rate of change of the constraints, helping testers understand the dynamic adjustment behavior of the element. Finally, the system can use a multi-sampling anti-aliasing algorithm to smooth the wireframe edges and overlay a text label containing the type abbreviation and unique identifier at a preset position on each target element, improving the readability and recognition of the wireframe. For example, in a real-world application scenario, if the system sets a decreasing transparency gradient and discovers that a deeply nested element has low transparency, the system can add periodic visual warning markers and draw a dashed warning border around its boundaries for further analysis and processing by testers.

[0069] The specific process of step S4 can be to generate an interactive layout analysis panel on the side of the display interface, and when the tester detects a click operation on the wireframe overlay layer, the hierarchical path information of the currently selected element, the pixel deviation rate between the actual rendering size and the design draft size, the text content overflow status mark, and the parent container constraint expression are displayed. Specifically, first, the system can generate an interactive layout analysis panel on the side of the display interface, which can display the detailed information of the currently selected element in real time. When the tester detects a click operation on the wireframe overlay layer, the system can display the hierarchical path information of the currently selected element to help the tester understand the nested relationship of the elements. Then, the system can display the pixel deviation rate between the actual rendering size and the design draft size to help the tester identify the size deviation problem. For target elements with text overflow, the system can display the text content overflow status mark to help the tester quickly identify the text overflow problem. Finally, the system can display the parent container constraint expression to help the tester understand the dynamic adjustment behavior of the element. For example, in an actual application scenario, the system generates an interactive layout analysis panel and finds that the actual rendering size of a button element deviates significantly from the design draft size. The system can display the hierarchical path information and parent container constraint expression of the button element for testers to further analyze and process.

[0070] The specific step S5 includes the following sub-steps:

[0071] When it is detected that the line height of a text element is less than a preset multiple of the font size, it is marked as a line spacing compression exception;

[0072] For elements using absolute positioning, the offset distance between the center point coordinates and the ideal layout grid nodes is calculated. When the offset distance exceeds a second preset threshold, it is marked as a positioning deviation abnormality.

[0073] When the cumulative size of the sub-elements in the scroll container exceeds the preset multiple of the visible area size, a scroll performance degradation warning is triggered;

[0074] The scaling mode of the image element is detected, and when forced stretching is detected and the aspect ratio change exceeds a third preset threshold, it is marked as an image distortion abnormality.

[0075] Specifically, the system can detect whether the line height of a text element is less than a preset multiple of the font size. If so, it is marked as a line spacing compression anomaly, which can help testers identify text line spacing problems. Then, for elements using absolute positioning, the system can calculate the offset distance between their center point coordinates and the ideal layout grid node. If the offset distance exceeds a second preset threshold, it is marked as a positioning deviation anomaly, which can help testers identify positioning deviation problems. If the cumulative size of subelements within a scrolling container exceeds a preset multiple of the visible area size, the system can trigger a scrolling performance degradation warning, which can help testers identify scrolling performance problems. Finally, the system can detect the scaling mode of image elements. If forced stretching is detected and the aspect ratio change exceeds a third preset threshold, it is marked as an image distortion anomaly, which can help testers identify image distortion problems. For example, in an actual application scenario, if the system detects the line height of a text element and finds that the line height of a text element is less than a preset multiple of the font size, the system can mark the text element as a line spacing compression anomaly for further analysis and processing by testers.

[0076] In some embodiments, the method for real-time rendering and displaying UI interface layout information based on the Hongmeng system may further include the following steps:

[0077] Establish wireless debugging channels with multiple Hongmeng test devices to obtain interface rendering data and layout analysis data of multiple Hongmeng test devices in real time;

[0078] Displaying comparison views of multiple HarmonyOS test devices side by side on the display interface, with the comparison views overlaying and displaying difference area marks of the wireframe layers of the multiple HarmonyOS test devices;

[0079] When a layout anomaly is detected on a specific device, the screen parameter set of the specific device is extracted, and historical optimization solutions are matched based on parameter similarity to generate layout adaptation suggestions.

[0080] Specifically, the system can establish a wireless debugging channel with multiple HarmonyOS test devices to obtain the interface rendering data and layout parsing data of these devices in real time, ensuring that real-time information of multiple devices is obtained. Then, the system can display a comparison view of multiple HarmonyOS test devices side by side on the display interface. The comparison view will superimpose the difference area marks of the wireframe layers of multiple HarmonyOS test devices, which can help testers quickly identify the layout differences between multiple devices. When a layout anomaly is detected in a specific device, the system can extract the screen parameter set of the device, match the historical optimization plan based on the parameter similarity, and generate layout adaptation suggestions, which can help testers quickly solve the layout anomaly problem. For example, in an actual application scenario, the system discovers that there is an anomaly in the interface rendering data of a HarmonyOS test device by establishing a wireless debugging channel. The system can extract the screen parameter set of the device and generate layout adaptation suggestions for further analysis and processing by testers.

[0081] In some embodiments, the method for real-time rendering and displaying UI interface layout information based on the Hongmeng system may further include the following steps:

[0082] The visual focus coordinates of the test person are captured at a preset sampling interval by a visual acquisition device;

[0083] According to the mapping relationship between the visual focus coordinates and the wireframe, a first transparency adjustment strategy is implemented for the wireframe within a preset radius centered on the coordinates, and a second transparency adjustment strategy is implemented for the wireframe in other areas;

[0084] When it is detected that the tester's gaze time on the same element exceeds a first preset time threshold, the historical change record of the element's constraint conditions is displayed on the interactive layout analysis panel, and the corresponding layout source program position in the integrated development environment is located;

[0085] The interface layout complexity index is calculated based on the visual focus distribution data. When the index exceeds the preset complexity threshold, an optimization suggestion is generated to split the complex layout into atomic components.

[0086] Specifically, the system can capture the visual focus coordinates of the tester at a preset sampling interval through a visual acquisition device to ensure that the visual focus information of the tester is obtained. Then, based on the mapping relationship between the visual focus coordinates and the wireframe, the system can implement a first transparency adjustment strategy for the wireframe within a preset radius centered on the coordinates, and implement a second transparency adjustment strategy for the wireframe in other areas, which can help the tester quickly focus on the key areas. When it is detected that the tester's gaze time on the same element exceeds the first preset time threshold, the system can display the historical change record of the constraint conditions of the element on the interactive layout analysis panel, and locate the corresponding layout source program location in the integrated development environment, which can help the tester quickly understand the adjustment history of the element. Finally, the system can calculate the interface layout complexity index based on the visual focus distribution data. If the index exceeds the preset complexity threshold, it will generate an optimization suggestion to split the composite layout into atomic components, which can help the tester optimize the interface layout. For example, in an actual application scenario, the system captures the tester's visual focus coordinates and finds that the tester's gaze time on an element exceeds a preset threshold. The system can display the historical change records of the element's constraints and generate optimization suggestions for the tester to further analyze and process.

[0087] In some embodiments, the method for real-time rendering and displaying UI interface layout information based on the Hongmeng system may further include the following steps:

[0088] Establish a version control database to record wireframe snapshot data, device adaptation reports, and corresponding source program change records after each layout modification;

[0089] When it is detected that the similarity between the current layout defect characteristics and historical cases exceeds the preset similarity threshold, the associated historical repair plan and verification test case set are pushed;

[0090] Build a layout defect prediction model based on graph neural network, input the current view tree structure data and device parameter set into the prediction model, and output the potential defect type identification and its confidence score;

[0091] For defect types whose confidence scores exceed the preset confidence threshold, a repair patch program containing constraint optimization rules and container logic improvement solutions is generated;

[0092] Verify the compatibility and performance improvement indicators of the repair patch in an isolated sandbox environment. After verification, deploy it to the target device through the hot update mechanism.

[0093] Specifically, the system can establish a version control database to record wireframe snapshot data, device adaptation reports, and corresponding source code change records after each layout modification, ensuring that a complete historical record of layout modifications is available. Next, when the similarity between the current layout defect characteristics and historical cases exceeds a preset similarity threshold, the system can push the associated historical repair plan and verification test case set, which can help testers quickly resolve similar issues. The system also builds a layout defect prediction model based on a graph neural network, inputs the current view tree structure data and device parameter set into the prediction model, and outputs a potential defect type identifier and its confidence score, which can help testers identify potential problems in advance. For defect types with confidence scores exceeding the preset confidence threshold, the system can generate a repair patch that includes constraint optimization rules and container logic improvement solutions, which can help testers quickly fix the problem. Finally, the system can verify the compatibility and performance improvement indicators of the repair patch in an isolated sandbox environment. Once verified, it is deployed to the target device through a hot update mechanism, which ensures the security and effectiveness of the repair patch. For example, in an actual application scenario, the system establishes a version control database and finds that the similarity between the current layout defect characteristics and historical cases exceeds the preset threshold. The system can push the associated historical repair plan and generate a repair patch program for testers to further analyze and process.

[0094] A real-time rendering and display system for UI interface layout information based on Hongmeng system, by applying a real-time rendering and display method for UI interface layout information based on Hongmeng system as described above, including a metadata extraction unit, a target element subset determination unit, a coordinate mapping unit, an analysis panel generation unit and a warning unit, with reference to Figure 2, obtain the interface rendering data stream of the target application when it is running in the Hongmeng system through the metadata extraction unit, and extract the layout metadata set containing element type identification, hierarchical depth value, absolute coordinate range, dynamic constraint expression and associated resource identifier from the interface rendering data stream through the underlying interface of the Hongmeng system; perform topological sorting and sensitive element marking on the layout metadata set through the target element subset determination unit to determine the target element subset containing percentage size attributes, nested scroll containers and data binding relationships; map the target element subset to the display interface through the coordinate mapping unit to perform spatial coordinate system mapping to generate a wireframe overlay layer with hierarchical transparency characteristics, wherein the color of the wireframe in the wireframe overlay layer is based on the hierarchical depth of the target element in the target element subset. The degree of the wireframe changes gradually according to the preset color scale rules, and the width of the wireframe is inversely proportional to the area of ​​the display area of ​​the target element; an interactive layout analysis panel is generated on the side of the display interface through the analysis panel generation unit. When a click operation of the tester on the wireframe overlay layer is detected, the hierarchical path information of the currently selected element, the pixel deviation rate between the actual rendering size and the design draft size, the text content overflow status mark and the parent container constraint expression are displayed; the warning unit calculates the deviation value between the actual rendering ratio of each target element in the target element subset and the preset standard ratio based on the screen pixel density, resolution parameters and safe display area boundary information of the target device. When the deviation value exceeds the first preset threshold, an adaptation abnormality alarm message is generated and the potential area with display conflict is marked.

[0095] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for real-time rendering and display of UI interface layout information based on Hongmeng system, characterized in that: The following steps are involved: Obtaining an interface rendering data stream of a target application when it is running on the Hongmeng system, and extracting a layout metadata set including an element type identifier, a layer depth value, an absolute coordinate range, a dynamic constraint expression, and an associated resource identifier from the interface rendering data stream through the underlying interface of the Hongmeng system; Performing topological sorting and sensitive element tagging on the layout metadata set to determine a subset of target elements including percentage size attributes, nested scroll containers, and data binding relationships; Mapping the target element subset to the display interface in a spatial coordinate system to generate a wireframe overlay layer with hierarchical transparency characteristics, wherein the color of the wireframe in the wireframe overlay layer gradually changes according to a preset color scale rule based on the hierarchical depth of the target elements in the target element subset, and the width of the wireframe is inversely proportional to the area of ​​the display region of the target element; An interactive layout analysis panel is generated on the side of the display interface. When a click operation of the tester on the wireframe overlay layer is detected, the panel displays the hierarchical path information of the currently selected element, the pixel deviation rate between the actual rendering size and the design draft size, the text content overflow status mark, and the parent container constraint condition expression; Based on the screen pixel density, resolution parameters and safe display area boundary information of the target device, the deviation value between the actual rendering ratio of each target element in the target element subset and the preset standard ratio is calculated. When the deviation value exceeds the first preset threshold, an adaptation abnormality alarm message is generated and the potential area where display conflict exists is marked.

2. According to the method of claim 1, the method is characterized in that: The steps of obtaining the interface rendering data stream of the target application when it is running on the Hongmeng system, and extracting a layout metadata set including an element type identifier, a layer depth value, an absolute coordinate range, a dynamic constraint expression, and an associated resource identifier from the interface rendering data stream through the underlying interface of the Hongmeng system are specifically as follows: Intercepting the layer synthesis data of the frame buffer through the graphics synthesis service interface of the Hongmeng system; Parsing the geometric vertex coordinates and texture mapping parameters of each drawing instruction in the layer synthesis data; For elements containing dynamic constraints, extract the anchor connection relationship between them and their parent container and the relative weight distribution ratio; Convert the parsing result into a structured data set containing a globally unique identifier, control type code, absolute coordinate range matrix, hierarchical depth index value, dynamic constraint condition logic expression and associated resource file feature hash value.

3. A method for real-time rendering and displaying UI interface layout information based on Hongmeng system according to claim 2, characterized in that: The steps of mapping the target element subset to the real-time screen image in a spatial coordinate system to generate a wireframe overlay layer with hierarchical transparency features are specifically as follows: Setting a transparency decreasing gradient according to the layer depth value, so that each increase in layer depth corresponds to a decrease in layer transparency by a preset percentage; Adding a periodic visual warning mark to the target element with text overflow, wherein the display frequency of the warning mark is positively correlated with the amount of text overflow; Draw a dotted warning boundary around the boundary of the target element with dynamic constraints, where the interval between the dotted lines is dynamically adjusted according to the rate of change of the constraints; The wireframe edges are smoothed using a multi-sampling anti-aliasing algorithm, and a text label containing the type abbreviation and unique identifier is superimposed and displayed at a preset position on each target element.

4. A method for real-time rendering and displaying UI interface layout information based on Hongmeng system according to claim 3, characterized in that: The steps of calculating the deviation between the actual rendering ratio of each interface element and a preset standard ratio based on the screen pixel density, resolution parameters, and safe display area boundary information of the target device, and generating an adaptation abnormality warning message and marking a potential area with display conflict when the deviation exceeds a first preset threshold are specifically as follows: When it is detected that the line height of a text element is less than a preset multiple of the font size, it is marked as a line spacing compression exception; For elements using absolute positioning, calculate the offset distance between the center point coordinates and the ideal layout grid nodes, and mark it as a positioning deviation abnormality when the offset distance exceeds a second preset threshold; When the cumulative size of the sub-elements in the scroll container exceeds the preset multiple of the visible area size, a scroll performance degradation warning is triggered; The scaling mode of the image element is detected, and when forced stretching is detected and the aspect ratio change exceeds a third preset threshold, it is marked as an image distortion abnormality.

5. The method for real-time rendering and displaying UI interface layout information based on Hongmeng system according to claim 1 is characterized in that: The method further comprises: Establish a wireless debugging channel with multiple Hongmeng test devices to obtain interface rendering data and layout analysis data of the multiple Hongmeng test devices in real time; Displaying comparison views of the multiple Hongmeng test devices side by side on the display interface, wherein the comparison views superimpose difference area marks of the wireframe layers of the multiple Hongmeng test devices; When a layout anomaly is detected on a specific device, a screen parameter set of the specific device is extracted, and historical optimization solutions are matched based on parameter similarity to generate a layout adaptation suggestion.

6. The method for real-time rendering and displaying UI interface layout information based on Hongmeng system according to claim 1 is characterized in that: The method further comprises: The visual focus coordinates of the test person are captured at a preset sampling interval by a visual acquisition device; According to the mapping relationship between the visual focus coordinates and the wireframe, a first transparency adjustment strategy is applied to the wireframe within a preset radius centered on the coordinates, and a second transparency adjustment strategy is applied to the wireframe in other areas; When it is detected that the tester's gaze time on the same element exceeds a first preset time threshold, the constraint condition history change record of the element is displayed on the interactive layout analysis panel, and the corresponding layout source program position in the integrated development environment is located; An interface layout complexity index is calculated based on visual focus distribution data. When the index exceeds a preset complexity threshold, an optimization suggestion is generated to split the complex layout into atomic components.

7. The method for real-time rendering and displaying UI interface layout information based on Hongmeng system according to claim 1 is characterized in that: The method further comprises: Establish a version control database to record wireframe snapshot data, device adaptation reports, and corresponding source program change records after each layout modification; When it is detected that the similarity between the current layout defect characteristics and historical cases exceeds the preset similarity threshold, the associated historical repair plan and verification test case set are pushed; Construct a layout defect prediction model based on a graph neural network, input the current view tree structure data and the device parameter set into the prediction model, and output the potential defect type identification and its confidence score; For defect types whose confidence scores exceed the preset confidence threshold, a repair patch program containing constraint optimization rules and container logic improvement solutions is generated; The compatibility and performance improvement indicators of the repair patch are verified in an isolated sandbox environment. After verification, it is deployed to the target device through the hot update mechanism.

8. A real-time rendering and display system for UI interface layout information based on Hongmeng system, characterized by: The system is used to implement the real-time rendering and display method of UI interface layout information based on the Hongmeng system according to any one of claims 1 to 7, including: A metadata extraction unit is used to obtain an interface rendering data stream of a target application when the target application is running on the Hongmeng system, and extract a layout metadata set including an element type identifier, a layer depth value, an absolute coordinate range, a dynamic constraint expression, and an associated resource identifier from the interface rendering data stream through the underlying interface of the Hongmeng system; a target element subset determining unit, configured to perform topological sorting and sensitive element tagging on the layout metadata set, and determine a target element subset including percentage size attributes, nested scroll containers, and data binding relationships; a coordinate mapping unit, configured to map the target element subset to a display interface in a spatial coordinate system to generate a wireframe overlay layer having a hierarchical transparency feature, wherein the color of the wireframe in the wireframe overlay layer gradually changes according to a preset color scale rule based on the hierarchical depth of the target elements in the target element subset, and the width of the wireframe is inversely proportional to the area of ​​the display region of the target element; An analysis panel generation unit is configured to generate an interactive layout analysis panel on the side of the display interface. When a click operation by the tester on the wireframe overlay layer is detected, the panel displays the hierarchical path information of the currently selected element, the pixel deviation rate between the actual rendering size and the design draft size, the text content overflow status mark, and the parent container constraint condition expression; A warning unit is used to calculate the deviation between the actual rendering ratio of each target element in the target element subset and the preset standard ratio based on the screen pixel density, resolution parameters and safe display area boundary information of the target device, and generate an adaptation abnormality alarm message and mark the potential area where display conflict exists when the deviation value exceeds a first preset threshold.

Images