A rendering display method, device and storage medium

Abstract

The application discloses a rendering display method and device and a storage medium, which are applied to a source terminal device and an augmented reality glasses, the source terminal device is used for extracting semantic information of a user interactive component; based on the semantic information, vector graphic description data corresponding to the user interactive component is generated; the vector graphic description data is sent to the augmented reality glasses; the augmented reality glasses are used for receiving the vector graphic description data sent by the source terminal device; graphic drawing is performed according to the vector graphic description data, so that the vector graphic corresponding to the user interactive component is obtained; and the vector graphic is displayed. Through extraction of the semantic information of the user interactive component, the user interactive component is converted into the vector graphic description data, and the traditional full-screen pixel flow is replaced by vector graphic description data transmission, so that the data transmission amount is greatly reduced, the bandwidth pressure is relieved, the transmission delay is reduced, and the problem of poor universality is overcome.

Description

Technical Field

[0001] This invention relates to the field of augmented reality display, and more particularly to a rendering display method, a computer device, and a computer-readable storage medium. Background Technology

[0002] With the continuous development of augmented reality (AR) technology, smart glasses are gradually becoming a new mainstream computing terminal after smartphones. These devices generally adopt display solutions such as optical waveguides and MicroLED (micron-level light-emitting diodes), which place stringent requirements on the contrast of the displayed image, edge sharpness, and overall power consumption.

[0003] Traditional smart glasses often feature static, preset user interfaces. However, with the widespread adoption of large language models and generative artificial intelligence, users expect smart glasses to integrate voice commands and environmental information to provide real-time, non-preset user interaction components, such as real-time navigation routes, translated subtitles, and dynamic data panels. This demand places higher requirements on the device's graphics rendering capabilities and data transmission efficiency.

[0004] Currently, there are two main transmission and display solutions in the industry: pixel streaming and custom command stream transmission. Pixel streaming renders the entire screen and encodes it for transmission on the mobile device, similar to wireless screen mirroring, but it suffers from high bandwidth consumption and high power consumption. Custom command stream transmission relies on custom commands to render the interface, but its protocol has poor universality. Summary of the Invention

[0005] This invention provides a rendering and display method, device, and storage medium that can reduce the bandwidth required for rendering and displaying user interaction components and overcome the problem of poor versatility.

[0006] In a first aspect, the present invention provides a rendering and display method applied to a source device, comprising: Extract semantic information from user interaction components; Based on semantic information, generate vector graphic description data corresponding to user interaction components; Vector graphics description data is sent to the augmented reality glasses, which then use the data to draw and display vector graphics corresponding to user interaction components.

[0007] Secondly, the present invention provides a rendering and display method for use in augmented reality glasses, comprising: Receive vector graphic description data sent by the source device. The vector graphic description data is generated by the source device based on semantic information. The semantic information is extracted by the source device from the user interaction components. Based on the vector graphics description data, the graphics are drawn to obtain the vector graphics corresponding to the user interaction components; Display vector graphics.

[0008] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described rendering and display method.

[0009] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described rendering and display method.

[0010] The aforementioned rendering and display method, device, and storage medium extract semantic information from user interaction components; based on the semantic information, generate vector graphic description data corresponding to the user interaction components; and send the vector graphic description data to augmented reality glasses, where the glasses execute the data to draw and display the vector graphics corresponding to the user interaction components. By extracting the semantic information of the user interaction components and converting them into vector graphic description data, and replacing the traditional full-screen pixel stream with vector graphic description data transmission, the amount of data transmission is significantly reduced. This alleviates bandwidth pressure, reduces transmission latency, and overcomes the problem of poor versatility. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of an application environment for a rendering and display method according to an embodiment of the present invention; Figure 2 This is a flowchart of a rendering and display method according to an embodiment of the present invention; Figure 3 yes Figure 2 A flowchart of step S101; Figure 4 yes Figure 2 A flowchart of step S102; Figure 5 yes Figure 4 A flowchart of step S1023; Figure 6 This is another flowchart of a rendering and display method according to one embodiment of the present invention; Figure 7 yes Figure 2 A flowchart of step S103; Figure 8 This is another flowchart of a rendering and display method according to one embodiment of the present invention; Figure 9 yes Figure 8 A flowchart of step S202; Figure 10 This is a schematic diagram of a rendering display device according to an embodiment of the present invention; Figure 11 This is another schematic diagram of a rendering display device according to one embodiment of the present invention; Figure 12 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0013] To make the technical problems solved, the technical solutions, and the beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0014] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0015] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0016] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0017] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0018] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention provides a rendering and display method that can be applied to, for example... Figure 1 The application environment shown is illustrated. Specifically, this rendering and display method is applied in a rendering and display system, which includes a source device and augmented reality glasses. The source device and the augmented reality glasses communicate via a communication link, transmitting source data, including user interaction components, from the source device to the augmented reality glasses for real-time rendering and display, thereby achieving an augmented reality interactive experience. The source device can be any computer device capable of providing a data source, such as a smartphone, tablet, or laptop. Furthermore, the communication link can be a low-latency, high-bandwidth wireless protocol such as Bluetooth, Wi-Fi, or UWB.

[0021] In one embodiment, please refer to Figure 2 This paper provides a rendering and display method for use on a source device, comprising the following steps: Step S101: Extract semantic information of user interaction components; Step S102: Based on semantic information, generate vector graphic description data corresponding to the user interaction components; Step S103: Send the vector graphics description data to the augmented reality glasses. The vector graphics description data is used by the augmented reality glasses to draw and display vector graphics corresponding to the user interaction components.

[0022] In this embodiment, the source device is used to provide a data source to the augmented reality glasses for display. The source device can be a mobile terminal device such as a smartphone or tablet, or a fixed terminal device such as a personal computer. This embodiment of the invention does not impose any limitations on this.

[0023] User interaction components are various interactive elements that need to be presented on the display interface in the system. In this embodiment, they are processed uniformly by the source device and then transmitted to the augmented reality glasses for rendering and display. Semantic information is structured basic attribute information extracted from user interaction components, including basic descriptive information such as text content, layout constraints, hierarchical relationships, and style parameters corresponding to the user interaction components. It is the input source for generating vector graphics description data.

[0024] As an example, in step S101, a semantic interception layer can be set up during the runtime of the extended reality framework of the source device to extract semantic information from user interaction components such as user interaction panels, dialog boxes, and text information that need to be displayed by augmented reality glasses, so as to generate vector graphic description data using the semantic information. The semantic interception layer is an intermediate logic layer deployed in the runtime environment, used to intercept relevant data of the output interface and extract its semantic information.

[0025] Among them, the vector graphics description data is a set of vector drawing instructions generated based on the semantic information of user interaction components. It is used to guide augmented reality glasses to complete the drawing of vector graphics. It can be in the form of SVG text, binary vector instructions, etc. This data is different from bitmap pixel streams. It has a small data size, supports lossless scaling, and finally renders a complete vector graphics for display.

[0026] As an example, in step S102, after establishing a communication connection with the augmented reality glasses, vector graphic description data matching the user interaction components is constructed and generated based on semantic information to guide the augmented reality glasses in drawing vector graphics adapted for display. Alternatively, the source device can generate the vector graphic description data, parse it, directly convert it into a binary drawing instruction sequence, and then transmit it to the augmented reality glasses, which can reduce the computational overhead of the augmented reality glasses in parsing the vector graphic description data.

[0027] As an example, in step S103, after generating the vector graphics description data, the vector graphics description data is sent to the augmented reality glasses so that the augmented reality glasses can parse and execute the vector graphics description data, render and display the vector graphics corresponding to the user interaction components, and the augmented reality glasses can perform drawing, layer composition and distortion correction, which can improve rendering efficiency.

[0028] As an example, extracting semantic information and generating vector graphics description data does not necessarily have to be done on the source device. It can also be generated using a cloud server. The cloud renders the generated vector graphics as a video stream with a transparent background or H.264 keyframes. The source device only acts as a relay, and the augmented reality glasses act as a video decoding terminal to adapt to scenarios where the computing power of augmented reality glasses is extremely low.

[0029] Please see Figure 3 In one embodiment, step S101, namely extracting semantic information of the user interaction component, includes: Step S1011: When the user interaction component includes text content, convert the text content into a vector path; Step S1012: Extract the semantic information of the transformed user interaction components.

[0030] The text content refers to textual information contained in user interaction components, such as Chinese characters, including prompts, annotations, and other character information. Vector paths are graphical data that uses vector geometric elements such as points, lines, and curves to describe the outline of text. This data no longer relies on font files for rendering; the shape of the text can be restored solely through geometric trajectories. It is the standard vector representation of text after conversion to curves.

[0031] As an example, in step S1011, after establishing a communication connection with the augmented reality glasses, if it is detected that the augmented reality glasses do not have a pre-installed complete font library, the source device can convert the target text into a vector path. The character content after path conversion is formatted in the same way as other graphic elements in the user interaction component, and there is no need to distinguish between text and graphic types in subsequent generation, transmission, and rendering stages. There are no restrictions on how to convert the target text into a vector path. For example, a font parsing library can be used to extract the glyph outline of the target text, or a vector font rendering engine can be called to generate a Bézier curve path corresponding to the target text. Alternatively, a deep learning model can be used to perform end-to-end vectorization modeling of the character stroke structure of the target text to obtain the corresponding vector path, thereby ensuring that the text form can be restored without distortion on any terminal.

[0032] As an example, in step S2012, after the text is processed into paths, semantic information is extracted from the user interaction components in a unified manner, and information such as text outlines, interface layout, and style are integrated into semantic information. Based on the transformed semantic information, vector graphic description data is generated, which can simplify the analysis and drawing logic of augmented reality glasses.

[0033] Please see Figure 4 In one embodiment, the user interaction component is a user interaction component used to anchor itself to a specific location in real space. Step S102, namely, generating vector graphic description data corresponding to the user interaction component based on semantic information, includes: Step S1021: Obtain the spatial anchor point coordinates of the real space corresponding to the user interaction component; Step S1022: Obtain the pose data of the augmented reality glasses, and calculate the two-dimensional projection coordinates of the spatial anchor point coordinates corresponding to the display screen of the augmented reality glasses based on the pose data, semantic information and spatial anchor point coordinates; Step S1023: Generate vector graphic description data based on semantic information and two-dimensional projection coordinates.

[0034] The user interaction components are user-interactive components displayed in vector graphics and anchored to specific locations in real space. These can include navigation icons, text prompts, translation subtitles, data panels, and other user interface content. Displayed as foreground content, these components are independent of the real-world background and dynamic video footage, and are a core component for the augmented reality glasses to achieve a seamless blend of virtual and real-world displays. Spatial anchor point coordinates are virtual coordinates obtained from spatial plotting. These spatial anchor points are virtual positioning anchor points created based on the augmented reality framework. In the three-dimensional spatial coordinate system of the real physical scene, the user interaction components are bound to target objects in the real-world scene through these spatial anchor points to achieve spatial positioning and display. This anchors and attaches the virtual user interaction content to designated objects or locations in the real-world scene, giving the virtual interface a realistic spatial identity.

[0035] As an example, in step S1021, for spatial placement requirements, the extended reality framework of the source device completes the three-dimensional space calculation to obtain the spatial anchor coordinates of the user interaction component corresponding to the real space. For example, the extended reality application places an anchor in the three-dimensional space, which corresponds to the position of the object in the real space, and obtains the spatial anchor coordinates.

[0036] The posture data consists of head posture data collected in real-time by the augmented reality glasses and reported to the source device. This includes angle, orientation, and position change parameters output by inertial measurement sensors and SLAM (Simultaneous Localization and Mapping) positioning, used to characterize changes in the user's viewpoint in real time. The two-dimensional projected coordinates are screen coordinates obtained by transforming the anchor point positions in real-world three-dimensional space through frustum projection and clipping operations. These coordinates are used to determine the final rendering position of user interaction components on the augmented reality glasses' display screen.

[0037] As an example, in step S1022, the head posture data reported in real time by the augmented reality glasses is obtained. Based on the head posture data, semantic information and spatial anchor point coordinates, frustum clipping and projection operations are performed to obtain the two-dimensional projection coordinates of the spatial anchor points corresponding to the display screen of the augmented reality glasses. When the source device detects a change in the head posture data sent by the augmented reality glasses, the two-dimensional projection coordinates are recalculated and updated.

[0038] As an example, in step S1023, vector graphic description data adapted for drawing with augmented reality glasses is generated based on semantic information and two-dimensional projection coordinates, and the vector graphic description data is dynamically corrected in real time based on the updated two-dimensional projection coordinates.

[0039] Please see Figure 5 In one embodiment, step S1023, namely generating vector graphic data based on semantic information and two-dimensional projection coordinates, includes: Step S10231: Construct constraint prompts based on semantic information and call a large language model to generate vector graphics description data adapted for augmented reality glasses; Step S10232: Correct the coordinate parameters of the vector graphic description data according to the two-dimensional projection coordinates.

[0040] Among them, constraint prompts are structured instruction texts constructed by the source device based on the semantic information of user interaction components and the display rules of augmented reality glasses. These texts are used to limit the output dimensions of the large language model, including graphic styles, element structures, layout rules, vector formats, and augmented reality interface constraints, enabling the large language model to stably, compliantly, and in a standardized manner output vector graphic data that can be parsed by augmented reality glasses. Constraint example: "Generate a navigation card in vector graphic format, including a left-turn arrow and distance text. Requirements: Canvas 640x480, using only..." <path> 、 <rect> 、 <text>This tag prohibits the use of external CSS stylesheets, complex filters, gradients, and opacity; only pure black / pure white (for monochrome displays) is allowed. The text content is 'Turn right 100 meters ahead,' centered, font size 48. The large language model is used to receive semantic information and constraint prompts from user interaction components, and autonomously reason to generate vector graphics description data that conforms to augmented reality display specifications. In this embodiment, the large language model is the large language model (LLM) of the source device or the cloud.

[0041] As an example, in step S10231, constraint prompts are constructed based on semantic information, and a local or cloud-based large language model is called to generate initial vector graphic description data adapted for the parsing and drawing of augmented reality glasses. The constraint prompts strongly standardize the output structure of the large oracle model, ensuring that the content generated by the large oracle model strictly conforms to the vector drawing standard of augmented reality glasses and can be directly parsed and executed by the augmented reality glasses hardware.

[0042] As an example, in step S10232, the coordinate parameters of the vector graphic description data are dynamically corrected in real time according to the updated two-dimensional projection coordinates. After the head posture changes, the two-dimensional projection coordinates are updated in real time, and the coordinates of the vector graphic description data are corrected synchronously. The augmented reality glasses are only responsible for parsing and drawing, which greatly reduces the computing power overhead of the glasses and is compatible with low-end lightweight augmented reality glasses hardware.

[0043] Please see Figure 6 In one embodiment, before sending the vector graphics description data to the augmented reality glasses in step S103, the rendering and display method further includes: Step S001: Preprocess the generated vector graphics description data. Preprocessing includes, but is not limited to, at least one of syntax validation, path simplification, and code reduction.

[0044] Preprocessing is an optimization process performed on the source device before sending vector graphics description data. It includes one or more of the following processes: syntax verification, path simplification, code reduction, and error correction. The purpose is to make the data legal, concise, and efficient, adapt to the resolution capabilities of augmented reality glasses and the low-bandwidth transmission environment, and improve rendering stability and efficiency.

[0045] As an example, in step S001, the source device parses and verifies the vector graphic description data, correcting errors, including syntax verification, path simplification, and code reduction. Syntax verification involves performing structural parsing, tag integrity checks, and syntax error detection on the vector graphic description data (such as SVG / XML format text) generated by the large language model, identifying and automatically correcting issues such as unclosed tags, syntax anomalies, and format errors, ensuring that the data can be parsed normally by augmented reality glasses. Path simplification involves using path optimization algorithms (such as the Douglas-Peucker algorithm) to reduce the number of path nodes in the vector graphics without changing the graphic outline, thereby reducing the computational load for augmented reality glasses rendering and improving drawing efficiency. Code reduction involves removing redundant characters from the vector code, including invalid content such as newlines, spaces, and comments, and optimizing the abbreviation of attributes and tags to further reduce data volume and transmission pressure.

[0046] Please see Figure 7 In one embodiment, step S103, which involves sending the vector graphics description data to the augmented reality glasses, includes: Step S1031: Obtain the hardware capability information of the augmented reality glasses, and determine whether the augmented reality glasses support hardware vector acceleration based on the hardware capability information to select the transmission mode; Step S1032: If hardware vector acceleration is supported, the vector graphics description data is transferred to the augmented reality glasses; Step S1033: If hardware vector acceleration is not supported, then draw vector graphics corresponding to the user interaction components according to the vector graphics description data, and then render the drawn vector graphics as bitmaps and transmit them to the augmented reality glasses.

[0047] Among them, the hardware capability information of augmented reality glasses refers to the hardware status parameters of augmented reality glasses obtained when the source device establishes a communication connection with the augmented reality glasses. It includes at least whether the augmented reality glasses support hardware vector acceleration, whether they support differential resolution or partial refresh, device memory parameters, display adaptation parameters, etc., and serves as the basis for the source device to dynamically select the data transmission method.

[0048] As an example, in step S1031, when the source device establishes a communication connection with the augmented reality glasses, it identifies whether the augmented reality glasses support hardware vector acceleration. If the augmented reality glasses support hardware vector acceleration, the vector graphic description data is directly transmitted to the augmented reality glasses so that the augmented reality glasses can parse and process it. If the augmented reality glasses do not support hardware vector acceleration, the source device draws vector graphics corresponding to the user interaction components according to the vector graphic description data, and renders the drawn vector graphics as a compressed bitmap or differential bitmap before transmitting it to the augmented reality glasses.

[0049] As an example, in step S103, when the source device transmits vector graphic description data to the augmented reality glasses, it compares the differences between the two frames of vector graphic description data and sends only the vector graphic description data or bitmap of the changed area to the augmented reality glasses. At the same time, the glasses driver only refreshes the changed area to maximize transmission efficiency.

[0050] As an example, in step S103, compressed bitmaps are used for transmission of static backgrounds and complex textures, while vector graphics description data is used for transmission of dynamically changing critical paths, such as navigation arrows and dynamic waveforms.

[0051] Please see Figure 8 This invention also provides a rendering and display method for augmented reality glasses, comprising: Step S201: Receive vector graphic description data sent by the source device. The vector graphic description data is generated by the source device based on semantic information. The semantic information is extracted by the source device from the user interaction components. Step S202: Draw the graphics according to the vector graphics description data to obtain the vector graphics corresponding to the user interaction components; Step S203: Display vector graphics.

[0052] When the source device establishes a communication connection with the augmented reality glasses, data transmission can be performed via Bluetooth. Based on the hardware capabilities of the augmented reality glasses, the transmission rules for two different data formats are dynamically switched: vector transmission and bitmap differential transmission. In bitmap differential transmission, only the local areas where the image has changed compared to the previous frame are identified and extracted, and only the image data of that area is transmitted, abandoning the method of transmitting the entire frame. Based on the Bluetooth Low Energy protocol and with the data length extension function enabled, the data to be transmitted is split into multiple sub-packets according to the rules and sent sequentially. After the augmented reality glasses receive all the sub-packets, the data is reassembled and restored.

[0053] As an example, in step S201, the source device and the augmented reality glasses establish a Bluetooth connection, receive vector graphics description data transmitted by the terminal device, and parse and process the vector graphics description data.

[0054] As an example, in step S202, before drawing according to the vector graphics description data, the vector graphics description data is parsed. If the parsed data is vector graphics description data, the vector core of the vector drawing unit (VGU) is called to directly draw lines and shapes in the frame buffer according to the path instructions to obtain the vector graphics corresponding to the user interaction component. If the parsed data is bitmap, the hardware layer compositor is used to scale it and perform the final layer composition. For text nodes, the vector drawing unit loads the vector font (True Type) for outline rendering.

[0055] As an example, in step S203, the obtained vector graphics or bitmaps are overlaid with the background layer and displayed on the display screen of the augmented reality glasses.

[0056] Please see Figure 9 In one embodiment, step S202, namely drawing graphics according to vector graphics description data, includes: Step S2021: Draw graphics in the user interaction layer according to the vector graphics description data; Step S2022: Display the user interaction layer overlaid on the background layer, where the background layer contains background image data.

[0057] The user interaction layer is a foreground interaction layer generated by the augmented reality glasses based on the vector graphic description data sent by the source device. It carries various interface elements that are dynamically generated and spatially anchored, including virtual user interaction content such as text prompts, icons, navigation labels, and data panels. It can independently perform graphic transformations, alpha blending, and distortion correction, and is a foreground layer for virtual-real fusion display.

[0058] As an example, in step S2021, after parsing the vector graphics description data, lines and shapes are drawn on the user interaction layer using the vector drawing unit, and distortion correction, scaling, alpha blending, pose compensation, etc. are performed on them.

[0059] The background layer is a display base layer that is independently generated locally by the augmented reality glasses. It consists of perspective images captured by the glasses' real-world camera or locally preset static images. It does not require transmission from the source device and serves as a real-world scene base, providing a real-world scene basis for the overlay of the user interaction layer.

[0060] As an example, in step S2022, the user interaction layer with completed distortion and posture correction is superimposed on the background layer through the augmented reality glasses hardware layer mixer and output to the augmented reality glasses display screen for display. In this embodiment, the virtual user interaction content and the real scene background are drawn in layers. The hardware layer mixer can perform rotation, scaling and mesh distortion correction on the user interaction layer separately, which can avoid the defect that traditional whole frame image rendering cannot correct user interaction separately and greatly improve the accuracy of augmented reality display adaptation. At the same time, the parsed user interaction layer is rendered to the off-screen buffer. Using the head posture data collected by the inertial sensor, geometric transformations such as translation and rotation are performed on the graphics in the off-screen buffer to offset the visual deviation caused by the user's head movement.

[0061] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0062] In one embodiment, the present invention provides a rendering and display system, including a source device and augmented reality glasses, which corresponds one-to-one with the rendering and display methods described in the above embodiments. This rendering and display system is applied to the source device and the augmented reality glasses.

[0063] The source device is used to extract semantic information of user interaction components; based on the semantic information, generate vector graphic description data corresponding to the user interaction components; and send the vector graphic description data to augmented reality glasses. Augmented reality glasses are used to receive vector graphics description data sent by a source device; draw graphics according to the vector graphics description data to obtain vector graphics corresponding to user interaction components; and display the vector graphics.

[0064] Optionally, in one embodiment, the source device is further configured to convert the text content into a vector path when the user interaction component includes text content; and extract the semantic information of the converted user interaction component.

[0065] Optionally, in one embodiment, the source device is further configured to obtain the spatial anchor point coordinates of the real space corresponding to the user interaction component; obtain the pose data of the augmented reality glasses, and calculate the two-dimensional projection coordinates of the spatial anchor point corresponding to the display screen of the augmented reality glasses based on the pose data, semantic information and spatial anchor point coordinates; and generate vector graphic description data based on semantic information and two-dimensional projection coordinates.

[0066] Optionally, in one embodiment, the source device is further configured to construct constraint prompts based on semantic information and call a large language model to generate vector graphic description data adapted for execution by augmented reality glasses; and correct the coordinate parameters of the vector graphic description data according to the two-dimensional projection coordinates.

[0067] Optionally, in one embodiment, the source device is further configured to preprocess the generated vector graphics description data, including but not limited to at least one of syntax verification, path simplification, and code reduction.

[0068] Optionally, in one embodiment, the source device is further configured to acquire hardware capability information of the augmented reality glasses, and determine whether the augmented reality glasses support hardware vector acceleration based on the hardware capability information to select a transmission mode; If hardware vector acceleration is supported, the vector graphics description data will be transferred to the augmented reality glasses; If hardware vector acceleration is not supported, vector graphics corresponding to user interaction components are drawn based on vector graphics description data, and the drawn vector graphics are rendered as bitmaps and then transmitted to the augmented reality glasses.

[0069] Optionally, in one embodiment, the augmented reality glasses are also used to draw graphics on the user interaction layer according to vector graphics description data; and to display the user interaction layer overlaid on the background layer, the background layer having background image data drawn on it.

[0070] For specific limitations on the rendering and display system, please refer to the limitations on the rendering and display methods mentioned above, which will not be repeated here.

[0071] Please refer to Figure 10 In one embodiment, the present invention provides a rendering and display device suitable for a source device, including an extraction module, a generation module, and a sending module; The extraction module is used to extract semantic information from user interaction components; The generation module is used to generate vector graphic description data corresponding to user interaction components based on semantic information; The sending module sends vector graphics description data to the augmented reality glasses. The vector graphics description data is used by the augmented reality glasses to draw and display vector graphics corresponding to user interaction components.

[0072] In one embodiment, the extraction module includes: A conversion unit is used to convert text content into vector paths when the user interaction component includes text content; The extraction unit is used to extract the semantic information of the transformed user interaction components.

[0073] In one embodiment, the generation module includes: The acquisition unit is used to obtain the spatial anchor point coordinates of the real space corresponding to the user interaction component; The computing unit is used to acquire the pose data of the augmented reality glasses, and calculate the two-dimensional projection coordinates of the spatial anchor points corresponding to the display screen of the augmented reality glasses based on the pose data, semantic information and spatial anchor point coordinates. The generation unit is used to generate vector graphics description data based on semantic information and two-dimensional projection coordinates.

[0074] In one embodiment, the generation unit includes: The calling unit is used to construct constraint prompts based on semantic information and call the large language model to generate vector graphics description data adapted for execution by augmented reality glasses; The correction unit corrects the coordinate parameters of the vector graphic description data based on the two-dimensional projection coordinates.

[0075] In one embodiment, the rendering display device further includes: The preprocessing module is used to preprocess the generated vector graphics description data. Preprocessing includes, but is not limited to, at least one of syntax validation, path simplification, and code reduction.

[0076] In one embodiment, the generation module includes: The judgment unit is used to obtain hardware capability information of augmented reality glasses and determine whether augmented reality glasses support hardware vector acceleration based on the hardware capability information to select the transmission mode. The first transmission unit is used to transmit vector graphics description data to augmented reality glasses if hardware vector acceleration is supported. The second transmission unit is used to draw vector graphics corresponding to user interaction components based on vector graphics description data if hardware vector acceleration is not supported, and then render the drawn vector graphics as bitmaps and transmit them to the augmented reality glasses.

[0077] Please refer to Figure 11 In one embodiment, a rendering display device suitable for augmented reality glasses is provided, including a receiving module, a drawing module, and a display module.

[0078] The receiving module is used to receive vector graphic description data sent by the source device. The vector graphic description data is generated by the source device based on semantic information, which is extracted by the source device from the user interaction components. The drawing module is used to draw graphics according to the vector graphics description data, and obtain vector graphics corresponding to the user interaction components; The display module is used to display vector graphics.

[0079] In one embodiment, the drawing module includes: The drawing unit is used to draw graphics in the user interaction layer according to the data described by vector graphics. The overlay unit is used to overlay the user interaction layer on top of the background layer for display. The background layer contains background image data.

[0080] It should be understood that each module in the above-mentioned rendering and display device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0081] In one embodiment, a computer device is provided, which may be a source device, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores rendering configuration data, vector graphics data, and terminal device information. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a rendering and display method.

[0082] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the rendering and display method described in the above embodiment. To avoid repetition, further details are omitted here. The computer-readable storage medium can be non-volatile or volatile.

[0083] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0084] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0085] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.< / text> < / rect> < / path>

Claims

1. A rendering and display method, applied to a source device, characterized in that, include: Extract semantic information from user interaction components; Based on the semantic information, vector graphic description data corresponding to the user interaction component is generated; The vector graphics description data is sent to the augmented reality glasses, and the vector graphics description data is used by the augmented reality glasses to draw and display vector graphics corresponding to the user interaction components.

2. The rendering and display method as described in claim 1, characterized in that, The extraction of semantic information from user interaction components includes: When the user interaction component includes text content, the text content is converted into a vector path; Extract the semantic information of the transformed user interaction component.

3. The rendering and display method as described in claim 1, characterized in that, The user interaction component is a user interaction component used to anchor itself to a specific location in real space. The step of generating vector graphic description data corresponding to the user interaction component based on the semantic information includes: Obtain the spatial anchor point coordinates of the real space corresponding to the user interaction component; Acquire the pose data of the augmented reality glasses, and calculate the two-dimensional projection coordinates of the spatial anchor point coordinates corresponding to the display screen of the augmented reality glasses based on the pose data, the semantic information and the spatial anchor point coordinates; The vector graphic description data is generated based on the semantic information and the two-dimensional projection coordinates.

4. The rendering and display method as described in claim 3, characterized in that, The step of generating the vector graphic description data based on semantic information and the two-dimensional projection coordinates includes: Based on the semantic information, constraint prompt words are constructed and a large language model is invoked to generate vector graphic description data adapted to the augmented reality glasses. The coordinate parameters of the vector graphic description data are corrected based on the two-dimensional projection coordinates.

5. The rendering and display method as described in claim 1, characterized in that, Before sending the vector graphics description data to the augmented reality glasses, the rendering and display method further includes: The generated vector graphics description data is preprocessed, including but not limited to at least one of syntax validation, path simplification, and code reduction.

6. The rendering and display method as described in claim 1, characterized in that, Sending the vector graphic description data to the augmented reality glasses includes: Obtain the hardware capability information of the augmented reality glasses, and determine whether the augmented reality glasses support hardware vector acceleration based on the hardware capability information to select the transmission mode; If hardware vector acceleration is supported, the vector graphics description data is transmitted to the augmented reality glasses; If hardware vector acceleration is not supported, then a vector graphic corresponding to the user interaction component is drawn based on the vector graphic description data, and the drawn vector graphic is rendered as a bitmap and then transmitted to the augmented reality glasses.

7. A rendering and display method applied to augmented reality glasses, characterized in that, The method includes: Receive vector graphic description data sent by the source device, wherein the vector graphic description data is generated by the source device based on semantic information, and the semantic information is extracted by the source device from user interaction components; The vector graphics are drawn according to the vector graphics description data to obtain the vector graphics corresponding to the user interaction component; The vector graphics are displayed.

8. A rendering and display method according to claim 7, characterized in that, The step of drawing graphics according to vector graphic description data includes: Based on the vector graphic description data, graphics are drawn in the user interaction layer; The user interaction layer is overlaid on the background layer for display, and the background layer contains background image data.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer device is configured as a source device, the processor executes the computer program to implement the rendering and display method according to any one of claims 1-6; or, when the computer device is configured as augmented reality glasses, the processor executes the computer program to implement the rendering and display method according to claim 7 or 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the rendering and display method as described in any one of claims 1-8.

Images