A heterogeneous data collaborative screen projection method and system based on dual-mode communication

Abstract

The present application relates to wireless projection and augmented reality technical field, specifically relates to a kind of heterogeneous data collaborative projection method and system based on dual-mode communication, comprising: S1: mobile terminal obtains the graphic text content to be projected, graphic text content is parsed, separates out picture data and text data;S2: mobile terminal passes through first communication link and transmits picture data to intelligent AR glasses;Mobile terminal passes through second communication link and transmits text data and layout instruction to intelligent AR glasses;S3: intelligent AR glasses receive picture data in real time and store in cache, while receiving text data and the layout instruction;Detect the receiving state of target picture, if has been received completely, render text and picture into display frame according to layout instruction;If not received completely, call local pre-stored placeholder, render into temporary display frame and output first;S4: intelligent AR glasses project the graphic text display frame generated to its display module;Solve the problem that the power consumption of video stream projection mode is high, and the endurance is short.

Description

Technical Field

[0001] This invention relates to the fields of wireless projection and augmented reality technology, specifically to a heterogeneous data collaborative projection method and system based on dual-mode communication, and more particularly to a method for transmitting images via Wi-Fi, transmitting text via Bluetooth, and synthesizing the data on the glasses. Background Technology

[0002] With the development of Augmented Reality (AR) technology, smart AR glasses, as wearable computing platforms, are widely used in scenarios such as reading assistance, real-time translation, navigation guidance, and industrial inspection. Currently, mobile terminals such as smartphones and tablets project content onto AR glasses primarily using the following two methods: The first method is screen mirroring via traditional wireless projection protocols, such as Miracast, AirPlay, and DLNA. This method encodes the entire screen of the mobile terminal as a video stream in real time and transmits it to the AR glasses. Its advantage is that it achieves a complete mirroring of the screen content and has good compatibility. However, its disadvantages are equally obvious: video stream transmission requires continuous high bandwidth, and the encoding and decoding process consumes extremely high power, leading to severe overheating of the AR glasses, significantly reduced battery life, and difficulty meeting the lightweight requirements of wearable devices.

[0003] The second method is to transmit data directly via narrowband communication links such as Bluetooth Low Energy (BLE). This method has low power consumption and is suitable for transmitting small amounts of data such as text and notifications. However, Bluetooth bandwidth is limited (usually only a few hundred Kbps to a few Mbps), resulting in extremely high latency when transmitting high-resolution images, sometimes reaching several seconds, which cannot meet the real-time display requirements of scenarios with both text and images.

[0004] Furthermore, existing technologies lack differentiated transmission schemes tailored to the characteristics of text and image content. They typically treat text and images as a whole, resulting in either sacrificing power consumption for image quality or sacrificing image quality for power consumption, making it impossible to strike a balance between the two.

[0005] Therefore, how to ensure high-definition and fast transmission of images, low-power real-time refresh of text information, reduce the decoding burden on the glasses, and ensure the synchronization of image and text display are technical problems that urgently need to be solved in this field. Summary of the Invention

[0006] To overcome the aforementioned technical problems in the prior art, this invention provides a heterogeneous data collaborative projection method and system based on dual-mode communication. By dividing the transmission of images and text through the dual-mode communication link, heterogeneous data collaboration is achieved. While ensuring high-definition and fast transmission of images and low-power real-time refresh of text, the processing flow on the glasses is simplified, the hardware load is reduced, and the smoothness of projection and user experience are improved.

[0007] To achieve the above objectives, this invention provides a heterogeneous data collaborative projection method based on dual-mode communication, applied to a system including a mobile terminal and smart AR glasses, comprising the following steps: S1: The mobile terminal acquires the graphic and text content to be projected, parses the graphic and text content, and separates the image data and text data; S2: The mobile terminal transmits the image data to the smart AR glasses through a first communication link; simultaneously, the mobile terminal transmits the text data and corresponding layout instructions to the smart AR glasses through a second communication link; S3: The smart AR glasses receive the image data in real time and store it in a cache, and simultaneously receive the text data and the layout instructions; detect the reception status of the target image, if the image has been completely received, directly render the text and image into a display frame according to the layout instructions; if the image has not been completely received, call the locally pre-stored placeholder image or dynamically generate a placeholder frame, render the text and placeholder image into a temporary display frame for output first; S4: The smart AR glasses project the generated graphic and text display frame onto its display module for display.

[0008] Preferably, step S2 further includes a synchronization mechanism: when the mobile terminal sends the text data through the second communication link, it attaches the image ID or timestamp corresponding to the text data; the smart AR glasses match the text with the cached image based on the image ID or timestamp.

[0009] Preferably, in step S3, after the image data is fully received, a refresh is automatically triggered to replace the placeholder image with the complete image and update the display.

[0010] Preferably, the first communication link is a high-bandwidth communication link, and the second communication link is a low-power communication link.

[0011] Preferably, the first communication link is a Wi-Fi P2P link, and the second communication link is a Bluetooth Low Energy (BLE) link.

[0012] Preferably, the image data is compressed and encoded before transmission, with the encoding format being JPEG, PNG, or HEIC; the text data is transmitted in plain text or rich text format.

[0013] Preferably, the layout instructions include one or more of the following: the display position of the text in the image, the font, the color, and the size.

[0014] Accordingly, the present invention also provides a heterogeneous data collaborative projection system based on dual-mode communication, including a mobile terminal and smart AR glasses; the mobile terminal includes: a content parsing module: used to separate the graphic and text content to be projected into image data and text data; a first communication module: used to send the image data to the smart AR glasses through a first communication link; a second communication module: used to send the text data and layout instructions to the smart AR glasses through a second communication link; the smart AR glasses include: a first receiving module: used to receive the image data through the first communication link; a second receiving module: used to receive the text data and the layout instructions through the second communication link; a graphic and text synthesis rendering module: used to detect the image reception status according to the layout instructions, and generate a temporary display frame using a placeholder image when the image is not fully received, or generate a complete graphic and text display frame when the image is fully received; a display driving module: used to drive the display module to work and transmit the synthesized graphic and text display frame to the display module; and a display module: used to project the graphic and text display frame to realize the visualization of the projected content.

[0015] Preferably, the first communication module and the first receiving module are Wi-Fi communication modules, and the second communication module and the second receiving module are Bluetooth Low Energy (BLE) communication modules.

[0016] Preferably, the image and text compositing rendering module is also used to automatically trigger a refresh after the image data is fully received, so as to replace the placeholder image with the complete image and update the display.

[0017] The present invention has at least the following technical effects through the technical solution provided by the present invention: This invention utilizes the high bandwidth of Wi-Fi for images that are not sensitive to real-time requirements but have large data volumes, and the low power consumption of BLE for text that has high real-time requirements but small data volumes. After the image transmission is complete, the Wi-Fi module can enter sleep mode, and text refresh only requires BLE operation, achieving an optimized balance between power consumption and performance. The processing load on the glasses is significantly reduced: the glasses do not need to perform complex image and text parsing and separation, but only need to perform combined rendering, without running the complete operating system projection protocol stack, reducing the load on the glasses' processor and further extending battery life. Real-time refresh experience: for text that needs to be refreshed frequently (such as notification messages, subtitles, translation results), millisecond-level refresh can be achieved because the BLE link is used, without retransmitting the entire image screen. For example, when turning pages in a news app, if the background image remains unchanged, new text can be sent simply via BLE, resulting in a smoother user experience. The image-text synchronization mechanism ensures display accuracy: by using image IDs or timestamps, it ensures correct matching of text and images, avoiding cross-page text and image mismatches; during image loading, a placeholder image and text are displayed first to avoid blank screens or long waiting times, significantly reducing the perceived latency for users; and images are automatically and seamlessly replaced after loading, providing a natural and smooth experience. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a heterogeneous data collaborative screen projection method based on dual-mode communication provided in an embodiment of the present invention; Figure 2 This is a structural block diagram of the collaborative screen projection system based on dual-mode communication in an embodiment of the present invention; Figure 3 This is a flowchart of a collaborative screen projection method based on dual-mode communication in one embodiment of the present invention; Figure 4 This is a schematic diagram of the separation and synthesis of text and image data in an embodiment of the invention.

[0019] Icons: Mobile terminal 100, Smart AR glasses 101, Content interpretation module 110, Wi-Fi transmission module 120, Bluetooth transmission module 130, Wi-Fi receiving module 210, Image and text synthesis and rendering module 230, Bluetooth receiving module 220, Display driver module 240, Display module 250. Detailed Implementation

[0020] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0021] In this invention, the terms "system" and "network" are used interchangeably. "Multiple" refers to two or more; therefore, in this invention, "multiple" can also be understood as "at least two." "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, it should be understood that in the description of this invention, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0022] Please see Figure 1 This invention provides a heterogeneous data collaborative projection method based on dual-mode communication, applicable to a system including a mobile terminal and smart AR glasses. The mobile terminal is a portable smart device such as a mobile phone or tablet with Wi-Fi P2P and Bluetooth Low Energy (BLE) communication capabilities. The smart AR glasses are wearable AR devices equipped with a display module and a cache module, and supporting Wi-Fi P2P and Bluetooth Low Energy (BLE) communication. The method specifically includes the following steps: S1: The mobile terminal acquires the image and text content to be projected, parses the image and text content, and separates the image data and text data. Specifically, the mobile terminal acquires the image and text content selected by the user through its built-in projection application or system-level projection function. The image and text content is a combination of static / dynamic images and text, such as news articles, novels with images and text, navigation maps and text guidance, translated text and accompanying images, etc. The content parsing module of the mobile terminal performs structured parsing of the image and text content, extracts the image data through image recognition technology, and separates the text data through character recognition and text extraction technology, achieving complete heterogeneous separation of image data and text data.

[0023] S2: The mobile terminal transmits the image data to the smart AR glasses via a first communication link; simultaneously, the mobile terminal transmits the text data and corresponding layout instructions to the smart AR glasses via a second communication link; wherein, the first communication link is a high-bandwidth communication link, specifically a Wi-Fi P2P peer-to-peer communication link, and the second communication link is a low-power communication link, specifically a low-power Bluetooth BLE communication link; the layout instructions are text display control instructions generated by the mobile terminal based on the original layout of the text and image content, including one or more of the following: the text's display position in the image, font, color, and size, such as the display position being the semi-transparent area in the lower left corner of the image, the font being bold, the color being white, and the font size being 14; the image data undergoes compression encoding processing by the mobile terminal before transmission, with the encoding format being one of JPEG, PNG, or HEIC, and the corresponding encoding format is selected according to the image type and image quality requirements, such as using JPEG format for photo images, PNG format for transparent background images, and HEIC format for high-definition large images. The format reduces the amount of image data transmitted through compression encoding, thereby improving transmission efficiency; the text data is transmitted in plain text or rich text format, where plain text format is suitable for ordinary text projection, and rich text format is suitable for projection scenarios that need to retain text bolding, italics, paragraphing, and other formatting.

[0024] Furthermore, step S2 also includes a synchronization mechanism: when the mobile terminal sends the text data through the second communication link, it attaches the image ID or timestamp corresponding to the text data; the image ID is a unique number or character identifier assigned by the mobile terminal to each image data, and the timestamp is the real-time time information of the mobile terminal sending the image data. The smart AR glasses accurately match the text with the cached image based on the image ID or timestamp, avoiding the problem of text and image mismatch in multi-image and text projection scenarios.

[0025] S3: The smart AR glasses receive the image data in real time and store it in a local cache pool. Simultaneously, they receive the text data and layout instructions via a Bluetooth Low Energy (BLE) link. The image compositing and rendering module of the smart AR glasses continuously monitors the reception status of the target image. If the image has been completely received and decoded, the text and image are directly rendered into a complete image and text display frame according to the layout instructions. If the image has not been completely received, or has been received but not yet decoded, a pre-stored placeholder image is immediately called, or a placeholder frame is dynamically generated using an algorithm. The text and placeholder image / frame are then rendered into a temporary display frame according to the layout instructions and output to the display driver module. After the image data is completely received and decoded, the image compositing and rendering module of the smart AR glasses automatically triggers a screen refresh, replacing the placeholder image / frame in the temporary display frame with the complete target image and updating it to a complete image and text display frame. The entire refresh process is seamless and does not affect the user's visual experience.

[0026] S4: The smart AR glasses project the generated temporary or complete graphic display frames onto its display module through the display driver module for display, thereby realizing the visualization of graphic content on the screen; the display module is the optomechanical display module of the smart AR glasses, and the display driver module converts the synthesized display frame into a signal adapted to the optomechanical display module, which is then projected onto the lens display area of ​​the AR glasses for the user to view.

[0027] In one implementation, a heterogeneous data collaborative screen projection method and system based on dual-mode communication is provided, applied to a news and image reading screen projection scenario using a mobile phone (mobile terminal 100) + smart AR glasses. Users project news and image content from their mobile phones onto the smart AR glasses for immersive reading. Figure 2 , Figure 3 , Figure 4 The specific implementation steps of this embodiment are as follows: 1. Data Parsing and Separation: When a user opens a news article containing a headline image and text content in a news client on their mobile phone, triggering the screen mirroring command, the phone's content parsing module performs structured parsing of the news article's image and text content. It extracts the high-resolution headline image from the news article using image recognition technology as image data, and separates the text paragraphs from the news article using text extraction technology as text data. Simultaneously, the content parsing module generates layout instructions based on the original news layout, specifying that the text be displayed in a semi-transparent black area below the image, using Microsoft YaHei font, white color, and a font size of 12. The phone's data preprocessing module compresses the headline image into JPEG format and converts the news text into plain text format. 2. Dual-mode dual-channel data transmission: Wi-Fi P2P Channel: The mobile phone's first communication module (Wi-Fi P2P module) sends the compressed and encoded headline image data to the first Wi-Fi receiving module 210 of the smart AR glasses through the Wi-Fi P2P point-to-point communication link. After receiving the data, the smart AR glasses store the image data in the local image cache pool. Bluetooth Low Energy (BLE) channel: The second communication module of the mobile phone (Bluetooth Low Energy (BLE) module) packages the plain text news text, layout instructions and the image ID (number: IMG2024001) corresponding to the text into a data packet and sends it to the second Wi-Fi receiving module 210 of the smart AR glasses through the BLE communication link. 3. Image and text composite with a placeholder buffer at the end of the glasses: After receiving the text, layout instructions and image ID, the second Wi-Fi receiving module 210 of the smart AR glasses transmits them to the image and text synthesis rendering module 230. This module matches the corresponding headline image in the image cache pool according to the image ID "IMG2024001" and detects the image reception status at the same time. If the headline image has not been fully received, the image and text compositing rendering module 230 immediately calls the locally pre-stored solid color placeholder image (gray), renders the news text and the gray placeholder image into a temporary display frame according to the layout instructions, and transmits it to the display driver module 240. After the headline image is fully received and decoded, the image and text synthesis rendering module 230 automatically triggers a screen refresh, replacing the gray placeholder image in the temporary display frame with the complete headline image, generating a complete image and text display frame and transmitting it to the display driver module 240. 4. Screen projection display: The display driver module 240 of the smart AR glasses converts temporary display frames / complete display frames into electrical signals that adapt to the optical engine display module 250, drives the display module 250 to work, and projects the graphic display frames onto the lens display area of ​​the AR glasses, so that the user can view the projected news graphic content. 5. Page Turning and Screen Casting: When a user triggers a news page turning command on their mobile phone, the phone detects that the background image of the next page is the same as the previous page. It only sends the new news text data and the updated layout command through the BLE communication link. The image and text synthesis rendering module 230 of the smart AR glasses directly retrieves the original image data in the image cache pool, synthesizes it with the new text data, and refreshes the display. There is no need to retransmit the image data, so as to realize fast screen casting for page turning.

[0028] In another embodiment, a heterogeneous data collaborative screen projection method and system based on dual-mode communication is provided, applied to an outdoor navigation screen projection scenario using a tablet (mobile terminal 100) + smart AR glasses. The user projects the navigation map and text guidance content from the tablet onto the smart AR glasses. The core difference of this embodiment is that the projected content is a dynamic map image + real-time updated navigation text guidance. The specific implementation steps are as follows: 1. Data Parsing and Separation: When a user sets a navigation route in the tablet's navigation software and triggers a screen mirroring command, the tablet's content parsing module parses the navigation image and text content, extracts the dynamic navigation map as image data, and separates the real-time updated navigation text guidance (such as "Turn left in 500 meters" and "Enter the main road") as text data; it generates layout instructions, specifying that the text is displayed in the semi-transparent white area in the upper right corner of the map, with the font being bold, the color being black, and the font size being 14; the data preprocessing module compresses and encodes the dynamic navigation map in PNG format and converts the navigation text into plain text format; 2. Dual-mode dual-channel data transmission: The tablet's Wi-Fi P2P module continuously transmits compressed dynamic navigation map data to the smart AR glasses, while the BLE module sends updated navigation text, layout instructions, and image timestamps (real-time update time of the navigation map) in real time. 3. Image and text synthesis and real-time refresh on the glasses: The image and text synthesis rendering module 230 of the smart AR glasses matches the dynamic map and navigation text according to the timestamp. When the map is not fully received, a navigation placeholder frame (simple road outline) is called to generate a temporary display frame; after the map is fully received, a complete display frame is generated; when the navigation text is updated in real time, new text data is received only through the BLE link, and the text content in the display frame is quickly refreshed without retransmitting map data. 4. Projection Display: The display module 250 of the smart AR glasses projects navigation graphics and text display frames onto the lenses, allowing users to view the projected navigation content in real time while walking outdoors, achieving immersive navigation.

[0029] In another implementation, as an extension of the above methods and systems, for screen projection scenarios involving lightweight SVG vector graphics and text (such as office document flowcharts and simple icons), the mobile terminal's content parsing module detects that the image data is a lightweight SVG vector graphic with a small data volume. In this case, there is no need to transmit via Wi-Fi P2P link; instead, the SVG vector graphic data, text data, and layout instructions are directly transmitted to the smart AR glasses via a low-power Bluetooth BLE link. The smart AR glasses' image and text synthesis rendering module directly receives and completes the image and text synthesis, further reducing communication power consumption and improving projection efficiency. For screen projection scenarios involving 8K ultra-high-definition landscape images and text, the mobile terminal performs high-definition compression encoding of the image in HEIC format and forces transmission via Wi-Fi P2P link to ensure the high-definition display effect of the image. The text data is still transmitted via BLE link, achieving smooth projection of high-definition images and text.

[0030] Please see Figure 2Based on the same inventive concept, this invention provides a heterogeneous data collaborative projection system based on dual-mode communication, used to implement the aforementioned heterogeneous data collaborative projection method based on dual-mode communication. The system includes a mobile terminal and smart AR glasses. A Wi-Fi P2P communication link and a Bluetooth Low Energy (BLE) communication link are established between the mobile terminal and the smart AR glasses to achieve bidirectional data transmission. The mobile terminal is a portable smart device such as a mobile phone or tablet, and it has a built-in hardware communication module and software function module, specifically including: a content parsing module: a software function module of the mobile terminal, developed based on image recognition and text extraction algorithms, used to perform structured parsing of the image and text content to be projected, separating it into independent image data and text data, and generating corresponding layout instructions according to the original layout of the image and text content. The layout instructions include one or more of the following: the display position of the text in the image, font, color, and size; and a first communication module: a Wi-Fi P2P hardware communication module of the mobile terminal, equipped with a Wi-Fi P2P communication chip and a corresponding communication driver, used to transmit data via Wi-Fi P2P. The P2P high-bandwidth communication link sends compressed image data to the smart AR glasses; the second communication module is the low-power Bluetooth BLE hardware communication module of the mobile terminal, equipped with a low-power Bluetooth BLE communication chip and corresponding communication driver, used to send text data and layout instructions to the smart AR glasses through the low-power Bluetooth BLE communication link, and attaching the image ID or timestamp corresponding to the text data when sending; the image and text synthesis rendering module is the core software function module of the smart AR glasses, electrically connected to the first receiving module, the second receiving module and the image cache pool, and has built-in image receiving status detection algorithm, image and text rendering algorithm and screen refresh algorithm, used to match text data with corresponding image data according to the received image ID or timestamp, detect the receiving and decoding status of the target image in real time, and call the locally pre-stored placeholder image or dynamically generate a placeholder when the image is not completely received / decoded. The frame generates a temporary display frame using placeholder images and text data according to layout instructions; or, after the image is fully received and decoded, it generates a complete graphic and text display frame using text and image data according to layout instructions. Simultaneously, this module is also used to automatically trigger a screen refresh after the image data is fully received and decoded, replacing the placeholder image / placeholder frame in the temporary display frame with the complete image and updating the display. The display driver module is the hardware driver module for the smart AR glasses, electrically connected to the graphic and text compositing rendering module and the display module. It carries a display driver chip and corresponding driver program, used to convert the temporary or complete display frame generated by the graphic and text compositing rendering module into electrical signals adapted to the display module, driving the display module to work and transmitting the composite graphic and text display frame to the display module.Display Module: This is the optomechanical display module for the smart AR glasses, electrically connected to the display driver module. It serves as the final display unit for the projected content, receiving signals from the display driver module and projecting image and text frames onto the lens display area of ​​the AR glasses to visualize the projected content. The smart AR glasses also include an image cache pool, a built-in flash memory cache module electrically connected to the first receiving module and the image / text synthesis and rendering module. This pool is specifically used to store image data received by the first receiving module, facilitating quick retrieval and matching by the image / text synthesis and rendering module. Furthermore, if the background image remains unchanged when the user flips through pages during projection, the image data in the cache pool can be directly retrieved without needing to be received again.

[0031] The beneficial effects of this invention are as follows: It utilizes the high bandwidth of Wi-Fi to transmit images that are not sensitive to real-time requirements but have large data volumes, and utilizes the low power consumption of BLE to transmit text that has high real-time requirements but small data volumes. After image transmission is complete, the Wi-Fi module can enter sleep mode, and text refresh only requires BLE operation, achieving an optimized balance between power consumption and performance. The glasses do not need to perform complex image and text parsing and separation, only combined rendering, without running a complete operating system projection protocol stack, reducing the load on the glasses' processor and further extending battery life. For text that requires frequent refresh (such as notification messages, subtitles, and translation results), millisecond-level refresh can be achieved due to the BLE link, without retransmitting the entire image screen. For example, when turning pages in a news app, if the background image remains unchanged, only new text needs to be sent via BLE, resulting in a smoother user experience. Image ID or timestamp mechanisms ensure correct matching of text and images, avoiding cross-page text and image mismatches. During image loading, a placeholder image and text are displayed first to avoid blank screens or long waiting times, significantly reducing the perceived latency for the user. Automatic and seamless replacement after image loading provides a natural and smooth experience.

[0032] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0033] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not describe the various possible combinations separately.

[0034] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0035] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. A heterogeneous data collaborative projection method based on dual-mode communication, applied to a system including a mobile terminal and smart AR glasses, characterized in that, Includes the following steps: S1: The mobile terminal acquires the graphic and text content to be projected, parses the graphic and text content, and separates the image data and text data; S2: The mobile terminal transmits the image data to the smart AR glasses via the first communication link; simultaneously, the mobile terminal transmits the text data and corresponding layout instructions to the smart AR glasses via the second communication link. S3: The smart AR glasses receive the image data in real time and store it in the cache, and at the same time receive the text data and the layout instructions; detect the receiving status of the target image. If the image has been completely received, the text and image are directly rendered into a display frame according to the layout instructions; if the image has not been completely received, the local pre-stored placeholder image is called or a placeholder frame is dynamically generated, and the text and placeholder image are rendered into a temporary display frame for output first. S4: The smart AR glasses project the generated image and text display frames onto its display module for display.

2. The heterogeneous data collaborative projection method based on dual-mode communication according to claim 1, characterized in that, The S2 also includes a synchronization mechanism: when the mobile terminal sends the text data through the second communication link, it attaches the image ID or timestamp corresponding to the text data; the smart AR glasses match the text with the cached image based on the image ID or timestamp.

3. The heterogeneous data collaborative projection method based on dual-mode communication according to claim 2, characterized in that, In step S3, after the image data is fully received, a refresh is automatically triggered to replace the placeholder image with the complete image and update the display.

4. The heterogeneous data collaborative projection method based on dual-mode communication according to claim 1, characterized in that, The first communication link is a high-bandwidth communication link, and the second communication link is a low-power communication link.

5. The heterogeneous data collaborative screen projection method based on dual-mode communication according to claim 4, characterized in that, The first communication link is a Wi-Fi P2P link, and the second communication link is a Bluetooth Low Energy (BLE) link.

6. The heterogeneous data collaborative projection method based on dual-mode communication according to claim 1, characterized in that, The image data is compressed and encoded before transmission, with the encoding format being JPEG, PNG, or HEIC; the text data is transmitted in plain text or rich text format.

7. The heterogeneous data collaborative projection method based on dual-mode communication according to claim 1, characterized in that, The layout instructions include one or more of the following: the position of the text in the image, the font, the color, and the size.

8. A heterogeneous data collaborative projection system based on dual-mode communication, used to implement the method according to any one of claims 1-7, characterized in that, Including mobile devices and smart AR glasses; The mobile terminal includes: Content parsing module: Used to separate the image and text content to be projected into image data and text data; First communication module: used to send the image data to the smart AR glasses via a first communication link; Second communication module: used to send the text data and layout instructions to the smart AR glasses through the second communication link; The smart AR glasses include: First receiving module: used to receive the image data through a first communication link; Second receiving module: used to receive the text data and the layout instructions through a second communication link; Image and text compositing rendering module: used to detect the image receiving status according to the layout instructions, and generate a temporary display frame using a placeholder image when the image is not fully received, or generate a complete image and text display frame when the image is fully received; Display driver module: Used to drive the display module to work and transmit the synthesized graphic display frame to the display module; Display module: Used to project the graphic display frame to visualize the projected content.

9. The heterogeneous data collaborative projection system based on dual-mode communication according to claim 8, characterized in that, The first communication module and the first receiving module are Wi-Fi communication modules, and the second communication module and the second receiving module are Bluetooth Low Energy (BLE) communication modules.

10. The heterogeneous data collaborative projection system based on dual-mode communication according to claim 8, characterized in that, The image and text compositing and rendering module is also used to automatically trigger a refresh after the image data is fully received, replacing the placeholder image with the complete image and updating the display.

Images