Rendering method and vehicle

Abstract

The present disclosure relates to a rendering method and a vehicle, the method comprising: determining a first rendering task of a mobile terminal and a second rendering task of a display terminal based on a communication quality between the mobile terminal and the display terminal; sending the first rendering task to the mobile terminal, receiving first rendering information sent by the mobile terminal; determining second rendering information based on the second rendering task; determining a display picture based on the first rendering information and the second rendering information. Wherein a cooperative rendering architecture of the mobile terminal and the display terminal is established, rendering tasks are dynamically allocated according to the communication quality, when the communication link between the mobile terminal and the display terminal fluctuates, the amount of tasks that the mobile terminal needs to render is reduced, thereby reducing the amount of transmitted data, reducing the burden of the communication link, and improving the stability and smoothness of picture display.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a rendering method and a vehicle. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] Through communication technology, applications on mobile terminals (such as mobile phones) can be projected onto the display screen of display terminals (such as in-vehicle systems).

[0004] In the communication solutions provided by related technologies, the mobile terminal acts as an independent rendering terminal, responsible for generating a complete display screen and transmitting it to the display terminal, while the display terminal only passively receives and decodes the screen data.

[0005] However, in complex environments, the communication link between the mobile terminal and the display terminal may fluctuate, which can reduce the stability and smoothness of the screen display. Summary of the Invention

[0006] To overcome or at least partially solve the above problems, this disclosure provides a rendering method and a vehicle.

[0007] A first aspect of this disclosure provides a rendering method applied to a display terminal, the method comprising: Based on the communication quality between the mobile terminal and the display terminal, a first rendering task of the mobile terminal and a second rendering task of the display terminal are determined, wherein when the communication quality decreases, the workload of the first rendering task decreases and the workload of the second rendering task increases. Send the first rendering task to the mobile terminal and receive the first rendering information sent by the mobile terminal; Based on the second rendering task, determine the second rendering information; The display screen is determined based on the first rendering information and the second rendering information.

[0008] Based on the same inventive concept, a second aspect of the present disclosure provides a vehicle including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect.

[0009] The technical solution provided in this disclosure has the following advantages: A collaborative rendering architecture between mobile terminals and display terminals was established. Rendering tasks were dynamically allocated based on communication quality. When the communication link between the mobile terminal and the display terminal fluctuated, the amount of rendering tasks required by the mobile terminal was reduced, thereby reducing the amount of data transmitted, alleviating the burden on the communication link, and improving the stability and smoothness of the screen display. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0011] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of the rendering method provided in the exemplary embodiments of this disclosure; Figure 2 This is a schematic flowchart of a rendering method provided in an exemplary embodiment of this disclosure; Figure 3 This is a schematic diagram of a rendering apparatus provided in an exemplary embodiment of the present disclosure; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of a vehicle provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0013] It is understood that the rendering-related data and communication-related data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and provisions.

[0014] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0015] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0016] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] To overcome the above problems, this disclosure provides a rendering method, and various non-limiting embodiments of this disclosure are described in detail below.

[0019] See Figure 1 This is a schematic diagram illustrating an application scenario of the rendering method provided in an exemplary embodiment of this disclosure.

[0020] In some exemplary embodiments, the application scenario includes a display terminal 110 and a mobile terminal 120.

[0021] Display terminal 110 includes vehicle-mounted terminals, smart TVs, TV boxes, conference screens, or other electronic devices that can be projected onto the screen.

[0022] As an example, the display terminal 110 is a terminal installed on the vehicle (i.e., an in-vehicle terminal) with data acquisition and processing capabilities. It can be a vehicle controller or a central processing unit in an autonomous driving system. The display terminal 110 can acquire relevant information and process the acquired information. In this embodiment of the disclosure, the display terminal 110 is the electronic device that ultimately controls the vehicle.

[0023] Mobile terminal 120 includes electronic devices that are close to the user and have data transmission and multimedia input / output functions, including but not limited to mobile phones, desktop computers, mobile computers, tablet computers, media players, smart wearable devices, personal digital assistants (PDAs) or other electronic devices that can perform the above functions.

[0024] The display terminal 110 and the mobile terminal 120 can be connected via wired or wireless communication networks to achieve data interaction.

[0025] As an example, when the display terminal 110 is an in-vehicle terminal, it can be connected to the mobile terminal 120 via Android Auto (Android for cars, a function for connecting mobile phones in cars).

[0026] The display terminal 110 determines the first rendering task of the mobile terminal 120 and the second rendering task of the display terminal 110 based on the communication quality between the mobile terminal 120 and the display terminal 110. Display terminal 110 sends the first rendering task to mobile terminal 120 and receives the first rendering information sent by mobile terminal 120. The display terminal 110 determines the second rendering information based on the second rendering task; The display terminal 110 determines the display screen based on the first rendering information and the second rendering information.

[0027] In some exemplary embodiments, the application scenario also includes a server 130.

[0028] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0029] The display terminal 110, mobile terminal 120 and server 130 can be connected via wired or wireless communication networks to achieve data interaction.

[0030] Server 130 can be used to provide display terminal 110 with functions such as communication quality prediction and rendering task parameter optimization suggestions.

[0031] The following is combined Figure 1 The above application scenarios are used to describe the rendering scheme according to exemplary embodiments of this disclosure. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this disclosure, and the embodiments of this disclosure are not limited in any way. Rather, the embodiments of this disclosure can be applied to any applicable scenario.

[0032] refer to Figure 2 This is a flowchart illustrating a rendering method provided by an exemplary embodiment of the present disclosure.

[0033] This method can be executed by a rendering device, which can be implemented in software and / or hardware and is generally integrated into an electronic device, such as a display terminal, a vehicle controller, or a central processing unit in an autonomous driving system.

[0034] like Figure 2 As shown, the method includes the following steps: Step 210: Based on the communication quality between the mobile terminal and the display terminal, determine the first rendering task of the mobile terminal and the second rendering task of the display terminal, wherein when the communication quality decreases, the workload of the first rendering task decreases and the workload of the second rendering task increases.

[0035] As an example, let's illustrate the communication between a mobile terminal and a display terminal using Android Auto technology: The display terminal runs the Android Automotive OS operating system and integrates the Android Auto Host module. The mobile terminal is a smartphone running the Android operating system (Android OS) with the Android Auto client application installed. The display terminal establishes a wireless connection with the mobile terminal via the Wi-Fi P2P (WLAN Direct) protocol.

[0036] It should be noted that in this embodiment, the communication between the mobile terminal and the display terminal via Android Auto technology is used as an example. It can be understood that this disclosure is also applicable to application scenarios in which the mobile terminal and the display terminal communicate via other car-connected mobile phone technologies, including but not limited to application scenarios in which the mobile terminal and the display terminal communicate via CarPlay technology, Samsung Auto technology, etc. The specific implementation methods are similar to those in this embodiment, and will not be repeated here.

[0037] In this embodiment, communication quality refers to the transmission performance of the Wi-Fi P2P wireless link between the mobile terminal and the display terminal.

[0038] As an example, communication quality can be quantitatively evaluated using the following parameters: Link bandwidth: The display terminal measures the currently available bandwidth by periodically sending probe packets, for example, at a measurement period of 100 milliseconds. The probe packets use the UDP protocol, and the real-time bandwidth is calculated by counting the number of probe packets successfully transmitted per unit time.

[0039] Transmission delay: The display terminal records the time difference (RTT) between the sending of each data packet from the mobile terminal to the receiving of the display terminal, such as calculating the average delay value within the most recent second.

[0040] Packet loss rate: The ratio of the number of data packets received by the display terminal in the last second to the total number of data packets sent by the mobile terminal is used to calculate the packet loss rate.

[0041] In some exemplary embodiments, determining a first rendering task for the mobile terminal and a second rendering task for the display terminal based on the communication quality between the mobile terminal and the display terminal, wherein when the communication quality deteriorates, the workload of the first rendering task decreases and the workload of the second rendering task increases, includes: Based on the quality level of the communication quality, the rendering object type corresponding to the first rendering task and the second rendering task and the rendering task corresponding to the rendering object type are determined respectively.

[0042] As an example, based on the three communication quality parameters in the example above, the display terminal divides the communication quality into three quality levels: First quality level: Link bandwidth > 15Mbps, transmission latency < 30ms, packet loss rate < 1%. The link is in excellent condition at this level.

[0043] Second quality level: Link bandwidth is between 8-15Mbps, transmission latency is between 30-80ms, and packet loss rate is between 1%-5%. At this level, the link status fluctuates moderately.

[0044] Third quality level: Link bandwidth < 8Mbps, transmission latency > 80ms, packet loss rate > 5%. The link is under strain at this level.

[0045] In some exemplary embodiments, the rendering object type includes dynamic objects and static objects; The step of determining the rendering object type corresponding to the first rendering task and the second rendering task, and the rendering task corresponding to the rendering object type, based on the quality level of the communication quality, includes: In response to the communication quality being a first quality level, the first rendering task includes rendering the dynamic object, and the second rendering task includes rendering the static object; In response to the communication quality being at the second quality level, the first rendering task includes rendering the dynamic object at a preset frame rate to obtain the first rendering information, and the second rendering task includes rendering the static object and post-processing the first rendering information, wherein the preset frame rate is lower than the standard frame rate. In response to the communication quality being at the third quality level, the first rendering task includes rendering keyframes of the dynamic object, and the second rendering task includes rendering the static object and rendering other frames of the dynamic object besides the keyframes. The communication quality represented by the first quality level, the second quality level, and the third quality level decreases sequentially.

[0046] In some exemplary embodiments, the determination of the dynamic object and the static object includes: Based on the type and update frequency of the objects in the display screen, the objects in the display screen are divided into dynamic objects and static objects.

[0047] As an example, the types of objects to be rendered include dynamic objects and static objects: Dynamic objects refer to frequently updated content on the display screen, including navigation map windows (real-time traffic conditions, vehicle location icons), media playback windows (album covers, playback progress bars), and voice control feedback windows (voice-to-text recognition). The update frequency of dynamic objects is typically ≥10 frames per second.

[0048] Static objects refer to content on the display screen that is updated very infrequently, including the status bar (time, signal strength), menu bar (navigation, music, phone, and other function icons), and fixed function icons (back, home, and other buttons). The update frequency of static objects is typically ≤ 1 time per minute.

[0049] Based on object type and update frequency, the display terminal classifies objects on the display screen into dynamic and static objects. Specifically, the display terminal analyzes the interface layout information of the Android Auto application to identify the type of each UI component (such as ImageView (a container for displaying images), TextView (a component for displaying text), SurfaceView (a canvas for drawing and video playback), etc.) and monitors the refresh rate of each component. Components with a refresh rate ≥ 10 frames / second are marked as dynamic objects; components with a refresh rate ≤ 1 time / minute are marked as static objects.

[0050] As an example, the rendering task allocation strategy corresponding to different quality levels is as follows: In response to a communication quality priority of 1, the first rendering task includes rendering all dynamic objects, and the second rendering task includes rendering all static objects. Under this strategy, the mobile terminal is responsible for rendering dynamic areas such as navigation maps and media playback at a standard frame rate (e.g., 30 frames per second), generating complete dynamic object rendering data. The display terminal is responsible for rendering static areas such as the status bar and menu bar, and persistently storing the static object rendering results in the GPU cache to avoid duplicate rendering.

[0051] Responding to a communication quality level of the second quality grade, the first rendering task includes rendering all dynamic objects at a preset frame rate to obtain first rendering information, and the second rendering task includes rendering all static objects and post-processing the first rendering information. Under this strategy, the mobile terminal reduces the rendering frame rate of dynamic objects to a preset frame rate (e.g., 15-20 frames / second) to reduce the amount of data transmitted. In addition to rendering static objects, the display terminal also performs post-processing on the first rendering information transmitted by the mobile terminal, including frame interpolation, sharpening, and other operations, to compensate for the stuttering caused by the reduced frame rate.

[0052] Responding to a communication quality level of three, the first rendering task includes rendering keyframes for dynamic objects, and the second rendering task includes rendering all static objects and rendering all frames of dynamic objects except for the keyframes. Under this strategy, the mobile terminal only renders and transmits keyframes for dynamic objects, for example, with a keyframe interval of 1 second (i.e., transmitting 1 keyframe per second). In addition to rendering static objects, the display terminal also needs to generate transition frames between keyframes based on the keyframe data using a motion estimation algorithm to achieve continuous display of dynamic objects.

[0053] Step 220: Send the first rendering task to the mobile terminal and receive the first rendering information sent by the mobile terminal.

[0054] As an example, the display terminal sends the first rendering task instruction to the mobile terminal through the preset message channel of Android Auto technology.

[0055] As an example, the instruction contains the following fields: The rendering task type, which can be full rendering, reduced frame rate rendering, or keyframe-only rendering.

[0056] Target frame rate, in frames per second. For the first quality level, the value is 30; for the second quality level, the value is 15-20; for the third quality level, the value is 1 (1 keyframe per second).

[0057] Target bitrate, in Mbps. Dynamically adjusted based on communication quality: Level 1 quality is 10-15 Mbps, Level 2 quality is 5-8 Mbps, and Level 3 quality is 2-3 Mbps.

[0058] A list of regions to be rendered, each region containing coordinates and region type.

[0059] As an example, the instruction is encapsulated in JSON format.

[0060] In some exemplary embodiments, the first rendering information includes: The differential data obtained by rendering the dynamic object; The differential data includes the difference between the target frame of the dynamic object and the previous frame of the target frame, and the difference is encoded using differential coding technology.

[0061] As an example, after receiving the first rendering task instruction, the mobile terminal calls the rendering engine of the Android Auto client application to perform GPU rendering of the dynamic object. After rendering is complete, the mobile terminal generates the first rendering information.

[0062] As an example, the first rendering information includes differential data obtained from rendering dynamic objects. Differential data refers to the difference between the target frame of the dynamic object and the previous frame, which is encoded using differential coding techniques.

[0063] As an example, the mobile device maintains a frame buffer queue that stores the rendering results of the most recent N frames (N=2). For the target frame being rendered, the mobile device compares the RGB values ​​of the target frame with those of the previous frame pixel by pixel. Only the differing pixels are encoded to generate differential data packets. The differential data packets contain the coordinates (x, y) of the differing pixels and their new RGB values. For dynamic object areas (such as navigation maps), only changing road lines, vehicle icons, and other elements are encoded; large areas of static content such as the sky background are not encoded.

[0064] As an example, the mobile terminal transmits differential data packets to the display terminal via a Wi-Fi P2P link. The UDP protocol is used to reduce transmission latency. To ensure data integrity, the mobile terminal adds a sequence number and a CRC (Cyclic Redundancy Check) to each differential data packet. The display terminal verifies the data upon receipt and discards any corrupted packets.

[0065] Through the above embodiments, differential coding technology significantly reduces the amount of data transmitted. Taking navigation maps as an example, the data size of a complete frame is approximately 2-3MB (1920×1080 resolution, 24-bit RGB), while the data size of differential data packets is typically only 100-300KB, reducing the data size by more than 90%.

[0066] Step 230: Determine the second rendering information based on the second rendering task.

[0067] In some exemplary embodiments, determining the second rendering information based on the second rendering task includes: Based on the second rendering task, the second rendering information is obtained locally from the display terminal; The second rendering information is obtained by the display terminal rendering the static object and stored locally on the display terminal.

[0068] In this embodiment, the display terminal obtains second rendering information locally based on the second rendering task. The second rendering information is obtained by the display terminal from rendering all static objects and stored locally on the display terminal.

[0069] As an example, the display terminal performs a complete static object rendering during the system initialization phase (e.g., within a preset time period after Android Auto connection is established). The rendering process is as follows: The display terminal obtains the interface layout information of the mobile terminal through the Android Auto protocol and identifies static object areas (status bar, menu bar, fixed function icons).

[0070] The display terminal calls the local GPU rendering engine to render static object areas.

[0071] After rendering is complete, the display terminal stores the rendering result (static frame) of the static object in the GPU cache.

[0072] The display terminal assigns a persistent storage identifier to static frames, which is then directly referenced during subsequent image composition without the need for re-rendering.

[0073] As an example, when a static object area is updated (such as due to changes in time or signal strength), the display terminal only updates the changed pixels, without re-rendering the entire static area. The update process is as follows: Mobile devices use the Android Auto protocol to notify the updated area (coordinates and size) of static objects on the device.

[0074] The display terminal only performs local rendering on the updated area to generate new pixel data.

[0075] The display terminal overwrites the old data at the corresponding position in the GPU cache with the new pixel data, completing the incremental update of the static frame.

[0076] Through the above embodiments, the display terminal avoids the repeated rendering of static objects, saving GPU computing resources and power consumption.

[0077] Step 240: Determine the display screen based on the first rendering information and the second rendering information.

[0078] In this embodiment, the display terminal combines the first rendering information (differential data) transmitted from the mobile terminal with the second rendering information (static frame) stored locally to generate the final display image. The combining process is as follows: The display terminal reads static frame data from the GPU cache as the underlying layer for image compositing.

[0079] The display terminal decodes the differential data packets in the first rendering information and extracts the coordinates and RGB values ​​of the differing pixels.

[0080] The display terminal overlays the difference pixels onto the corresponding positions of the static frame to generate a composite frame.

[0081] The display terminal outputs the composite frame to the display screen, completing one screen refresh.

[0082] In some exemplary embodiments, determining the display screen based on the first rendering information and the second rendering information includes: In response to the detection of frame loss and / or delay in the first rendering information, a compensation frame is generated; The first rendering information, the second rendering information, and the compensation frame are combined to obtain the display screen.

[0083] In this embodiment, the display terminal also needs to handle frame loss and / or delay. When the display terminal detects frame loss and / or delay in the first rendering information, it generates a compensation frame and synthesizes the first rendering information, the second rendering information, and the compensation frame to obtain the display image.

[0084] Frame loss detection: The display terminal determines whether frame loss exists by checking the sequence number continuity of differential data packets. For example, if data packets with sequence numbers 100 and 102 are received, then the data packet with sequence number 101 is determined to be lost.

[0085] Delay detection: The display terminal records the receive timestamp of each differential data packet and compares it with the send timestamp carried in the data packet to calculate the transmission delay. For example, if the delay exceeds 100 milliseconds, a delay is determined to exist.

[0086] As an example, the process for generating compensation frames is as follows: The terminal displays the key frames before and after the lost frame (i.e., frames with sequence numbers 100 and 102) extracted from the frame buffer queue.

[0087] The display terminal extracts motion vectors from keyframes to identify the motion trajectory of dynamic objects. For example, for a vehicle icon in a navigation map, its displacement vector (dx, dy) between two keyframes is extracted.

[0088] The display terminal generates compensation frames for the lost frames using a linear interpolation algorithm based on the motion vectors. The position of the vehicle icon in the compensation frame is: x_compensation = x_100 + 0.5 × dx, y_compensation = y_100 + 0.5 × dy (assuming the lost frame is located between keyframes 100 and 102).

[0089] The display terminal inserts compensation frames into the frame sequence to maintain the continuity of the display frame rate.

[0090] Through the above embodiments, the display terminal can maintain a display frame rate of 25-30 frames per second in scenarios with link fluctuations, thus avoiding screen stuttering.

[0091] In some embodiments, the display terminal also performs priority region partitioning and rendering parameter adjustment based on content characteristics to further optimize the display effect: In some exemplary embodiments, the method further includes: Based on the content characteristics of the displayed screen, the displayed screen is divided into a high-priority area and a low-priority area; Based on the communication quality and the content characteristics, the rendering parameters for the high-priority region and the low-priority region are determined; The rendering parameters are sent to the mobile terminal as part of the first rendering task; The rendering parameters are used to indicate the adjustment of the rendering bitrate and resolution of different regions based on the rendering parameters.

[0092] In this embodiment, the display terminal analyzes the content characteristics of the displayed screen and divides the displayed screen into high-priority areas and low-priority areas.

[0093] As an example, content feature analysis employs image processing algorithms, including edge detection, optical character recognition (OCR), and color histogram analysis. The display terminal performs edge detection on the synthesized frame to identify regions containing a large amount of edge information. For example, areas with high edge density, such as road lines and traffic signs in a navigation map, are marked as high-priority areas.

[0094] The display terminal performs text recognition on the composite frame, identifying areas containing text information. For example, text areas such as road names and distance prompts in a navigation map are marked as high-priority areas.

[0095] The display terminal performs color histogram analysis on the composite frame to identify large areas with a single color. For example, areas such as the sky background in a navigation map or the blurred background in a media playback window are marked as low-priority areas.

[0096] As an example, high-priority areas include: road lines, text labels, traffic signs, and subtitles in a media playback window.

[0097] Low-priority areas include: the sky background and solid color areas in the navigation map, and the blurred background in the media playback window.

[0098] In this embodiment, the display terminal determines the rendering parameters for high-priority and low-priority regions based on communication quality and content characteristics.

[0099] As an example, rendering parameters include rendering bitrate and resolution. The display terminal formulates differentiated rendering parameter strategies based on the communication quality level and regional priority: For high-priority areas: Maintain high bitrate and high resolution. For example, set the bitrate to 8-10 Mbps and the resolution to 1080P (1920×1080). Ensure display clarity in high-priority areas by allocating more transmission bandwidth and rendering resources.

[0100] For low-priority areas: reduce bitrate and resolution. For example, adjust the bitrate to 2-4 Mbps and the resolution to 720P (1280×720). By reducing resource consumption in low-priority areas, bandwidth and GPU computing resources are saved.

[0101] In this embodiment, the display terminal sends the rendering parameters to the mobile terminal as part of the first rendering task.

[0102] As an example, the display terminal can add the following subfields to the first rendering task instruction: A list of high-priority regions, each containing coordinates, target bitrate, and target resolution.

[0103] A list of low-priority regions, each containing coordinates, target bitrate, and target resolution.

[0104] In this embodiment, after receiving the rendering parameters, the mobile terminal uses differentiated encoding parameters for different regions when rendering dynamic objects. For example, for low-priority regions, the encoding complexity and bitrate are reduced.

[0105] When compositing images, the display terminal synchronously adjusts the resolution of the corresponding areas. For example, it downsamples low-priority areas, scaling the 1080P rendering result down to 720P before compositing it into the final image.

[0106] Through the above content feature analysis and priority area division mechanism, the display terminal prioritizes the display quality of high-priority areas under limited transmission bandwidth, achieving a technical effect of improving the clarity of navigation text by 35%.

[0107] In some embodiments, the display terminal also implements a connection interruption and recovery mechanism to cope with momentary disconnections of the Wi-Fi P2P link: In some exemplary embodiments, the method further includes: In response to detecting a connection interruption between the mobile terminal and the display terminal, the cached first rendering information and second rendering information are invoked; In response to the detection that the connection between the mobile terminal and the display terminal has been restored, the display screen is determined based on the cached first rendering information and the second rendering information.

[0108] In this embodiment, in response to the detection of a connection interruption between the mobile terminal and the display terminal, the cached first rendering information and second rendering information are invoked.

[0109] The display terminal determines whether a connection interruption has occurred by monitoring the connection status of the Wi-Fi P2P link.

[0110] As an example, the conditions for determining a connection interruption include: no data packets are received from the mobile terminal for 3 consecutive seconds; the Wi-Fi P2P protocol stack reports a connection disconnection event.

[0111] During normal operation, the display terminal maintains a rendering information cache queue, periodically (e.g., every 500 milliseconds) storing the latest first rendering information (differential data) and second rendering information (static frames) into the local cache. The cache queue has a capacity of rendering data for the most recent 10 seconds, with a data size of approximately 100-200MB.

[0112] When a connection interruption is detected, the display terminal stops receiving the first rendering information from the mobile terminal and instead reads the latest first and second rendering information from the local cache queue to continue image compositing and display output. At this time, the display screen retains the state of the last received dynamic objects, while static objects remain unchanged.

[0113] In this embodiment, in response to the detection that the connection between the mobile terminal and the display terminal has been restored, the display screen is determined based on the cached first rendering information and second rendering information.

[0114] The display terminal continuously monitors the connection status of the Wi-Fi P2P link. When connection restoration is detected (receiving a data packet from the mobile terminal or a connection establishment event reported by the Wi-Fi P2P protocol stack), the display terminal executes the following restoration procedure: The display terminal sends a synchronization request to the mobile terminal, requesting the mobile terminal to transmit the latest keyframe data.

[0115] Upon receiving the synchronization request, the mobile terminal immediately renders and transmits a complete keyframe containing complete rendering data for all dynamic objects.

[0116] After receiving the keyframe, the display terminal combines it with the locally cached second rendering information (static frame) to generate the restored first frame display image.

[0117] The display terminal clears the old data in the cache queue, restarts receiving differential data from the mobile terminal, and resumes the normal collaborative rendering process.

[0118] The entire process from connection restoration to display refresh typically takes 300-500 milliseconds, resulting in extremely short perceived screen interruption time and significantly improved user experience.

[0119] As an example, let's illustrate this by applying it to an urban business district scenario: This embodiment describes the collaborative rendering process when a vehicle is driving in an urban commercial district. The electromagnetic environment of urban commercial districts is complex, and the bandwidth of Wi-Fi P2P links fluctuates frequently, typically ranging from 5 to 20 Mbps.

[0120] Initial stage: The user starts the vehicle, and the mobile terminal establishes a connection with the display terminal via Wi-Fi P2P. The display terminal detects a link bandwidth of 18Mbps, a latency of 25ms, and a packet loss rate of 0.5%, classifying the communication quality as the highest quality level. The display terminal sends the first rendering task instruction to the mobile terminal, specifying the rendering task type as full rendering, the target frame rate as 30 frames per second, and the target bitrate as 12Mbps. The mobile terminal begins rendering the navigation map (dynamic object) at a frame rate of 30 frames per second, while the display terminal renders the status bar and menu bar (static objects), storing the static frames in the GPU cache.

[0121] Stable operation phase: As the vehicle enters the main road of the commercial district, the link bandwidth remains at 15-18 Mbps. The mobile terminal continuously transmits differential data of the navigation map, with a data volume of approximately 200-300 KB / frame. The display terminal receives the differential data, combines it with local static frames, and outputs the displayed image. The display frame rate remains stable at 30 frames per second, resulting in smooth and lag-free visuals.

[0122] Link fluctuation phase: As the vehicle enters the underground parking lot of the shopping district, the link bandwidth drops to 10Mbps, latency rises to 50ms, and packet loss rate increases to 3%. The display terminal determines that the communication quality has dropped to the second quality level and sends a new first rendering task instruction to the mobile terminal, specifying the rendering task type as reduced frame rate rendering, with a target frame rate of 18 frames per second and a target bitrate of 6Mbps. The mobile terminal reduces the rendering frame rate to 18 frames per second, and the data volume drops to 150KB / frame. The display terminal activates the frame rate compensation mechanism, interpolating frames from the received 18 frames per second data to generate a 30 frames per second display. Through motion estimation algorithms, the interpolation quality is good, and the user does not perceive a decrease in frame rate.

[0123] Link strain phase: As the vehicle entered the depths of the underground parking lot, the link bandwidth further decreased to 6Mbps, latency increased to 100ms, and packet loss rate rose to 8%. The display terminal determined that the communication quality had dropped to the third quality level and sent a new first rendering task instruction to the mobile terminal, specifying the rendering task type as keyframe-only rendering, with a target frame rate of 1 frame / second and a target bit rate of 3Mbps. The mobile terminal transmitted only one keyframe per second, reducing the data volume to 80KB / frame. Based on the keyframe data, the display terminal used a motion estimation algorithm to generate 29 intermediate transition frames, maintaining a display frame rate of 30 frames / second. Although image detail was slightly reduced, the basic functions of the navigation map (vehicle position, road lines) remained clearly visible.

[0124] Connection recovery phase: The vehicle exited the underground parking lot, and the link bandwidth returned to 16Mbps. The display terminal determined that the communication quality had returned to the highest quality level and sent a synchronization request to the mobile terminal. The mobile terminal transmitted a complete keyframe, which the display terminal received and combined with its local static frame. The displayed image returned to high quality within 300 milliseconds. The normal collaborative rendering process then resumed.

[0125] In the context of urban commercial districts, the proposed solution achieves the following technical effects: The amount of data transmitted is reduced by 65%. Compared with the traditional solution of 15-20 Mbps full video stream transmission, the average amount of data transmitted in this application is 5-8 Mbps during the link fluctuation phase, which is a reduction of 65%.

[0126] The display frame rate is consistently maintained at 25-30 frames per second. Even during periods of high link strain, the frame rate compensation mechanism ensures that the display frame rate remains above 25 frames per second, avoiding the black screen and stuttering issues common in traditional solutions.

[0127] Navigation text clarity is improved by 35%. Through content feature analysis and priority area division, higher rendering bitrates are allocated to high-priority areas such as road names and distance prompts in the navigation map, resulting in a significant improvement in text clarity.

[0128] As an example, this disclosure is illustrated using a high-speed driving scenario: This embodiment describes the collaborative rendering process when a vehicle is traveling on a highway. The electromagnetic environment of a highway is relatively simple, but the vehicle speed is high (80-120km / h), and the link bandwidth fluctuates between 3-15Mbps.

[0129] Initial stage: The vehicle enters the highway with a link bandwidth of 14Mbps, a latency of 30ms, and a packet loss rate of 1%. The display terminal determines the communication quality to be at the highest level and sends a first rendering task instruction to the mobile terminal, specifying the rendering task type as full rendering and the target frame rate as 30 frames per second. The mobile terminal renders the navigation map at 30 frames per second, while the display terminal renders static objects and stores them in the GPU cache.

[0130] High-speed driving phase: The vehicle travels at a speed of 100 km / h, and the link bandwidth fluctuates between 8 and 15 Mbps. The display terminal dynamically adjusts the parameters of the first rendering task based on the link fluctuation. When the link bandwidth is ≥12 Mbps, it maintains full rendering at 30 frames per second; when the link bandwidth is between 8 and 12 Mbps, it reduces the frame rate to 20 frames per second and starts frame interpolation; when the link bandwidth is <8 Mbps, it switches to keyframe-only mode.

[0131] In high-speed driving scenarios, the dynamic objects on the navigation map are updated more frequently (vehicle positions move approximately 28 meters per second), placing stricter requirements on frame rate compensation. The display terminal employs an optimized motion estimation algorithm to improve frame interpolation quality for high-speed moving scenes: The display terminal identifies vehicle icons in the navigation map and extracts their displacement vectors between keyframes.

[0132] The display terminal predicts the position of the vehicle icon in the next frame based on the vehicle speed information (obtained from the vehicle's CAN bus).

[0133] The display terminal uses a non-linear interpolation algorithm (such as cubic spline interpolation) to generate smoother transition frames and avoid the abruptness of vehicle icons.

[0134] Through the above optimizations, even during periods of high link strain (keyframe mode only), the display terminal can still generate high-quality interpolated frames, maintaining a display frame rate of 28 frames per second, with no noticeable stuttering perceived by the user.

[0135] Tunnel crossing phase: As the vehicle enters the highway tunnel, the link bandwidth drops sharply to 3-5 Mbps, latency rises to 150 ms, and packet loss rate increases to 12%. The display terminal determines the communication quality has dropped to the third quality level and sends a first rendering task command in keyframe-only mode to the mobile terminal. The mobile terminal transmits one keyframe every 2 seconds, reducing the data volume to 50 KB / frame. The display terminal utilizes cached rendering information and motion estimation algorithms to generate 59 intermediate transition frames, maintaining a display frame rate of 30 frames per second. Although image detail decreases, the basic functions of the navigation map (vehicle location, road lines, exit prompts) remain clearly visible, meeting the navigation needs of high-speed driving.

[0136] Connection recovery phase: The vehicle exited the tunnel, and the link bandwidth returned to 13Mbps. The display terminal sent a synchronization request to the mobile terminal, the mobile terminal transmitted the complete keyframe, and the display terminal restored a high-quality display within 400 milliseconds.

[0137] In high-speed driving scenarios, the solution proposed in this application achieves the following technical effects: The amount of data transmitted is reduced by 70%. Compared with the traditional solution of 15-20 Mbps full video stream transmission, the average amount of data transmitted in this application is 3-5 Mbps during the tunnel crossing phase, a reduction of 70%.

[0138] The display frame rate remains stable at 28-30 frames per second. Even in extreme link-stress scenarios involving tunnel traversal, the display frame rate is maintained above 28 frames per second thanks to optimized motion estimation algorithms and frame interpolation mechanisms.

[0139] The navigation function was 100% intact. Throughout the entire highway driving process, key information such as the vehicle's location, road lines, and exit prompts on the navigation map remained clearly visible without any blackouts or information loss, ensuring driving safety.

[0140] This application's solution achieves collaborative rendering optimization while maintaining full compatibility with the Android Auto protocol, and can pass PCTS (Protocol Conformance Test Suite) certification testing.

[0141] PCTS certification is an official Android Auto device certification standard required by Google, ensuring that the communication protocol between the display terminal and the mobile terminal conforms to the Android Auto specification. This application achieves PCTS compatibility through the following technical means: Protocol extension rather than modification: The proposed solution implements collaborative rendering functionality through a custom message channel of the Android Auto protocol without modifying the core protocol stack of Android Auto (such as Video Streaming, Audio Streaming, Input Handling, etc.).

[0142] Backward compatibility: For mobile terminals that do not support collaborative rendering, the display terminal automatically reverts to the traditional mobile terminal exclusive rendering mode to ensure that basic screen casting functions work normally.

[0143] State synchronization mechanism: The display terminal and the mobile terminal periodically synchronize and coordinate the rendering state through heartbeat packets to ensure that both parties keep in line with key parameters such as rendering task allocation and frame sequence number.

[0144] Error recovery mechanism: When an anomaly occurs during collaborative rendering (such as mobile terminal crash, data packet corruption, etc.), the display terminal automatically triggers the error recovery process to re-establish the collaborative rendering connection or revert to the traditional mode.

[0145] Through the above technical means, the solution in this application can pass all test cases of PCTS 2.0, including video stream transmission test, audio stream transmission test, input processing test, connection stability test, etc.

[0146] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0147] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] Based on the same inventive concept, corresponding to any of the above-described embodiments, this disclosure also provides a rendering apparatus.

[0149] refer to Figure 3This is a schematic diagram of a rendering apparatus provided in an exemplary embodiment of the present disclosure.

[0150] The device can be implemented by software and / or hardware, and is generally integrated into an electronic device, such as a display terminal, a vehicle controller, or a central processing unit in an autonomous driving system.

[0151] like Figure 3 As shown, the device includes the following modules: The rendering task allocation module 310 is configured to determine a first rendering task of the mobile terminal and a second rendering task of the display terminal based on the communication quality between the mobile terminal and the display terminal, wherein when the communication quality decreases, the workload of the first rendering task decreases and the workload of the second rendering task increases. The first rendering information determination module 320 is configured to send the first rendering task to the mobile terminal and receive the first rendering information sent by the mobile terminal. The second rendering information determination module 330 is configured to determine the second rendering information based on the second rendering task; The display screen determination module 340 is configured to determine the display screen based on the first rendering information and the second rendering information.

[0152] In some exemplary embodiments, the rendering task allocation module 310 is configured to: Based on the quality level of the communication quality, the rendering object type corresponding to the first rendering task and the second rendering task and the rendering task corresponding to the rendering object type are determined respectively.

[0153] In some exemplary embodiments, the rendering object type includes dynamic objects and static objects; Rendering task allocation module 310 is configured as follows: In response to the communication quality being a first quality level, the first rendering task includes rendering the dynamic object, and the second rendering task includes rendering the static object; In response to the communication quality being at the second quality level, the first rendering task includes rendering the dynamic object at a preset frame rate to obtain the first rendering information, and the second rendering task includes rendering the static object and post-processing the first rendering information, wherein the preset frame rate is lower than the standard frame rate. In response to the communication quality being at the third quality level, the first rendering task includes rendering keyframes of the dynamic object, and the second rendering task includes rendering the static object and rendering other frames of the dynamic object besides the keyframes. The communication quality represented by the first quality level, the second quality level, and the third quality level decreases sequentially.

[0154] In some exemplary embodiments, the rendering task allocation module 310 is configured to: Based on the type and update frequency of the objects in the display screen, the objects in the display screen are divided into dynamic objects and static objects.

[0155] In some exemplary embodiments, the first rendering information includes: The differential data obtained by rendering the dynamic object; The differential data includes the difference between the target frame of the dynamic object and the previous frame of the target frame, and the difference is encoded using differential coding technology.

[0156] In some exemplary embodiments, the second rendering information determination module 330 is configured to: Based on the second rendering task, the second rendering information is obtained locally from the display terminal; The second rendering information is obtained by the display terminal rendering the static object and stored locally on the display terminal.

[0157] In some exemplary embodiments, the display screen determination module 340 is configured to: In response to the detection of frame loss and / or delay in the first rendering information, a compensation frame is generated; The first rendering information, the second rendering information, and the compensation frame are combined to obtain the display screen.

[0158] In some exemplary embodiments, the rendering task allocation module 310 is configured to: Based on the content characteristics of the displayed screen, the displayed screen is divided into a high-priority area and a low-priority area; Based on the communication quality and the content characteristics, the rendering parameters for the high-priority region and the low-priority region are determined; The rendering parameters are sent to the mobile terminal as part of the first rendering task; The rendering parameters are used to indicate the adjustment of the rendering bitrate and resolution of different regions based on the rendering parameters.

[0159] In some exemplary embodiments, the display screen determination module 340 is configured to: In response to detecting a connection interruption between the mobile terminal and the display terminal, the cached first rendering information and second rendering information are invoked; In response to the detection that the connection between the mobile terminal and the display terminal has been restored, the display screen is determined based on the cached first rendering information and the second rendering information.

[0160] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0161] It should be noted that, Figure 3 The rendering apparatus shown can execute the various steps in the above method embodiments and achieve the various processes and effects in the above method embodiments, which will not be elaborated here.

[0162] Based on the same inventive concept, corresponding to any of the above-described embodiments, this disclosure also provides an electronic device.

[0163] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0164] In this embodiment of the disclosure, Figure 4 The electronic device shown can be a server or a terminal, wherein the terminal specifically includes display terminals, etc., without limitation.

[0165] like Figure 4 As shown, the electronic device may include a processor 410 and a memory 420 storing computer program instructions.

[0166] Specifically, the processor 410 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this disclosure.

[0167] Memory 420 may include a large-capacity storage device for information or instructions. For example, and not limitingly, memory 420 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 420 may include removable or non-removable (or fixed) media. Where appropriate, memory 420 may be internal or external to the integrated gateway device. In a particular embodiment, memory 420 is a non-volatile solid-state memory. In a particular embodiment, memory 420 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0168] The processor 410 performs the steps of the rendering method provided in the embodiments of this disclosure by reading and executing computer program instructions stored in the memory 420.

[0169] In one example, the electronic device may also include a transceiver 430 and a bus 440. Wherein, as... Figure 4 As shown, the processor 410, memory 420 and transceiver 430 are connected via bus 440 and communicate with each other.

[0170] Bus 440 may include hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 440 may include one or more buses.

[0171] This disclosure also provides a computer-readable storage medium that can store a computer program, which, when executed by a processor, causes the processor to implement the rendering method provided in this disclosure.

[0172] The aforementioned storage medium may, for example, include a memory 420 containing computer program instructions, which can be executed by a processor 410 to complete the rendering method provided in the embodiments of this disclosure. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), external cache memory, compact disc ROM (CD-ROM), magnetic tape, floppy disk, flash memory, and optical data storage device. By way of illustration and not limitation, RAM is available in various forms, such as static random access memory (SRAM) and dynamic random access memory (DRAM).

[0173] refer to Figure 5 This disclosure also provides a vehicle 500, which includes one or more processors 510 and one or more memories 520.

[0174] Processor 510 may include one or more processing cores, such as a quad-core processor, a deca-core processor, etc. Processor 510 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 510 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 510 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 510 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0175] The memory 520 may include one or more computer-readable storage media, which may be non-transitory. The memory 520 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 520 are used to store at least one computer program, which is executed by the processor 510 to implement the driveable lane identification method provided in the method embodiments of this disclosure.

[0176] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on vehicle 500 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0177] Based on the same inventive concept, corresponding to the rendering method described in any of the above embodiments, this disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to perform the rendering method. Corresponding to the execution entity for each step in each embodiment of the rendering method, the processor executing the corresponding step can belong to the corresponding execution entity.

[0178] The computer program products of the above embodiments are used to cause the computer and / or the processor to execute the rendering method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0179] Those skilled in the art will recognize that embodiments of this disclosure can be implemented as a system, method, or computer program product. Therefore, this disclosure can be implemented as entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this disclosure can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0180] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (not exhaustive) of a computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0181] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0182] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0183] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0184] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine that, when executed by a computer or other programmable data processing device, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0185] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce a product comprising an instruction apparatus that implements the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0186] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus can provide a process for implementing the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0187] Furthermore, although the operations of the methods of this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be executed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0188] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0189] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0190] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0191] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0192] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0193] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

[0194] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the appended claims is to be interpreted in the broadest sense, thereby encompassing all such modifications and equivalent structures and functions.

Claims

1. A rendering method, characterized in that, Applied to a display terminal, the method includes: Based on the communication quality between the mobile terminal and the display terminal, a first rendering task of the mobile terminal and a second rendering task of the display terminal are determined, wherein when the communication quality decreases, the workload of the first rendering task decreases and the workload of the second rendering task increases. Send the first rendering task to the mobile terminal and receive the first rendering information sent by the mobile terminal; Based on the second rendering task, determine the second rendering information; The display screen is determined based on the first rendering information and the second rendering information.

2. The method according to claim 1, characterized in that, The method of determining a first rendering task for the mobile terminal and a second rendering task for the display terminal based on the communication quality between the mobile terminal and the display terminal, wherein when the communication quality deteriorates, the workload of the first rendering task decreases and the workload of the second rendering task increases, includes: Based on the quality level of the communication quality, the rendering object type corresponding to the first rendering task and the second rendering task and the rendering task corresponding to the rendering object type are determined respectively.

3. The method according to claim 2, characterized in that, The types of rendered objects include dynamic objects and static objects; The step of determining the rendering object type corresponding to the first rendering task and the second rendering task, and the rendering task corresponding to the rendering object type, based on the quality level of the communication quality, includes: In response to the communication quality being a first quality level, the first rendering task includes rendering the dynamic object, and the second rendering task includes rendering the static object; In response to the communication quality being at the second quality level, the first rendering task includes rendering the dynamic object at a preset frame rate to obtain the first rendering information, and the second rendering task includes rendering the static object and post-processing the first rendering information, wherein the preset frame rate is lower than the standard frame rate. In response to the communication quality being at the third quality level, the first rendering task includes rendering keyframes of the dynamic object, and the second rendering task includes rendering the static object and rendering other frames of the dynamic object besides the keyframes. The communication quality represented by the first quality level, the second quality level, and the third quality level decreases sequentially.

4. The method according to claim 3, characterized in that, The methods for determining the dynamic object and the static object include: Based on the type and update frequency of the objects in the display screen, the objects in the display screen are divided into dynamic objects and static objects.

5. The method according to claim 3, characterized in that, The first rendering information includes: The differential data obtained by rendering the dynamic object; The differential data includes the difference between the target frame of the dynamic object and the previous frame of the target frame, and the difference is encoded using differential coding technology.

6. The method according to claim 3, characterized in that, The step of determining the second rendering information based on the second rendering task includes: Based on the second rendering task, the second rendering information is obtained locally from the display terminal; The second rendering information is obtained by the display terminal rendering the static object and stored locally on the display terminal.

7. The method according to claim 1, characterized in that, The step of determining the display screen based on the first rendering information and the second rendering information includes: In response to the detection of frame loss and / or delay in the first rendering information, a compensation frame is generated; The first rendering information, the second rendering information, and the compensation frame are combined to obtain the display screen.

8. The method according to claim 1, characterized in that, The method further includes: Based on the content characteristics of the displayed screen, the displayed screen is divided into a high-priority area and a low-priority area; Based on the communication quality and the content characteristics, the rendering parameters for the high-priority region and the low-priority region are determined; The rendering parameters are sent to the mobile terminal as part of the first rendering task; The rendering parameters are used to indicate the adjustment of the rendering bitrate and resolution of different regions based on the rendering parameters.

9. The method according to claim 1, characterized in that, The method further includes: In response to detecting a connection interruption between the mobile terminal and the display terminal, the cached first rendering information and second rendering information are invoked; In response to the detection that the connection between the mobile terminal and the display terminal has been restored, the display screen is determined based on the cached first rendering information and the second rendering information.

10. A vehicle, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

Images