Display method and system, vehicle, electronic equipment and computer readable storage medium

By transferring the rendering task to the user equipment side, using the powerful GPU performance of the user equipment, the problem of insufficient hardware resources in the on-board system is solved, and efficient rendering of 3D vehicle models is achieved, improving user experience and system performance, while reducing costs and design Complexity.

CN119048681APending Publication Date: 2024-11-29POLESTAR TIMES TECHNOLOGY (NANJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411197142.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The hardware resources of the on-board system are limited, especially the insufficient GPU performance, which leads to lag, delay and other phenomena when displaying 3D vehicle models, affecting the user experience, and upgrading the hardware configuration increases cost and design complexity.

Method used

Transfer the rendering generation task to the user equipment side, utilize the powerful GPU performance of the user equipment, work in collaboration with the wallpaper service of the on-board system through the resident rendering service on the user equipment side, realize efficient rendering and real-time interaction, and reduce the hardware performance requirements of the on-board system. .

Benefits of technology

It improves the overall performance and user experience of the on-board system, releases hardware resources, simplifies system design, reduces production costs, extends equipment life and improves stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048681A_ABST
    Figure CN119048681A_ABST
Patent Text Reader

Abstract

The invention relates to a display method and system, electronic equipment and a computer readable storage medium, and relates to the technical field of computers. The display method comprises the steps that first state information of a vehicle is sent to user equipment, and the first state information is used for indicating the current states of multiple components of the vehicle; receiving encoded first image data of the vehicle returned by the user equipment, wherein the first image data is generated by rendering according to the first state information; monitoring the creation of a drawing surface instance corresponding to the wallpaper service of the system carried by the vehicle; in response to the creation of the drawing surface instance, starting a coding and decoding class to decode the coded first image data; and drawing the decoded first image data to a drawing surface instance, so that a display device of the vehicle displays a virtual object of the vehicle. According to the technical scheme, the display effect can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a display method, a display system, a vehicle, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the advancement of automobile technology, intelligent vehicle systems have become an important part of vehicles. These systems not only need to handle basic functions such as navigation and entertainment, but also need to display various vehicle status information and complex 3D (three-dimensional) vehicle models.

[0003] In the related art, a 3D vehicle model is usually generated by hardware rendering of an onboard system. Summary of the invention

[0004] According to some embodiments of the present disclosure, a display method is provided, which is applied to a vehicle, and includes: sending first status information of the vehicle to a user device, the first status information being used to indicate the current status of multiple components of the vehicle; receiving encoded first image data of the vehicle returned by the user device, the first image data being rendered and generated according to the first status information; monitoring the creation of a drawing surface instance corresponding to a wallpaper service of a system carried by the vehicle; in response to the creation of the drawing surface instance, starting a codec class to decode the encoded first image data; and drawing the decoded first image data to the drawing surface instance, so that the display device of the vehicle displays a virtual object of the vehicle.

[0005] In some embodiments, the display method also includes: in response to initiating an interactive operation on a virtual object of a vehicle at a first position of a display device, sending the first position to a user device; receiving a user instruction corresponding to the interactive operation returned by the user device, the user instruction being generated based on the first position and first state information; and according to the user instruction, controlling a component on the vehicle corresponding to the user instruction to change the current state of the component.

[0006] In some embodiments, the display method also includes: sending second status information to a user device, the second status information is used to indicate the current status of the changed component; receiving encoded second image data of the vehicle returned by the user device, the second image data is rendered and generated based on the second status information; and updating the virtual object of the vehicle on the display device based on the decoded second image data.

[0007] In some embodiments, the display method further includes: sending screen information of the display device to the user device, and the first image data is generated by rendering according to the screen information.

[0008] In some embodiments, the display method further includes: creating an engine instance in the wallpaper service, the engine instance being used to monitor the creation of a drawing surface instance related to the wallpaper service, wherein the multimedia codec instance created by calling the codec class shares the corresponding drawing surface instance with the wallpaper service.

[0009] According to some other embodiments of the present disclosure, a display method is provided, which is applied to a user device, and includes: receiving first status information of a vehicle sent by the vehicle, the first status information being used to indicate current statuses of multiple components of the vehicle; rendering and generating first image data of the vehicle based on the first status information; and sending the encoded first image data to the vehicle so as to display a first virtual object of the vehicle on a display device mounted on the vehicle.

[0010] In some embodiments, the display method also includes: in response to initiating an interactive operation on a first virtual object of a vehicle at a first position of a display device, receiving a first position sent by the vehicle; generating a user instruction corresponding to the interactive operation based on the first position and first state information; and sending the user instruction to the vehicle so that the vehicle controls the component on the vehicle corresponding to the user instruction according to the user instruction to change the current state of the component.

[0011] In some embodiments, generating a user instruction corresponding to an interactive operation based on the first position and the first state information includes: determining a component of the vehicle corresponding to the user instruction based on the first position; determining a current state of the component based on the first state information; and determining the user instruction based on the current state of the component.

[0012] In some embodiments, determining the component of the vehicle corresponding to the user instruction based on the first position includes: coordinate mapping the first position to determine a second position on a second virtual object of the vehicle, the second virtual object being rendered and generated based on the first state information; and determining the component on the second virtual object based on the second position.

[0013] In some embodiments, the display method also includes: receiving second state information of the vehicle sent by the vehicle, the second state information being used to indicate the current state of the changed component; rendering and generating second image data of the vehicle based on the second state information; and returning the encoded second image data to the vehicle so that the vehicle updates the first virtual object of the vehicle based on the encoded second image data.

[0014] In some embodiments, rendering and generating first image data of the vehicle according to the first state information includes: rendering and generating the first image data according to screen information of a display device sent by the vehicle.

[0015] In some embodiments, rendering and generating first image data according to screen information of a display device sent by a vehicle includes: rendering and generating a second virtual object of the vehicle according to the screen information and the first state information; and generating the first image data according to the second virtual object.

[0016] In some embodiments, the media codec instance used for rendering and generating the second virtual object is configured with screen information.

[0017] In some embodiments, the first image data includes an encoded video stream generated based on the second virtual object.

[0018] According to some further embodiments of the present disclosure, a display system is provided, comprising: a vehicle, for executing the vehicle-side display method in any of the above embodiments; and a user device, for executing the user device-side display method in any of the above embodiments.

[0019] According to some further embodiments of the present disclosure, a vehicle is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the vehicle-side display method in any one of the above embodiments based on instructions stored in the memory device.

[0020] According to some further embodiments of the present disclosure, there is provided an electronic device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the display method on the user device side of any one of the above embodiments based on instructions stored in the memory device.

[0021] According to some further embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the display method on the vehicle side or the display method on the user device side in any of the above-mentioned embodiments is implemented.

[0022] According to some further embodiments of the present disclosure, a computer program product is also provided, comprising instructions, which, when executed by a processor, cause the processor to execute the display method on the vehicle side or the display method on the user device side according to any of the above embodiments.

[0023] In the above embodiment, by interacting with the user device, the image data of the vehicle is generated by rendering using the user device, and a drawing surface is shared with the wallpaper service through the decoding service, so that the display of the virtual object of the vehicle is realized. In this way, the hardware performance requirements of the vehicle system are reduced, thereby improving the display effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0026] Figure 1 A flowchart showing some embodiments of the vehicle side display method disclosed herein;

[0027] Figure 2 Schematic diagrams showing some embodiments of the display system of the present disclosure;

[0028] Figure 3 A flowchart showing some embodiments of the user device side display method disclosed in the present invention;

[0029] Figure 4 A block diagram showing some embodiments of the display system of the present disclosure;

[0030] Figure 5 A block diagram showing a portion of an exemplary vehicle including a battery provided by at least one embodiment of the present disclosure;

[0031] Figure 6 A block diagram showing some embodiments of the electronic device of the present disclosure;

[0032] Figure 7 A block diagram showing some other embodiments of the electronic device of the present disclosure. DETAILED DESCRIPTION

[0033] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0034] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0035] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0036] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0037] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0038] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0039] As mentioned above, in order to display various vehicle status information, complex 3D vehicle models and the vehicle's surrounding 3D environment, higher requirements are placed on the hardware performance of the vehicle system, especially the demand for GPU (Graphics Processing Unit, graphics processor) has increased significantly. However, the hardware resources of the vehicle systems of many vehicles are limited, especially in terms of GPU performance, which cannot meet the requirements of efficient rendering of complex 3D graphics.

[0040] On the other hand, the hardware resources of the vehicle system are limited, and the GPU performance is usually not as good as some user devices (such as mobile devices), making it difficult to smoothly run highly complex 3D rendering tasks. This leads to freezes and delays when displaying 3D vehicle models, poor display effects, and affects user experience.

[0041] In addition, if higher-performance hardware is used to improve the rendering performance of the in-vehicle system, it will not only increase the power consumption of the in-vehicle system, but may also cause the system to overheat, affecting the life and stability of the equipment; moreover, upgrading the hardware configuration of the in-vehicle system, especially the GPU, significantly increases the manufacturing cost; the introduction of high-performance hardware increases the design complexity of the in-vehicle system, requiring more design and debugging work to ensure the stable operation of the system under different conditions.

[0042] In response to the above technical problems, the present disclosure proposes a display technology solution that can efficiently render and generate virtual objects of the vehicle (such as a 3D model of the vehicle) without increasing the hardware burden of the vehicle system.

[0043] For example, the technical solution disclosed in the present invention can efficiently render and generate a virtual object of a vehicle by transferring the rendering generation task to the user device side, making full use of the powerful GPU performance of the user device.

[0044] In this way, the overall performance and user experience of the vehicle system can be improved through the collaboration of the resident rendering service on the user device side and the wallpaper service of the vehicle system, as well as efficient data transmission and real-time interaction (such as through P2P connection). Moreover, it can not only release the hardware resources of the vehicle system, but also simplify the system design and reduce production costs.

[0045] For example, the technical solution of the present disclosure can be implemented through the following embodiments.

[0046] Figure 1 A flow chart showing some embodiments of the vehicle side display method of the present disclosure.

[0047] like Figure 1 As shown, in step 110, first state information of the vehicle is sent to the user equipment, and the first state information is used to indicate the current state of multiple components of the vehicle. For example, the multiple components of the vehicle include windows, lights, doors, vehicle air conditioners, etc. The current state may include open, closed, set temperature, etc.

[0048] In step 120, the encoded first image data of the vehicle returned by the user device is received, and the first image data is rendered and generated according to the first state information. For example, the user device can be an electronic device with image processing and communication functions such as a mobile phone, a tablet computer, and a wearable device; the first image data can be a video, an image, etc.

[0049] In step 130, the creation of a drawing surface instance corresponding to a wallpaper service of a system mounted on the vehicle is monitored. For example, the wallpaper service may be a wallpaperservice in a vehicle operating system.

[0050] In some embodiments, the system installed in the vehicle is also referred to as a smart cockpit system, a smart vehicle system, a vehicle computer system, etc., which collects vehicle users' instructions such as gestures, voice, and touch commands through sensors installed on the vehicle, processes user commands through a processor (such as a computing platform), and controls various on-board hardware to realize various functions of the vehicle cockpit or other body parts.

[0051] In some embodiments, by creating an engine instance corresponding to a custom engine class in the wallpaper service of the system service, the engine instance will listen to the creation of a drawing surface instance corresponding to the wallpaper service. It can be understood that the drawing surface instance is related to the wallpaper service, and the engine instance will draw the image data onto the drawing surface instance.

[0052] In step 140, in response to the creation of the drawing surface instance, the codec class is started to decode the encoded first image data. For example, the codec class may be MediaCodec.

[0053] It should be noted that starting the codec class to decode the encoded first image data can be understood as the codec service creating a first multimedia codec instance, and configuring the codec type to be performed and the corresponding vehicle screen parameters for the first multimedia codec instance, configuring the input buffer to receive the encoded first image data, and configuring the output buffer to pass the decoded first image data. In step 150, the decoded first image data is drawn to the drawing surface instance so that the vehicle's display device displays the vehicle's virtual object. For example, the first virtual object of the vehicle is displayed on the vehicle's display device, and the first virtual object can be a 3D vehicle model.

[0054] It should be noted that the above-mentioned first multimedia codec instance is also configured to share a drawing surface instance of the wallpaper service, that is, the output buffer of the first multimedia codec instance is bound to the drawing surface instance, and the decoded first image data can be directly drawn onto the drawing surface instance of the wallpaper service, and the first image data can be displayed on the display device of the vehicle cabin through a surface synthesizer or after hardware synthesis.

[0055] In some embodiments, an engine instance is created in the wallpaper service, and the engine instance is used to monitor the creation of a drawing surface instance corresponding to the wallpaper service. The multimedia codec instance created by calling the codec class shares the corresponding drawing surface instance with the wallpaper service.

[0056] In the above embodiment, by interacting with the user device, the image data of the vehicle is generated by rendering using the user device, and a drawing surface is shared by the first multimedia codec instance and the wallpaper service, thereby realizing efficient display of the virtual object of the vehicle. In this way, the hardware performance requirements of the vehicle-mounted system are reduced, thereby improving the display effect.

[0057] The following describes, through some embodiments, a manner in which a vehicle-mounted display device sends the first status information to a user device.

[0058] In some embodiments, the vehicle-mounted system and the user device can interact by establishing a wireless connection. The wireless connection can be a short-range wireless connection, such as WiFi (such as WiFi-P2P), Bluetooth (classic Bluetooth or low-power Bluetooth), Near Field Communication (NFC) and Ultra Wide Band (UWB). For example, the vehicle-mounted system initializes the P2P service and establishes a P2P channel with the user device as a P2P server; the user device starts the service, initializes the Unity rendering pipeline, etc., and searches for network signals at the same time, and establishes a P2P channel with the vehicle-mounted system as a P2P client.

[0059] In some embodiments, the vehicle-mounted system sends the first state information and the screen information of the display device to the user device, and the user device renders and generates the first image data according to the first state information and the screen information of the display device. For example, the screen information may include screen resolution information (such as DPI (Dots Per Inch, number of pixels per inch) information, etc.), screen size information (such as width and height information, etc.), etc.

[0060] For example, the user device receives and parses the screen information and the first state information sent by the vehicle system through a short-range wireless connection. For example, after the short-range wireless connection between the vehicle system and the user device is successfully established, the vehicle system sends the screen resolution information to the screen projection service of the user device through the short-range wireless connection, and the screen projection service of the user device applies for the relevant graphics buffer according to the resolution information, etc.

[0061] In the above embodiment, after the vehicle-mounted system and the user device establish a short-range wireless connection, before the user device starts the screen projection service to the vehicle-mounted display device, the two parties of the screen projection service negotiate the screen information to be projected. In this way, it can be ensured that the display size of the first image data rendered and generated by the user device is compatible with the vehicle-mounted display device, thereby improving the display effect.

[0062] The following uses some embodiments to exemplarily illustrate the manner in which the user equipment renders and generates the first image data after receiving the first state information.

[0063] In some embodiments, the user device renders and generates a second virtual object of the vehicle according to the screen information and the first state information; generates first image data according to the second virtual object; and sends the encoded first image data to the onboard display device. It can be understood that the 3D model of the vehicle rendered on the user device is used as the second virtual object.

[0064] In this way, by migrating the vehicle-side rendering generation tasks that require higher computing power to the user device side, the hardware performance requirements of the vehicle-mounted system are reduced, thereby improving the display effect.

[0065] In some embodiments, the second multimedia codec instance used to render and generate the second virtual object is configured with the type of codec to be performed, the parameters of the corresponding vehicle screen, the input buffer to receive the first image data, and the output buffer to output the encoded first image data. For example, the user device creates a second MediaCodec (multimedia codec) instance, in which the width and height information of the vehicle screen and other required parameters are configured; createInputSurface (create input drawing surface instance) is called through the instance to create an input drawing surface instance, and the input drawing surface instance is passed to a 3D graphics rendering program (such as Unity program) or other suitable 3D drawing programs (such as OpenGL ES, etc.); the 3D graphics rendering program can use off-screen rendering to generate a 3D vehicle model in combination with the current application scenario of the vehicle and the current state of the vehicle; the rendered content is output to the input drawing surface instance.

[0066] For example, the rendering task of the above-mentioned three-dimensional graphics rendering program can be run in a background service, and by configuring the system whitelist of the user device, the background service can be prevented from being frozen or killed in the background, thereby realizing off-screen rendering of the 3D vehicle model. It can be understood that the above-mentioned background service can be configured to participate in the foreground display of the user device or to be configured not to participate in the foreground display of the user device or to be configured to controllably participate in or not participate in the foreground display of the user device, so that the user device can determine whether to display the 3D vehicle model rendered in the background in the foreground or not according to demand.

[0067] In this way, the user equipment achieves high-quality rendering through off-screen rendering, and the user equipment can present or not present the rendering effect according to demand, thereby improving the user experience.

[0068] In other instances, rendering on the user device can be accomplished by real-time rendering.

[0069] In some embodiments, the first image data includes a coded video stream generated according to the second virtual object. For example, the input drawing surface instance acquired by the user device is bound to the content input buffer of the second multimedia codec instance created by the user device; in response to the 3D vehicle model generated by the user device being rendered into the input drawing surface instance by a three-dimensional graphics rendering program, that is, the first image data of the 3D vehicle model is generated on the input drawing surface, the second multimedia codec instance performs an encoding operation on the first image data input data including the 3D vehicle model, and sends the encoded first image data to the P2P channel.

[0070] The following uses some embodiments to exemplarily illustrate how the vehicle-mounted display device displays the first virtual object after receiving the first image data returned by the user device.

[0071] In some embodiments, the wallpaperservice in the vehicle operating system is started; in the wallpaperservice, a custom Engine class is created, and the creation of the surface instance is listened to; in response to the completion of the creation of the surface instance, the MediaCodec is configured to be started, a corresponding first MediaCodec instance is created, and the first MediaCodec instance is set to output the decoded first image data (such as video frames, etc.) to the surface instance; the decoded first image data is drawn to the surface instance of the wallpaperservice, and displayed on the vehicle screen.

[0072] For example, the launcher of the vehicle operating system can be set to a transparent background, so that the first virtual object serving as the wallpaper is clearer, thereby improving the display effect.

[0073] In some embodiments, after the display device completes drawing of the first image data on the drawing surface, it submits the corresponding buffer to the image stream consumer; and submits the corresponding buffer to the drawing surface synthesizer or hardware for synthesis and display.

[0074] In the above embodiment, the multimedia codec instance created by the multimedia codec service (such as MediaCodec) and the wallpaper service (such as WallpaperService) share the same surface; the content buffer output by the multimedia codec instance after decoding is bound to the surface corresponding to the wallpaper service, and the surface of the wallpaper service is bound to the graphic buffer displayed by the wallpaper service, so that the content output after decoding can be drawn on the surface of the wallpaper service, thereby realizing that the vehicle-mounted display device can display the image data rendered and generated by the user device, reducing the hardware performance requirements of the vehicle-mounted system and improving the display effect.

[0075] The following uses some embodiments to exemplify the manner in which the vehicle-mounted system controls vehicle components based on interactive operations on the display device.

[0076] In some embodiments, in response to initiating an interactive operation on a virtual object of a vehicle at a first position of the display device, the display device sends the first position to a user device; the user device receives the first position sent by the display device, generates a user instruction corresponding to the interactive operation based on the first position and first status information, and sends the user instruction to the display device; the display device receives the user instruction corresponding to the interactive operation returned by the user device, and controls the component on the vehicle corresponding to the user instruction according to the user instruction to change the current status of the component.

[0077] For example, interactive operations include user touch operations (such as clicking, sliding, etc.) on the vehicle's display screen to trigger touch events; touch operations can be user control operations on vehicle doors, windows and other parts, such as a user wants to close the vehicle's door by touching the door position of the 3D vehicle model displayed on the screen.

[0078] For example, the vehicle-mounted display device waits to receive a touch event of the user on the screen; and sends the vehicle-mounted screen coordinates (ie, the first position) corresponding to the touch event to the rendering service of the user device through the P2P channel.

[0079] In some embodiments, the user device determines the component of the vehicle corresponding to the user instruction based on the first position; determines the current state of the component based on the first state information; and determines the user instruction based on the current state of the component. For example, the user device performs coordinate mapping on the first position to determine a second position on a second virtual object of the vehicle, the second virtual object is rendered and generated based on the first state information; and determines the component on the second virtual object based on the second position.

[0080] For example, the user device receives the vehicle screen coordinates corresponding to the touch event sent by the vehicle system; the user device maps the vehicle screen coordinates to the coordinate system of the 3D vehicle model generated by the user device to determine the user device screen coordinates (i.e., the second position) corresponding to the vehicle screen coordinates; and determines the vehicle component corresponding to the user command based on the component on the 3D vehicle model corresponding to the user device screen coordinates.

[0081] For example, the user device determines the user instruction based on the current state of the component (e.g., if the component is a car door and the current state is open, it can be determined that the user instruction is to close the car door); the user instruction is sent to the vehicle system through the P2P channel for execution. The vehicle system receives the user instruction sent by the rendering service of the user device; sends the user instruction to the vehicle control module to control the component corresponding to the user instruction to change the current state (such as controlling the door to close, etc.); and synchronizes the state of the component after the user instruction is executed to the user device.

[0082] In the above embodiment, through interaction with the user device, the vehicle can be controlled based on the image data rendered by the user device according to the interaction between the user and the display device. In this way, the hardware performance requirements of the vehicle system are reduced, thereby improving the display effect.

[0083] The following describes, through some embodiments, the manner in which the vehicle-mounted system updates the displayed vehicle virtual object.

[0084] In some embodiments, the vehicle-mounted display device sends second status information to the user device, where the second status information is used to indicate the current status of the changed component; the user device renders and generates second image data of the vehicle based on the second status information, and returns the encoded second image data to the display device; the display device updates the first virtual object of the vehicle on the display device based on the decoded second image data.

[0085] In this way, the image data of the vehicle after the state change can be re-rendered by using the user device, which reduces the hardware performance requirements of the vehicle system and improves the display effect.

[0086] Figure 2 Schematic diagrams showing some embodiments of the display system of the present disclosure.

[0087] like Figure 2 As shown, the vehicle 21 may run the operating system mentioned above in this article, and the system may exemplarily include modules such as wallpaper service 211, touch screen 212, vehicle control API (Application Programming Interface) 213, UI (User Interface) 214, etc.; the user device 22 may run an operating system, and the operating system may include modules such as 3D service 221 and Wifi (Wireless-Fidelity) connection layer 222.

[0088] In some embodiments, the user device 22 starts the rendering service, the vehicle 21 starts the wallpaper service 211, and establishes a P2P connection channel for both parties; the wallpaper service 211 sends the current status of the vehicle components to the user device 22; the rendering service of the user device 22 receives the current status and initializes the 3D vehicle model for rendering; the user device 22 transmits the rendered 3D vehicle model image (i.e., the first image data, such as a video stream, etc.) to the wallpaper service 211 through the P2P channel; the wallpaper service 211 receives the video stream and displays it on the touch screen 212.

[0089] In some embodiments, in response to a user triggering a touch event on the touch screen 212, the vehicle 21 transmits the corresponding vehicle screen coordinates to the rendering service of the user device 22 through the P2P channel. For example, the touch event on the touch screen 212 (including the vehicle screen coordinates) is sent to the message communication module of the Wifi connection layer 222 through the transparent transmission module of the wallpaper service 211; the message communication module sends the touch event to the Unity application in the 3D service 221.

[0090] In some embodiments, the rendering service of the user device 22 maps the touch event to the 3D vehicle model and sends the user command to the vehicle 21. For example, the interactive logic module of the Unity application is used to convert the coordinates of the vehicle screen into the coordinates on the 3D vehicle model to determine the component corresponding to the user command; the interactive logic module is used to determine the user command according to the current state of the component; and the control signal sending module of the Unity application is used to send the control event (including the user command) to the vehicle control logic module of the wallpaper service 211 through the message communication module.

[0091] In some embodiments, the vehicle 21 executes the user instruction and synchronizes the updated state of the corresponding component to the user device 22. For example, the vehicle control logic module sends a control signal (including the user instruction) to the vehicle control API 213 to control the change of the current state of the corresponding component; and sends the updated state to the message communication module through the state monitoring module.

[0092] In some embodiments, the user device 22 re-renders the 3D vehicle model according to the updated state, and returns the updated picture (i.e., the second image data) to the vehicle 21. For example, through the state sending interface, the updated state is sent to the Unity application for rendering to update the 3D skeletal animation; through the screen projection Source (source end) via the screen projection module, the updated picture is sent to the vehicle 21; the vehicle 21 sends the updated picture to the UI 214 for display through the screen projection Sink (receiving end).

[0093] In the above embodiment, the 3D rendering task is transferred to the user device, and the powerful performance of the user device (such as GPU, etc.) is utilized to release the hardware resources of the vehicle system (such as GPU, etc.), so that it can more efficiently utilize the hardware resources to process other tasks, thereby improving the overall performance of the vehicle system. Moreover, by handing over the high-power rendering task to the user device, the power consumption and heat dissipation burden of the vehicle system are reduced, the device life is extended, and the system stability is improved.

[0094] In addition, rendering is performed using user devices that users already own, avoiding the need to upgrade the vehicle system hardware and reducing manufacturing and maintenance costs. Moreover, by handing over complex rendering tasks to user devices, the complexity of the vehicle system's hardware and software design is simplified, reducing the workload of the vehicle system's design and debugging, and improving the vehicle system's reliability and maintainability.

[0095] Figure 3 A flowchart showing some embodiments of the user device side display method disclosed in the present invention.

[0096] like Figure 3 As shown, in step 310, first status information of the vehicle sent by the vehicle is received, and the first status information is used to indicate the current status of multiple components of the vehicle.

[0097] In step 320 , first image data of the vehicle is rendered and generated according to the first state information.

[0098] In step 330, the encoded first image data is sent to the vehicle so as to display the first virtual object of the vehicle on the display device mounted on the vehicle. For example, the encoded first image data is sent to the vehicle so that the vehicle monitors the creation of a drawing surface instance corresponding to the wallpaper service of the system mounted on the vehicle, and in response to the creation of the drawing surface instance, the codec class is started to decode the encoded first image data, and the decoded first image data is drawn to the drawing surface instance so that the display device of the vehicle displays the first virtual object of the vehicle.

[0099] In some embodiments, the first multimedia codec instance created by calling the encoder class shares the corresponding drawing surface instance with the wallpaper service.

[0100] In some embodiments, in response to initiating an interactive operation on a first virtual object of a vehicle at a first position on a display device, a first position sent by the vehicle is received; a user instruction corresponding to the interactive operation is generated based on the first position and first state information; and the user instruction is sent to the vehicle so that the vehicle controls a component on the vehicle corresponding to the user instruction according to the user instruction to change the current state of the component.

[0101] In some embodiments, the component of the vehicle corresponding to the user instruction is determined based on the first position; the current state of the component is determined based on the first state information; and the user instruction is determined based on the current state of the component.

[0102] In some embodiments, coordinate mapping is performed on the first position to determine a second position on a second virtual object of the vehicle, and the second virtual object is rendered and generated according to the first state information; and a component is determined on the second virtual object according to the second position.

[0103] In some embodiments, second state information of the vehicle sent by the vehicle is received, the second state information being used to indicate a current state of a changed component; second image data of the vehicle is rendered and generated based on the second state information; and the encoded second image data is returned to the vehicle so that the vehicle updates a first virtual object of the vehicle based on the encoded second image data.

[0104] In some embodiments, first image data is rendered and generated based on screen information of a display device sent from a vehicle.

[0105] In some embodiments, a second virtual object of the vehicle is rendered and generated based on the screen information and the first state information; and the first image data is generated based on the second virtual object.

[0106] In some embodiments, the media codec instance used for rendering and generating the second virtual object is configured with screen information.

[0107] In some embodiments, the first image data includes an encoded video stream generated based on the second virtual object.

[0108] Figure 4 A block diagram showing some embodiments of the display system of the present disclosure.

[0109] like Figure 4 As shown, the display system 4 includes: a vehicle 41, used to execute the vehicle-side display method in any of the above embodiments; and a user device 42, used to execute the user device-side display method in any of the above embodiments.

[0110] Figure 5 A block diagram of a portion of an exemplary vehicle including a battery provided by at least one embodiment of the present disclosure is shown.

[0111] exist Figure 5 In the example of FIG. 5 , vehicle 510 is a type of vehicle that can carry passengers (e.g., a car, truck, or other automotive vehicle). Configurations in which vehicle 510 is a robot (e.g., an autonomous robot) or other vehicle that does not carry human passengers may also be used. Figure 5 As shown, vehicle 510 may be operating on, for example, road 511 .

[0112] Vehicle 510 may be driven manually (e.g., by a human driver), may be operated via remote control, and / or may be operated autonomously (e.g., by an autonomous driving system or other autonomous propulsion system). Using vehicle sensors such as lidar, radar, visible light and / or infrared cameras (e.g., two-dimensional and / or three-dimensional cameras), proximity (range) sensors, and / or other sensors, an autonomous driving system and / or driver assistance system in vehicle 510 may perform automatic braking, steering, and / or other operations to help avoid pedestrians, inanimate objects, and / or other external structures, such as exemplary obstacle 512 on road 511.

[0113] Vehicle 510 may include a vehicle body, such as body 513. Body 513 may include vehicle structure, such as body panels formed of metal and / or other materials, and may include doors, a hood, a trunk, fenders, a chassis with wheels mounted, a roof, etc. Windows may be formed in doors 514 (e.g., on the sides of body 513, on the roof of vehicle 510, and / or in other parts of vehicle 510). Windows, doors 514, and other parts of body 510 may separate the interior of vehicle 510 from the external environment surrounding vehicle 510. Doors 514 may be opened and closed to allow people to enter and exit vehicle 510. Seats and other structures may be formed in the interior of body 513.

[0114] Vehicle 510 may have automotive lighting, such as one or more headlights (sometimes referred to as headlamps), driving lights, fog lights, daytime running lights, turn signals, brake lights, and / or other lights. Figure 5 As shown, for example, vehicle 510 may have lights, such as light 515. Generally, light 515 may be mounted on the front F of vehicle 510, on the rear R of vehicle 510, on the left and / or right sides W of vehicle 510, and / or on other parts of body 510 (e.g., the interior of vehicle 510). As an example, there may be left and right headlights 515 located on the left and right sides of vehicle 510, respectively, to provide illumination in a forward direction (e.g., in a Figure 5 In the example, the vehicle 510 is provided with illumination 517 in the X direction (in the example, the X direction along which the vehicle 510 moves when traveling forward). By shining the left and right headlights 515 on the exterior surface 516 in front of the vehicle 510, such as the road 511 and the exemplary obstacle 512, the driver or passenger of the vehicle 510 can see even in dim ambient lighting conditions (e.g., at night). The operation of sensors in the vehicle 510, such as image sensors and other sensors that use light, can also be supported by providing illumination to the road 511.

[0115] The vehicle 510 may have an onboard terminal (e.g., a vehicle computer) 518. The onboard terminal 518 may include control circuits and / or input-output devices. The control circuit in the onboard terminal 518 may be configured to run the underlying vehicle computer operating system, autonomous driving applications, navigation applications (e.g., applications for displaying maps on a display), and software for controlling vehicle temperature control devices, lighting, media playback, window lifting, door operation, sensor operation, and / or other vehicle operations. For example, a portion of the vehicle computer system may form an autonomous driving system that uses data such as sensor data to automatically drive the vehicle 510 on a road such as road 511. The control circuit may include a processing circuit and a memory, and may be configured to perform operations in the vehicle 510 using hardware (e.g., dedicated hardware or circuits), firmware, and / or software. Software codes and other data for performing operations in the vehicle 510 are stored on a non-transitory computer-readable storage medium (e.g., a tangible computer-readable storage medium) in the control circuit. Software code may sometimes be referred to as software, data, program instructions, computer instructions, or code. The non-transitory computer-readable storage medium may include non-volatile memory, such as non-volatile random access memory, one or more hard disk drives (e.g., magnetic disk drives or solid-state drives), one or more removable flash drives or other removable media or other memory. The software stored on the non-transitory computer-readable storage medium may be executed on the processing circuit of the vehicle terminal 518. The processing circuit may include a dedicated integrated circuit having a processing circuit, one or more microprocessors, a central processing unit (CPU), or other processing circuits.

[0116] The input-output device of the vehicle terminal 518 may include a display, a button, a microphone, a speaker and / or user input and other devices for providing output. The input device can be used to receive input digital or character information, and to generate signal input related to the settings and function control of the driver, passengers and other people of the vehicle 510. Specifically, the input device may include a touch panel and other input devices (such as a microphone, buttons in the vehicle, etc.). The touch panel, also known as a touch screen, can collect touch operations on or near the driver, passengers and other people of the vehicle 510 (such as operations performed by a user using a finger (such as a single finger or multiple fingers), a stylus or any other suitable object or accessory on or near the touch panel), and drive the corresponding connection device according to a pre-set program.

[0117] The output device of the vehicle terminal 518 may be used to provide audio output, visual output (eg, display content, etc.), and / or other suitable output to the driver, passengers, and others of the vehicle 510 .

[0118] The visual output device of the vehicle terminal 518 can be divided into two categories. The first category is the vehicle display screen; the second category is the projection display screen, such as a head-up display device (HUD). The vehicle display screen is a physical display screen and an important part of the vehicle system. The vehicle 510 can be provided with multiple display screens, such as a digital instrument display screen, a central control screen, a display screen in front of the passenger on the co-pilot seat (also called the front passenger), a display screen in front of the left rear passenger, and a display screen in front of the right rear passenger. Even the window can be used as a display screen for display. Head-up display, also known as a head-up display system. It is mainly used to display driving information such as speed and navigation on a display device in front of the driver (such as a windshield). To reduce the driver's line of sight transfer time, avoid pupil changes caused by the driver's line of sight transfer, and improve driving safety and comfort. HUD includes, for example, a combined head-up display (C-HUD) system, a windshield head-up display (W-HUD) system, and an augmented reality head-up display system (AR-HUD).

[0119] One or more vehicle-mounted display screens (or vehicle-mounted screens) are arranged inside the vehicle 510, including but not limited to the main driver's display screen (or, it can also be called the central control screen), the co-driver's display screen (or, it can also be called the co-driver's entertainment screen), the rear seat display screen (for example, the rear screen of the main driver's headrest, or, the rear screen of the co-driver's headrest) and the instrument screen. In some possible implementations, the main driver's display screen can also be a long screen (for example, a fish screen) extending to the co-driver area. Any of the above display screens can display a graphical user interface (Graphical User Interface, GUI), and the GUI can include icons / running interfaces of one or more applications, and / or one or more cards. For example, an audio playback window of the vehicle system can be displayed on a display screen (for example, a main driver's display screen or a co-driver's display screen or a rear seat display screen or other display screens), and the above audio playback window can be displayed on the display screen in the form of a card, or displayed on the display screen in the form of a floating window. For example, the audio playback window can be dragged on the display screen, and the user or any person in the vehicle 510 can drag the audio playback window to any area of ​​the display screen. Alternatively, the audio playback window can be enlarged or reduced on the display screen. In some examples, when the audio playback window is enlarged to a certain ratio (e.g., more than 50% or other ratio) of the display screen, the audio playback window is displayed in full screen on the display screen. When the audio playback window is reduced to a certain ratio (e.g., less than 10% or other ratio) of the display screen, the audio playback window is minimized or displayed as a card in a fixed area of ​​the display screen. The virtual object of the vehicle described in the present disclosure can be displayed on one or more vehicle-mounted display screens set inside the vehicle 510.

[0120] The vehicle 510 may have components 519, which may include, for example, a propulsion and steering system (e.g., a manually adjustable steering system and / or a steering system having wheels (not shown) coupled to the vehicle body 513, steering controls, one or more motors for driving the wheels, etc.) and other vehicle components. The components 519 may also include other input-output devices within the vehicle 510, such as sensors, haptic devices, speakers, and / or other devices for collecting environmental measurements, information about vehicle operation, and / or user input, and for providing output. The sensors in component 519 may include ambient light sensors, touch sensors, force sensors, proximity sensors, optical sensors such as cameras operating at visible, infrared and / or ultraviolet wavelengths (e.g., fisheye cameras, two-dimensional cameras, three-dimensional cameras, and / or other cameras), capacitive sensors, resistive sensors, ultrasonic sensors (e.g., ultrasonic distance sensors), microphones, radio frequency sensors such as radar sensors, lidar (light detection and ranging) sensors, door open / close sensors, seat pressure sensors and other vehicle occupant sensors, window sensors, position sensors for monitoring position, orientation and movement, speedometers, satellite positioning system sensors, and / or other sensors. The output devices in component 519 may be used to provide tactile outputs (e.g., vibration of vehicle seats), audio outputs (e.g., speakers), visual outputs (e.g., lights, etc.), and / or other suitable outputs to vehicle occupants and other persons or the environment outside the vehicle.

[0121] The three-dimensional sensor in component 519 can be formed by a pair of two-dimensional image sensors (e.g., a pair of binocular cameras formed at a three-dimensional camera) that together act as a stereo depth sensor. A three-dimensional sensor can also be formed using an image sensor system that emits structured light (e.g., an array of dots, lines, grids, and / or other structured light patterns of infrared and / or visible wavelengths) and captures images (e.g., two-dimensional images) for analysis. The captured image shows how the structured light pattern is distorted by the three-dimensional surface illuminated by the structured light pattern. By analyzing the distortion of the structured light, the three-dimensional shape of the surface can be reconstructed. If desired, the three-dimensional sensor of vehicle 510 may include one or more time-of-flight sensors. For example, time-of-flight measurements can be made using light (e.g., lidar sensor measurements) and radio frequency signals (e.g., three-dimensional radar).

[0122] During operation, component 519 may collect information from sensors and / or other input-output devices, such as lidar data, camera data (e.g., two-dimensional images), radar data, and / or other sensor data. For example, three-dimensional image data may be captured using a three-dimensional image sensor. Two-dimensional images may also be collected (e.g., in Figure 5 headlight illumination image on one or more areas of road 511).

[0123] The vehicle driver, passenger or other user of the vehicle 510 can provide user input to the vehicle terminal 518 and / or component 519. Cameras, touch sensors, physical controls (e.g., steering wheels, brakes, etc.) and other input devices can be used to collect user input. The remote data source can provide database information to the vehicle terminal 518 and / or component 519 by wirelessly communicating with the vehicle 510. Display screens, speakers and other output devices can be used to provide content to the user, such as interactive on-screen menu options and audio. The user can interact with this interactive content by providing touch input to the touch sensor in the display and / or by providing user input with other input devices. If necessary, the control circuit of the vehicle 510 can use sensor data, user input, information from a remote database and / or other information (e.g., information about nearby obstacles in the road and / or other environments around the vehicle 510) to provide driver assistance information and / or autonomous driving vehicle 510 to the driver.

[0124] Component 519 may include a forward sensor circuit, such as Figure 5 The forward sensor circuit may include one or more sensors facing a surface in front of the vehicle 510 (e.g., oriented Figure 5 One or more sensors in the X direction of the vehicle 510 are used to detect structures in front of the vehicle 510, such as obstacles 512 and the surface 516 of the road 511). For example, the forward sensor 520 can be a laser radar, a millimeter wave radar, an ultrasonic radar, or any other sensor that can detect the structure in front of the vehicle 510.

[0125] The vehicle 510 may have a battery 5A, which may be disposed at the bottom, head, or tail of the vehicle 510. The battery 5A may be used to power a lighting system (e.g., a lamp 515), a component 519 (e.g., a motor in a propulsion system, an operating system of a vehicle computer, etc.), or other parts of the vehicle 510 that are not shown. For example, the battery 5A may be used for starting the vehicle 510, the vehicle computer system, the air conditioning system, and power consumption during driving. For example, the battery 5A may include a battery module composed of a plurality of batteries or a plurality of battery cells described in any example of the present disclosure, or may include a battery pack composed of a plurality of battery modules. In some examples, the battery pack includes a plurality of battery modules, a mechanism system (e.g., a battery pack cover, a tray, a metal bracket, etc.), an electrical system (e.g., a high-voltage wiring harness, a low-voltage wiring harness, and a relay, etc.), a thermal management system (e.g., a water-cooling, liquid-cooling, air-cooling system, etc.), and a BMS (Battery management system). In other examples, the battery pack may be directly composed of more batteries or more battery cells. For example, more batteries or more battery cells may be integrated on the chassis of the vehicle 510 using CTC (Cell To Chassis) technology, thereby eliminating the design of the battery module.

[0126] Figure 6 A block diagram showing some embodiments of the electronic device of the present disclosure.

[0127] like Figure 6 As shown, the electronic device 6 can be a station (STATION, ST) in a WLAN, and can be a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a computer, a laptop computer, a handheld communication device, a handheld computing device, and / or other devices for communicating on a wireless system and a next-generation communication system, for example, a mobile terminal in a 5G network, a mobile terminal in a future-evolved Public Land Mobile Network (PLMN) or a mobile terminal in a future-evolved Non-terrestrial Network (NTN), etc.

[0128] The electronic device 6 of this embodiment includes: a memory 61 and a processor 62 coupled to the memory 61 , and the processor 62 is configured to execute the display method on the user device side in any of the above embodiments based on the instructions stored in the memory 61 .

[0129] The memory 61 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0130] In some embodiments, the user command is determined based on a component of the vehicle corresponding to the user command and a current state of the component, the component is determined based on a first position, and the current state of the component is determined based on first state information.

[0131] In some embodiments, the component corresponding to the user instruction is determined according to a second position on a second virtual object of the vehicle, the second position is determined by coordinate mapping the first position, and the second virtual object is generated by rendering by the user device according to the first state information.

[0132] In some embodiments, the first image data is generated based on a second virtual object of the vehicle, and the second virtual object is rendered based on the screen information and the first state information.

[0133] In some embodiments, the media codec instance used to render the second virtual object is configured with screen information.

[0134] In some embodiments, the first image data includes an encoded video stream generated based on the second virtual object.

[0135] Figure 7 A block diagram showing some other embodiments of the electronic device of the present disclosure.

[0136] like Figure 7 As shown, the electronic device 7 of this embodiment includes: a memory 710 and a processor 720 coupled to the memory 710, and the processor 720 is configured to execute the display method on the user device side in any of the above embodiments based on the instructions stored in the memory 710.

[0137] The memory 710 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0138] The electronic device 7 may also include an input / output interface 730, a network interface 740, a storage interface 750, etc. These interfaces 730, 740, 750, the memory 710, and the processor 720 may be connected, for example, via a bus 760. The input / output interface 730 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 740 provides a connection interface for various networked devices. The storage interface 750 provides a connection interface for external storage devices such as an SD card and a USB flash drive.

[0139] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] So far, the display method, display system, vehicle, electronic device, computer-readable storage medium and computer program product according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0141] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0142] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A display method, applied to a vehicle, comprising: Sending first status information of the vehicle to a user device, where the first status information is used to indicate current status of multiple components of the vehicle; receiving encoded first image data of the vehicle returned by the user equipment, where the first image data is generated by rendering according to the first state information; Monitoring the creation of a drawing surface instance corresponding to a wallpaper service of a system carried by the vehicle; In response to the creation of the drawing surface instance, starting the codec class to decode the encoded first image data; The decoded first image data is drawn to the drawing surface instance, so that a display device of the vehicle displays a virtual object of the vehicle.

2. The display method according to claim 1, further comprising: In response to initiating an interactive operation on the virtual object of the vehicle at a first position of the display device, sending the first position to the user device; receiving a user instruction corresponding to the interactive operation returned by the user equipment, where the user instruction is generated according to the first position and the first state information; According to the user instruction, a component on the vehicle corresponding to the user instruction is controlled to change a current state of the component.

3. The display method according to claim 2, further comprising: Sending second state information to the user equipment, where the second state information is used to indicate the changed current state of the component; receiving encoded second image data of the vehicle returned by the user equipment, where the second image data is generated by rendering according to the second state information; The virtual object of the vehicle is updated on the display device according to the decoded second image data.

4. The display method according to any one of claims 1 to 3, further comprising: The screen information of the display device is sent to the user equipment, and the first image data is generated by rendering according to the screen information.

5. The display method according to any one of claims 1 to 3, further comprising: An engine instance is created in the wallpaper service, and the engine instance is used to monitor the creation of a drawing surface instance corresponding to the wallpaper service, wherein the multimedia codec instance created by calling the codec class shares the corresponding drawing surface instance with the wallpaper service.

6. A display method, applied to a user device, comprising: Receiving first status information of the vehicle sent by the vehicle, where the first status information is used to indicate current status of multiple components of the vehicle; Rendering and generating first image data of the vehicle according to the first state information; The encoded first image data is sent to the vehicle so as to display a first virtual object of the vehicle on a display device carried by the vehicle.

7. The display method according to claim 6, further comprising: In response to initiating an interactive operation on a first virtual object of the vehicle at a first position of the display device, receiving the first position sent by the vehicle; generating a user instruction corresponding to the interactive operation according to the first position and the first state information; The user instruction is sent to the vehicle so that the vehicle controls the component corresponding to the user instruction on the vehicle according to the user instruction to change the current state of the component.

8. The display method according to claim 7, wherein: The generating, according to the first position and the first state information, a user instruction corresponding to the interactive operation includes: determining, according to the first position, a component of the vehicle corresponding to the user instruction; determining a current state of the component according to the first state information; The user instruction is determined according to the current state of the component.

9. The display method according to claim 8, wherein: Determining, based on the first position, the component of the vehicle corresponding to the user instruction includes: Performing coordinate mapping on the first position to determine a second position on a second virtual object of the vehicle, the second virtual object being rendered and generated according to the first state information; The component is determined on the second virtual object according to the second position.

10. The display method according to claim 7, further comprising: receiving second state information of the vehicle sent by the vehicle, where the second state information is used to indicate a current state of the component after the change; Rendering and generating second image data of the vehicle according to the second state information; The encoded second image data is returned to the vehicle so that the vehicle updates the first virtual object of the vehicle according to the encoded second image data.

11. The display method according to any one of claims 6 to 10, wherein: The rendering and generating the first image data of the vehicle according to the first state information includes: The first image data is generated by rendering according to the screen information of the display device sent by the vehicle.

12. The display method according to claim 11, wherein: The rendering and generating the first image data according to the screen information of the display device sent by the vehicle includes: Rendering and generating a second virtual object of the vehicle according to the screen information and the first state information; The first image data is generated according to the second virtual object.

13. The display method according to claim 12, wherein: The screen information is configured in the media codec instance used for rendering and generating the second virtual object.

14. The display method according to claim 12, wherein: The first image data includes an encoded video stream generated according to the second virtual object.

15. A display system comprising: A vehicle, used to execute the display method according to any one of claims 1 to 5; User equipment, used to execute the display method described in any one of claims 6-14.

16. A vehicle comprising: Memory; and A processor coupled to the memory, wherein the processor is configured to execute the display method according to any one of claims 1 to 5 based on instructions stored in the memory.

17. An electronic device comprising: Memory; and A processor coupled to the memory, wherein the processor is configured to execute the display method according to any one of claims 6 to 14 based on instructions stored in the memory.

18. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the display method according to any one of claims 1 to 5 or the display method according to any one of claims 6 to 14 is implemented.

19. A computer program product, comprising instructions, which, when executed by a processor, cause the processor to perform the display method according to any one of claims 1 to 5, or the display method according to any one of claims 6 to 14.