Rendering method and apparatus, and vehicle

Abstract

The present application is applicable to the field of intelligent driving, and provides a rendering method and apparatus, and a vehicle. The method comprises: acquiring perception data, the perception data comprising vehicle surrounding environment data and vehicle status data; on the basis of the perception data, generating first rendering data by means of a first application, wherein the first rendering data corresponds to a first viewing angle, and the first viewing angle is an observation viewing angle for observing a vehicle; and, by means of the first application, performing secondary rendering of the first rendering data onto a first window of a first screen. The present application can enhance the flexibility and scalability of rendering, optimize rendering strategies, and improve user experience.

Description

Rendering method, device and vehicle

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 8, 2024, with application number 202410276831.5 and application name “Rendering Method, Device and Vehicle”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of intelligent driving, and in particular to a rendering method, device, and vehicle. Background Art

[0003] With the rapid development of intelligent driving technology, vehicles are now equipped with a variety of display devices (also called display screens), such as instrument panels and central control screens. These different display devices work together to provide users with a rich set of functions and services, thereby improving driving safety and convenience, and enhancing the user experience.

[0004] Real-time reconstruction and rendering of the vehicle's surroundings to various display devices has become an essential feature for assisting users in determining safe driving. For example, three-dimensional (3D) reconstruction and simulation rendering (SR) of the vehicle's surroundings are required. However, rendering the vehicle's surroundings is a complex and critical task, and flexibility in rendering has become a pressing technical challenge. Summary of the Invention

[0005] This application provides a rendering method, device, and vehicle that can improve the flexibility and scalability of rendering, optimize rendering strategies, and enhance user experience.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] In a first aspect, the present application provides a rendering method applied to a vehicle, the method comprising: obtaining perception data; the perception data comprising vehicle surrounding environment data and vehicle status data; generating first rendering data through a first application based on the perception data; wherein the first rendering data corresponds to a first perspective, which is an observation perspective for observing the vehicle; and secondary rendering the first rendering data to a first window of a first screen through the first application.

[0008] In this application, in the same application, the flexibility and scalability of rendering are increased by rendering twice, and the rendering strategy is optimized.

[0009] According to the first aspect, the vehicle surrounding environment data is used to represent the environmental conditions around the vehicle; and the vehicle status data is used to represent the status information of the vehicle itself.

[0010] In some embodiments, perception data is acquired through various sensors installed on the vehicle.

[0011] In other embodiments, sensory data is obtained from various subsystems of the vehicle.

[0012] In this application, the reconstruction and rendering of the vehicle and the surrounding environment are completed based on multiple perception devices to improve the rendering accuracy and rendering effect.

[0013] In some embodiments, based on the correspondence between the pre-set perception data and the observation perspective, the first application determines the first perspective based on the perception data, and the first application performs simulation rendering processing based on the constructed vehicle-surrounding environment scene and the first perspective to generate first rendering data.

[0014] In some embodiments, after the first rendering data is obtained, the first rendering data is cached in a texture buffer so that it can be quickly applied to a corresponding display device later.

[0015] According to the first aspect, or any implementation of the first aspect above, the method further includes: rendering the first rendering data a second time to a second window on the second screen through the first application.

[0016] In some embodiments, window information for displaying environment rendering data in each display device is pre-set. After generating rendering data, the display device currently displayed by the vehicle is obtained, the window information of the currently displayed display device is determined, and the rendering data is re-rendered into the corresponding window.

[0017] In some embodiments, the secondary rendering is to transmit the first rendering data generated by the rendering from the texture buffer to the window buffer of the display device, so that the window processes the rendering data and displays the display content corresponding to the rendering data.

[0018] In this application, the first rendering data can be rendered twice into the display windows of different display devices to achieve diversified display effects, improve the flexibility of rendering, meet the usage requirements of different application scenarios, and provide personalized display solutions.

[0019] According to the first aspect, or any implementation of the first aspect above, the method further includes: generating second rendering data through the first application based on the perception data; wherein the second rendering data corresponds to a second perspective; the second perspective is an observation perspective for observing the vehicle, and the second perspective is different from the first perspective; and secondary rendering the second rendering data to a third window of a third screen through the first application.

[0020] In some embodiments, based on the pre-set correspondence between the display device and the observation perspective, the observation perspective corresponding to each display device is determined, rendering data is generated based on the observation perspective, and the rendering data is rendered a second time to the window of the display device corresponding to the observation perspective.

[0021] In some embodiments, based on the display requirements of each display device, the rendering content and viewing angle corresponding to each display device are customized.

[0022] In this application, the first application is based on the same data source and shares the same rendering logic to complete rendering from different perspectives, which can save system memory and system computing power, synchronously update the rendering content of each display device, ensure that the rendering data is fully synchronized, and improve the utilization rate of rendering resources and rendering efficiency.

[0023] According to the first aspect, or any implementation of the first aspect above, the method further includes: secondary rendering the first rendering data to the first window of the first screen through the first application, and secondary rendering the second rendering data to the fourth window of the first screen.

[0024] In this application, the first application renders different rendering data twice to different windows of the same display device, enriching the display content of the display device and better ensuring safe driving.

[0025] According to the first aspect, or any implementation of the first aspect above, first rendering data is generated through a first application based on the perception data, including: based on the perception data, constructing a vehicle surrounding environment scene through the first application; based on the first perspective, rendering the vehicle surrounding environment scene through the first application to generate the first rendering data.

[0026] In this application, the first application constructs and generates a vehicle-surrounding environment scene based on perception data. It then performs simulation rendering based on the user's viewing angle to obtain rendering data corresponding to that viewing angle and outputs it to the corresponding display device. The display device then displays the vehicle-surrounding environment scene from different viewing angles. The rendering data on each display device is derived from the same vehicle-surrounding environment scene. Reusing the same rendering resources when rendering data from different viewing angles saves computing power and memory, and synchronizes the rendering data.

[0027] According to the first aspect, or any implementation of the first aspect, the first rendering data is first 3D rendering data, and the second rendering data is second 3D rendering data.

[0028] In this application, the rendering data may be 3D rendering data, which vividly shows the current condition of the vehicle and the surrounding environment so that the driver can make clear judgments and drive safely.

[0029] According to the first aspect, or any implementation of the first aspect above, the acquiring of the perception data includes: acquiring the perception data through a sensor.

[0030] In this application, perception data is obtained through various sensors to provide high-precision, real-time perception data, so that the vehicle and surrounding environment can be accurately constructed and rendered in the future to ensure safe travel.

[0031] In a second aspect, the present application provides a rendering device for use in a vehicle. The device comprises: an acquisition module for acquiring perception data; the perception data includes vehicle surrounding environment data and vehicle status data; a processing module for generating first rendering data based on the perception data via a first application; wherein the first rendering data corresponds to a first perspective, which is the perspective from which the vehicle is observed; and the processing module for secondary rendering the first rendering data to a first window on a first screen via the first application.

[0032] According to the second aspect, the processing module is further configured to secondary render the first rendering data to a second window on the second screen through the first application.

[0033] According to the second aspect, or any implementation of the second aspect above, the processing module is further used to generate second rendering data through the first application based on the perception data; wherein the second rendering data corresponds to a second perspective; the second perspective is the observation perspective of observing the vehicle, and the second perspective is different from the first perspective; the processing module is further used to secondary render the second rendering data to a third window of a third screen through the first application.

[0034] According to the second aspect, or any implementation of the above second aspect, the processing module is also used to secondary render the first rendering data to the first window of the first screen through the first application, and secondary render the second rendering data to the fourth window of the first screen.

[0035] According to the second aspect, or any implementation method of the above second aspect, the processing module is also used to construct the vehicle surrounding environment scene through the first application based on the perception data; the processing module is also used to render the vehicle surrounding environment scene through the first application based on the first perspective to generate the first rendering data.

[0036] According to the second aspect, or any implementation of the second aspect, the first rendering data is first 3D rendering data, and the second rendering data is second 3D rendering data.

[0037] According to the second aspect, or any implementation of the second aspect above, the acquisition module is further configured to acquire the perception data through a sensor.

[0038] In a third aspect, the present application provides a rendering device. The device includes a processor and a memory, the memory being coupled to the processor and configured to store computer-readable instructions. When the processor reads the computer-readable instructions from the memory, the rendering device executes the method of the first aspect and any one of the embodiments of the first aspect.

[0039] In a fourth aspect, the present application provides a vehicle, comprising the rendering device as described in the third aspect.

[0040] Exemplary vehicles include cars, trucks, motorcycles, buses, lawn mowers, recreational vehicles, amusement park vehicles, construction equipment, trams, golf carts, trains, etc., which are not particularly limited in this application. The power of the above-mentioned vehicles can be provided by gasoline, diesel, electricity, solar energy, or hydrogen energy.

[0041] In a fifth aspect, the present application provides a chip system comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions, and the at least one processor is used to execute the method of the first aspect and any one of the embodiments of the first aspect.

[0042] In a sixth aspect, the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is executed by a processor, it implements the method of the first aspect and any one of the embodiments of the first aspect.

[0043] In a seventh aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, implements the method of the first aspect and any one of the implementation methods of the first aspect.

[0044] The technical effects corresponding to the second to seventh aspects and any implementation method of each aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation method of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] FIG1 is a schematic diagram of a first flow chart of an existing rendering output according to an embodiment of the present application;

[0046] FIG2 is a second schematic diagram of a conventional rendering output process provided by an embodiment of the present application;

[0047] FIG3A is a schematic diagram of the architecture of a rendering system provided in an embodiment of the present application;

[0048] FIG3B is a first structural diagram of a rendering device provided in an embodiment of the present application;

[0049] FIG4 is a schematic diagram of the architecture of the vehicle-mounted system provided in an embodiment of the present application;

[0050] FIG5 is a schematic diagram of the vehicle structure provided in an embodiment of the present application;

[0051] FIG6 is a schematic diagram of a flow chart of a rendering method provided in an embodiment of the present application;

[0052] FIG7 is a first schematic diagram of a rendering scene provided in an embodiment of the present application;

[0053] FIG8A is a second schematic diagram of a rendering scene provided in an embodiment of the present application;

[0054] FIG8B is a third schematic diagram of a rendering scene provided in an embodiment of the present application;

[0055] FIG8C is a fourth schematic diagram of a rendering scene provided in an embodiment of the present application;

[0056] FIG9 is a fifth schematic diagram of a rendering scene provided in an embodiment of the present application;

[0057] FIG10 is a sixth schematic diagram of a rendering scene provided in an embodiment of the present application;

[0058] FIG11 is a seventh schematic diagram of a rendering scene provided in an embodiment of the present application;

[0059] FIG12 is a second structural diagram of a rendering device provided in an embodiment of the present application;

[0060] FIG13 is a schematic structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0062] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0063] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more. In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0064] In some embodiments, the solution for reconstructing and rendering the vehicle's surrounding environment generally adopts a rendering output mode of a single application to a single display device. Exemplarily, as shown in FIG1 , the display devices in the vehicle may include an instrument screen, a central control screen and a head-up display (HUD), as well as an entertainment screen (not shown in FIG1 ). Accordingly, each display device in the vehicle has a corresponding application (APP), namely, an instrument APP, a central control APP and a head-up display APP. Each APP obtains perception data (such as vehicle surrounding environment data or camera data, etc.) from the vehicle, reconstructs and renders based on the perception data to obtain rendering data, and outputs the rendering data to the corresponding display device.

[0065] However, in the above embodiment, the rendering of each display device in the vehicle is independently processed by different APPs, and each APP calls a process to independently complete the rendering of the corresponding display device. On the one hand, it will cause a huge waste of system memory and system computing power, and rendering is inflexible. For example, if the instrument panel and the central control screen need to display content from the same perspective, the instrument APP and the central control APP perform the same calculation based on the same perception data to obtain the same rendering data. The instrument APP and the central control APP perform repeated processing, which wastes system computing power and makes rendering inflexible. On the other hand, it will also lead to rendering asynchrony. Each APP completes the rendering independently, and the timing of each APP receiving and processing the perception data is not synchronized, resulting in asynchrony of the display content corresponding to the rendering data, which in turn leads to asynchrony of display on different display devices.

[0066] In other embodiments, the solution for reconstructing and rendering the vehicle's surrounding environment is implemented using a customized operating system. Exemplarily, as shown in Figure 2, the display device in the vehicle includes an instrument screen, a central control screen, and a head up display (HUD). The vehicle also includes an intelligent driving APP that controls the display of the vehicle display device, an operating system (not shown in the figure), and a virtual screen buffer (buffer). Specifically, the intelligent driving APP obtains perception data from the vehicle, generates rendering data based on the perception data, and outputs the rendering data to the virtual screen buffer. The operating system divides the rendering data in the virtual screen buffer according to the customized function, and outputs the divided rendering data (such as rendering data corresponding to the instrument screen, rendering data corresponding to the central control screen, and rendering data corresponding to the head up display) to the corresponding display devices respectively.

[0067] It is understandable that in the above embodiment, although only one APP is used for rendering, which saves system computing power, the rendering data generated by the APP is panoramic data from the same perspective. The operating system cuts the panoramic data based on the customization function to obtain the rendering data of each display device. After cutting, the rendering data output to each display device is all from the same perspective. Moreover, the coupling between the smart driving APP and the operating system is very high, and the functions of the operating system and the logic of the application are mixed together, lacking flexibility and scalability. When it is necessary to modify or add functions, the entire system needs to be significantly modified, which increases the complexity of development and maintenance.

[0068] In order to solve the technical problems described above, an embodiment of the present application provides a rendering method, which includes: obtaining perception data; the perception data includes vehicle surrounding environment data and vehicle status data; based on the perception data, generating first rendering data and second rendering data through a first application; wherein the first rendering data corresponds to a first perspective, and the second rendering data corresponds to a second perspective, the first perspective and the second perspective are observation perspectives for observing the vehicle, and the first perspective is different from the second perspective; and the first rendering data and the second rendering data are respectively rendered twice to windows of different display devices through the first application. The method provided in the embodiment of the present application can improve the flexibility and scalability of rendering and optimize the rendering strategy through two renderings. Moreover, the rendering processing of multiple display devices and different perspectives is completed by the same application, which saves system memory and computing power and optimizes system performance.

[0069] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0070] The rendering method in the embodiments of the present application can be applied to any rendering scenario. Optionally, the rendering scenario includes vehicle-mounted rendering, aviation rendering, game rendering, entertainment rendering, conference, exhibition, performance, and other scenarios. By rendering, the environment simulation is completed to provide a more realistic and immersive experience and realize various functions. For example, the vehicle-mounted rendering scenario includes an autonomous driving scenario, in which the display device in the vehicle displays a simulated rendered image of the surrounding environment to assist the user in driving. The embodiments of the present application do not impose any special restrictions on the specific form of the rendering scenario.

[0071] The rendering method in the embodiments of the present application can also be applied to various devices that use rendering functions. The various devices can include various means of transportation such as new energy vehicles, electric vehicles, buses, and cars. The various devices can also include various electronic devices such as mobile phones, computers, tablet computers, personal computers (PCs), and wearable devices. The embodiments of the present application do not impose any particular restrictions on the specific form of the device.

[0072] The following uses the in-vehicle rendering scene as an example to illustrate the rendering method of this application.

[0073] For example, refer to Figure 3A, which shows a schematic diagram of the architecture of a rendering system 30 provided in an embodiment of the present application. As shown in Figure 3A, rendering system 30 may include a sensing device 31, a rendering device 32, and a display device (also described as a display device) 33. Sensing device 31, rendering device 32, and display device 33 are communicatively connected.

[0074] In the embodiment of the present application, the sensing device 31 is used to obtain sensing data, which includes vehicle surrounding environment data and vehicle status data. The sensing device 31 is also used to send the sensing data to the rendering device 32 so that the rendering device 32 can render.

[0075] Optionally, the sensing device 31 includes a vehicle sensor, which is typically located inside the vehicle. The vehicle sensor can be used to sense vehicle surrounding environment data and vehicle status data. The vehicle sensor can include an accelerometer, a gyroscope, a wheel speed sensor, an air pressure sensor, an ultrasonic sensor, a camera sensor, a positioning sensor, a radar sensor, and the like.

[0076] It can be understood that since the vehicle sensor is set in the vehicle, when the perception device 31 is a vehicle sensor, the perception device 31 moves with the vehicle, and the perception device 31 and the rendering device 32 can communicate through the wireless communication network.

[0077] It should be understood that the above description of the sensing device 31 is merely an example, and the embodiment of the present application does not specifically limit the specific form and implementation method of the sensing device 31.

[0078] In the embodiment of the present application, the rendering device 32 is used to execute the rendering method provided by the present application, generate rendering data, and output the rendering data to the display device 33. The rendering device 32 includes a first application.

[0079] Alternatively, rendering device 32 may be an intelligent driving computing platform. This platform implements intelligent driving, decision-making, planning, and control functions and is a core component of the vehicle. The platform interacts with various components in the vehicle, acquiring real-time data from them and controlling their operation.

[0080] Optionally, the rendering device 32 may be a server. The server may be a Linux server, a Windows server, or other server device that can provide simultaneous access to multiple devices. It may also be a server cluster consisting of multiple regions, multiple computer rooms, or multiple servers. As an example, the rendering device 32 may be a server of an intelligent transportation system, such as a physical server or a cloud server, but this embodiment of the application is not limited to this.

[0081] In the embodiment of the present application, the display device 33 receives the rendering data sent by the rendering device 32 .

[0082] In this embodiment of the present application, rendering device 32 includes a first application, and display device 33 includes a first screen. Specifically, rendering device 32 obtains perception data from perception device 31; based on this perception data, rendering device 32 generates rendering data using the first application; the first rendering data corresponds to a first perspective, which is the perspective from which the vehicle is viewed; and rendering device 32 performs secondary rendering of this first rendering data onto a first window of the first screen of display device 33 using the first application.

[0083] It is understood that the sensing device 31, rendering device 32, and display device 33 in the rendering system 30 are independent components that interact with each other to achieve rendering. In actual applications, the rendering system 30 may also include only the rendering device 32, with the other components controlled by other systems in the vehicle, and the systems interacting to achieve rendering. This embodiment of the present application does not limit the rendering system and its specific implementation.

[0084] It should be understood that the above description of the rendering system is merely an example, and the embodiment of the present application does not specifically limit the specific form and implementation of the rendering system 30.

[0085] It is understandable that the modules in the above-mentioned rendering system are divided according to functional logic, and other division methods may be used in practice. In addition, the above-mentioned modules can be named by other names. In addition, each module can be implemented by hardware, software, or a combination of hardware and software. Whether a specific module is implemented in hardware, software, or a combination of hardware and software depends on the specific application and design constraints of the technical solution. Different modules can be implemented by different hardware, and multiple modules can also be implemented by the same hardware. The embodiments of the present application do not specifically limit this.

[0086] It can be understood that the system architecture and business scenarios described in this application are intended to more clearly illustrate the technical solutions of this application, and do not constitute the sole limitation on the technical solutions provided by this application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by this application are also applicable to similar technical problems.

[0087] Refer to FIG3B , which shows a schematic structural diagram of a rendering device 32 provided in an embodiment of the present application.

[0088] In this embodiment of the present application, the first application of the rendering device 32 includes a rendering generation module and an on-screen rendering output module. The rendering generation module includes a meter data rendering unit, a central control data rendering unit, and a head-up data rendering unit. Correspondingly, the on-screen rendering output module includes a meter data output unit, a central control data output unit, and a head-up data output unit.

[0089] In an embodiment of the present application, the rendering generation module is used to generate rendering data based on the perception data. The rendering generation module is also used to send the rendering data to the upper screen rendering output module according to usage requirements. The upper screen rendering output module is used to render the rendering data twice and output it to a window of a display device (not shown in Figure 3B) according to usage requirements.

[0090] Specifically, taking the instrument data rendering unit as an example, the instrument data rendering unit is used to render and generate instrument data (also described as instrument rendering data) based on the perception data and the observation angle corresponding to the instrument screen. The instrument data rendering unit sends the instrument data to the instrument data output unit. The instrument data output unit then renders the instrument data again and outputs it to the window on the instrument screen.

[0091] Referring to Figure 4, Figure 4 shows a schematic diagram of the architecture of an in-vehicle system provided in an embodiment of the present application. The in-vehicle system includes a perception system, a rendering system, and a display system. The perception system, the rendering system, and the display system are connected and communicated with each other.

[0092] In an embodiment of the present application, a perception system is used to obtain perception data, and a rendering system is used to execute the rendering method provided by the present application, generate first rendering data based on the perception data, and secondary render the first rendering data to a first window of a first screen in a display system. The display system includes multiple display devices (not shown in FIG4 ), and a screen buffer module corresponding to a window in each display device. That is, the screen buffer module includes screen buffers (surfaces) corresponding to different windows of different display devices in the display system.

[0093] Taking the rendering system as an example, the system structure of the rendering system is introduced in detail.

[0094] In an embodiment of the present application, the rendering system includes an application layer, a framework layer, and a kernel layer. The application layer includes a first application; the framework layer includes a window manager module and an activity manager module; and the kernel layer includes a graphics rendering application programming interface (API).

[0095] Optionally, the first application is used to generate first rendering data based on the perception data obtained from the perception system, and then render the first rendering data to a window of a display device in the display system. The window manager module is used to manage and control windows in a graphical user interface (GUI). For example, it manages windows displayed on all display devices in the vehicle. The activity manager module is used to manage the life cycle and task stack of the active page of the application. For example, it coordinates and controls the active window of the first application, and manages the creation, startup, switching and destruction of the active page of the first application. Among them, the window manager module and the activity manager module respectively store the screen buffer corresponding to each window in different display devices in the vehicle. The graphics rendering application program interface is used to perform graphics rendering and image processing in the first application. For example, the graphics rendering API can be OpenGL (a cross-platform graphics rendering API), Vulkan (a cross-platform low-level graphics rendering and computing API), etc.

[0096] Specifically, the first application obtains perception data from the perception system and obtains the screen buffer corresponding to each window in different display devices from the window manager module or the activity manager module. The first application reconstructs and simulates the perception data through the graphics rendering application program interface to obtain first rendering data. The first application re-renders the first rendering data according to the different requirements of different display devices and outputs it to the screen buffer corresponding to the first rendering data in the screen buffer module (not shown in Figure 4). The screen buffer continues to process and display it on the window of the corresponding display device. For example, the first rendering data is re-rendered to the first window of the first screen.

[0097] FIG5 is a schematic diagram of the structure of a vehicle 500 provided in an embodiment of the present application. Referring to FIG5 , vehicle 500 may include various subsystems, such as a travel system 510, a sensor system 520, a control system 530, one or more peripheral devices 540, a power supply 550, a computer system 560, and a user interface 570. Optionally, vehicle 500 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 500 may be interconnected via wired or wireless connections.

[0098] Propulsion system 510 may include components that provide powered motion for vehicle 500. Engine 511 may be an electric motor or other types of engine combinations. Engine 511 converts energy source 512 into mechanical energy. Examples of energy source 512 include solar panels, batteries, and other sources of electricity. Transmission 513 may transmit mechanical power from engine 511 to wheels 514.

[0099] The sensor system 520 may include a number of sensors that sense information about the environment surrounding the vehicle 500. For example, the sensor system 520 may include a positioning system 521, such as a global positioning system (GPS), a BeiDou system, or other positioning systems, an inertial measurement unit (IMU) 522, a radar 523, a laser rangefinder 524, and a camera 525.

[0100] Control system 530 controls the operation of vehicle 500 and its components. Control system 530 may include various components, including a steering system 531, a throttle 532, a brake unit 533, a computer vision system 534, a path control system 535, and an obstacle avoidance system 536, which may also be referred to as an obstacle avoidance system.

[0101] Vehicle 500 interacts with external sensors, other vehicles, other computer systems, or users via peripheral devices 540. Peripheral devices 540 may include a wireless communication system 541, an onboard computer 542, a microphone 543, and / or a speaker 544.

[0102] Power source 550 may provide power to various components of vehicle 500 .

[0103] Some or all functions of vehicle 500 are controlled by computer system 560. Computer system 560 may include at least one processor 561 that executes instructions 5621 stored in a non-transitory computer-readable medium such as memory 562. Computer system 560 may also be a plurality of computing devices that control individual components or subsystems of vehicle 500 in a distributed manner.

[0104] The processor 561 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor may be a dedicated device such as an application-specific integrated circuit (ASIC) or other hardware-based processor.

[0105] In some embodiments, memory 562 may include instructions 5621 (e.g., program logic) that are executable by processor 561 to perform various functions of vehicle 500. Memory 562 may also include additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of travel system 510, sensor system 520, control system 530, and peripherals 540.

[0106] In addition to instructions 5621, memory 562 may also store data such as road maps, route information, the vehicle's location, direction, speed, and other vehicle data, and other information. This information may be used by vehicle 500 and computer system 560 during operation of vehicle 500 in autonomous, semi-autonomous, and / or manual modes.

[0107] The user interface 570 is used to provide information to or receive information from a user of the vehicle 500 .

[0108] Computer system 560 may control functions of vehicle 500 based on input received from various subsystems (eg, travel system 510 , sensor system 520 , and control system 530 ) and from user interface 570 .

[0109] In some embodiments, vehicle 500 may also include a vehicle controller (not shown in FIG. 5 ), which can also be described as a powertrain controller or intelligent driving computing platform. It is the core control component of the entire vehicle. It collects input information from various systems and components, makes decisions based on this input, and controls the operation of various components in vehicle 500 to drive vehicle 500.

[0110] Specifically, as the command and management center for vehicle 500, the vehicle controller's primary functions include: driving torque control, optimized braking energy control, vehicle energy management, maintenance and management of the Controller Area Network (CAN), fault diagnosis and troubleshooting, and vehicle status monitoring. It controls vehicle operation. Therefore, the quality of the vehicle controller directly determines the stability and safety of the vehicle.

[0111] Alternatively, one or more of the above components may be installed or associated separately from the vehicle 500. For example, the memory 562 may be partially or completely separate from the vehicle 500. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0112] Optionally, the above components are only an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 5 should not be understood as a limitation on the embodiments of the present application.

[0113] The vehicle 500 may be a new energy vehicle, an electric vehicle, a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawn mower, an amusement vehicle, an amusement park vehicle, construction equipment, a tram, a golf cart, or a train, etc., and is not particularly limited in this embodiment of the present application. The vehicle may be powered by gasoline, diesel, electricity, solar energy, hydrogen energy, etc.

[0114] In other embodiments of the present application, the vehicle may further include hardware structures and / or software modules to implement the aforementioned functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular one of the aforementioned functions is implemented in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0115] The method provided in the embodiments of the present application is described below with reference to the accompanying drawings.

[0116] In order to optimize the rendering strategy and improve the flexibility of rendering, this application proposes a rendering method, which can be executed by a vehicle or other devices outside the vehicle, such as mobile phones, computers and other electronic devices. It can also be a processor on the vehicle or other devices outside the vehicle, such as the processor 561 mentioned above.

[0117] The present application embodiment is described using an in-vehicle rendering scenario as an example. Referring to FIG6 , FIG6 shows a flowchart of a rendering method provided by the present application embodiment. The rendering method includes the following steps S601-S603:

[0118] S601. The vehicle controller obtains perception data; the perception data includes vehicle surrounding environment data and vehicle status data.

[0119] In the embodiment of the present application, the vehicle controller can also be an intelligent driving computing platform, which is the core control component of the vehicle. For example, the vehicle controller verifies data in communication, storage, and other scenarios in the vehicle to ensure data security in the vehicle.

[0120] In the embodiment of the present application, the vehicle controller may also be a controller of a component or system in the vehicle, for example, the controller of the computer vision system 534 in FIG5 .

[0121] In the embodiment of the present application, the perception data is the vehicle's surrounding environment data and vehicle status data at the current moment.

[0122] In the embodiments of the present application, vehicle-surrounding environmental data is used to represent the environmental conditions surrounding the vehicle. Exemplarily, the vehicle-surrounding environmental data includes, but is not limited to, road information (such as road type and lane information), traffic sign information (such as signboard information and traffic light information), obstacle information (such as obstacle location, speed, and predicted trajectory), other vehicle information, terrain information (such as geographic information of the vehicle's location, terrain height, and map data), and other environmental information.

[0123] In the embodiments of the present application, vehicle status data is used to represent the status information of the vehicle itself. For example, the vehicle status data includes, but is not limited to, vehicle speed, vehicle acceleration, vehicle angular velocity, vehicle attitude angle, vehicle position, vehicle gear status, vehicle throttle and brake status, vehicle steering wheel speed, road slope of the vehicle, radar information, and other status information.

[0124] In some embodiments of the present application, the vehicle controller obtains perception data through various sensors installed on the vehicle.

[0125] Optionally, sensors installed on the vehicle include but are not limited to speed sensors, inertial measurement units (IMUs), cameras, radars, lidars, global positioning systems (GPSs), wheel speed sensors, etc.

[0126] For example, the vehicle is equipped with multiple cameras to collect image data of the vehicle's surrounding environment; the vehicle is equipped with radar to detect obstacle information around the vehicle (such as position and speed); the vehicle is equipped with lidar to provide high-precision three-dimensional point cloud data of the surrounding environment for subsequent drawing, reconstruction, and simulation rendering of 3D images; the vehicle is equipped with GPS to provide accurate vehicle location information; the vehicle is equipped with IMU to provide vehicle posture information and motion status, etc. The vehicle controller obtains perception data based on various sensors installed on the vehicle.

[0127] In some embodiments, the perception data may be detection data from sensors in the vehicle, for example, images captured by each camera in the vehicle.

[0128] In other embodiments, the perception data may also be pre-processed detection data from sensors in the vehicle and then processed afterwards, for example, a 360° panoramic image generated based on images captured by each camera in the vehicle.

[0129] In some other embodiments of the present application, the vehicle controller obtains perception data from various subsystems of the vehicle. As in the example of vehicle 500 described above, the vehicle controller obtains perception data from sensor system 520 , control system 530 and other subsystems of the vehicle 500 .

[0130] It is understandable that the embodiments of the present application do not limit the perception data.

[0131] S602. The vehicle controller generates first rendering data through a first application based on the perception data; wherein the first rendering data corresponds to a first perspective, which is an observation perspective for observing the vehicle.

[0132] In the embodiment of the present application, the first application is an application for performing image rendering. The number of the first application is one.

[0133] In some embodiments, the first application analyzes and processes the perception data, including rendering processes such as computer vision, deep learning, path planning, 3D reconstruction, and simulation rendering. In this process, the first application can perform corresponding processing based on the perception data to generate first rendering data.

[0134] In an embodiment of the present application, the first rendering data is rendering data generated by a first application at a first perspective. The rendering data can be understood as images or simulation data of the vehicle state and surrounding environment. Exemplarily, the first rendering data is a 3D image, and the first rendering data is cached in a texture buffer.

[0135] In the embodiment of the present application, the first viewing angle is the viewing angle for observing the vehicle.

[0136] It is understood that in the embodiments of the present application, the first application reconstructs and renders the perception data to simulate a realistic 3D scene of the vehicle's current environment. In this scene, the vehicle serves as the observed object, and the user can observe the vehicle from different positions and angles, such as a bird's-eye view, a head-on view, and a panoramic view. For example, a head-on view can be used to observe the road traffic conditions in front of the vehicle. Alternatively, a panoramic view can be used to observe the broader surrounding environment.

[0137] It is understood that the first rendering data is an image or simulation of the vehicle's state and surrounding environment generated based on a first-person perspective of the vehicle. The first rendering data can be used for a visual interface for autonomous driving, as well as for recording, analyzing, testing, and verifying data.

[0138] In some embodiments of the present application, the vehicle controller generates first rendering data through a first application based on the perception data, including: the vehicle controller constructs a vehicle-surrounding environment scene through the first application based on the perception data; the vehicle controller renders the vehicle-surrounding environment scene through the first application based on the first perspective to generate the first rendering data.

[0139] In the embodiment of the present application, the vehicle surrounding environment scene is a simulated schematic diagram of the environment that the vehicle is currently in. Exemplarily, the vehicle surrounding environment scene is generated by 3D reconstruction technology, and the vehicle surrounding environment scene is a 3D image.

[0140] In some embodiments, the first application uses computer vision, deep learning, 3D reconstruction, and other technologies to model and reconstruct the perception data, combining the vehicle's surrounding environment data with the vehicle's own state data to construct the vehicle's current surrounding environment scene. Based on the constructed vehicle surrounding environment scene, the first application performs simulation rendering processing based on the first perspective to generate first rendering data (e.g., a 3D image).

[0141] Exemplarily, as shown in FIG7 , a first rendering data is provided in an embodiment of the present application, and the first rendering data is a 3D picture. The 3D picture 700 includes a 3D image 701 of the vehicle itself and 3D images of various objects in the current vehicle's surrounding environment. The first perspective corresponding to the 3D picture 700 is the perspective of observing the vehicle from the rear and top of the vehicle. For example, it includes but is not limited to 3D images 702 of one or more other vehicles (two are shown in FIG7 ), 3D images 703 of road lines (five are shown in FIG7 ), 3D images 704 of pedestrians, and one or more of 3D images of road signs not shown in FIG7 , 3D images of buildings, 3D images of roadblocks, etc.

[0142] In one possible implementation, a first application models and reconstructs the vehicle's current surrounding environment based on the perception data. The first application determines a first perspective based on the perception data and performs simulation rendering based on the constructed vehicle surrounding environment, using the first perspective, to generate first rendering data.

[0143] Specifically, in the embodiment of the present application, the vehicle controller may store a correspondence between pre-set perception data and observation perspectives.

[0144] For example, the optimal observation perspective corresponding to different perception data is determined based on experiments or experience, such as the optimal observation perspective when the vehicle is in driving state is a bird's-eye view from the rear and above of the vehicle or a horizontal view in front of the vehicle, so as to display environmental information such as road conditions and vehicles in front, and assist users in driving safely; or, the optimal observation perspective when the vehicle is parked is a bird's-eye view from directly above the vehicle, so as to assist users in grasping the position and size of the parking space and accurately parking the vehicle; or, the optimal observation perspective when the vehicle is in high-speed driving state is a panoramic view, which can display the entire vehicle's surroundings, facilitating the driver to react quickly to avoid danger; or, the optimal observation perspective when the vehicle is reversing is a rear view, so as to clearly display obstacles and distances behind the vehicle, and to reverse safely; or, the optimal observation perspective when the vehicle is turning is a side view, so as to clearly display the conditions on the side and at the turning point of the vehicle, and to turn safely, etc.

[0145] It can be understood that the observation perspective corresponding to the perception data is determined based on the preset correspondence, so as to output rendering data that is more in line with the current state of the vehicle to the display device, assisting the driver to better understand the surrounding environment, make accurate judgments and operations, and better ensure driving safety.

[0146] In some embodiments, after the first rendering data is obtained, the first rendering data is cached in a texture buffer so that it can be quickly applied to a corresponding display device later.

[0147] It is understandable that the first rendering data in the embodiment of the present application is not directly transmitted to the display device, but is first cached in the texture buffer so that the rendering data can be quickly obtained according to the usage requirements of each display device. Caching the rendering data in the texture buffer, on the one hand, facilitates further processing, operation and modification of these rendering data, can achieve more image effects, increase interactivity, and can also dynamically adjust the display content according to demand to adapt to different needs and scenarios. On the other hand, the rendering data cached in the texture buffer can be used for display on different screens, which increases the utilization rate of the rendering data. Moreover, in the prior art, the generated rendering data is directly transmitted to the display screen, and rendering is performed each time it is displayed. However, the present application caches the rendering data in the texture buffer, and the rendering is completed once. The rendering data can be subsequently transmitted to multiple display devices by copying or mapping, which reduces repeated rendering and improves rendering performance.

[0148] It is understandable that the above example uses the first viewing angle as an example to illustrate the generation process of the first rendering data. In actual applications, the first application can also generate different rendering data based on different viewing angles.

[0149] In other embodiments of the present application, the vehicle controller may further generate second rendering data based on the perception data through the first application, wherein the second rendering data corresponds to a second perspective, which is a perspective from which the vehicle is viewed, and is different from the first perspective.

[0150] In some embodiments, the vehicle controller constructs the vehicle surrounding environment scene through the first application based on the perception data; the vehicle controller renders the vehicle surrounding environment scene through the first application based on the second perspective to generate the second rendering data.

[0151] In other embodiments, the vehicle controller determines the observation angle corresponding to each display device based on the pre-set correspondence between the display device and the observation angle, generates rendering data based on the observation angle, and renders the rendering data a second time to the window of the display device corresponding to the observation angle.

[0152] Specifically, in the embodiment of the present application, the vehicle controller may store a pre-set correspondence between display devices and viewing angles, and customize the rendering content and viewing angle corresponding to each display device based on the display requirements of each display device.

[0153] For example, for the instrument panel, the corresponding rendering content can be surrounding vehicle and pedestrian road information (surrounding environment data), and the corresponding observation perspective can be a top-down observation perspective from behind the vehicle or a horizontal perspective in front of the vehicle. Alternatively, for the central control panel, the corresponding rendering content can be all-round surrounding environment data, such as a 360-degree surround image stitched together from images captured by four fisheye cameras in front, behind, and on both sides of the vehicle, and the corresponding observation perspective can be a bird's-eye view from directly above the vehicle. Alternatively, for the head-up display, the corresponding rendering content can be navigation prompts and road condition warning information, and the corresponding observation perspective can be a perspective in front of the vehicle or a panoramic perspective.

[0154] It can be understood that by setting the corresponding observation angle of each display device, each display device can clearly present key information in the driver's field of view, clearly display the environment around the vehicle, and improve the driver's perception of the surrounding environment; better meet the driver's information needs and improve driving safety and comfort.

[0155] It is understandable that in the embodiments of the present application, the entire vehicle is used as the observation object. First, based on the perception data, a simulated environment scene of the vehicle is constructed with the vehicle as the center (which can also be understood as simulating and constructing the world in which the current vehicle is located in the form of a three-dimensional model, which world includes the vehicle and the surrounding environment). In this simulated environment scene, the user can observe the vehicle from different perspectives, and based on the user's observation perspective, simulation rendering is performed to obtain rendering data corresponding to the observation perspective. For example, based on the first perspective, rendering generates first rendering data; based on the second perspective, rendering generates second rendering data.

[0156] In the embodiment of the present application, the first rendering data may be first 3D rendering data, and the second rendering data may be second 3D rendering data.

[0157] It is understandable that the above example takes 3D reconstruction rendering as an example. In actual applications, the first application can also perform two-dimensional (2D) rendering on the perception data to obtain corresponding 2D rendering data based on different perspectives.

[0158] It is understandable that observation perspectives can be set in advance for different display devices in the vehicle. When rendering, the vehicle surrounding environment scene is first constructed based on the perception data, and then the rendering data corresponding to each display device is rendered separately according to the observation perspective pre-set for each display device. In this way, the vehicle controller obtains the rendering data corresponding to each display device based on the same perception data and preset observation perspective. The rendering data corresponding to each display device comes from the same data source (the vehicle surrounding environment scene). Different display devices share the same rendering logic, which can save system memory and system computing power and avoid resource waste. Moreover, different display devices reuse the same rendering data source, and the rendering content transmitted in each display device is updated synchronously, which can ensure that the rendering data is fully synchronized, thereby improving the utilization rate of rendering resources and rendering efficiency. The processing logic of the user interface display content of each display device is consistent (this consistency means that the vehicle status and surrounding environment on different display devices are consistent (for example, if the vehicle is driving normally and there is an obstacle on the right side of the vehicle, the obstacle will be displayed on the right side of the vehicle on different display devices). The specific display content on different display devices is different based on the viewing angle (for example, the obstacle on the right side of the vehicle is displayed according to the top-down or horizontal viewing angle, and the specific shape or size of the obstacle may be different), which reduces user confusion and improves the overall user experience. For example, when the vehicle makes a quick turn or brakes suddenly, the vehicle's turning condition or braking condition is displayed synchronously on different display devices.

[0159] In this application, when rendering data corresponding to different perspectives, rendering is completed based on the same rendering logic and the same perception data, which saves system computing power, improves rendering efficiency, and enhances user experience.

[0160] S603: The vehicle controller re-renders the first rendering data to the first window of the first screen through the first application.

[0161] In the embodiment of the present application, the first screen is a display device in the vehicle, and the number of the first screen is one. The first window is a pre-divided area in the first screen for displaying an image corresponding to the first rendering data.

[0162] In the embodiment of the present application, secondary rendering can be understood as transmitting the first rendering data generated by rendering from the texture buffer to the window buffer of the display device, so that the window processes the rendering data and displays the display content corresponding to the rendering data.

[0163] It's understandable that if a display device needs to present multiple rendering data, it simply transfers the rendering data stored in the texture buffer to the window on the screen to be displayed, which then handles subsequent processing and display. By performing two rendering operations, the vehicle controller can quickly access rendering data based on user needs, flexibly and efficiently allocating rendering data transmission. This improves rendering flexibility, enhances system performance, and optimizes rendering strategies to meet diverse user needs and enhance the user experience.

[0164] For example, as shown in FIG8A , the first screen is an instrument panel. A first window (dashed box) in the instrument panel displays a 3D image corresponding to the first rendering data. The first perspective corresponding to the first rendering data is a rear-upper-view perspective. The instrument panel may also display information such as vehicle speed, mileage, charging status, and time.

[0165] In an embodiment of the present application, the vehicle controller may also store preset window information for each display device displaying environment rendering data. After the vehicle controller generates rendering data, the vehicle controller obtains the display device currently displaying the vehicle, determines the window information of the currently displaying display device, and re-renders the rendering data into the corresponding window.

[0166] For example, the vehicle controller can obtain the vehicle's current display device while acquiring the perception data, and determine the window information and corresponding viewing angle of the current display device. For example, if the vehicle's current display device only has a first screen, the window information of the first screen is determined to be the first window based on the stored information, and the viewing angle corresponding to the first screen is determined to be the first viewing angle. Based on the perception data, the vehicle controller generates first rendering data through a first application, and then uses the first application to re-render the first rendering data to the first window of the first screen.

[0167] It is understood that in the embodiment of the present application, the vehicle controller or the first application stores the window information and corresponding viewing angle of each display device in the vehicle. During the rendering process, the rendering data corresponding to each display device can be personalized based on the preset corresponding relationship to meet the user's personalized needs, improve the user experience as well as rendering efficiency, rendering flexibility and rendering effect. Moreover, in the present application, the rendering is completed independently by the first application, and the first application directly reuses the multi-screen display multiplexing mechanism of the operating system to output accurate and complete rendering data to multiple display devices.

[0168] It is understandable that obtaining window information while obtaining perception data can avoid resource waste caused by repeated rendering and improve rendering efficiency.

[0169] In this application, a first application generates rendering data based on the perception data, and then a second rendering is performed to output the rendering data to the display device. The two renderings increase the flexibility and scalability of the rendering. The rendering data generated by the first rendering can be saved in the system so that it can be rendered twice to different display devices to achieve diverse display effects. This improves the flexibility of the rendering, can meet the usage requirements of different application scenarios, and provide personalized display solutions.

[0170] It can be understood that in the above example, each display device receives one type of rendering data, that is, the relationship between the display device and the rendering data is one-to-one. In actual applications, each display device can also receive multiple types of rendering data, or multiple display devices can also receive the same type of rendering data.

[0171] In an embodiment of the present application, the rendering method further includes: rendering the first rendering data a second time to a second window on the second screen through the first application.

[0172] In an embodiment of the present application, based on the example of the second rendering data in S602 above, the vehicle controller may also perform secondary rendering of the second rendering data to a third window of a third screen through the first application.

[0173] For example, as shown in FIG8B , the third screen is the central control screen. The third window (dashed box) in the central control screen displays a 3D image corresponding to the second rendering data. The second perspective corresponding to the second rendering data is a bird's-eye view directly above the vehicle. The central control screen can also display information such as applications, time, temperature, and prompts.

[0174] In an embodiment of the present application, the vehicle controller may also secondary render the first rendering data to the first window of the first screen through the first application, and secondary render the second rendering data to the fourth window of the first screen.

[0175] Exemplarily, based on the examples of Figures 8A and 8B above, as shown in Figure 8C, the first screen is an instrument screen, the first window (dashed box) of the instrument screen displays a 3D image corresponding to the first rendering data, and the fourth window (solid box) of the instrument screen displays a 3D image corresponding to the second rendering data.

[0176] It is understandable that the above example only renders the rendering data into the window of the display device, and does not involve the subsequent synthetic display of the rendering data by the display device. The synthetic display of the rendering data by the rendering device can be found in the prior art and will not be repeated here.

[0177] For example, taking the vehicle including the instrument screen, central control screen and head-up display as an example, as shown in Figures 9, 10 and 11, they are implementation framework diagrams of different usage scenarios provided in the embodiments of the present application.

[0178] As shown in Figure 9, the first application obtains perception data, and renders the instrument texture buffer (such as the first rendering data, corresponding to the first perspective), the central control texture buffer (such as the second rendering data, corresponding to the second perspective) and the head-up display texture buffer (such as the third rendering data, corresponding to the third perspective) based on the perception data. According to the preset one-to-one relationship between the rendering data and the display device, the rendering data is rendered twice and output to the window of the corresponding display device. For example, the instrument texture buffer is rendered twice to obtain instrument output data, and the instrument output data is transmitted to the first window of the instrument screen (not shown in the figure), the central control texture buffer is rendered twice to obtain central control output data, and the central control output data is transmitted to the second window of the central control screen (not shown in the figure), the head-up display texture buffer is rendered twice to obtain head-up display output data, and the head-up display output data is transmitted to the third window of the head-up display screen (not shown in the figure).

[0179] As shown in Figure 10, the first application obtains perception data, and renders the instrument texture buffer (such as the first rendering data, corresponding to the first perspective) and the central control texture buffer (such as the second rendering data, corresponding to the second perspective) based on the perception data. According to the preset many-to-one relationship between the rendering data and the display device, the instrument texture buffer is rendered twice to obtain the instrument output data, and the instrument output data is transmitted to the first window of the instrument screen (not shown in the figure). The central control texture buffer is rendered twice to obtain the central control output data, and the central control output data is transmitted to the second window of the central control screen (not shown in the figure). The central control texture buffer is rendered twice to obtain the head-up display output data, and the head-up display output data is transmitted to the third window of the head-up display screen (not shown in the figure).

[0180] As shown in Figure 11, the first application obtains perception data and renders the instrument texture buffer (such as the first rendering data, corresponding to the first perspective), the central control texture buffer (such as the second rendering data, corresponding to the second perspective) and the head-up display texture buffer (such as the third rendering data, corresponding to the third perspective) based on the perception data. According to the preset many-to-many relationship between rendering data and display devices, the instrument output data is obtained by secondary rendering based on the instrument texture buffer and the central control texture buffer, and the instrument output data is transmitted to the instrument screen. The first window of the instrument screen (not shown in the figure) receives the instrument texture buffer, and the fourth window of the instrument screen (not shown in the figure) receives the central control texture buffer. The central control output data is obtained by secondary rendering based on the central control texture buffer and the head-up display texture buffer, and the central control output data is transmitted to the central control screen. The second window of the central control screen (not shown in the figure) receives the central control texture buffer, and the fifth window of the instrument screen (not shown in the figure) receives the head-up display texture buffer. The head-up display texture buffer is secondary rendered to obtain the head-up display output data, and the head-up display output data is transmitted to the third window of the head-up display screen (not shown in the figure).

[0181] It can be understood that based on the correspondence between rendering data and display devices, personalized display can be achieved, the display content of the display device can be enriched, and safe driving can be better guaranteed.

[0182] In this application, through the above-mentioned method, in the same application, based on the same perception data, the same rendering engine and rendering process are used to complete the rendering of different display devices with different viewing angles. The rendering method provided by this application increases the flexibility and scalability of rendering through two renderings. During the second rendering, the rendering data can be rendered to different display devices to achieve diversified display and optimize the rendering strategy. Moreover, in the same application, multi-angle rendering is completed, and the same rendering data source is reused to ensure that the rendering data is fully synchronized, saving system computing power, improving rendering efficiency, meeting the personalized needs of users, and improving user experience.

[0183] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0184] Figure 12 shows a schematic diagram of the structure of another rendering device provided in an embodiment of the present application. The rendering device 1200 includes an acquisition module 1201 and a processing module 1202. The rendering device 1200 is used to execute the aforementioned rendering method, for example, the rendering method shown in Figures 6-11. Of course, the rendering device 1200 may also include other modules, or the rendering device 1200 may include fewer modules. The embodiments of the present application do not specifically limit the specific form and implementation of the rendering device.

[0185] The acquisition module 1201 is used to acquire perception data; the perception data includes vehicle surrounding environment data and vehicle status data.

[0186] The processing module 1202 is configured to generate first rendering data based on the perception data through a first application, wherein the first rendering data corresponds to a first perspective, which is a perspective for observing the vehicle;

[0187] The processing module 1202 is further configured to re-render the first rendering data to the first window of the first screen through the first application.

[0188] The operations and / or functions of each module in the rendering device 1200 are respectively for implementing the corresponding processes of the rendering method described in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional unit. For the sake of brevity, they will not be repeated here.

[0189] Optionally, the rendering apparatus 1200 shown in FIG12 may further include a storage module (not shown in FIG12 ) storing a program or instruction. When the acquisition module 1201 and the processing module 1202 execute the program or instruction, the rendering apparatus 1200 shown in FIG12 may perform the rendering method described in the above method embodiment.

[0190] The technical effects of the rendering device 1200 shown in FIG12 may refer to the technical effects of the rendering method described in the above method embodiment, and will not be described in detail here.

[0191] An embodiment of the present application also provides a chip system, as shown in Figure 13, the chip system 1300 includes at least one processor 1301 and at least one interface circuit 1302. As an example, when the chip system 1300 includes one processor and one interface circuit, the one processor may be the processor 1301 shown in the solid box in Figure 13 (or the processor 1301 shown in the dotted box), and the one interface circuit may be the interface circuit 1302 shown in the solid box in Figure 13 (or the interface circuit 1302 shown in the dotted box). When the chip system 1300 includes two processors and two interface circuits, the two processors include the processor 1301 shown in the solid box in Figure 13 and the processor 1301 shown in the dotted box, and the two interface circuits include the interface circuit 1302 shown in the solid box in Figure 13 and the interface circuit 1302 shown in the dotted box. This is not limited.

[0192] The processor 1301 and the interface circuit 1302 can be interconnected via a line. For example, the interface circuit 1302 can be used to receive a signal. For another example, the interface circuit 1302 can be used to send a signal to another device (such as the processor 1301). Exemplarily, the interface circuit 1302 can read an instruction stored in a memory and send the instruction to the processor 1301. When the instruction is executed by the processor 1301, the rendering device can perform the various steps in the above embodiment. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiment of the present application.

[0193] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system on a chip (SoC), a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chip.

[0194] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0195] An embodiment of the present application further provides a computer-readable storage medium storing one or more computer programs, wherein the one or more computer programs include instructions, which, when executed by a computer, enable the computer to execute the corresponding process of the rendering method in the above embodiment.

[0196] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0197] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the rendering method in the above-mentioned embodiment.

[0198] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0199] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A rendering method, characterized in that: Applied to a vehicle, the method comprises: Acquiring sensory data, wherein the sensory data includes vehicle surrounding environment data and vehicle status data; Based on the perception data, generating first rendering data by a first application; wherein the first rendering data corresponds to a first perspective, and the first perspective is an observation perspective for observing the vehicle; The first rendering data is secondary rendered to a first window of a first screen through the first application.

2. The method according to claim 1, characterized in that The method further comprises: The first rendering data is secondary rendered to a second window of a second screen through the first application.

3. The method according to claim 1 or 2, characterized in that The method further comprises: generating, by the first application, second rendering data based on the perception data, wherein the second rendering data corresponds to a second perspective; the second perspective is a perspective for observing the vehicle, and the second perspective is different from the first perspective; The second rendering data is secondary rendered to a third window of a third screen through the first application.

4. The method according to claim 3, characterized in that The method further comprises: The first rendering data is secondary rendered to the first window of the first screen through the first application, and the second rendering data is secondary rendered to the fourth window of the first screen.

5. The method according to claim 1, wherein The generating, by a first application, first rendering data based on the perception data includes: Based on the perception data, construct a vehicle surrounding environment scene through the first application; Based on the first perspective, the vehicle surrounding environment scene is rendered by the first application to generate the first rendering data.

6. The method according to claim 3, characterized in that The first rendering data is first 3D rendering data, and the second rendering data is second 3D rendering data.

7. The method according to any one of claims 1 to 6, characterized in that The acquiring of the perception data includes: The perception data is acquired through sensors.

8. A rendering device, characterized in that: Applied to a vehicle, the device comprises: An acquisition module is used to acquire perception data; the perception data includes vehicle surrounding environment data and vehicle status data; a processing module, configured to generate first rendering data based on the perception data through a first application; wherein the first rendering data corresponds to a first perspective, and the first perspective is an observation perspective for observing the vehicle; The processing module is further configured to secondary render the first rendering data to the first window of the first screen through the first application.

9. The device according to claim 8, characterized in that The processing module is further configured to re-render the first rendering data to a second window on a second screen through the first application.

10. The device according to claim 8 or 9, characterized in that The processing module is further configured to generate second rendering data based on the perception data through the first application; wherein the second rendering data corresponds to a second perspective; the second perspective is a perspective for observing the vehicle, and the second perspective is different from the first perspective; The processing module is further configured to perform secondary rendering of the second rendering data to a third window of a third screen through the first application.

11. The device according to claim 10, characterized in that The processing module is further configured to perform secondary rendering of the first rendering data to the first window of the first screen through the first application, and perform secondary rendering of the second rendering data to the fourth window of the first screen.

12. The device according to claim 8, characterized in that The processing module is further configured to construct a vehicle surrounding environment scene through the first application based on the perception data; The processing module is further configured to render the vehicle surrounding environment scene based on the first perspective through the first application to generate the first rendering data.

13. The device according to claim 10, characterized in that The first rendering data is first 3D rendering data, and the second rendering data is second 3D rendering data.

14. The device according to any one of claims 8 to 13, characterized in that The acquisition module is further configured to acquire the perception data through a sensor.

15. A rendering device, characterized in that: include: A processor and a memory, wherein the memory is coupled to the processor, and the memory is used to store computer-readable instructions. When the processor reads the computer-readable instructions from the memory, the rendering device executes the method according to any one of claims 1 to 7.

16. A vehicle, characterized in that: The vehicle includes the rendering device of claim 15 .

17. A chip system, characterized in that: The method comprises at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions, and the at least one processor is used to perform the method according to any one of claims 1 to 7.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

19. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images