Panoramic image rendering method and device, vehicle, medium and product

Abstract

The invention provides a panoramic image rendering method and device, a vehicle, a medium and a product, and relates to the field of vehicle-mounted images. The rendering method comprises the following steps: initializing a first panoramic image process and a second panoramic image process; under the condition that initialization of the first panoramic image process is completed, the current state of the second panoramic image process is obtained through the first panoramic image process; wherein the state of the second panoramic image process at least comprises a display state and a superposition state; and under the condition that the current state is a display state or a superposition state, rendering the panoramic image through the first panoramic image process and the second panoramic image process. Through the method, a proper rendering mode is matched according to the current state of the second panoramic image process. And when the second panoramic image process is in the display state or the superposition state, the panoramic image is rendered according to the synergistic effect of the first panoramic image process and the second panoramic image process, so that the acquisition efficiency of the panoramic image can be improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-mounted images, and in particular to a method, device, vehicle, medium and product for rendering panoramic images. Background Art

[0002] The Around View Monitor (AVM) is composed of four surround-view cameras to provide a 360-degree field of view to assist users in driving and parking.

[0003] With the rapid iterative development of intelligent driving technology, the autonomous driving domain controller is merged with the intelligent cockpit controller to form a cabin-parking and cabin-driving integrated architecture. Therefore, the panoramic imaging function is moved from the autonomous driving domain controller to the intelligent cockpit controller, thereby completing the implementation of this function under the intelligent cockpit system.

[0004] The smart cockpit uses the Android system, and most panoramic image functions are deployed in the Apk layer. However, there is a big problem in this scenario, that is, the slow startup time. The reason is that the Android system starts slowly, and the Apk application can only be started after the Android system starts. Therefore, the panoramic image acquisition speed is slow. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method, device, vehicle, medium and product for rendering panoramic images, so that the efficiency of obtaining panoramic images can be improved.

[0006] In a first aspect, an embodiment of the present application provides a panoramic image rendering method, which is applied to a vehicle, wherein the vehicle includes a bottom layer and an application layer, wherein the bottom layer is provided with a first panoramic image process, and the application layer is provided with a second panoramic image process; the first panoramic image process includes a first rendering function and a first display window function, and the second panoramic image process includes a second rendering function and a second display window function, and both the first display window function and the second display window function include a function of providing a panoramic image display interface; the rendering method includes:

[0007] Initialize the first panoramic image process and the second panoramic image process;

[0008] When the first panoramic image process is initialized, the current state of the second panoramic image process is obtained through the first panoramic image process; wherein the state of the second panoramic image process includes at least a display state and an overlay state; the display state indicates that the display window function of the second panoramic image process is initialized, but the rendering function is not initialized; the overlay state indicates that both the rendering function and the display window function of the second panoramic image process are initialized;

[0009] When the current state is a display state or an overlay state, the panoramic image is rendered through the first panoramic image process and the second panoramic image process.

[0010] In some embodiments, the second display window function includes a third rendering function, and when the current state is the display state, the panoramic image is rendered through the first panoramic image process and the second panoramic image process, including:

[0011] When the current state is the display state, performing off-screen rendering on the first image data through a first rendering function to obtain a first panoramic image video stream;

[0012] Sending the first panoramic image video stream to the second panoramic image process through the first panoramic image process;

[0013] The first panoramic image video stream is screen rendered through the panoramic image display interface function of the second display window function and the third rendering function to obtain a first target panoramic image.

[0014] In some implementations, sending the panoramic image to the second panoramic image process through the first panoramic image process includes:

[0015] Based on the Shmem shared memory method, the first panoramic image video stream is sent to the second panoramic image process through the first panoramic image process; or

[0016] Based on the DMA transmission mode, the first panoramic image video stream is sent to the second panoramic image process through the first panoramic image process.

[0017] In some implementations, the second panoramic image process further includes a fourth rendering function, which, when the current state is an overlay state, performs panoramic image rendering through the first panoramic image process and the second panoramic image process, including:

[0018] When the current state is the overlay state, the second image data is screen rendered by the first rendering function to obtain a second panoramic image video stream;

[0019] Sending the second panoramic image video stream to the second panoramic image process through the first panoramic image process;

[0020] Performing screen rendering on the second panoramic image video stream through the panoramic image display interface function of the second rendering function and the second display window function to obtain a third panoramic image video stream;

[0021] Rendering the target two-dimensional material in the material library through the fourth rendering function to obtain a three-dimensional model corresponding to the target two-dimensional material;

[0022] The third panoramic image video stream and the three-dimensional model are superimposed through a second panoramic image process to obtain a second target panoramic image.

[0023] In some implementations, the state of the second panoramic image process further includes an initial state and a destroyed state, the initial state indicating that both the rendering function and the display window function of the second panoramic image process are not initialized, the destroyed state indicating that the second panoramic image process is destroyed, and the rendering method further includes:

[0024] When the current state is the initial state, the third image data is screen rendered through the panoramic image display interface function of the first display window function and the first rendering function to obtain a third target panoramic image;

[0025] When the current state is the destroyed state, the fourth image data is screen rendered through the panoramic image display interface function of the first display window function and the first rendering function to obtain a fourth target panoramic image.

[0026] In some implementations, before obtaining the current state of the second panoramic image process through the first panoramic image process, the rendering method includes:

[0027] An inter-process communication service is created, where the inter-process communication service is used for information exchange between the first panoramic image process and the second panoramic image process.

[0028] In a second aspect, an embodiment of the present application provides a panoramic image rendering device, which is applied to a vehicle. The vehicle includes a bottom layer and an application layer. The bottom layer is provided with a first panoramic image process, and the application layer is provided with a second panoramic image process. The first panoramic image process includes a first rendering function and a first display window function, and the second panoramic image process includes a second rendering function and a second display window function. Both the first display window function and the second display window function include a function of providing a panoramic image display interface. The rendering device includes:

[0029] An initialization module, initializing the first panoramic image process and the second panoramic image process;

[0030] The acquisition module acquires the current state of the second panoramic image process through the first panoramic image process when the first panoramic image process is initialized; wherein the state of the second panoramic image process includes at least a display state and an overlay state; the display state indicates that the display window function of the second panoramic image process is initialized, but the rendering function is not initialized; the overlay state indicates that both the rendering function and the display window function of the second panoramic image process are initialized;

[0031] The rendering module renders the panoramic image through the first panoramic image process and the second panoramic image process when the current state is the display state or the overlay state.

[0032] In a third aspect, an embodiment of the present application provides a vehicle, including:

[0033] a memory configured to store instructions; and

[0034] The processor is configured to call the instructions from the memory and implement the panoramic image rendering method provided in the first aspect of the embodiment of the present application when executing the instructions.

[0035] In a fourth aspect, an embodiment of the present application provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the panoramic image rendering method described above.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the panoramic image rendering method according to the above-mentioned implementation manner.

[0037] In an embodiment of the present application, a first panoramic image process and a second panoramic image process are initialized. When the first panoramic image process is initialized, the current state of the second panoramic image process is obtained through the first panoramic image process, and the state of the second panoramic image process includes at least a display state and an overlay state. When the current state is a display state or an overlay state, the panoramic image is rendered through the first panoramic image process and the second panoramic image process. In this way, a suitable rendering method is matched according to the current state of the second panoramic image process. And when the current state of the second panoramic image process is a display state or an overlay state, the panoramic image is rendered according to the synergy of the first panoramic image process and the second panoramic image process, which can improve the efficiency of obtaining the panoramic image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is one of the flowcharts of the panoramic image rendering method provided in the embodiment of the present application;

[0039] Figure 2 This is a second flow chart of a panoramic image rendering method provided in an embodiment of the present application;

[0040] Figure 3 is a schematic diagram of the structure of a panoramic image rendering device provided in an embodiment of the present application;

[0041] Figure 4 It is a schematic diagram of components for multi-process joint rendering of the Native layer and APK layer of a panoramic image provided in an embodiment of the present application;

[0042] Figure 5 It is a schematic diagram of the structure of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0044] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0045] In the following, in combination with the accompanying drawings, the panoramic image rendering method, device, vehicle, medium and product provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0046] See also Figure 1 , is a flow chart of a panoramic image rendering method provided in an embodiment of the present application, the rendering method is applied to a vehicle, the vehicle includes a bottom layer and an application layer, the bottom layer is provided with a first panoramic image process, and the application layer is provided with a second panoramic image process; the first panoramic image process includes a first rendering function and a first display window function, the second panoramic image process includes a second rendering function and a second display window function, and the first display window function and the second display window function both include a function of providing a panoramic image display interface.

[0047] In one example, the bottom layer and application layer of the vehicle may be located in the same system, such as the bottom layer and application layer of the vehicle are both located in the Android system. In another example, the bottom layer and application layer of the vehicle may be located in different systems, such as the bottom layer of the vehicle may be located in the Qnx system or the Linux system, and the application layer of the vehicle may be located in the Android system. It should be noted that the above two examples only provide two possibilities and are not used to limit the system to which the vehicle, including the bottom layer and application layer, belongs.

[0048] like Figure 1 As shown, the rendering method includes the following steps S100 to S300.

[0049] Step S100: Initializing a first panoramic image process and a second panoramic image process.

[0050] In an embodiment of the present application, the first panoramic image process is set at the bottom layer of the panoramic image rendering device, that is, the Native layer, and the second panoramic image process is set at the application layer of the panoramic image rendering device, that is, the APK layer.

[0051] The first panoramic image process includes a first rendering function and a first display window function, and the second panoramic image process includes a second rendering function and a second display window function. The first rendering function may include but is not limited to rendering functions such as view stitching, track lines, and 2D car models. The second rendering function may include but is not limited to rendering functions such as 3D car models. Both the first display window function and the second display window function include the function of providing a panoramic image display interface.

[0052] Before rendering the panoramic image, the first panoramic image process and the second panoramic image process are initialized. Through initialization, the previous state, such as cache or temporary data, can be cleared before rendering begins, which can avoid affecting the quality of the current image and ensure the accuracy of rendering.

[0053] Step S200: When the first panoramic imaging process is initialized, the current state of the second panoramic imaging process is obtained through the first panoramic imaging process; wherein the state of the second panoramic imaging process includes at least a display state and an overlay state; the display state indicates that the display window function of the second panoramic imaging process has been initialized, but the rendering function has not been initialized; the overlay state indicates that both the rendering function and the display window function of the second panoramic imaging process have been initialized.

[0054] In the embodiment of the present application, in general, the initialization speed of the first panoramic image process is greater than the initialization speed of the second panoramic image process. Therefore, after the initialization of the first panoramic image process is completed, the current state of the second panoramic image process is obtained through the first panoramic image process.

[0055] In one example, the first panoramic image process may obtain the current state of the second panoramic image process based on the inter-process communication service. The inter-process communication service may be understood as a module for communication between processes.

[0056] In the embodiment of the present application, the state of the second panoramic image process may include an initial state and a destroyed state in addition to a display state and an overlay state. The initial state may be used to indicate that the second panoramic image process has not been initialized. The destroyed state may be used to indicate that the second panoramic image process has been destroyed.

[0057] Different panoramic image rendering methods may be determined according to different current states of the second panoramic image process.

[0058] Step S300: When the current state is a display state or an overlay state, the panoramic image is rendered through a first panoramic image process and a second panoramic image process.

[0059] After obtaining the current state of the second panoramic image process, the functions initialized and completed by the second panoramic image process are different due to the different current states of the second panoramic image process. The first panoramic image process determines the rendering method of the panoramic image according to the current state of the second panoramic image process. The rendering method of the panoramic image may include but is not limited to rendering the panoramic image based on the first panoramic image process alone, and rendering the panoramic image based on the first panoramic image process and the second panoramic image process together.

[0060] In the embodiment of the present application, when the current state of the second panoramic image process is a display state or an overlay state, the panoramic image can be rendered jointly by the first panoramic image process and the second panoramic image process.

[0061] In the embodiment of the present application, when the current state of the second panoramic image process is an initial state or a destroyed state, the panoramic image can be rendered independently through the first panoramic image process.

[0062] In one example, when the current state of the second panoramic image process is the display state, the display window function also includes a third rendering function. The third rendering function can be used to render the video stream. During rendering, the first rendering function can be used to perform off-screen rendering on multiple image data to obtain a first panoramic image video stream. The first panoramic image video stream is then sent to the second panoramic image process through the first panoramic image process. Finally, through the second panoramic image process, the first panoramic image video stream is screen rendered according to the display window function to complete the rendering of the panoramic image. Screen rendering can be understood as rendering in the screen cache area currently used for display. Off-screen rendering can be understood as opening up an additional frame cache area and rendering to that area.

[0063] In one example, when the current state of the second panoramic image process is the superposition state, the second image data can be rendered off-screen through the first rendering function to obtain a second panoramic image video stream. Then, the second panoramic image video stream is sent to the second panoramic image process through the first panoramic image process. Then, the second panoramic image video stream is screen rendered through the second rendering function and the panoramic image display interface function of the second display window function to obtain a third panoramic image video stream. And the target two-dimensional material in the material library is rendered through the fourth rendering function to obtain a three-dimensional model corresponding to the target two-dimensional material. Finally, the third panoramic image video stream and the three-dimensional model can be superimposed through the second panoramic image process to obtain a second target panoramic image.

[0064] In one example, when the current state of the second panoramic image process is the initial state, the third image data can be screen rendered through the panoramic image display interface function of the first display window function and the first rendering function to obtain a third target panoramic image.

[0065] In one example, when the current state of the second panoramic image process is a destroyed state, the fourth image data can be screen rendered through the panoramic image display interface function of the first display window function and the first rendering function to obtain a fourth target panoramic image.

[0066] It should be noted that the first, second, third and fourth mentioned above are only used to distinguish similar objects and are not used to describe a specific order or sequence.

[0067] Through the above steps S100-S300, the first panoramic image process and the second panoramic image process are initialized. When the first panoramic image process is initialized, the current state of the second panoramic image process is obtained through the first panoramic image process, and the state of the second panoramic image process at least includes a display state and an overlay state. When the current state is a display state or an overlay state, the panoramic image is rendered through the first panoramic image process and the second panoramic image process. In this way, according to the current state of the second panoramic image process, a suitable rendering method is matched. And when the current state of the second panoramic image process is a display state or an overlay state, the panoramic image is rendered according to the synergy of the first panoramic image process and the second panoramic image process, which can improve the efficiency of obtaining the panoramic image.

[0068] In some embodiments, the second display window function includes a third rendering function, and when the current state is the display state, the panoramic image is rendered through the first panoramic image process and the second panoramic image process, including:

[0069] When the current state is the display state, performing off-screen rendering on the first image data through a first rendering function to obtain a first panoramic image video stream;

[0070] Sending the first panoramic image video stream to the second panoramic image process through the first panoramic image process;

[0071] The first panoramic image video stream is screen rendered through the panoramic image display interface function of the second display window function and the third rendering function to obtain a first target panoramic image.

[0072] Specifically, the display state indicates that the display window function of the second panoramic image process has been initialized, but the rendering function has not been initialized. When the current state of the second panoramic image process is the display state, the first panoramic image process and the second panoramic image process can perform rendering together.

[0073] When the current state of the second panoramic image process is the display state, the first panoramic image process has been initialized, that is, the display window function and the rendering function of the first panoramic image process have been initialized. Since the rendering function included in the first panoramic image process is different from the rendering function included in the second panoramic image process, the first image data can be rendered off-screen through the first rendering function of the first panoramic image process to obtain a first panoramic image video stream. The first panoramic image video stream is sent to the second panoramic image process through the first panoramic image process. Then, the first panoramic image video stream is screen rendered through the third rendering function of the second panoramic image process and the panoramic image display interface function of the second display window function, that is, the obtained first panoramic image video stream is rendered to the screen to obtain the first target panoramic image.

[0074] In this embodiment, when the current state of the second panoramic image process is the display state, the first panoramic image process and the second panoramic image process jointly render the panoramic image, so as to achieve functional complementarity between the two processes and improve resource utilization.

[0075] In some implementations, sending the first panoramic image video stream to the second panoramic image process through the first panoramic image process includes:

[0076] Based on the Shmem shared memory method, the first panoramic image video stream is sent to the second panoramic image process through the first panoramic image process; or

[0077] Based on the DMA transmission mode, the first panoramic image video stream is sent to the second panoramic image process through the first panoramic image process.

[0078] Specifically, after performing off-screen rendering on multiple image data through the first rendering function to obtain a first panoramic image video stream, the first panoramic image process sends the first panoramic image video stream to the second panoramic image process.

[0079] In one example, the first panoramic image video stream may be sent through a Shmem shared memory method. The shared memory method (Shmem) may be understood as an anonymous shared memory customized and developed based on the Linux shared memory.

[0080] In one example, the first panoramic image video stream may be sent via DMA transmission. Direct memory access (DMA) can be understood as copying data from one address space to another address space, providing high-speed data transmission between peripherals and memory or between memory and memory.

[0081] In this embodiment, the Shmem shared memory mode or the DMA transmission mode can realize zero-copy cross-process read and write operations in the frame buffer area. By adopting the Shmem shared memory mode or the DMA transmission mode to transmit the video stream, the efficiency and accuracy of the video stream transmission can be improved.

[0082] In some implementations, the second panoramic image process further includes a fourth rendering function, which, when the current state is an overlay state, performs panoramic image rendering through the first panoramic image process and the second panoramic image process, including:

[0083] When the current state is the overlay state, the second image data is rendered off-screen by the first rendering function to obtain a second panoramic image video stream;

[0084] Sending the second panoramic image video stream to the second panoramic image process through the first panoramic image process;

[0085] Performing screen rendering on the second panoramic image video stream through the panoramic image display interface function of the second rendering function and the second display window function to obtain a third panoramic image video stream;

[0086] Rendering the target two-dimensional material in the material library through the fourth rendering function to obtain a three-dimensional model corresponding to the target two-dimensional material;

[0087] The third panoramic image video stream and the three-dimensional model are superimposed through the second panoramic image process to obtain a second target panoramic image.

[0088] Specifically, the superimposed state indicates that the rendering function and the display window function of the second panoramic image process are initialized. When the current state of the second panoramic image process is the superimposed state, the first panoramic image process and the second panoramic image process can perform rendering together.

[0089] When the current state of the second panoramic image process is the superposition state, the first panoramic image process has been initialized, that is, the first display window function and the first rendering function of the first panoramic image process have been initialized, so the second image data can be first rendered off-screen through the first rendering function to obtain the second panoramic image video stream. That is, the second image data is preliminarily rendered through the first rendering function of the first panoramic image process. Then the second panoramic image video stream is sent to the second panoramic image process through the first panoramic image process, and further rendered through the second rendering function of the second panoramic image process. Specifically, the second panoramic image video stream can be screen rendered through the panoramic image display interface function of the second rendering function and the second display window function to obtain a third panoramic image video stream; the target two-dimensional material in the material library is rendered through the fourth rendering function to obtain a three-dimensional model corresponding to the target two-dimensional material. The fourth rendering function can be understood as a function for rendering a three-dimensional model. Finally, the third panoramic image video stream and the three-dimensional model are superimposed through the second panoramic image process to obtain a second target panoramic image.

[0090] In this embodiment, when the current state of the second panoramic image process is the overlay state, the first panoramic image process and the second panoramic image process jointly render the panoramic image, so as to achieve functional complementarity between the two processes and improve resource utilization.

[0091] In some implementations, the state of the second panoramic image process further includes an initial state and a destroyed state, the initial state indicating that both the rendering function and the display window function of the second panoramic image process are not initialized, the destroyed state indicating that the second panoramic image process is destroyed, and the rendering method further includes:

[0092] When the current state is the initial state, the third image data is screen rendered through the panoramic image display interface function of the first display window function and the first rendering function to obtain a third target panoramic image;

[0093] When the current state is the destroyed state, the fourth image data is screen rendered through the panoramic image display interface function of the first display window function and the first rendering function to obtain a fourth target panoramic image.

[0094] Specifically, the initial state can be used to indicate that the second panoramic image process has not been initialized. The destroyed state can be used to indicate that the second panoramic image process has been destroyed. Destruction can be understood as a cleanup operation being performed, such as releasing memory, closing files, etc.

[0095] In one example, when the current state of the second panoramic image process is the initial state, it means that the rendering function and the display window function of the second panoramic image process have not been initialized, that is, it does not have the display window function and the rendering function. Therefore, in the initial state, it can only be rendered by the first panoramic image process alone. The third target panoramic image can be obtained by screen rendering the third image data through the first rendering function of the first panoramic image process.

[0096] In one example, when the current state of the second panoramic image process is the destroyed state, it means that the second panoramic image process has been destroyed and destructed, and therefore, it does not have the display window function and the rendering function, and can only be rendered by the first panoramic image process alone. The fourth target panoramic image can be obtained by screen rendering the fourth image data through the first rendering function of the first panoramic image process.

[0097] In this embodiment, when the current state of the second panoramic image process is the initial state or the destroyed state, the panoramic image rendering is performed only through the first panoramic image process, which can improve the efficiency of the panoramic image rendering.

[0098] In some implementations, before acquiring the current state of the second panoramic image process through the first panoramic image process, the rendering method further includes:

[0099] An inter-process communication service is created, where the inter-process communication service is used for information exchange between the first panoramic image process and the second panoramic image process.

[0100] Specifically, the cross-process communication service can be understood as a module for communication between user processes. The cross-process communication service may include but is not limited to the binder service, etc. In one example, when the first panoramic image process is initialized, a binder service is created; the second panoramic image process can send the current state to the first panoramic image process through the binder service.

[0101] In this embodiment, by creating a cross-process communication service, information transmission between processes can be achieved, thereby improving the correlation between two processes.

[0102] See also Figure 2 , is a flowchart of a method for rendering a panoramic image provided by a specific embodiment of the present application. Figure 2 As shown, the rendering method may include:

[0103] Step S201: The vehicle is ignited, the vehicle system is started, and the Native layer AVM process 1 is initialized first, and then the APK layer AVM process 2 is initialized.

[0104] Step S202: After the Native layer AVM process 1 is initialized, a binder service is created to monitor the status signal of the APK layer AVM process 2, which is updated from the initial state, display state, overlay state, and destruction state as the APK layer AVM process 2 is updated. When the user clicks to trigger the AVM, the Native layer AVM process 1 determines the current state, implements different conditional logic, and then completes operations such as functional interaction, switching, and complementation.

[0105] Step S203: When the initial state or the destruction state is met, the Native layer AVM process 1 renders and outputs the AVM video stream to the vehicle screen.

[0106] Step S204: When the display status is met, the Native layer AVM process 1 switches the display mode from screen rendering to off-screen rendering, and sends the AVM video stream to the APK layer AVM process 2 through Shmem shared memory or DMA. The APK layer AVM process 2 renders the AVM video stream and outputs it to the vehicle screen.

[0107] Step S205: When the overlay state is met, the APK layer AVM process 2 renders the AVM function on the AVM video stream, and outputs it to the vehicle screen after overlay synthesis.

[0108] In this specific embodiment, the specific implementation can refer to the above description and will not be repeated here.

[0109] See also Figure 3 , is a schematic diagram of the structure of a panoramic image rendering device provided by an embodiment of the present application. A second aspect of an embodiment of the present application provides a panoramic image rendering device 30, the rendering device 30 is applied to a vehicle, the vehicle includes a bottom layer and an application layer, the bottom layer is provided with a first panoramic image process, the application layer is provided with a second panoramic image process; the first panoramic image process includes a first rendering function and a first display window function, the second panoramic image process includes a second rendering function and a second display window function, the first display window function and the second display window function both include a function of providing a panoramic image display interface; the rendering device 30 includes:

[0110] An initialization module 31 is used to initialize a first panoramic image process and a second panoramic image process;

[0111] The acquisition module 32 acquires the current state of the second panoramic image process through the first panoramic image process when the first panoramic image process is initialized; wherein the state of the second panoramic image process includes at least a display state and an overlay state; the display state indicates that the display window function of the second panoramic image process is initialized, but the rendering function is not initialized; the overlay state indicates that both the rendering function and the display window function of the second panoramic image process are initialized;

[0112] The rendering module 33 renders the panoramic image through the first panoramic image process and the second panoramic image process when the current state is the display state or the overlay state.

[0113] The panoramic image rendering device 30 provided in the second aspect of the embodiment of the present application can implement each process implemented in the above-mentioned method embodiment and achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0114] See also Figure 4 , is a schematic diagram of components for multi-process joint rendering of a panoramic image at the Native layer and the APK layer provided in a specific embodiment of the present application. Figure 4 As shown, the panoramic image rendering device includes a Native layer and an APK layer. The Native layer contains AVM1 (i.e., the first panoramic image process in this application), and the APK layer contains AVM2 (i.e., the second panoramic image process in this application). AVM1 and AVM2 only have rendering functions and display window functions. AVM2 of the APK layer sends status information to AVM1 of the Native layer through the Binder service. Video streams can be transmitted between AVM1 of the Native layer and AVM2 of the APK layer through Shmem shared memory or DMA transmission. Finally, the video stream is output to the car screen for display of the panoramic image.

[0115] The specific implementation can be found in the above description and will not be repeated here.

[0116] See also Figure 5 , is a structural diagram of a vehicle provided in an embodiment of the present application. An embodiment of the present application provides a vehicle 5000, including a processor 5100 and a memory 5200. The memory 5200 stores machine executable instructions that can be executed by the processor 5100. The processor 5100 can execute the machine executable instructions to implement the above-mentioned panoramic image rendering method.

[0117] In some implementations, the embodiments of the present application further provide a machine-readable storage medium, on which instructions are stored, and when the instructions are executed by a processor, the processor implements the above-mentioned panoramic image rendering method.

[0118] In some implementations, the embodiments of the present application further provide a computer program product, including a computer program, which, when executed by a processor, implements the panoramic image rendering method according to the above implementation.

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0123] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0125] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

[0126] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A method for rendering a panoramic image, characterized in that: Applied to a vehicle, the vehicle includes a bottom layer and an application layer, the bottom layer is provided with a first panoramic image process, the application layer is provided with a second panoramic image process; the first panoramic image process includes a first rendering function and a first display window function, the second panoramic image process includes a second rendering function and a second display window function, the first display window function and the second display window function both include a panoramic image display interface function; the rendering method includes: Initializing the first panoramic image process and the second panoramic image process; When the first panoramic image process is initialized, the current state of the second panoramic image process is obtained through the first panoramic image process; wherein the state of the second panoramic image process includes at least a display state and an overlay state; the display state indicates that the display window function of the second panoramic image process is initialized, but the rendering function is not initialized; the overlay state indicates that both the rendering function and the display window function of the second panoramic image process are initialized; When the current state is the display state or the overlay state, the panoramic image is rendered through the first panoramic image process and the second panoramic image process.

2. The rendering method according to claim 1, characterized in that: The second display window function includes a third rendering function, and when the current state is the display state, the panoramic image is rendered through the first panoramic image process and the second panoramic image process, including: When the current state is the display state, performing off-screen rendering on the first image data by using the first rendering function to obtain a first panoramic image video stream; Sending the first panoramic image video stream to the second panoramic image process through the first panoramic image process; The first panoramic image video stream is screen rendered through the panoramic image display interface function of the second display window function and the third rendering function to obtain a first target panoramic image.

3. The rendering method according to claim 2, characterized in that: The sending the first panoramic image video stream to the second panoramic image process through the first panoramic image process includes: Based on the Shmem shared memory method, the first panoramic image video stream is sent to the second panoramic image process through the first panoramic image process; or Based on the DMA transmission method, the first panoramic image video stream is sent to the second panoramic image process through the first panoramic image process.

4. The rendering method according to claim 1, characterized in that: The second panoramic image process further includes a fourth rendering function, and when the current state is the superimposed state, the panoramic image is rendered through the first panoramic image process and the second panoramic image process, including: When the current state is the overlay state, performing off-screen rendering on the second image data by using the first rendering function to obtain a second panoramic image video stream; Sending the second panoramic image video stream to the second panoramic image process through the first panoramic image process; Performing screen rendering on the second panoramic image video stream through the panoramic image display interface function of the second rendering function and the second display window function to obtain a third panoramic image video stream; Rendering the target two-dimensional material in the material library by means of the fourth rendering function to obtain a three-dimensional model corresponding to the target two-dimensional material; The third panoramic image video stream and the three-dimensional model are superimposed through the second panoramic image process to obtain a second target panoramic image.

5. The rendering method according to claim 1, characterized in that: The state of the second panoramic image process further includes an initial state and a destroyed state, wherein the initial state indicates that the rendering function and the display window function of the second panoramic image process are not initialized, and the destroyed state indicates that the second panoramic image process is destroyed. The rendering method further includes: When the current state is the initial state, performing screen rendering on the third image data through the panoramic image display interface function of the first display window function and the first rendering function to obtain a third target panoramic image; When the current state is the destroyed state, the fourth image data is screen rendered through the panoramic image display interface function of the first display window function and the first rendering function to obtain a fourth target panoramic image.

6. The rendering method according to claim 1, characterized in that: Before acquiring the current state of the second panoramic image process through the first panoramic image process, the rendering method further includes: An inter-process communication service is created, where the inter-process communication service is used for information exchange between the first panoramic image process and the second panoramic image process.

7. A panoramic image rendering device, characterized in that: Applied to a vehicle, the vehicle includes a bottom layer and an application layer, the bottom layer is provided with a first panoramic image process, the application layer is provided with a second panoramic image process; the first panoramic image process includes a first rendering function and a first display window function, the second panoramic image process includes a second rendering function and a second display window function, the first display window function and the second display window function both include a function of providing a panoramic image display interface; the rendering device includes: An initialization module, initializing the first panoramic image process and the second panoramic image process; an acquisition module, wherein, when the first panoramic image process is initialized, the current state of the second panoramic image process is acquired through the first panoramic image process; wherein the state of the second panoramic image process includes at least a display state and an overlay state; the display state indicates that the display window function of the second panoramic image process is initialized, but the rendering function is not initialized; the overlay state indicates that both the rendering function and the display window function of the second panoramic image process are initialized; The rendering module renders the panoramic image through the first panoramic image process and the second panoramic image process when the current state is the display state or the overlay state.

8. A vehicle, characterized in that: include: a memory configured to store instructions; as well as A processor is configured to call the instruction from the memory and implement the panoramic image rendering method according to any one of claims 1 to 6 when executing the instruction.

9. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions, and the instructions are used to enable a machine to execute the panoramic image rendering method according to any one of claims 1 to 6.

10. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of a vehicle, the vehicle executes the panoramic image rendering method as described in any one of claims 1 to 6.

Images