Multi-window display method and system based on 3D rendering engine

Abstract

The invention provides a multi-window display method based on a 3D rendering engine, and the method comprises the steps: starting a 3D rendering process, removing SurfaceView in the 3D engine, and carrying out the rendering of 3D contents; starting a display process according to the 3D rendering content, and binding the display process with a 3D rendering process; the display process newly builds a TextureView, and the Surface and the Touch Event are transmitted to the rendering process through the AIDL to be drawn; and after the rendering process finishes drawing, displaying the 3D drawing effect through the display process. According to the method and the device, the SurfaceView is replaced by the TextureView, so that the problems that a double-cache mechanism exists when the SurfaceView is adopted and double images exist probabilistically when the SurfaceView is quickly switched among the pages are solved, and the problems that the SurfaceView can only be provided with the highest level or the lowest level and the view is shielded for the display application are also solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of 3D rendering, in particular to a multi-window display method and system based on a 3D rendering engine. BACKGROUND

[0002] In traditional computer graphics display, a single window display mode is usually adopted, that is, a 3D scene or view is presented in one display window. However, with the continuous development of computer graphics technology and the increasing complexity of application scenarios, the single window display mode has been unable to meet some specific needs. For example, in game development, developers may need to view game scenes from different perspectives at the same time in order to perform scene arrangement, character action debugging, etc.; in virtual reality applications, in order to provide a more comprehensive immersive experience, multiple associated virtual scenes may need to be displayed simultaneously; in architectural design and industrial simulation fields, designers and engineers need to observe building models or industrial product models from different angles to evaluate their design effects and performance. At present, although there are some multi-window display technologies, there are still problems such as different synchronization display of multiple windows, abnormal view hierarchy when integrating 3D functions into existing Android projects, insufficient system resources for single-process rendering and display, etc. Therefore, there is an urgent need for a multi-window display method and system based on a 3D rendering engine to solve the problems in the prior art. SUMMARY

[0003] The present application aims to solve at least one of the technical problems in the prior art and proposes a multi-window display method and system based on a 3D rendering engine.

[0004] In a first aspect, an embodiment of the present application provides a multi-window display method based on a 3D rendering engine, comprising:

[0005] starting a 3D rendering process, removing a SurfaceView in the 3D engine, and rendering 3D content;

[0006] starting a display process according to the 3D rendering content, and binding the display process with the 3D rendering process;

[0007] newly creating a TextureView in the display process, and passing a Surface and a TouchEvent to the rendering process for drawing through AIDL;

[0008] after the rendering process completes drawing, displaying the 3D drawing effect through the display process.

[0009] Further, the 3D rendering process is an independent application process in a car machine, and the step of starting the 3D rendering process comprises:

[0010] Receiving a start instruction, creating an independent process for the 3D rendering process, initializing the independent process; wherein the process initialization includes: allocating an independent memory space for the 3D rendering process, physically isolating the 3D rendering process from other processes; initializing a log module, registering an exception capture mechanism, the log module is used to record the start process and rendering errors; the exception capture mechanism is used to automatically restart the process when the process crashes;

[0011] Binding the 3D engine, according to the GPU model of the car machine, the engine interaction module loads the corresponding 3D engine dynamic library from the car machine storage partition, and creates a rendering context.

[0012] Further, load the SurfaceView when initializing the 3D engine, remove the SurfaceView in the 3D engine through the removeView method of the original layout, the specific steps include: obtaining the parent layout of the SurfaceView, using the getParent method of the SurfaceView to obtain its parent layout object, and casting it to the ViewGroup type; After obtaining the parent layout, call the removeView method of the parent layout and pass the SurfaceView instance to be removed as a parameter.

[0013] Further, the display process and the rendering process are bound by using the AIDL communication mechanism, the specific steps include:

[0014] Define the AIDL interface, declare the communication method between the rendering process and the display process, including passing the Surface and controlling the rendering frame;

[0015] Implement the rendering process service, create an AIDL Service in the rendering process, and handle the request of the display process;

[0016] Display process binding service, bind the AIDL Service of the rendering process in the display process, get the rendering result and display.

[0017] Further, the display process creates a new TextureView, the specific steps include:

[0018] Add TextureView in the layout file or dynamically create TextureView through code;

[0019] Set SurfaceTextureListener to listen to the creation and update of SurfaceTexture;

[0020] Pass the SurfaceTexture to the rendering process, and the rendering process writes frame data through the SurfaceTexture;

[0021] Trigger view redraw in onSurfaceTextureUpdated, update the processing frame.

[0022] Further, the display process passes the Surface and the TouchEvent to the rendering process through AIDL, specifically including: passing the Surface of the newly created TextureView and the user directly to the rendering engine, the rendering engine draws the picture on the Surface, and the TouchEvent obtained can be passed to the engine, the engine is used for responding to the click or drag action of the user.

[0023] Further, after the display process displays the 3D drawing effect, the user's click or drag operation is obtained in the form of a TouchEvent and is passed to the rendering process, and the 3D engine responds to the Surface and the TouchEvent passed by the display process to achieve the effect of user operation of 3D.

[0024] In a second aspect, the embodiment of the present application further discloses a multi-window display system based on a 3D rendering engine, comprising: a 3D content rendering module, a display process and 3D rendering process binding module, a rendering process drawing module and a 3D drawing effect display module; wherein:

[0025] The 3D content rendering module is used for starting the 3D rendering process, removing the SurfaceView in the 3D engine, and rendering the 3D content.

[0026] The display process and 3D rendering process binding module is used for starting the display process according to the 3D rendering content, and binding the display process and the 3D rendering process.

[0027] The rendering process drawing module is used for the display process to newly create a TextureView, pass the Surface and the TouchEvent to the rendering process through AIDL for drawing.

[0028] The 3D drawing effect display module is used for displaying the 3D drawing effect through the display process after the rendering process drawing is completed.

[0029] In a third aspect, the embodiment of the present application further discloses an electronic device, comprising:

[0030] One or more processors;

[0031] A memory for storing one or more programs;

[0032] When the one or more programs are executed by the one or more processors, the one or more processors implement the display method.

[0033] In a fourth aspect, the embodiments of the present application further disclose a computer readable medium, and a computer program is stored on the computer readable medium, and the computer program is executed by a processor to implement the steps in the display method.

[0034] The present application provides a multi-window display method based on a 3D rendering engine, which comprises the following steps: starting a 3D rendering process, removing a SurfaceView in the 3D engine, and rendering 3D content; starting a display process according to the 3D rendering content, and binding the display process with the 3D rendering process; newly creating a TextureView in the display process, and transferring a Surface and a TouchEvent to the rendering process for drawing through AIDL; and displaying the 3D drawing effect through the display process after the rendering process is completed. The present application replaces the SurfaceView with the TextureView, solves the problem of ghosting in the rapid switching between pages due to the double buffering mechanism of the SurfaceView, and solves the problem of view blocking in the display application due to the highest or lowest level of the SurfaceView. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 A flowchart of a multi-window display method based on a 3D rendering engine provided by the embodiments of the present application;

[0036] Figure 2 A running logic diagram of a multi-window display method based on a 3D rendering engine provided by the embodiments of the present application;

[0037] Figure 3 A structure block diagram of a multi-window display system based on a 3D rendering engine provided by the embodiments of the present application;

[0038] Figure 4 A structure block diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0039] In order to make the skilled in the art better understand the technical solutions of the present application, the following describes exemplary embodiments of the present application in conjunction with the drawings, which include various details of the embodiments of the present application to help understanding, and should be considered only as exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, in order to be clear and concise, the description in the following description omits the description of known functions and structures.

[0040] In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0041] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0042] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. "Coupled" or "connected" or similar terms are not restricted to physical or mechanical connections or associations, but can also include electrical connections, whether direct or indirect.

[0043] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0044] In the technical solutions of the present application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs. The use of user data in the technical solutions complies with relevant national laws and regulations (for example, "Information Security Technology Personal Information Security Specification" and the like). For example, appropriate measures are taken for personal information access control; the display of personal information is limited by regulations; the use purpose of personal information does not exceed the direct or reasonably related range; the use of personal information eliminates explicit identity pointing and avoids precise positioning to a specific individual.

[0045] In the related art, a multi-screen linkage method is disclosed. When a user slides on a car machine system screen, the sliding event is converted into a change parameter of a camera view angle, the rendering parameters are updated in a rendering engine to obtain image data, and then the image data is displayed on another screen to realize view changes under different view angles, multi-screen image following changes and synchronization, thereby realizing multi-screen linkage.

[0046] However, although this scheme realizes multi-window display synchronization on the car machine, other windows mainly display images, and image data needs to be regenerated and displayed every time, which consumes a large amount of system resources and is prone to time delay in the display process, resulting in different synchronization of user visual experience.

[0047] In the related art, a non-integrated single-engine background rendering method for multiple applications is also disclosed, including the following steps: a single engine provides creation of an off-screen rendering buffer and off-screen rendering, so that the single engine is normally operated under an external on-screen off-screen rendering buffer to produce a rendering picture; a multi-view rendering output is provided, and input click processing is isolated; a general view component is abstracted; a cross-process communication mechanism and a general protocol are formulated; a rendering service library is provided; and one-key packaging and project generation are performed.

[0048] However, although this scheme provides multiple views for different applications for integrated display, the view used by the engine is based on SurfaceView, which may cause ghosting problems caused by the original double-buffering mechanism, and SurfaceView cannot have a hierarchical relationship like a normal View, which may cause hierarchical problems on the view, resulting in abnormal display of the original project view.

[0049] To solve at least one of the technical problems existing in the related art, the present application provides a multi-window display method and system based on a 3D rendering engine.

[0050] The present embodiment discloses a multi-window display method based on a 3D rendering engine, which includes the following steps: Figure 1 and 2 , including:

[0051] S100. Start a 3D rendering process, remove SurfaceView in the 3D engine, and render 3D content; wherein the 3D rendering process is an independent application process in the car machine, and directly interacts with the 3D engine, starts the rendering process, and prepares for subsequent display and interaction logic;

[0052] In the present embodiment, the 3D rendering process is an independent application process in the car machine, and the step of starting the 3D rendering process includes:

[0053] Receive a start instruction, create an independent process for the 3D rendering process, and initialize the independent process; wherein the process initialization includes: allocating an independent memory space for the 3D rendering process to physically isolate the 3D rendering process from other processes; initializing a log module and registering an exception capture mechanism, the log module is used to record the start process and rendering errors; and the exception capture mechanism is used to automatically restart the process when the process crashes;

[0054] Bind the 3D engine according to the GPU model of the car machine, and the engine interaction module loads the corresponding 3D engine dynamic library from the storage partition of the car machine, and creates a rendering context. The specific method of creating a rendering context includes: passing the Surface to the rendering process through AIDL by the display process. The rendering process creates an independent EGLContext in its own GLThread, and binds to the Surface.

[0055] The method disclosed in the embodiment isolates the rendering task to an independent process, avoids blocking the main thread (UI thread), and supports high-performance rendering and crash isolation.

[0056] In the embodiment, a SurfaceView is used in the 3D engine to load a 3D view, and the SurfaceView has a double-buffering mechanism. Rapid switching between pages may probabilistically cause ghosting problems, and the SurfaceView can only be set to the highest or lowest level, which may cause view blocking problems for display applications. Therefore, the SurfaceView loaded at the initialization of the 3D engine needs to be removed from the original layout by using the removeView method.

[0057] The specific steps of removing the SurfaceView in the 3D engine include: loading the SurfaceView at the initialization of the 3D engine, and removing the SurfaceView in the 3D engine by using the removeView method of the original layout. The specific steps include: obtaining the parent layout of the SurfaceView, using the getParent method of the SurfaceView to obtain the parent layout object, and casting it to the ViewGroup type; after obtaining the parent layout, calling the removeView method of the parent layout and passing the SurfaceView instance to be removed as a parameter. Through the above steps, the SurfaceView can be safely removed from the layout, and common resource leakage and crash problems can be avoided.

[0058] S200. Start a display process according to the 3D rendering content, and bind the display process with the 3D rendering process; in the embodiment, the AIDL (Android Interface Definition Language) communication mechanism is used to bind the display process and the rendering process, wherein the AIDL is an efficient mechanism for realizing cross-process communication (IPC, Inter-Process Communication) in an Android system. It allows components (such as Activity, Service, etc.) in different processes to exchange data and remotely call methods through a clear interface, and is especially suitable for scenarios (such as system services, plug-in architecture, etc.) that require high performance or security isolation.

[0059] In S100 of the embodiment, only the rendering of the 3D content is performed, and different processes can be started to display the 3D effect according to different rendering contents in S200, which are usually other applications of the vehicle machine, such as air conditioning, vehicle control, etc. Specifically, the AIDL communication mechanism is used to bind the display process and the rendering process, and the specific steps include:

[0060] Defining an AIDL interface, declaring the communication method between the rendering process and the display process, including passing Surface and controlling the rendering frame;

[0061] Implementing a rendering process service, creating an AIDL Service in the rendering process, and processing the request of the display process;

[0062] Binding the service in the display process, binding the AIDL Service of the rendering process in the display process, obtaining the rendering result and displaying it.

[0063] Through the above method, the display process and the rendering process can be efficiently bound.

[0064] S300. The display process newly creates a TextureView, and passes Surface and TouchEvent to the rendering process through AIDL for drawing; in Android, when the display process displays the rendering result of the rendering process (or other process) through TextureView, it is necessary to correctly initialize TextureView and set its SurfaceTexture callback to receive frame data.

[0065] In the embodiment, the display process newly creates a TextureView, and the specific steps include:

[0066] Adding TextureView in the layout file or dynamically creating TextureView through code;

[0067] SurfaceTextureListener is set to monitor the creation and update of SurfaceTexture;

[0068] SurfaceTexture is passed to the rendering process, and the rendering process writes frame data through SurfaceTexture;

[0069] In onSurfaceTextureUpdated, view redrawing is triggered to update the processed frame. Through the above steps, the display process can efficiently display the rendering result of the rendering process through TextureView.

[0070] In the embodiment, Surface and TouchEvent are passed to the rendering process (such as Service or independent thread) for drawing through TextureView. Two problems need to be solved, i.e., cross-process transmission of Surface (used for rendering target) and cross-process transmission of TouchEvent (used for interactive control). In the embodiment, the Surface of the created TextureView and the user are directly passed to the rendering engine. The rendering engine can draw pictures on the Surface, and can pass the obtained TouchEvent to the engine, so that the engine can respond to some click or drag actions of the user, and realize visual synchronization of 3D effect.

[0071] In the embodiment, the display process passes Surface and TouchEvent to the rendering process through AIDL. Specifically, the Surface of the newly created TextureView and the user are directly passed to the rendering engine. The rendering engine draws pictures on the Surface, and can pass the obtained TouchEvent to the engine. The engine is used to respond to click or drag actions of the user.

[0072] S400. After the rendering process is completed, the 3D drawing effect is displayed through the display process. Specifically, the Surface of the display process is passed to the rendering process, so that the rendering effect of the 3D engine can be displayed, and the separation of rendering resources and display resources is realized, so as to achieve the purpose of resource optimization.

[0073] In some preferred embodiments, a multi-window display method based on a 3D rendering engine further comprises:

[0074] S500. After the display process displays the 3D rendering effect, the user clicks or drags the operation in the form of TouchEvent to the rendering process, and the 3D engine achieves the effect of user operation 3D by responding to the Surface and TouchEvent transmitted by the display process.

[0075] The embodiment provides a multi-window display method based on a 3D rendering engine, removes a SurfaceView in a 3D engine by starting a 3D rendering process, and renders 3D content; starts a display process according to the 3D rendering content, binds the display process with the 3D rendering process; the display process newly creates a TextureView, and transmits Surface and TouchEvent to the rendering process for drawing through AIDL; and the 3D rendering effect is displayed by the display process after the rendering process completes drawing. The TextureView is used to replace the SurfaceView, so that the double-buffering mechanism caused by the SurfaceView is solved, the ghosting problem caused by fast switching between pages is solved, and the problem of view blocking caused by the SurfaceView being only able to be set to the highest or lowest level is solved.

[0076] Based on the same inventive concept, the embodiment of the present application also provides a multi-window display system based on a 3D rendering engine, which comprises a 3D content rendering module, a display process and 3D rendering process binding module, a rendering process drawing module and a 3D rendering effect display module. Figure 3

[0077] The 3D content rendering module is used to start a 3D rendering process, remove a SurfaceView in a 3D engine, and render 3D content.

[0078] The display process and 3D rendering process binding module is used to start a display process according to 3D rendering content, and bind the display process with the 3D rendering process.

[0079] The rendering process drawing module is used for the display process to newly create a TextureView, and transmit Surface and TouchEvent to the rendering process for drawing through AIDL.

[0080] The 3D rendering effect display module is used to display the 3D rendering effect by the display process after the rendering process completes drawing.

[0081] The 3D content rendering module, the display process and 3D rendering process binding module, the rendering process drawing module and the 3D rendering effect display module have been described in detail in the above multi-window display method based on a 3D rendering engine, and will not be described again in the embodiment. ​

[0082] Based on the same inventive concept, the embodiment of the present application also provides an electronic device. Figure 3 The structure block diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the electronic device provided by the embodiment of the present application comprises one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the display method in any of the above embodiments. The one or more I / O interfaces 103 are connected between the processor and the memory, and are configured to realize the information interaction between the processor and the memory. Figure 3 The electronic device provided by the embodiment of the present application comprises one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the display method in any of the above embodiments. The one or more I / O interfaces 103 are connected between the processor and the memory, and are configured to realize the information interaction between the processor and the memory.

[0083] The processor 101 is a device with data processing capability, including but not limited to a central processing unit (CPU) and the like; the memory 102 is a device with data storage capability, including but not limited to a random access memory (RAM, more specifically SDRAM, DDR, etc.), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), and a flash memory (FLASH); and the I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102, and can realize the information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus) and the like.

[0084] In some embodiments, the processor 101, the memory 102, and the I / O interface 103 are connected with each other through a bus 104, and further connected with other components of the computing device.

[0085] In some embodiments, the one or more processors 101 comprise a field programmable gate array.

[0086] The embodiment of the present application also provides a computer readable medium. The computer readable medium stores a computer program, and when the program is executed by a processor, the steps in the display method in any of the above embodiments are implemented. The computer readable storage medium can be a volatile or non-volatile computer readable storage medium.

[0087] The embodiment of the present application also provides a computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying the computer readable code. When the computer readable code is run in the processor of the electronic device, the processor in the electronic device executes the display method.

[0088] Those of ordinary skill in the art will realize and understand that all or some of the steps in the methods disclosed above and the functional modules / units in the systems and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementation, the division between the functional modules / units referred to in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on computer readable storage media, which can include computer storage media (or non-transitory media) and communication media (or transitory media).

[0089] As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read only memory (CD-ROM), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Further, it is well known to those of ordinary skill in the art that communication media typically embodies computer readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. As used herein, the term "modulated data signal" means a signal that has one or more of its characteristics changed or set in a manner so as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as wireless networks, cellular telephone networks, code division multiple access (CDMA) networks, and other terrestrial and satellite radio frequency communication networks or frequency (RF) media. The computer software can be downloaded to the respective computing / processing device from a computer software distribution medium or to an external computer or external storage device from a network, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer software from the network and forwards the computer software for storage in a computer readable storage medium within the respective computing / processing device.

[0090] The computer software described herein can be downloaded to the respective computing / processing device from a computer software distribution medium or to an external computer or external storage device from a network, for example, the Internet, a local area network, a wide area network, and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer software from the network and forwards the computer software for storage in a computer readable storage medium within the respective computing / processing device.

[0091] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0092] The computer program product described herein can be embodied in a specific manner by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), and the like.

[0093] The computer program product described herein can be embodied in a specific manner by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), and the like.

[0094] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage viruses or other code or instructions implementing a functionally equivalent to that of the software manual reproduction process. The instructions might also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0095] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0096] The flow diagrams and block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions ("instructions"). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0097] Example embodiments have been disclosed herein and, although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics or elements described with respect to a particular embodiment can be used, alone or in combination, with other embodiments unless specifically recited otherwise in the detailed description. Accordingly, it will be understood that various changes in form and details can be made without departing from the scope of the present application as set forth in the appended claims.

Claims

1. A multi-window display method based on a 3D rendering engine, characterized by, The application comprises the following steps: starting a 3D rendering process, removing a SurfaceView in a 3D engine, and rendering 3D content; starting a display process according to the 3D rendering content, and binding the display process to the 3D rendering process; creating a TextureView in the display process, and transmitting a Surface and a TouchEvent to the rendering process for drawing through AIDL; displaying the 3D drawing effect through the display process after the rendering process is completed.

2. The method of claim 1, wherein, The 3D rendering process is an independent application process in the car machine, and the starting steps of the 3D rendering process comprise the following steps: receiving a starting instruction, creating an independent process for the 3D rendering process, and initializing the independent process; wherein the process initialization comprises the following steps: allocating an independent memory space for the 3D rendering process, physically isolating the 3D rendering process from other processes; initializing a log module and registering an exception capturing mechanism, the log module is used for recording a starting process and a rendering error, and the exception capturing mechanism is used for automatically restarting the process when the process crashes; binding a 3D engine, loading a corresponding 3D engine dynamic library from a car machine storage partition according to a car machine GPU model, and creating a rendering context.

3. The method of claim 1, wherein, The SurfaceView is loaded when the 3D engine is initialized, and the SurfaceView in the 3D engine is removed through a removeView method of an original layout, and the specific steps comprise the following steps: obtaining a parent layout of the SurfaceView, using a getParent method of the SurfaceView to obtain a parent layout object, and converting the parent layout object into a ViewGroup type; after the parent layout is obtained, a removeView method of the parent layout is called, and a SurfaceView instance to be removed is transmitted as a parameter.

4. The method of claim 1, wherein, The display process and the rendering process are bound through an AIDL communication mechanism, and the specific steps comprise the following steps: defining an AIDL interface, declaring a communication method between the rendering process and the display process, and transmitting a Surface and controlling a rendering frame; implementing a rendering process service, creating an AIDL Service in the rendering process, and processing a request of the display process; binding the service in the display process, binding an AIDL Service of the rendering process in the display process, obtaining a rendering result, and displaying the rendering result.

5. The method of claim 1, wherein, The TextureView is newly created in the display process, and the specific steps comprise the following steps: adding the TextureView in a layout file or dynamically creating the TextureView through code; setting a SurfaceTextureListener to listen to the creation and update of the SurfaceTexture; transmitting the SurfaceTexture to the rendering process, and writing frame data by the rendering process through the SurfaceTexture; triggering view redrawing in the onSurfaceTextureUpdated, and updating the processed frame.

6. The method of claim 1, wherein, The display process transmits Surface and TouchEvent to the rendering process through AIDL, specifically including: transmitting the Surface of the newly created TextureView and user directly to the rendering engine, which draws pictures on the Surface; and transmitting the obtained TouchEvent to the engine, which responds to the user's click or drag action.

7. The method of claim 1, wherein, After the display process displays the 3D drawing effect, it obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D.

8. A multi-window display system based on a 3D rendering engine, characterized in that, The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D.

9. An electronic device, comprising: The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D.

10. A computer readable medium having stored thereon a computer program, characterized in that The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process displays the 3D drawing effect, and obtains the user's click or drag operation in the form of TouchEvent and transmits it to the rendering process, and the 3D engine responds to the Surface and TouchEvent transmitted by the display process to achieve the effect of user operation of 3D. The display process

Images