Transparent chassis architecture display method and system based on openGL

Abstract

The invention discloses a transparent chassis architecture display method and system based on openGL, and relates to the technical field of vehicle panoramic look-around, and the method comprises the steps: obtaining a camera image as input; splicing the camera images into a panoramic view; coordinates of the chassis area are calculated through the real-time signals; intercepting a chassis area image from the previous frame of image; synthesizing the chassis area and the current frame panoramic image; and rendering the synthesized image to a display screen. According to the openGL-based transparent chassis architecture display method provided by the invention, the algorithm is divided into different small modules, and the modules can be repeatedly used in different projects, so that the expandability is high, and the development period and cost can be reduced; a panoramic looking-around system with a transparent chassis function is developed by using a simple architecture, so that the overall development efficiency is improved; development is carried out based on openGL, the parallel computing capability of the GPU is applied, and the algorithm frame rate and the resource consumption can meet the software development requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle panoramic surround view, and in particular to an OpenGL-based transparent chassis architecture display method and system. Background Art

[0002] The vehicle's panoramic view system is a driving assistance system for automobiles. With the continuous development of science and technology, panoramic view has become one of the indispensable software for vehicles. Among them, stitching the four camera images into a 360-degree panoramic view image has become the basic algorithm, but there is still room for research and improvement on more advanced technologies such as transparent chassis. Summary of the invention

[0003] In view of the above-mentioned problems, the present invention is proposed.

[0004] Therefore, the technical problems solved by the present invention are: first, there are few relevant literatures and studies on the field of transparent chassis in the prior art. The present invention provides an effective technical solution to make up for the lack of systematic summary of transparent chassis technology in the prior art; second, since the system involves driving safety, it has high real-time requirements, and the algorithm frame rate and resource consumption need to meet software development requirements; finally, the system architecture is complex and lacks good scalability, which leads to increased development cycle and cost, and the modules are tightly coupled, which increases the difficulty of development and maintenance and limits the scalability of the system and the efficiency of multi-developer collaboration.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a transparent chassis architecture display method based on OpenGL, comprising:

[0006] Get the camera image as input;

[0007] Stitching camera images into a panoramic ring view;

[0008] Calculate the coordinates of the chassis area through real-time signals;

[0009] Capture the chassis area image from the previous frame image;

[0010] The synthesized image is combined with the chassis area and the current frame panoramic image;

[0011] Render to the display.

[0012] As a preferred solution of the openGL-based transparent chassis architecture display method described in the present invention, the cameras are arranged in four directions: front, rear, left and right of the vehicle.

[0013] As a preferred solution of the transparent chassis architecture display method based on openGL described in the present invention, wherein: the stitching into a panoramic view includes inputting the camera image into an openGL shader;

[0014] A panoramic image without a transparent bottom plate is obtained by shader image stitching;

[0015] The size of the panoramic image is w*h, and there are no pixels in the a*b area;

[0016] Use OpenGL to create a frame buffer named FBO0 with a size of w*h, and save the image to FBO0;

[0017] Use glBindTexture to bind the image texture of FBO0. After the image is processed by the shader, the image is rendered directly to the screen, and FBO0 is retained as the input for the next frame of the image.

[0018] As a preferred solution of the transparent chassis architecture display method based on openGL of the present invention, wherein: the calculation of the coordinates of the chassis area by real-time signals includes using the previous frame of panoramic surround image FBO0 as image end input;

[0019] Taking the timestamp of the vehicle system and the vehicle signal as input parameters;

[0020] The chassis area size is a*b. In the openGL shader, the accurate chassis area coordinates are calculated according to the input parameters of the signal acquisition module, and the corresponding a*b size image is intercepted from FBO0 according to the coordinates. This image is the transparent chassis image corresponding to the current frame image;

[0021] Use openGL to create a frame buffer named FBO1 with a size of w*h, and save the image of the corresponding a*b size to the corresponding a*b area of ​​FBO1. The valid area is a*b, and there are no pixels in the non-a*b area of ​​FBO1.

[0022] As a preferred solution of the transparent chassis architecture display method based on openGL of the present invention, wherein: the capturing of the chassis area image from the previous frame image includes obtaining a camera image as input;

[0023] The camera image is input into the openGL shader, and after the shader image stitching processing, a panoramic image without a transparent bottom plate is obtained. The effective pixel area is all areas except the a*b area in the w*h size, and there is no pixel in the a*b area;

[0024] Use OpenGL to create a frame buffer named FBO2 with a size of w*h, and save the panoramic image without a transparent bottom plate into FBO2.

[0025] As a preferred solution of the transparent chassis architecture display method based on openGL of the present invention, wherein: the synthesis of the chassis area and the current frame panoramic image includes taking the texture images corresponding to FBO1 and FBO2 as input;

[0026] If the vertex coordinates of the shader are in the a*b area of ​​the chassis, the texture is sampled from FBO1; if the vertex coordinates are in the non-chassis a*b area, the texture is sampled from FBO2;

[0027] Save the image after texture sampling to FBO0.

[0028] As a preferred solution of the openGL-based transparent chassis architecture display method described in the present invention, wherein: rendering the synthesized image to the display screen includes rendering the FBO0 image directly to the screen.

[0029] Another object of the present invention is to provide a transparent chassis architecture display system based on OpenGL, which can solve the problem of poor effect of transparent chassis in existing vehicle panoramic view by constructing a remote safety communication system for distribution network operations.

[0030] To solve the above technical problems, the present invention provides the following technical solutions: a transparent chassis architecture display system based on OpenGL, comprising: an image input module, a transparent chassis calculation and interception module, a current frame panoramic image generation module, a fusion module and a final rendering module; the image input module is used to obtain camera images as input; the camera images are spliced ​​into a panoramic surround view; the transparent chassis calculation and interception module is used to calculate the coordinates of the chassis area through real-time signals and intercept the chassis area image; the current frame panoramic image generation module is used to generate a panoramic image of the current frame that does not contain the transparent chassis area; the fusion module is used to synthesize the chassis area and the current frame panoramic image; the final rendering module is used to render the synthesized image onto a display screen.

[0031] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned transparent chassis architecture display method based on OpenGL when executing the computer program.

[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned transparent chassis architecture display method based on OpenGL.

[0033] Beneficial effects of the present invention: the transparent chassis architecture display method based on openGL provided by the present invention divides the algorithm into various small modules, each module is independent and can be reused in different projects, has strong scalability and can reduce development cycle and cost; the panoramic view system with transparent chassis function is developed using a simple architecture with a clear structure and easy to understand, so that developers can get started faster, thereby improving overall development efficiency; based on openGL development and using the parallel computing capabilities of GPU, the algorithm frame rate and resource consumption can meet software development requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0035] Figure 1 An overall flow chart of an OpenGL-based transparent chassis architecture display method provided by an embodiment of the present invention.

[0036] Figure 2 A diagram showing the sizes of various regions of a panoramic surround image of a transparent chassis architecture display method based on OpenGL provided by an embodiment of the present invention.

[0037] Figure 3 An overall structural diagram of an OpenGL-based transparent chassis architecture display system provided by an embodiment of the present invention.

[0038] Figure 4 A diagram of a surround-view transparent chassis architecture based on OpenGL and a transparent chassis architecture display method based on OpenGL provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0041] Example 1

[0042] Reference Figure 1-Figure 2 , as an embodiment of the present invention, provides a transparent chassis architecture display method based on OpenGL, comprising:

[0043] Get the camera image as input;

[0044] Stitching camera images into a panoramic ring view;

[0045] Calculate the coordinates of the chassis area through real-time signals;

[0046] Capture the chassis area image from the previous frame image;

[0047] Synthesize the chassis area and the current frame panoramic image;

[0048] Render the synthesized image to the display.

[0049] The cameras are arranged in four directions: front, rear, left and right of the vehicle.

[0050] The stitching into a panoramic ring view comprises inputting the camera image into an openGL shader;

[0051] A panoramic image without a transparent bottom plate is obtained by shader image stitching;

[0052] like Figure 2 As shown, the size of the panoramic image is w*h, wherein there are no pixels in the a*b area;

[0053] Use OpenGL to create a frame buffer named FBO0 with a size of w*h, and save the image to FBO0;

[0054] Use glBindTexture to bind the image texture of FBO0. After the image is processed by the shader, the image is rendered directly to the screen, and FBO0 is retained as the input for the next frame of the image.

[0055] The calculating the coordinates of the chassis area by the real-time signal includes taking the previous frame of panoramic view image FBO0 as the image end input;

[0056] Taking the timestamp of the vehicle system and the vehicle signal as input parameters;

[0057] The chassis area size is a*b. In the openGL shader, the accurate chassis area coordinates are calculated according to the input parameters of the signal acquisition module, and the corresponding a*b size image is intercepted from FBO0 according to the coordinates. This image is the transparent chassis image corresponding to the current frame image;

[0058] Use openGL to create a frame buffer named FBO1 with a size of w*h, and save the image of the corresponding a*b size to the corresponding a*b area of ​​FBO1. The valid area is a*b, and there are no pixels in the non-a*b area of ​​FBO1.

[0059] The capturing of the chassis area image from the previous frame of image includes obtaining a camera image as input;

[0060] The camera image is input into the openGL shader, and after the shader image stitching processing, a panoramic image without a transparent bottom plate is obtained. The effective pixel area is all areas except the a*b area in the w*h size, and there is no pixel in the a*b area;

[0061] Use OpenGL to create a frame buffer named FBO2 with a size of w*h, and save the panoramic image without a transparent bottom plate into FBO2.

[0062] The synthesizing the chassis area and the current frame panoramic image includes taking the texture images corresponding to FBO1 and FBO2 as input;

[0063] If the vertex coordinates of the shader are in the a*b area of ​​the chassis, the texture is sampled from FBO1; if the vertex coordinates are in the non-chassis a*b area, the texture is sampled from FBO2;

[0064] Save the image after texture sampling to FBO0.

[0065] Rendering the synthesized image onto the display screen includes rendering the FBO0 image directly onto the screen.

[0066] Example 2

[0067] Reference Figure 3 , as an embodiment of the present invention, provides a transparent chassis architecture display system based on OpenGL, comprising:

[0068] Image input module 100, transparent chassis calculation and interception module 200, current frame panorama generation module 300, fusion module 400 and final rendering module 500;

[0069] The image input module 100 is used to obtain camera images as input; stitch the camera images into a panoramic view;

[0070] The transparent chassis calculation and interception module 200 is used to calculate the coordinates of the chassis area through real-time signals and intercept the chassis area image;

[0071] The current frame panoramic image generation module 300 is used to generate a panoramic image of the current frame without a transparent bottom plate area;

[0072] The fusion module 400 is used to synthesize the chassis area and the current frame panoramic image;

[0073] The final rendering module 500 is used to render the synthesized image onto a display screen.

[0074] Example 3

[0075] An embodiment of the present invention is different from the first two embodiments in that:

[0076] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0077] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0078] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0079] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0080] Example 4

[0081] Reference Figure 4 , as an embodiment of the present invention, provides a transparent chassis architecture display method based on OpenGL.

[0082] 1. Initialization module: The present invention is based on a panoramic view system without a permanent memory function. When the software is turned on, the first frame of panoramic image needs to be generated as input for subsequent transparent chassis processing. This module is only executed once when the software is turned on. The steps are as follows:

[0083] 1.1. Image input module: real-time acquisition of images from the front, back, left, and right cameras as input;

[0084] 1.2. Current frame panorama generation module: pass the input of step 2.1 to the openGL shader, and obtain a panoramic image without transparent bottom plate after the shader image stitching processing. The image size is w*h, and there are no pixels in the a*b area; use openGL to create a frame buffer named FBO0 with a size of w*h, and save the image to FBO0;

[0085] 1.3. Final rendering module: Use glBindTexture to bind the image texture of FBO0, render the image directly to the screen after shader image processing, and retain FBO0 as the input of the next frame image.

[0086] 2. Transparent chassis calculation and interception module: The goal of this module is to perform coordinate calculation and image sampling based on the previous frame of panoramic view image, and finally intercept the image that only belongs to the chassis area in the current frame image. The steps are as follows:

[0087] 2.1. Image input module: use the previous frame of panoramic view image FBO0 as the image input;

[0088] 2.2. Real-time signal input module: The timestamp of the vehicle system and signals such as vehicle speed, gear position, and wheel angle are used as input parameters;

[0089] 2.3. Calculation and interception module: The chassis area size is a*b. In the openGL shader, the accurate chassis area coordinates are calculated according to the input parameters of the signal acquisition module, and the corresponding a*b size image is intercepted from FBO0 according to the coordinates. This image is the transparent chassis image corresponding to the current frame image;

[0090] 2.4. Output module: Use openGL to create a frame buffer named FBO1 with size w*h, and save the image of size a*b in step 2.3 to the corresponding a*b area of ​​FBO1. The valid area is a*b, and there are no pixels in the non-a*b area of ​​FBO1.

[0091] 3. Current frame panorama generation module: This module is used to generate a panoramic image of the current frame without the transparent bottom plate area. The steps are as follows:

[0092] 3.1. Real-time image input module: obtains images from the front, back, left, and right cameras as input;

[0093] 3.2. Current frame acquisition module: pass the input of step 3.1 to the openGL shader, and obtain a panoramic image without a transparent bottom plate after the shader image stitching process. The effective pixel area is all areas except the a*b area in the w*h size, and there is no pixel in the a*b area;

[0094] 3.3. Output module: Use openGL to create a frame buffer named FBO2 with a size of w*h, and save the image in step 3.2 to FBO2;

[0095] 4. Fusion module: This module is used to integrate the output image of the transparent chassis calculation and interception module with the output image of the current frame panoramic image acquisition module into a complete panoramic image with a transparent chassis. The steps are as follows:

[0096] 4.1. Input module: Take the texture images corresponding to FBO1 and FBO2 as input;

[0097] 4.2. Shader image processing module: If the vertex coordinates of the shader are located in the a*b area of ​​the chassis, the texture sampling is performed on FBO1; if the vertex coordinates are located in the non-chassis a*b area, the texture sampling is performed on FBO2;

[0098] 4.3. Output module: Save the image in step 4.2 to FBO0.

[0099] 5. Final rendering module: Render the FBO0 image directly to the screen, and use the FBO0 as the input for the next frame of image. Repeat the algorithms of modules 2.&3.&4 for subsequent rendering.

[0100] The relationship between the initialization module and the real-time rendering module is that the initialization module obtains FBO0 and then inputs it into the real-time rendering module; the real-time signal input module uses the timestamp of the vehicle system and signals such as vehicle speed, gear, wheel angle, etc. as input parameters to calculate the accurate chassis area coordinates; the real-time image input module obtains images from the front, back, left and right four-way cameras as input to the module, and the data comes from the image input module in the initialization module.

[0101] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A transparent chassis architecture display method based on openGL, characterized in that: include: Get the camera image as input; Stitching camera images into a panoramic ring view; Calculate the coordinates of the chassis area through real-time signals; Capture the chassis area image from the previous frame image; Synthesize the chassis area and the current frame panoramic image; Render the synthesized image to the display.

2. The method for displaying a transparent chassis structure based on OpenGL as claimed in claim 1, characterized in that: The cameras are arranged in four directions: front, rear, left and right of the vehicle.

3. The method for displaying a transparent chassis structure based on OpenGL as claimed in claim 2, characterized in that: The stitching into a panoramic ring view comprises inputting the camera image into an openGL shader; A panoramic image without a transparent bottom plate is obtained by shader image stitching; The size of the panoramic image is w*h, and there are no pixels in the a*b area; Use OpenGL to create a frame buffer named FBO0 with a size of w*h, and save the image to FBO0; Use glBindTexture to bind the image texture of FBO0. After the image is processed by the shader, the image is rendered directly to the screen, and FBO0 is retained as the input for the next frame of the image.

4. The method for displaying a transparent chassis structure based on OpenGL as claimed in claim 3, characterized in that: The calculating the coordinates of the chassis area by the real-time signal includes taking the previous frame of panoramic view image FBO0 as the image end input; Taking the timestamp of the vehicle system and the vehicle signal as input parameters; The chassis area size is a*b. In the openGL shader, the accurate chassis area coordinates are calculated according to the input parameters of the signal acquisition module, and the corresponding a*b size image is intercepted from FBO0 according to the coordinates. This image is the transparent chassis image corresponding to the current frame image; Use openGL to create a frame buffer named FBO1 with a size of w*h, and save the image of the corresponding a*b size to the corresponding a*b area of ​​FBO1. The valid area is a*b, and there are no pixels in the non-a*b area of ​​FBO1.

5. The method for displaying a transparent chassis structure based on OpenGL as claimed in claim 4, characterized in that: The capturing of the chassis area image from the previous frame of image includes obtaining a camera image as input; The camera image is input into the openGL shader, and after the shader image stitching processing, a panoramic image without a transparent bottom plate is obtained. The effective pixel area is all areas except the a*b area in the w*h size, and there is no pixel in the a*b area; Use OpenGL to create a frame buffer named FBO2 with a size of w*h, and save the panoramic image without a transparent bottom plate into FBO2.

6. The method for displaying a transparent chassis structure based on OpenGL as claimed in claim 5, characterized in that: The synthesizing the chassis area and the current frame panoramic image includes taking the texture images corresponding to FBO1 and FBO2 as input; If the vertex coordinates of the shader are in the a*b area of ​​the chassis, the texture is sampled from FBO1; if the vertex coordinates are in the non-chassis a*b area, the texture is sampled from FBO2; Save the image after texture sampling to FBO0.

7. The method for displaying a transparent chassis structure based on OpenGL as claimed in claim 6, characterized in that: Rendering the synthesized image onto the display screen includes rendering the FBO0 image directly onto the screen.

8. A system using the openGL-based transparent chassis architecture display method as claimed in any one of claims 1 to 7, characterized in that: include: An image input module (100), a transparent chassis calculation and interception module (200), a current frame panorama generation module (300), a fusion module (400) and a final rendering module (500); The image input module (100) is used to obtain a camera image as input; Stitching camera images into a panoramic ring view; The transparent chassis calculation and interception module (200) is used to calculate the coordinates of the chassis area through real-time signals and intercept the chassis area image; The current frame panoramic image generation module (300) is used to generate a panoramic image of the current frame without a transparent bottom plate area; The fusion module (400) is used to synthesize the chassis area and the current frame panoramic image; The final rendering module (500) is used to render the synthesized image onto a display screen.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the transparent chassis architecture display method based on OpenGL described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the transparent chassis architecture display method based on OpenGL described in any one of claims 1 to 7 are implemented.

Images