Image rendering method, device and equipment
By using image rendering methods, DVR and AVM windows are created, image data is processed uniformly and stored in a preset frame buffer object, solving the problem of sharing image streams between the panoramic surround view system and the dashcam, and achieving efficient resource utilization and reduced power consumption.
Patent Information
- Application Number
- CN202610068523.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-19
- Publication Date
- 2026-05-15
AI Technical Summary
When existing panoramic surround view systems and dashcams work simultaneously, they cannot share the image stream from the vehicle's camera, resulting in duplicated hardware resources, increased system power consumption, and higher costs.
By using image rendering methods, in response to user interaction, a DVR window and/or an AVM window are created, the original image data is acquired, texture conversion is performed, and the data is uniformly stored in a preset frame buffer object. The data is then dynamically rendered to the display window according to the event type, thereby enabling the sharing and reuse of image data.
It solves the problem of image stream resource contention, reduces redundant occupation of system resources, optimizes hardware access channels and data transmission bandwidth, and reduces the overall power consumption and cost of the system.
Smart Images
Figure CN122044725A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to an image rendering method, apparatus and device. Background Technology
[0002] With the increasing intelligence of automobiles, Around View Monitors (AVMs) and dashcams have become important in-vehicle functions for enhancing driving safety and convenience. AVMs use multiple wide-angle cameras to capture images of the vehicle's surroundings, stitching and transforming them in real time to provide the driver with a panoramic view from above, assisting with parking and navigating narrow areas. In practical scenarios, Digital Video Recorders (DVRs) are typically used as dashcams, primarily for continuously recording video data during driving to provide evidence for accident investigation. Integrating both of these functions into the same vehicle has become a mainstream configuration.
[0003] Currently, when implementing panoramic surround view display and dashcam recording, the AVM and DVR typically run as two independent processes. However, since the camera image streams can only be accessed exclusively by a single process in most system architectures, competition for image stream resources arises when the AVM and DVR need to work simultaneously. Therefore, existing methods typically employ a method where the AVM and DVR directly acquire image streams from different cameras to prevent this image stream resource contention.
[0004] However, because the multiple camera resources deployed on a vehicle cannot be reused between the AVM and DVR, this results in the duplication and waste of hardware access channels, data transmission bandwidth, and processor computing power in the vehicle's infotainment system, thereby increasing the overall power consumption and cost of the system. Therefore, there is an urgent need for an image rendering method that enables the AVM and DVR to share image streams from the vehicle's cameras. Summary of the Invention
[0005] The main objective of this application is to provide an image rendering method, apparatus, and device, which aims to solve the technical problem that existing AVMs and DVRs are difficult to share image streams to collaboratively achieve image rendering when working simultaneously.
[0006] To achieve the above objectives, this application proposes an image rendering method, the method comprising: In response to user interaction, determine the current event type; A corresponding display window is created based on the current event type, and the display window is a DVR window and / or an AVM window; The system acquires raw image data collected by cameras in each vehicle, performs texture conversion processing on the raw image data to obtain target image texture, and renders the target image texture to a preset frame buffer object. The target image texture is read from the preset frame buffer object and rendered to the display window.
[0007] In one embodiment, the current event type includes: collaborative event, recording-only event, monitoring-only event, and irrelevant event. The step of creating a corresponding display window based on the current event type includes: Determine whether the current event type is the irrelevant event; If not, then when the current event type is the collaborative event, create the DVR window and the AVM window; When the current event type is the "record only" event, the DVR window is created; When the current event type is the monitored event only, the AVM window is created.
[0008] In one embodiment, the steps of acquiring raw image data collected by cameras of each vehicle, performing texture conversion processing on the raw image data to obtain a target image texture, and rendering the target image texture to a preset frame buffer object include: The single-frame images captured by the cameras of each vehicle are obtained as raw image data, and the image texture object is determined based on the raw image data; The image texture object is encoded and converted using a fragment shader to obtain a target image texture, which is an RGBA format image. The target image texture is rendered to a preset frame buffer object.
[0009] In one embodiment, when the current event type is the collaborative event, the step of reading the target image texture from the preset frame buffer object and rendering the target image texture to the display window includes: Obtain the vehicle's operating condition information, and determine the rendering priority corresponding to the DVR window and the AVM window based on the operating condition information; The target image texture is read from the preset frame buffer object and rendered to the display window according to the rendering priority.
[0010] In one embodiment, the step of rendering the target image texture to the display window according to the rendering priority includes: Based on the rendering priority, a priority rendering window and a secondary rendering window are determined between the DVR window and the AVM window; Bind the priority rendering window to the current rendering environment; In the current rendering environment that is bound to the priority rendering window, the preset frame buffer object is read to obtain the target image texture, and the target image texture is rendered to the priority rendering window; Unbind the primary rendering window from the current rendering environment, and bind the secondary rendering window to the current rendering environment; In the current rendering environment that is bound to the secondary rendering window, the preset frame buffer object is read to obtain the target image texture, and the target image texture is rendered to the secondary rendering window.
[0011] In one embodiment, when the current event type is the recorded event only, the step of reading the target image texture from the preset frame buffer object and binding the target image texture to the display window to render the target image texture to the display window includes: Initialize the current rendering environment and bind the DVR window to the current rendering environment; In the current rendering environment bound to the DVR window, the preset frame buffer object is read to obtain the target image texture, and the target image texture is rendered to the DVR window.
[0012] In one embodiment, when the current event type is the monitored-only event, the step of reading the preset frame buffer object to render the target image texture to the current window includes: Initialize the current rendering environment and bind the AVM window to the current rendering environment; In the current rendering environment bound to the AVM window, the preset frame buffer object is read to obtain the target image texture, and the target image texture is rendered to the AVM window.
[0013] In one embodiment, the step of rendering the target image texture to the display window includes: When the display window is the AVM window, the target image texture is sampled by the AVM shader to obtain a first sampling result. The first sampling is a sampling method based on the geometric calibration parameters of each vehicle camera combined with nonlinear perspective projection. Based on the first sampling result, a bird's-eye view rendering view and a current rendering single view are obtained. The bird's-eye view rendering view and the current rendering single view are stitched together to form an AVM panoramic image and rendered to the display window. When the display window is the DVR window, the target image texture is sampled a second time through the DVR shader to obtain a second sampling result. The second sampling is a sampling method based on preset layout parameters and linear scaling. Based on the second sampling result, a multi-grid panoramic view is generated to be identified as the DVR panoramic view and rendered to the display window.
[0014] Furthermore, to achieve the above objectives, this application also proposes an image rendering apparatus, the apparatus comprising: The interaction response module is used to respond to user interaction operations and determine the current event type; A window creation module is used to create a corresponding display window based on the current event type, wherein the display window is a DVR window and / or an AVM window; A general processing module is used to acquire raw image data, perform texture conversion processing on the raw image data to obtain a target image texture, and render the target image texture to a preset frame buffer object. The raw image data is acquired by cameras in each vehicle. The window rendering module is used to read the target image texture from the preset frame buffer object and bind the target image texture to the display window so as to render the target image texture to the display window.
[0015] In addition, to achieve the above objectives, this application also proposes an image rendering device, which includes: a memory, a processor, and an image rendering program stored in the memory and executable on the processor. When the image rendering program is executed by the processor, it implements the image rendering method described above.
[0016] This application discloses an image rendering method, comprising: responding to a user interaction operation, determining the current event type; creating a corresponding display window according to the current event type, wherein the display window is a DVR window and / or an AVM window; acquiring raw image data collected by cameras of each vehicle, performing texture conversion processing on the raw image data to obtain a target image texture, and rendering the target image texture to a preset frame buffer object; reading the target image texture from the preset frame buffer object, and rendering the target image texture to the display window.
[0017] Because this application can uniformly process the raw image data from each vehicle camera based on user interaction events, and store the converted target image texture into a preset frame buffer object, the DVR and AVM windows can directly read the texture from the preset frame buffer object for rendering based on the event type. This solves the problem of duplicate system resource occupation caused by the AVM and DVR having to independently acquire and process image data because they cannot share the same image stream, and realizes one-time processing and multiple reuse of image data. Attached Figure Description
[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a flowchart illustrating the first embodiment of the image rendering method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the image rendering method of this application; Figure 3 This is a flowchart illustrating the third embodiment of the image rendering method of this application; Figure 4 This application presents a schematic diagram of the AVM panoramic view and a schematic diagram of the DVR surround view. Figure 5 This is a schematic diagram of the entire process of the image rendering method in this application; Figure 6 This is a schematic diagram of the module structure of the image rendering apparatus of this application. Figure 7 This is a schematic diagram of the image rendering device of this application.
[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0024] This application provides an image rendering method, referencing... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the image rendering method of this application. In this embodiment, the method includes steps S10 to S40: Step S10: In response to user interaction, determine the current event type.
[0025] It should be noted that the executing entity in this embodiment can be a computing electronic device with data processing, network communication, program execution, and interactive display functions, such as an in-vehicle system, a vehicle image stream management system, etc., or a vehicle assistance electronic control unit (ECU) integrated with GPU video memory in an in-vehicle system. This embodiment does not limit this. Here, a vehicle image stream management system associated with GPU video memory (hereinafter referred to as "the system") is selected as an example to illustrate the various embodiments of this application.
[0026] It should be understood that user interaction operations can be user input actions to the system. That is, users can trigger different types of operation events through the vehicle's central control screen or physical buttons, including: collaborative event A, recording-only event B, monitoring-only event C, and irrelevant event D.
[0027] Among them, the collaborative event can be when the user turns on both the DVR software recording switch and the AVM software at the same time; the record-only event can be when the user turns on only the DVR software recording switch; the monitor-only event can be when the user turns on only the AVM software; and the irrelevant event can be when the user does not turn on either the DVR software or the AVM software.
[0028] In practice, the system can perform logical judgments and classifications based on the captured user input actions to determine the specific current event type.
[0029] Step S20: Create a corresponding display window according to the current event type, wherein the display window is a DVR window and / or an AVM window.
[0030] It should be noted that the display window can be a logical area prepared by the system for subsequent image rendering, which can receive pixel data. For example, the system can use the embedded graphics library interface functions (EGL API functions) in OpenGL for Embedded Systems (OpenGL ES) to create different display windows: DVR window (SurfaceDVR) and / or AVM window (SurfaceAVM).
[0031] The DVR window (SurfaceDVR) can be an off-screen rendering window. This DVR window can not only be directly displayed to the user, but also serve as the input source for the DVR encoder. The image rendered to this DVR window can be encoded in real time and written to the memory card to achieve driving recording.
[0032] The AVM window (SurfaceAVM) can be a display window. The image rendered onto this AVM window can be directly output to the vehicle's central control screen through the display compositor for the user to view.
[0033] It should be understood that the current event type can be collaborative event A, recording-only event B, monitoring-only event C, or irrelevant event D. Irrelevant event D does not require the system to create a display window. Therefore, the current event type can be filtered first: if the current event type is A / B / C, the display window creation process can be triggered; if the event type is D, no action is required.
[0034] In the specific implementation, the system can first determine whether the current event type is an irrelevant event; if it is not an irrelevant event, it can create a DVR window and an AVM window when the current event type is a collaborative event; create a DVR window when the current event type is a recording-only event; and create an AVM window when the current event type is a monitoring-only event.
[0035] Step S30: Obtain the raw image data captured by the cameras of each vehicle, perform texture conversion processing on the raw image data to obtain the target image texture, and render the target image texture to a preset frame buffer object.
[0036] It should be understood that the raw image data can be single-frame images captured by each camera on the vehicle. For example, if a camera is installed on each of the four viewpoints (front, rear, left, and right) of the vehicle, then single-frame images captured by each viewpoint camera at the same timestamp can be obtained as raw image data for subsequent image processing based on the timestamp identifier.
[0037] It can be understood that since the original image data is a YUVA format data stream directly output from the camera, it is not convenient for subsequent rendering and display. Therefore, the original image data can be first converted in format and texture, and then the converted target image texture can be rendered into a Framebuffer Object (FBO). As a rendering target, the FBO can store the converted target image texture with different color attachments for subsequent window rendering.
[0038] Furthermore, to illustrate in detail how the original image data is processed, step S30 specifically includes: steps S301~S303: Step S301: Obtain single-frame images captured by the cameras of each vehicle as raw image data, and determine the image texture object based on the raw image data.
[0039] It should be noted that a single frame can be the smallest unit processed by the system. The system can process the image frames captured by each vehicle camera in a loop, frame by frame. The raw image data is the unprocessed image frame directly output from each vehicle camera, such as four single-frame images from cameras with different viewing angles.
[0040] It should also be noted that the raw image data can be in YUV encoding format, which can separate luminance (Y) and chrominance (UV), which is beneficial for compression and transmission, but is not suitable for direct use in color calculation and display.
[0041] In the specific implementation, the system can request an empty texture storage space in the GPU memory through the OpenGL ES API and set basic parameters (such as filtering method and wrapping mode) to obtain an initial texture object. Then, the texture object can be bound to a texture unit, which can be a set of independent hardware resources set up by the GPU for sampling textures. Each texture unit can bind and cache a texture object (including image data, sampling parameters, format, etc.) for subsequent shader access. Finally, the original image data in the aforementioned YUV encoding format can be uploaded to the aforementioned initial texture object to obtain an image texture object (YUV texture object), thus realizing the uploading of original graphics data and texture.
[0042] Step S302: Convert the encoding format of the image texture object using a fragment shader to obtain the target image texture, wherein the target image texture is an RGBA format image.
[0043] It's important to note that fragment shaders can be used to determine the final color of each output pixel. Specifically, a fragment shader can receive the previously uploaded YUV texture object as input, then calculate and convert the Y and UV components of each pixel in real time into four components: R (red), G (green), B (blue), and A (transparency). This maps the data from the YUV color space to the RGBA color space, obtaining the converted RGBA color value. This RGBA color value is the pixel content of the target image texture, and the subsequent RGBA pixel values of the target image texture can be output by the fragment shader.
[0044] Step S303: Render the target image texture to a preset frame buffer object.
[0045] It should be understood that before rendering, the system can call the OpenGL ES API to switch the current rendering target from the default system window to the preset FBO, thereby ensuring that the output stream of the subsequent fragment shader flows to that FBO.
[0046] It is understood that an FBO can have multiple color attachments attached, which can be considered as multiple storage locations within the FBO. For example, the target image textures corresponding to the four viewpoints of the vehicle can be rendered to four different color attachments of a preset FBO.
[0047] In the specific implementation, after configuring the FBO and running the fragment shader that performs format conversion, the system can write the RGBA pixel values output by the fragment shader into the color attachment currently bound to the preset FBO. This preset FBO then centrally stores the target image texture, providing a unified, high-performance read-only data source for subsequent rendering modules with different purposes.
[0048] Step S40: Read the target image texture from the preset frame buffer object and render the target image texture to the display window.
[0049] It should be noted that before rendering, the current OpenGL rendering context can be bound to a specific display window (i.e., the previously created SurfaceDVR or SurfaceAVM) through a thread binding function (such as eglMakeCurrent).
[0050] It should be understood that, depending on the current event type, the fragment shader corresponding to the AVM or DVR can be run to read the image textures of the four perspectives from the aforementioned preset FBO, stitch them together according to the corresponding mode, and generate the final display screen; then, the rendered display screen can be submitted to the corresponding display window through the thread rendering display function (such as eglSwapBuffers) and displayed on the vehicle's central control screen for the user to view.
[0051] It should also be noted that in OpenGL ES, the GPU can only render to one target at a time. Therefore, when the current event type is Cooperative Event A, after binding the AVM window and completing the rendering and display in the AVM window, it is possible to switch the binding to the DVR window to complete the rendering and display in the DVR window. This enables the same frame of image stream to achieve both "real-time viewing" and "full-time recording" functions, which helps reduce the overall power consumption of the vehicle's infotainment system.
[0052] This embodiment can uniformly process raw image data from various vehicle cameras based on a single user interaction event, and store the converted target image texture into a preset frame buffer object. This allows the DVR and AVM windows to directly read the texture from the preset frame buffer object for rendering based on the event type. This solves the problem of duplicate system resource occupation caused by the AVM and DVR having to independently acquire and process image data because they cannot share the same image stream, and realizes one-time processing and multiple reuse of image data.
[0053] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2This is a flowchart illustrating the second embodiment of the image rendering method of this application.
[0054] In this embodiment, to specifically illustrate how image rendering is performed based on different event types, when the current event type is a collaborative event, step S40 includes: steps S401~S402: Step S401: Obtain the vehicle's operating condition information and determine the rendering priority corresponding to the DVR window and the AVM window based on the operating condition information.
[0055] It should be noted that the operating condition information can be the vehicle's operating status and environmental parameters at different times, which can include dynamic driving signals and environmental signals. The dynamic driving signals can include: vehicle speed (obtained from the CAN bus, such as 0km / h, 30km / h, 60km / h), current gear (P Park, R Reverse, N Neutral, D Drive), turn signal status (left turn, right turn), and door / trunk opening status; the environmental signals can be the driving mode from the body domain controller (such as "Parking Mode", "Economy Mode", "Sport Mode"), or geographical location information linked to the navigation system (such as about to enter a narrow alley).
[0056] Understandably, priority strategies for different operating conditions can be pre-set based on historical experience data, including: AVM high rendering priority, DVR high rendering priority, and historical performance feedback strategy.
[0057] For example, if the vehicle is determined to be in a low-speed or parking condition based on the operating condition information, considering that the driver relies heavily on the AVM panoramic view for precise operation, rendering delay may lead to collision risk, while DVR recording can accept millisecond-level lag. Therefore, a strategy of high AVM rendering priority can be selected.
[0058] If the vehicle is determined to be in a high-speed driving condition based on the operating condition information, then considering that an accident can occur instantly at high speed, the continuity and integrity of the DVR image are more important. Therefore, a strategy of high rendering priority for the DVR can be selected.
[0059] If the vehicle is determined to be in a medium-speed driving condition based on the operating condition information, the historical performance feedback strategy can be directly adopted to give higher rendering priority to display windows with a longer average rendering time in the previous cycle.
[0060] Step S402: Read the target image texture from the preset frame buffer object, and render the target image texture to the display window according to the rendering priority.
[0061] It should be understood that, based on the rendering priority mentioned above, the display window that is bound and rendered first when the current event type is a collaborative event can be determined. Therefore, to specifically illustrate the rendering process of collaborative events, step S402 specifically includes: steps S4021~S4025: Step S4021: Determine the priority rendering window and the secondary rendering window between the DVR window and the AVM window according to the rendering priority.
[0062] For example, this embodiment can be described using the AVM window as the primary rendering window and the DVR window as the secondary rendering window.
[0063] Step S4022: Bind the priority rendering window to the current rendering environment.
[0064] It should be noted that the current rendering environment can be an OpenGL rendering context, which is a set of states and a workbench upon which the GPU depends for executing all drawing commands. Furthermore, at any given time, a rendering context can only be bound to one drawing surface, i.e., the display window.
[0065] In practice, the system can use the eglMakeCurrent function to bind SurfaceAVM to the current environment.
[0066] Step S4023: In the current rendering environment bound to the priority rendering window, read the preset frame buffer object to obtain the target image texture, and render the target image texture to the priority rendering window.
[0067] It should be understood that in this rendering environment, the corresponding fragment shader (AVM shader) can be enabled to read the target image texture (corresponding to the four viewpoints) from the aforementioned preset FBO; then, the fragment shader will crop and stitch the target image texture into the display mode corresponding to AVM to obtain the AVM panoramic image and render it to SurfaceAVM. At this time, the user can view the AVM window on the AVM software display interface in the vehicle's central control screen.
[0068] Step S4024: Unbind the primary rendering window from the current rendering environment, and bind the secondary rendering window to the current rendering environment.
[0069] It should be understood that when the AVM window finishes rendering, the SurfaceAVM can be unbound from the current rendering environment, and the SurfaceDVR can be bound to the current rendering environment.
[0070] Step S4025: In the current rendering environment that is bound to the secondary rendering window, read the preset frame buffer object to obtain the target image texture, and render the target image texture to the secondary rendering window.
[0071] Understandably, in this rendering environment, the corresponding fragment shader (DVR shader) can be enabled to read the target image texture (corresponding to the four viewpoints) from the aforementioned preset FBO. The fragment shader then clips and stitches the target image texture to the DVR's corresponding display mode, obtaining the DVR surround view image and rendering it to the SurfaceDVR. This DVR surround view image can then be simultaneously recorded to the DVR software, and the user can view this image on the DVR software display interface of the vehicle's central control screen.
[0072] It should also be noted that when a user simultaneously selects both AVM and DVR software to view images on the vehicle's central control screen, the AVM panoramic view and the DVR surround view can be displayed in windows of varying sizes. For example, the AVM panoramic view can occupy the center or most of the left side of the screen as the main view, while the DVR surround view can be displayed as a shrunk floating window in the lower right or upper right corner. This embodiment does not impose any limitations on this.
[0073] It should be noted that, when the current event type is a collaborative event, the system can dynamically determine the rendering priority of the AVM and DVR based on the vehicle's real-time operating information, and execute the window binding and rendering process according to this priority order. This allows the system to prioritize the allocation of limited GPU rendering resources to more critical functional modules in the current driving state, even when the AVM and DVR need to work together. Thus, while ensuring the reuse of image data from the same source and the collaborative operation of the AVM and DVR, it further achieves intelligent and minimal allocation of rendering latency, thereby optimizing the real-time output of key safety functions and the overall reliability of the system.
[0074] Furthermore, when the current event type is "record only", step S40 includes: steps S401'~S402': Step S401': Initialize the current rendering environment and bind the DVR window to the current rendering environment.
[0075] Step S402': In the current rendering environment bound to the DVR window, read the preset frame buffer object to obtain the target image texture, and render the target image texture to the DVR window.
[0076] In the specific implementation, the system can directly bind the SurfaceDVR to the current rendering environment. Then, in this rendering environment, the target image texture in the preset FBO is used as input to obtain the imageDVR, i.e., the DVR surround view. The eglSwapBuffers function can be used to render the DVR surround view to the SurfaceDVR. At this time, the DVR surround view can be recorded in the DVR software to realize driving recording.
[0077] Furthermore, when the current event is a monitored event only, step S40 includes: steps S401''~S402'': Step S401'': Initialize the current rendering environment and bind the AVM window to the current rendering environment; Step S402'': In the current rendering environment bound to the AVM window, read the preset frame buffer object to obtain the target image texture, and render the target image texture to the AVM window.
[0078] In the specific implementation, the system can directly bind SurfaceAVM to the current rendering environment. Then, in this rendering environment, the target image texture in the preset FBO is used as input to obtain the imageAVM, which is the AVM panoramic image. The eglSwapBuffers function can be used to render the AVM panoramic image to SurfaceAVM. At this time, the user can view the AVM window image on the AVM software display interface in the vehicle's central control screen.
[0079] This embodiment employs differentiated window binding and rendering strategies for different event types: when only recording or monitoring events, the current rendering environment is bound to the corresponding DVR or AVM window at once, and single-function rendering is performed based on a shared preset frame buffer object; while during collaborative events, the binding is dynamically switched and the two windows are rendered sequentially based on the priority determined by the operating conditions. This allows the system to operate with minimal overhead and latency in single-function scenarios, and intelligently allocate rendering resources in collaborative scenarios, thus achieving optimal utilization of system resources and improved energy efficiency across all operating conditions while ensuring real-time functionality.
[0080] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the image rendering method of this application.
[0081] In this embodiment, to specifically illustrate how to generate the final display image on the AVM window or DVR window from a shared image texture, the step of rendering the target image texture to the display window includes: steps S4001~S4004: Step S4001: When the display window is the AVM window, the target image texture is sampled first by the AVM shader to obtain the first sampling result.
[0082] It should be noted that the AVM shader is a fragment shader program that runs on the GPU and is specific to AVM.
[0083] It should be understood that the first sampling can be a sampling method based on the geometric calibration parameters of each vehicle camera combined with nonlinear perspective projection. The geometric calibration parameters can be mathematical model parameters obtained beforehand through the vehicle camera calibration process, describing the optical characteristics of the vehicle camera itself and its position and orientation in the vehicle coordinate system, and may include intrinsic and extrinsic parameters. The nonlinear perspective projection can be a standard camera model that maps three-dimensional spatial points to a 2D imaging plane, and the relationship is nonlinear.
[0084] In its implementation, the AVM shader can calculate the sampling coordinates based on the vehicle camera's geometric calibration parameters and through nonlinear perspective projection inverse transformation to achieve texture sampling from a virtual bird's-eye view and obtain the first sampling result.
[0085] Step S4002: Based on the first sampling result, obtain the bird's-eye view rendering view and the current rendering single view, stitch the bird's-eye view rendering view and the current rendering single view into an AVM panoramic image and render it to the display window.
[0086] It should be understood that, based on the texture sampling coordinates of each virtual bird's-eye view in the first sampling result above, a virtual panoramic image, i.e., a bird's-eye view rendering view, can be formed by looking vertically down from directly above the vehicle.
[0087] Furthermore, the current rendered single view can refer to a view of a vehicle camera that is dynamically selected based on real-time vehicle signals (such as gear position R / N / D, turn signal status) or a view preset by the user. This view can be obtained by processing the image texture of this view in the target image texture through processes such as distortion correction, magnification, and overlaying dynamic guide lines.
[0088] In its implementation, the system can use the AVM shader to sample the target image textures corresponding to different viewpoints in the FBO, and then crop and stitch the images into a bird's-eye view + single view mode, which is then rendered as the ImageAVM to the display window SurfaceAVM. A schematic diagram of this ImageAVM can be found in [reference needed]. Figure 4 4-A in Figure 4 These are schematic diagrams of the AVM panoramic view and the DVR surround view view of this application.
[0089] Step S4003: When the display window is the DVR window, the target image texture is sampled a second time using the DVR shader to obtain a second sampling result.
[0090] It should be noted that the DVR shader can be a dedicated fragment shader program that runs on the GPU.
[0091] It should be understood that the second sampling can be a sampling method based on preset layout parameters and linear scaling. The preset layout parameters can be predefined constants used to describe a 2x2 (or multi-grid) layout, including sub-region divisions (the precise range of each quadrant in the output image) and mapping relationships (e.g., the front view corresponding to the upper left quadrant). Linear scaling can be an operation that scales the target image texture to fit the size of the sub-regions in the output image.
[0092] In a practical implementation, the DVR shader can calculate the sampling coordinates based on a fixed layout mapping relationship through linear scaling and translation to achieve texture sampling for multi-screen tiled display and obtain a second sampling result.
[0093] Step S4004: Generate a multi-grid panoramic view based on the second sampling result, determine it as the DVR panoramic view, and render it to the display window.
[0094] It should be understood that the system can directly place the pixels of each camera obtained by the above linear sampling into the corresponding positions of the output screen, thereby obtaining the final 2*2 grid screen, i.e., the multi-grid panoramic screen.
[0095] In its implementation, the system can use a DVR shader to sample the target image textures corresponding to different viewpoints in the FBO, stitching the images into a 2x2 grid view, which is then rendered as the ImageDVR to the display window SurfaceDVR. A schematic diagram of this ImageDVR can be found in [reference needed]. Figure 4 4-B in.
[0096] This embodiment defines and implements fundamentally different texture sampling methods for AVM and DVR functions: the AVM shader uses camera-based geometric calibration and non-linear sampling to synthesize a bird's-eye view, while the DVR shader uses fixed-layout and linear scaling sampling to generate multi-grid images. This allows the same set of target image textures to be reused and, through two independent, highly optimized GPU processing paths, generates in real-time enhanced panoramic images that conform to human visual perception and surround-view recording images for complete recording purposes. This maximizes the utilization of hardware computing power and data bandwidth while ensuring optimal output quality for both AVM and DVR.
[0097] Furthermore, you can also refer to this section. Figure 5 The entire process of the image rendering method in this application is described. Figure 5 This is a schematic diagram of the entire process of the image rendering method in this application.
[0098] Depend on Figure 5 It can be seen that the system can divide the entire image rendering process into different module processing stages.
[0099] First, the user interaction event recognition module can respond to user interaction operations and determine the current event type; The event filtering module can filter the current event type to ensure that subsequent processes are executed when the current event type is not an irrelevant event. The image processing module is the main module, which includes: window creation module, image capture module, and general image processing module; The window creation module can create corresponding display windows based on the current event type, including: DVR window and / or AVM window; The image capture module can acquire raw image data collected by the cameras of each vehicle; The general image processing module can perform texture conversion on the raw image data to obtain the target image texture, and store the target image texture in a preset frame buffer object. Figure 5 (represented as FBO0 in the middle). Next, branches can be created based on the event type, including: Event A (cooperative event): entering both AVM and DVR processing branches simultaneously; Event B (recording event only): entering the DVR processing branch; Event C (monitoring event only): entering the AVM processing branch; The window environment binding module can bind the current rendering environment to the corresponding display window (DVR window and / or AVM window). In the AVM and DVR processing branch, the window environment binding module can first bind the current rendering environment to the AVM window; then, use FBO0 as the input of the AVM image processing module to obtain the AVM panoramic image and render it to SurfaceAVM; next, the window environment binding module can unbind the AVM window from the current rendering environment to bind the current rendering environment to the DVR window; thus, use FBO0 as the input of the DVR image processing module to obtain the DVR surround view image and render it to SurfaceDVR.
[0100] In the DVR processing branch, the window environment binding module can directly bind the current rendering environment to the DVR window; then, FBO0 is used as the input of the DVR image processing module to obtain the DVR surround view image and render it to the SurfaceDVR; In the AVM processing branch, the window environment binding module can first bind the current rendering environment to the AVM window; then use FBO0 as the input of the AVM image processing module to obtain the AVM panoramic image and render it to SurfaceAVM.
[0101] This application's image rendering method employs an event-driven process. The user interaction event recognition module and event filtering module determine the working condition, while the window environment binding module dynamically schedules the rendering environment. This allows the image processing module to execute the AVM and DVR image processing modules separately or sequentially based on the FBO0 data output from the same general processing module, according to the event type (A / B / C). This achieves single-time acquisition and unified processing of image data, multi-path reuse, and on-demand binding and switching of rendering targets. While ensuring that panoramic image display and surround view recording functions can be executed independently and in parallel, it significantly improves system resource utilization, rendering real-time performance, and multi-task collaborative stability.
[0102] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the image rendering method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.
[0103] This application also provides an image rendering apparatus, please refer to... Figure 6 , Figure 6 This is a schematic diagram of the module structure of the image rendering apparatus of this application, the apparatus comprising: The interaction response module 601 is used to respond to user interaction operations and determine the current event type; The window creation module 602 is used to create a corresponding display window according to the current event type, wherein the display window is a DVR window and / or an AVM window; The general processing module 603 is used to acquire raw image data, perform texture conversion processing on the raw image data to obtain a target image texture, and render the target image texture to a preset frame buffer object. The raw image data is acquired by the cameras of each vehicle. The window rendering module 604 is used to read the target image texture from the preset frame buffer object and bind the target image texture to the display window so as to render the target image texture to the display window.
[0104] The device in this embodiment can uniformly process the raw image data from each vehicle camera based on user interaction events, and store the converted target image texture into a preset frame buffer object. This allows the DVR and AVM windows to directly read the texture from the preset frame buffer object for rendering based on the event type. This solves the problem of duplicate system resource occupation caused by the AVM and DVR having to independently acquire and process image data because they cannot share the same image stream, and realizes one-time processing and multiple reuse of image data.
[0105] This application also provides an image rendering apparatus, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the image rendering method in the first embodiment described above.
[0106] The following is for reference. Figure 7 , Figure 7 This is a schematic diagram of the image rendering device of this application. The image rendering device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable media players (PMPs), and fixed terminals such as digital TVs and desktop computers. Figure 7 The image rendering device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0107] like Figure 7As shown, the image rendering device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the image rendering device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the image rendering device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows an image rendering device with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.
[0108] The image rendering device provided in this application, employing the image rendering method described in the above embodiments, can solve the technical problem of image rendering. Compared with the prior art, the beneficial effects of the image rendering device provided in this application are the same as those of the image rendering method described in the above embodiments, and other technical features of the image rendering device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.
[0109] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other elements in the process, method, article, or system that includes that element.
[0110] The above embodiment numbers are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. They are only some embodiments of this application and do not limit the scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of this application.
Claims
1. An image rendering method, characterized in that, The method includes: In response to user interaction, determine the current event type; A corresponding display window is created based on the current event type, and the display window is a DVR window and / or an AVM window; The system acquires raw image data collected by cameras in each vehicle, performs texture conversion processing on the raw image data to obtain target image texture, and renders the target image texture to a preset frame buffer object. The target image texture is read from the preset frame buffer object and rendered to the display window.
2. The method as described in claim 1, characterized in that, The current event type includes: collaborative event, recording-only event, monitoring-only event, and irrelevant event. The step of creating a corresponding display window based on the current event type includes: Determine whether the current event type is the irrelevant event; If not, then when the current event type is the collaborative event, create the DVR window and the AVM window; When the current event type is the "record only" event, the DVR window is created; When the current event type is the monitored event only, the AVM window is created.
3. The method as described in claim 1, characterized in that, The steps of acquiring raw image data from each vehicle's camera, performing texture conversion processing on the raw image data to obtain a target image texture, and rendering the target image texture to a preset frame buffer object include: The single-frame images captured by the cameras of each vehicle are obtained as raw image data, and the image texture object is determined based on the raw image data; The image texture object is encoded and converted using a fragment shader to obtain a target image texture, which is an RGBA format image. The target image texture is rendered to a preset frame buffer object.
4. The method as described in claim 2, characterized in that, When the current event type is the collaborative event, the step of reading the target image texture from the preset frame buffer object and rendering the target image texture to the display window includes: Obtain the vehicle's operating condition information, and determine the rendering priority corresponding to the DVR window and the AVM window based on the operating condition information; The target image texture is read from the preset frame buffer object and rendered to the display window according to the rendering priority.
5. The method as described in claim 4, characterized in that, The step of rendering the target image texture to the display window according to the rendering priority includes: Based on the rendering priority, a priority rendering window and a secondary rendering window are determined between the DVR window and the AVM window; Bind the priority rendering window to the current rendering environment; In the current rendering environment that is bound to the priority rendering window, the preset frame buffer object is read to obtain the target image texture, and the target image texture is rendered to the priority rendering window; Unbind the primary rendering window from the current rendering environment, and bind the secondary rendering window to the current rendering environment; In the current rendering environment that is bound to the secondary rendering window, the preset frame buffer object is read to obtain the target image texture, and the target image texture is rendered to the secondary rendering window.
6. The method as described in claim 2, characterized in that, When the current event type is the recorded event only, the step of reading the target image texture from the preset frame buffer object and binding the target image texture to the display window to render the target image texture to the display window includes: Initialize the current rendering environment and bind the DVR window to the current rendering environment; In the current rendering environment bound to the DVR window, the preset frame buffer object is read to obtain the target image texture, and the target image texture is rendered to the DVR window.
7. The method as described in claim 2, characterized in that, When the current event type is the monitored event only, the step of reading the preset frame buffer object to render the target image texture to the current window includes: Initialize the current rendering environment and bind the AVM window to the current rendering environment; In the current rendering environment bound to the AVM window, the preset frame buffer object is read to obtain the target image texture, and the target image texture is rendered to the AVM window.
8. The method as described in claim 4, characterized in that, The step of rendering the target image texture to the display window includes: When the display window is the AVM window, the target image texture is sampled by the AVM shader to obtain a first sampling result. The first sampling is a sampling method based on the geometric calibration parameters of each vehicle camera combined with nonlinear perspective projection. Based on the first sampling result, a bird's-eye view rendering view and a current rendering single view are obtained. The bird's-eye view rendering view and the current rendering single view are stitched together to form an AVM panoramic image and rendered to the display window. When the display window is the DVR window, the target image texture is sampled a second time through the DVR shader to obtain a second sampling result. The second sampling is a sampling method based on preset layout parameters and linear scaling. Based on the second sampling result, a multi-grid panoramic view is generated to be identified as the DVR panoramic view and rendered to the display window.
9. An image rendering apparatus, characterized in that, The device includes: The interaction response module is used to respond to user interaction operations and determine the current event type; A window creation module is used to create a corresponding display window based on the current event type, wherein the display window is a DVR window and / or an AVM window; A general processing module is used to acquire raw image data, perform texture conversion processing on the raw image data to obtain a target image texture, and render the target image texture to a preset frame buffer object. The raw image data is acquired by cameras in each vehicle. The window rendering module is used to read the target image texture from the preset frame buffer object and bind the target image texture to the display window so as to render the target image texture to the display window.
10. An image rendering device, characterized in that, The image rendering device includes: a memory, a processor, and an image rendering program stored in the memory and executable on the processor, wherein the image rendering program, when executed by the processor, implements the image rendering method as described in any one of claims 1 to 8.