An image processing method, system, device, medium, equipment and vehicle

Abstract

Embodiments of the present application disclose an image processing method, system, device, medium, equipment and vehicle, which can solve the problem of redundancy and clutter in visual driving information. The method comprises: acquiring a first image and a second image, the first image being a current frame image rendered by a surround display perception system (SR) of a vehicle, and the second image being a current frame image rendered by an all-view monitoring system (AVM) of the vehicle; and fusing the first image and the second image to obtain a fused image, the image content of the fused image comprising: image content of the first image or the second image outside a preset display range, and image content of the first image and the second image fused within the preset display range.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202510594954.8, filed on May 9, 2025, entitled "An Image Processing Method, System, Apparatus, Medium, Device and Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of graphics rendering technology, and in particular to an image processing method, system, apparatus, medium, device, and vehicle. Background Technology

[0004] Surround reality technology refers to the perception and visualization of driving information such as traffic participants and road facilities around a vehicle based on onboard perception capabilities. It can provide visual decision-making basis for advanced autonomous driving functions and help drivers observe the surrounding environment of the vehicle, thereby improving driving safety.

[0005] In related technologies, the real-world video stream rendered using an Around View Monitor (AVM) system is overlaid as an independent layer on the scene rendered using a Surrounding Reality (SR) system to achieve a visual display of driving information.

[0006] However, this image processing method of overlaying two layers of images can easily cause redundancy and clutter in the visual field of driving information, which in turn can lead to the risk of obscuring key driving information and increase the cognitive load on the driver. Summary of the Invention

[0007] This application provides an image processing method, system, apparatus, medium, device, and vehicle that can solve the problem of visual redundancy and clutter in driving information. The technical solution is as follows:

[0008] In a first aspect, an image processing method is provided, the method comprising:

[0009] Acquire a first image and a second image, wherein the first image is a current frame image rendered using the vehicle's Surround Display Perception System (SR), and the second image is a current frame image rendered using the vehicle's All-Around View Monitoring System (AVM).

[0010] The first image and the second image are merged to obtain a merged image. The image content of the merged image includes: the image content of the first image or the second image outside the preset display range, and the image content of the first image and the second image after merging within the preset display range.

[0011] Secondly, an image processing system is provided, the system comprising an AVM module, an image rendering engine, and a frame buffer object (FBO), wherein the FBO is communicatively connected to the AVM module and the image rendering engine, respectively, wherein:

[0012] The AVM module is configured to render consecutive frame images based on AVM and transmit the rendered current frame image to the FBO for storage;

[0013] The image rendering engine is configured to render consecutive frame images based on SR, and to fuse the current frame image stored in the FBO with the current frame image rendered based on SR to obtain a fused image; the image content of the fused image includes: the image content of the current frame image rendered by the AVM module or the current frame image rendered based on SR outside the preset display range, and the image content of the current frame image rendered by the AVM module and the current frame image rendered based on SR fused within the preset display range.

[0014] Thirdly, an image processing apparatus is provided, the apparatus comprising:

[0015] The communication module is used to acquire a first image and a second image, wherein the first image is a current frame image rendered using the vehicle's Surround View Perception System (SR), and the second image is a current frame image rendered using the vehicle's Panoramic Surround View Monitoring System (AVM).

[0016] The processing module is used to fuse the first image and the second image to obtain a fused image. The image content of the fused image includes: the image content of the first image or the second image outside a preset display range, and the image content of the first image and the second image after fusion within the preset display range.

[0017] Fourthly, a computer-readable storage medium is provided, the storage medium storing at least one instruction, the at least one instruction being executed by a processor to implement the image processing method as described in the first aspect.

[0018] Fifthly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the image processing method as described in the first aspect.

[0019] In a sixth aspect, a vehicle is provided, the vehicle including electronic equipment as described in the fifth aspect, or the vehicle including an image processing system as described in the second aspect.

[0020] In the image processing method provided in this application embodiment, by retaining the image content represented by the first image or the second image outside the preset display range, and fusing the image content represented by the first image and the second image within the preset display range, the same driving information perceived by the SR and AVM can be avoided by using the mutual complementarity of the perception range to ensure the comprehensiveness of the perceived driving information. This avoids redundant and cluttered driving information in terms of perspective, thereby reducing the risk of key driving information being obscured and significantly reducing the cognitive load on the driver. Attached Figure Description

[0021] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of a driving information visualization technique in related technologies.

[0023] Figure 2 A structural block diagram of an electronic device provided in an embodiment of this application;

[0024] Figure 3 A flowchart illustrating an image processing method provided in an embodiment of this application;

[0025] Figure 4 A schematic diagram of a near-field obstacle-triggered fusion scene provided in an embodiment of this application;

[0026] Figure 5 A schematic diagram of an intersection turning activation fusion scenario provided in an embodiment of this application;

[0027] Figure 6 A schematic diagram of an automated parking fusion scenario provided in an embodiment of this application;

[0028] Figure 7 A schematic diagram of a fused image under a default viewpoint fusion scheme provided in an embodiment of this application;

[0029] Figure 8 A schematic diagram of a top-down view fusion scheme provided in an embodiment of this application;

[0030] Figure 9 A schematic diagram of a fused image under a forward-view fusion scheme provided in an embodiment of this application;

[0031] Figure 10 A schematic diagram of a fused image under a left-view fusion scheme provided in an embodiment of this application;

[0032] Figure 11 A schematic diagram of a fused image under a right-view fusion scheme provided in an embodiment of this application;

[0033] Figure 12 A schematic diagram of a fused image under a close-up fusion scheme provided in an embodiment of this application;

[0034] Figure 13 A schematic diagram of a fused image under a left rear view fusion scheme provided in an embodiment of this application;

[0035] Figure 14 A schematic diagram of a fused image under a right-side rear-view fusion scheme provided in an embodiment of this application;

[0036] Figure 15 A schematic diagram illustrating the sensing range distribution of SR, AVM, and SD Map provided in an embodiment of this application;

[0037] Figure 16 A schematic diagram illustrating the sensing distance of SR, AVM, and SD Map provided in an embodiment of this application;

[0038] Figure 17 An architecture diagram of an image processing system provided in this application embodiment;

[0039] Figure 18 An architectural diagram of another image processing system provided in this application embodiment;

[0040] Figure 19 A real-time rendering timing diagram of a system provided in this application embodiment;

[0041] Figure 20 A schematic diagram illustrating an opaque blending effect provided in an embodiment of this application;

[0042] Figure 21 A schematic diagram illustrating a semi-transparent blending effect provided in an embodiment of this application;

[0043] Figure 22 This is a block diagram of an image processing apparatus provided in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0045] Please refer to Figure 1 The diagram illustrates a visualization of driving information in related technologies. In surround reality technology, the current approach primarily involves overlaying the real-world video stream rendered by AVM onto the SR-rendered scene as an independent layer. For example... Figure 1 The image shows an AVM window overlaid on the full-screen SR image to visualize the driving information perceived by the SR and AVM.

[0046] However, this dual-layer image processing method can easily lead to redundancy and clutter in the visual perspective of driving information, resulting in confusion in the visual priority of key driving information (such as lane lines and obstacles), posing a risk of occlusion, and increasing the cognitive load on the driver, potentially delaying decision-making in emergency situations. Furthermore, the spatial coordinate system deviation caused by the dual-layer image overlay can easily lead to inconsistencies in human-computer interaction logic, resulting in interaction contradictions (for example, the predicted path displayed in the SR image may be difficult to correlate with the AVM real-world view), impacting the user experience.

[0047] Based on the above analysis, and addressing the problem of redundant and cluttered driving information in related technologies from a visual perspective, this application provides an image processing method, system, device, medium, equipment, and vehicle. It performs deep data fusion on the driving information perceived by the SR and AVM to avoid repetitive visualization of the same driving information perceived by the SR and AVM. Compared to image processing methods that overlay two layers of images, this application can effectively avoid redundancy and clutter in the visual perspective of driving information, thereby reducing the risk of key driving information being obscured and significantly reducing the cognitive load on the driver.

[0048] First, to facilitate understanding of the technical solutions provided in this application, the main technical concepts involved in the embodiments of this application will be briefly explained below.

[0049] SR: This is an intelligent driving visualization system based on advanced in-vehicle perception systems and artificial intelligence (AI) algorithms. Through multi-sensor fusion (including cameras, millimeter-wave radar, and lidar), it constructs and dynamically renders a 360° traffic environment around the vehicle in real time. With high-precision 3D modeling technology at its core, the system digitizes driving information such as road facilities, traffic participants (vehicles, pedestrians, and non-motorized vehicles), lane lines, and traffic signals in real time. This information is then intuitively displayed through the in-vehicle central control screen, instrument panel, or augmented reality head-up display (AR-HUD), providing drivers with a global road condition perception and providing visual decision-making support for advanced autonomous driving functions.

[0050] AVM, also known as a 360° panoramic imaging system, is a panoramic vision assistance technology for automobiles. This technology stitches together images from multiple perspectives using onboard cameras to provide the driver with a 360° overhead view of driving information, helping the driver to more clearly observe the surrounding environment and improving safety during parking, low-speed driving, and navigating narrow roads.

[0051] Standard Definition Map (SD MAP): This is a map type that is different from High Definition Map (HD Map) and is usually suitable for general navigation or basic geographic information query scenarios.

[0052] Image rendering engines, such as the Kanzi engine, are mainly used to render 2D / 3D models, textures, lighting and other data into high-fidelity images in real time or offline, based on computer graphics.

[0053] Please refer to Figure 2 This diagram illustrates a structural block diagram of an electronic device provided in an exemplary embodiment of this application. The electronic device can be an in-vehicle central control unit, instrument cluster controller, smart cockpit, or multi-screen digital cockpit, capable of running applications. The electronic device in this application may include one or more of the following components: a processor 110, a memory 120, and an input / output device 130.

[0054] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 110 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0055] The memory 120 may include random access memory (RAM) or read-only memory. Optionally, the memory 120 may include a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be an Android system (including systems deeply developed based on the Android system), an iOS system (including systems deeply developed based on the iOS system), or other systems. The data storage area may also store data created by the electronic device during use (such as audio and video data).

[0056] Please refer to Figure 3 The diagram illustrates a flowchart of an image processing method according to an embodiment of this application. This method can be applied to the electronic device described above. The method may include the following steps:

[0057] Step S101: Acquire a first image and a second image, wherein the first image is a current frame image rendered using the vehicle's Surround View Perception System (SR), and the second image is a current frame image rendered using the vehicle's Panoramic Surround View Monitoring System (AVM).

[0058] In practice, a FrameBuffer Object (FBO) is created in advance, and the first image received from the SR algorithm module and the second image received from the AVM algorithm module are stored in the FBO for subsequent fusion.

[0059] Optionally, the first image is the current frame image rendered using the vehicle's image rendering engine, which integrates an SR algorithm module and a map (such as SD MAP) algorithm module.

[0060] Step S102: Merge the first image and the second image to obtain a merged image. The image content of the merged image includes: the image content of the first image or the second image outside the preset display range, and the image content of the first image and the second image after merging within the preset display range.

[0061] In practice, the preset display range can be flexibly adjusted according to user instructions or the vehicle's driving status. For example, the preset display range can be adjusted to the full (or partial) perception range of the SR or AVM to support different interaction needs.

[0062] The image content represented by the second image within a preset display range is merged with the image content represented by the first image within the same preset display range. This merges two sets of driving information that are visualized using SR and AVM respectively into one set, thereby avoiding the repeated visualization of the same driving information and effectively reducing the information pressure on the driver.

[0063] For example, for image content rendered using SR and AVM for the same driving information (i.e., the same driving information visualized, such as the same obstacle visualized), only the image content rendered by SR can be retained, or only the image content rendered by AVM can be retained.

[0064] For example, for image content rendered using SR and AVM for the same driving information, new image content can be generated based on the two image contents (using a generative model). This allows for a more accurate visualization of the relevant driving information by combining the three-dimensional contours, colors, and textures reflected in the respective image contents by SR and AVM.

[0065] If the image content to be included within and outside the preset display range comes from different images (for example, if the image content to be included outside the preset display range comes from the first image, and the image content to be included within the preset display range comes from the second image), then the corresponding image content within and outside the preset display range is processed by coordinate alignment and other methods to merge the two into a complete image, thereby obtaining a fused image.

[0066] Therefore, by deeply integrating SR virtual scenes with AVM real scenes (such as overlaying lane line prediction information displayed by SR visualization with obstacles displayed by AVM visualization), the cognitive load on drivers can be reduced, creating an immersive experience of "what you see is what you get".

[0067] Optionally, blind spot risks (such as the risk of a child suddenly running into the image) can be marked in the fused image to transform the safety features of the Advanced Driving Assistance System (ADAS) into value points that users can perceive, supporting the upward pricing of the vehicle.

[0068] In summary, the technical solution provided in this application retains the image content represented by the first image or the second image outside the preset display range, and fuses the image content represented by the first image and the second image within the preset display range. Thus, while utilizing the mutual complementarity of the perception range of SR and AVM to ensure the comprehensiveness of the perceived driving information, it can avoid the repeated visualization of the same driving information perceived by SR and AVM, thereby avoiding redundancy and clutter in the perspective of driving information. This can reduce the risk of key driving information being obscured and significantly reduce the cognitive load on the driver.

[0069] In an optional embodiment, the method further includes:

[0070] Detect whether the vehicle meets the preset activation conditions;

[0071] When the vehicle is detected to meet the preset activation conditions, the fused image is displayed through the vehicle's display device;

[0072] If the vehicle is detected to not meet the preset activation conditions, the first image or the second image is displayed on the vehicle's display device.

[0073] In this embodiment, by setting preset activation conditions, the fusion rendering function of SR and AVM can be automatically turned on and off according to the specific scene in which the vehicle is located.

[0074] When the vehicle does not meet the preset activation conditions, the fusion rendering function of SR and AVM is automatically turned off. At this time, only the current frame image (i.e., the first image) rendered using SR (and map, such as SD MAP) or the current frame image (i.e., the second image) rendered using AVM is displayed on the vehicle's display device (such as the in-vehicle central control screen, instrument screen or AR-HUD). This can reduce the computational overhead and ensure display efficiency.

[0075] When the vehicle meets the preset activation conditions, the fusion rendering function of SR and AVM is automatically activated. At this time, the fused image obtained based on the first image and the second image is displayed on the display device to ensure the comprehensiveness of driving information.

[0076] Optionally, the preset activation condition includes at least one of the following:

[0077] The target object was detected within a preset range where the vehicle was located, such as a static obstacle, a moving pedestrian, or a low object.

[0078] When the vehicle speed is less than the set speed and the turn signal is turned on, for example, when the vehicle speed is less than 15 km / h and the turn signal is turned on.

[0079] The vehicle is in a parked state.

[0080] In an optional embodiment, the image content after fusing the first image and the second image within a preset display range includes at least one of the following:

[0081] For the first driving information, the first image is the image content visualized within the preset display range; and for the second driving information, the second image is the image content visualized within the preset display range, wherein the second driving information is different from the first driving information. For example, the image content of the first image included in the preset display range of the fused image may be a 3D scene rendered by SR (and map, such as SD Map), and the image content of the second image included in the preset display range of the fused image may be the image content rendered by AVM for driving information not perceived by SR (and map, such as SD Map) (corresponding to the second driving information, such as static obstacles, moving objects, etc.).

[0082] For the same driving information, the first image or the second image visualizes the image content within the preset display range; for example, within the preset display range, the fused image may include SR (and map, such as SD Map) or AVM rendering of the image content for the same obstacle.

[0083] Optionally, the image content included in the fused image can be flexibly adjusted according to the vehicle's driving state (which can be used to characterize different scenarios in which the vehicle is located).

[0084] For example, when a vehicle passes a pedestrian crossing, the fused image (such as the image content of the fused image within a preset display range) may include a 3D road scene map rendered by SR (and a map, such as SD Map) (including building models and road models, etc.) and pedestrian images rendered by AVM, so that the fused image can accurately describe moving pedestrians on the road based on AVM, thereby improving driving safety.

[0085] For example, when the vehicle is driving normally, the fused image (such as the image content of the fused image within a preset display range) may include the image content rendered by SR (and map, such as SD Map) for buildings and roads, so that the fused image can provide a more comprehensive description of the buildings and roads around the vehicle based on SR (and map, such as SD Map), thereby improving the driving experience.

[0086] For example, when a vehicle is in a parking scenario, the fused image (such as the image content of the fused image within a preset display range) may include the image content rendered by AVM for the ground, so that the fused image can accurately describe the ground of the parking lot based on AVM, thereby improving the parking experience.

[0087] Optionally, in response to detecting that the vehicle's driving state conforms to a first preset state (e.g., parking, low-speed driving, or a target object detected within a preset range of the vehicle), the fused image includes obstacles visualized in the second image within the preset display range. These obstacles are those not perceived by the SR (due to the lack of a pre-set model), ensuring an accurate description of obstacles surrounding the vehicle. Optionally, the target object includes at least one of static obstacles, moving pedestrians, and low-lying objects.

[0088] Optionally, in response to detecting that the vehicle's driving state conforms to a second preset state (such as the vehicle passing through a turning intersection or pedestrian crossing), the fused image includes the moving objects visualized in the second image within the preset display range, so as to achieve enhanced feedback on moving objects around the vehicle.

[0089] For example, refer to Figure 4The diagram illustrates a near-obstacle-triggered fusion scenario. When a target object is detected within a preset range of the vehicle, such as a static obstacle within 1.0 meter (adjustable from 0.3 to 1.0 meter), a moving pedestrian within 3.0 meters (adjustable from 1.0 to 3.0 meters), or a low-lying object within 0.5 meters, it indicates the vehicle is in a near-obstacle-triggered fusion scenario. In this case, the obstacle visualized in the second image within the preset display range supplements the obstacle information visualized in the first image within the same preset display range. This allows AVM imagery to supplement details of physical world obstacles that SR (and maps, such as SDMap) cannot perceive, making driving decisions easier for the user.

[0090] For example, refer to Figure 5 The diagram illustrates a scenario where a vehicle is turning at an intersection (e.g., when the vehicle's speed is detected to be less than a set speed and the turn signal is activated, it can be determined that the vehicle is turning at the intersection) and / or crossing a pedestrian crossing. The second image, within the preset display range, visualizes moving objects, replacing the first image's visualized moving objects within the preset display range. For example, AVM images are used to display moving pedestrians in a 3D scene rendered by SR (and maps, such as SD Map). This enhances driving safety by combining AVM images with the scene rendered by SR (and maps, such as SD Map) to provide stronger feedback on moving objects in the physical world.

[0091] For example, refer to Figure 6 The diagram illustrates an automated parking fusion scenario. When a vehicle is in a parking state, it can be determined that the vehicle is in an automated parking fusion scenario. At this time, the obstacles visualized in the second image within the preset display range are used to supplement the obstacles visualized in the first image within the preset display range. This allows AVM images to supplement the details of physical world obstacles that cannot be perceived by SR (and maps, such as SD Map) in parking lots with complex environments, thereby enhancing the sense of control and trust in parking and improving the user's parking experience.

[0092] In an optional embodiment, fusing the first image and the second image includes:

[0093] Adjust the image content of the first image within the first target range and the image content outside the first target range to different transparency levels, and / or adjust the image content of the second image within the second target range and the image content outside the second target range to different transparency levels;

[0094] The first image and the second image are superimposed according to a preset superposition order.

[0095] Optionally, at least one of the first target range, the second target range, and the preset overlay order is determined based on at least one of the user instruction, the vehicle's driving status, and the preset display range.

[0096] Taking the second target range as the preset display range as an example, the transparency of the image content of the second image within the preset display range can be adjusted to 1 so that the image content within the preset display range is fully displayed; and the transparency of the image content of the second image outside the preset display range can be adjusted to 0 so that the image content outside the preset display range is completely transparent; finally, the second image is superimposed on the first image to obtain a fused image, thereby enabling the fused image to include the image content of the first image outside the preset display range, as well as the image content of the second image within the preset display range.

[0097] In this embodiment, image fusion is achieved by adjusting transparency and combining image overlay. By introducing a clear range division (such as a first target range and a second target range), the degree of influence between different images within the corresponding range can be effectively controlled. For example, in the example above, the interference of AVM images outside the preset display range on SR (and maps, such as SD MAP) images can be effectively avoided.

[0098] Optionally, before superimposing the first image and the second image, the first image and / or the second image are fully distorted to ensure the clarity of the image (such as the image rendered after a viewpoint switch) and to avoid image distortion.

[0099] Optionally, an image within a preset display range can be cropped from the fused image and output to a display device for display.

[0100] In an optional embodiment, the method further includes:

[0101] Based on the vehicle's driving status (such as road conditions and vehicle speed, which can be determined based on the detection results of various onboard sensors to reflect the vehicle's location), the preset display range is adjusted, for example, by adjusting the shape, size, and angle range of the preset display range; or,

[0102] Based on the vehicle's driving status, target information is sent to the vehicle's SR and AVM, the target information indicating at least one of the following: the required image dimension to be rendered, the lens information for image rendering, and the viewpoint information for image rendering.

[0103] The image dimension can be two-dimensional or three-dimensional, the lens information can include the lens position, and the viewpoint information can include the viewpoint direction (such as top view, left / right view, front view and default view), the viewpoint size (such as the magnification or reduction factor required for the viewpoint, i.e. the distance required to zoom in or out), the position of the viewpoint, and the position to be focused on.

[0104] For example, when it is necessary to switch the fused image displayed by the vehicle to a normal view (i.e., the default view), after sending the corresponding target information to the SR and AVM, the system receives a first image rendered by the SR (and a map, such as an SD Map) based on its own default view, and a second image rendered by the AVM based on its own default view. During the fusion phase, the original viewpoints of the first and second images are maintained, and the second image is overlaid on the first image after adjusting its transparency, so as to preserve the image content rendered by the AVM within a preset display range, thereby using the AVM image to emphasize and provide feedback on driving information within the preset display range.

[0105] When it is necessary to switch the fused image displayed on the vehicle to a top-down view, after sending the corresponding target information to the SR (Surveillance Renderer) and AVM (Autonomous View Renderer), the system receives a first image rendered by the SR (and a map, such as an SD Map) based on the top-down view, and a second image (a two-dimensional image) rendered by the AVM based on the same top-down view. During the fusion phase, the transparency of the second image can be adjusted and superimposed on the first image to preserve the image content rendered by the AVM within a preset display range, so as to use the AVM's two-dimensional real-scene imagery to emphasize and provide feedback on driving information within the preset display range.

[0106] When it's necessary to switch the fused image displayed on the vehicle to a forward-facing fused state, after sending target information to the SR (Signal Renderer) and AVM (Autonomous Rendering Machine), the SR (and the map, such as SD Map) zooms in on its own viewpoint based on the target information, simultaneously and quickly zooms in to the front of the vehicle, and changes its viewpoint to eye level to obtain the SR forward-facing view (i.e., the first image); the AVM renders its forward-facing AVM view based on the target information to obtain the second image. During the fusion phase, after fully correcting the distortion of the forward-facing AVM view, it is then fused with the SR forward-facing view to achieve a reasonable visualization of driving information in the area near the front of the vehicle.

[0107] When it's necessary to switch the fused image displayed by the vehicle to a left-facing view, after sending target information to the SR (and map, such as SD Map), the SR (and map, such as SD Map) magnifies its own viewpoint based on the target information, simultaneously rotates clockwise to the left side of the vehicle, and changes its viewpoint to eye level to obtain the SR left-facing view (i.e., the first image); the AVM renders the left-facing AVM view based on the target information to obtain the second image. During the fusion phase, after fully correcting the distortion of the left-facing AVM view, it is fused with the SR left-facing view. During the fusion process, the vehicle model is set to transparent, and ground information is visualized using the SR (and map, such as SD Map). This allows for a reasonable visualization of driving information in the area near the left side of the vehicle.

[0108] When it's necessary to switch the fused image displayed by the vehicle to a right-facing view, after sending target information to the SR (and map, such as SD Map), the SR (and map, such as SD Map) magnifies its own viewpoint based on the target information, simultaneously rotates counter-clockwise to the right side of the vehicle, and changes its viewpoint to eye level to obtain the SR right-facing view (i.e., the first image); the AVM renders the right-facing AVM view based on the target information to obtain the second image. During the fusion phase, after fully correcting the distortion of the right-facing AVM view, it is fused with the SR right-facing view. During the fusion process, the vehicle model is set to be transparent, and ground information is visualized using the SR (and map, such as SD Map). This allows for a reasonable visualization of driving information in the area near the right side of the vehicle.

[0109] When it is necessary to switch the fused image displayed by the vehicle to a close-up view, after sending target information to the SR (Surveillance Renderer) and AVM (Autonomous View Renderer), the SR (and map, such as SD Map) and AVM zoom in on the viewpoint according to the target information, focus on the preset display range, and render a first image and a second image (which are 3D images) that conform to their default viewpoints. During the fusion stage, the transparency of the second image is adjusted and superimposed on the first image to retain the image content rendered by the AVM within the preset display range, so as to use the AVM's 3D real-world imagery to emphasize and provide feedback on driving information within the preset display range.

[0110] When it's necessary to switch the fused image displayed on the vehicle to a left rear view, after sending target information to the SR (Side Mirror) and AVM (Autonomous View Monitor), the SR (and the map, such as SD Map) zooms out and shrinks the viewpoint based on the target information, simultaneously rotating clockwise by a certain angle (e.g., about 100 degrees), and zooms the camera to the rearview mirror position (the specific coordinates can be flexibly determined) to obtain the SR left-facing view (i.e., the first image). The AVM renders the left-facing AVM view based on the target information to obtain the second image. During the fusion stage, the left-facing AVM view is fully distorted and then fused with the SR left-facing view. An image within a preset display range is cropped from the fused image and displayed through a display device. This allows for a reasonable visualization of driving information in the area near the left rear of the vehicle.

[0111] When it's necessary to switch the fused image displayed on the vehicle to a right-side rear view, after sending target information to the SR (Side View Controller) and AVM (Auto View Controller), the SR (and the map, such as SD Map) zooms out and shrinks the viewpoint based on the target information, simultaneously rotating counterclockwise by a certain angle (e.g., about 100 degrees), and zooms the camera to the rearview mirror position (the specific coordinates can be flexibly determined) to obtain the SR right-side view (i.e., the first image). The AVM renders the right-side AVM view based on the target information to obtain the second image. During the fusion stage, the right-side AVM view is fully distorted and then fused with the SR right-side view. An image within a preset display range is cropped from the fused image and displayed through a display device. This allows for a reasonable visualization of driving information in the area near the right rear of the vehicle.

[0112] Optionally, the perspective fusion scheme used by the vehicle is determined according to the vehicle's driving status, and the perspective information of the fused image to be displayed by the vehicle is determined accordingly. The perspective information may include information such as perspective direction, perspective size, and perspective position.

[0113] For example, when the vehicle uses a default view fusion scheme (which is used to obtain a fused image under the default view), the preset display range is a circular area centered at the center point of the vehicle's rear and conforming to a set radius. For example, the fused image under the default view fusion scheme can be referenced... Figure 7 As shown, the value range of the set radius can be 3.13 meters to 6 meters;

[0114] For example, when the vehicle uses a top-down view fusion scheme (used to obtain a fused image from a top-down perspective), the preset display range is an area of ​​a first predetermined size where the vehicle is located, such as a circular area of ​​a predetermined radius where the vehicle is located. Exemplarily, the fused image under the top-down view fusion scheme can refer to... Figure 8 As shown;

[0115] For example, when the vehicle uses a forward-view fusion scheme (used to obtain a fused image from a forward view), the preset display range is an area that includes the area in front of the vehicle's front end and conforms to a second preset size. Exemplarily, the fused image under the forward-view fusion scheme can refer to... Figure 9 As shown;

[0116] For example, when the vehicle uses a left-view fusion scheme (used to obtain a fused image from a left-view perspective), the preset display range is an area that includes the left side of the vehicle and conforms to a third preset size. Exemplarily, the fused image under the left-view fusion scheme can refer to... Figure 10 As shown;

[0117] For example, when the vehicle uses a right-view fusion scheme (used to obtain a fused image from a right-view perspective), the preset display range is an area that includes the right side of the vehicle and conforms to a fourth preset size. Exemplarily, the fused image under the right-view fusion scheme can refer to... Figure 11 As shown;

[0118] For example, if no new moving objects or static obstacles are detected around the vehicle (and the vehicle uses a close-up fusion scheme to obtain a fused image from a close-up perspective), the preset display range is a fan-shaped area centered on the vehicle's center point and conforming to a first angle range. For instance, the fused image under the close-up fusion scheme can refer to... Figure 12 As shown.

[0119] Optionally, when a new moving object or static obstacle is detected around the vehicle (and the vehicle uses a near-field fusion scheme), the preset display range is a fan-shaped area centered on the vehicle's center point and conforming to a second angle range. The second angle range includes and is greater than the first angle range, thereby increasing the angle range of the fan-shaped area. This provides users with richer visual information when a new moving object or static obstacle appears, making it easier for the driver to avoid the newly appearing moving object or static obstacle, thus improving driving safety. For example, the first angle range can be 120° to 180°, the second angle range can be 120° to 360°, and the radius of the fan-shaped area can range from 3.13 meters to 6 meters.

[0120] Optionally, when the vehicle uses a left rear-view fusion scheme (which is used to obtain a fused image from the left rear view), the preset display range is an area that includes the left rear region of the vehicle and conforms to a fifth preset size. For example, the fused image under the left rear-view fusion scheme can be referenced... Figure 13As shown;

[0121] Optionally, when the vehicle uses a right-side rear-view fusion scheme (which is used to obtain a fused image from the right-side rear view), the preset display range is an area that includes the right rear region of the vehicle and conforms to a sixth preset size. For example, the fused image under the right-side rear-view fusion scheme can be referenced... Figure 14 As shown.

[0122] In an optional embodiment, the first image is the current frame image obtained by SR arranging and rendering a preset 3D model based on perception data and by standard definition map SD MAP arranging and rendering a preset 3D building model based on road surrounding data.

[0123] The pre-set 3D model includes at least one of the following: vehicle model, adjacent vehicle model (such as sedan, SUV, truck, ambulance, motorcycle, electric vehicle and bicycle model), pedestrian model, road model (such as lane, lane line, parking space, lane marking model), facility model (such as toll station, newsstand, trash can, cone, guardrail, lawn and tree model), and traffic sign model (such as speed limit sign, traffic light, parking lot, no-parking zone, gas / charging station and highway sign). The pre-set 3D building model includes at least one of the following: residential building model, office building model, shop building model, city landmark building model, park building model, and bridge building model.

[0124] In practical implementation, SR (Sensitive Surface View) performs real-time placement and rendering of the 3D model based on perception data; AVM (Ambient Visualization View) collects real-time image data of the physical world through cameras (typically capturing any object in the vehicle's surrounding environment), thus effectively supplementing the 3D model pre-set in SR; SD Map (Site Map) performs real-time placement and rendering of the 3D building model based on data from the surrounding roads. By merging and rendering the image content displayed by SR, AVM, and maps (such as SD Map) in real time on the same display device, the complementary perception ranges of the three products enable blind-spot-free environmental perception, making it more intelligent and reducing the need for multiple applications (APPs) to combine information, thus reducing the user's decision-making burden. For example, the perception range distribution and perception distance reference of SR, AVM, and SD Map are shown below. Figure 15 and Figure 16 As shown.

[0125] Understandably, under the influence of the trend of one-shot three-dimensional scenes and the combination of virtual and reality, the fusion rendering scheme of SR, AVM and map (such as SD MAP) provided in this application embodiment (also known as IntelliSphere Reality) can effectively enhance the visual impact of the combination of virtual and reality, thereby greatly enhancing product competitiveness.

[0126] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0127] Please refer to Figure 17 This diagram illustrates the architecture of an image processing system according to an embodiment of this application. The system is used to implement the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application. The system includes an AVM module, an image rendering engine, and a frame buffer object (FBO). The FBO is communicatively connected to both the AVM module and the image rendering engine, wherein:

[0128] The AVM module is configured to render consecutive frame images based on AVM and transmit the rendered current frame image to the FBO for storage;

[0129] The image rendering engine is configured to render consecutive frame images based on SR, and to fuse the current frame image stored in the FBO with the current frame image rendered based on SR to obtain a fused image; the image content of the fused image includes: the image content of the current frame image rendered by the AVM module or the current frame image rendered based on SR outside the preset display range, and the image content of the current frame image rendered by the AVM module and the current frame image rendered based on SR fused within the preset display range.

[0130] In the technical solution provided in this application embodiment, the image rendering engine retains the image content represented by the current frame image rendered by itself or the AVM module outside the preset display range, and fuses the image content represented by the current frame image rendered by itself and the AVM module within the preset display range. Thus, while utilizing the mutual complementarity of the perception range of SR and AVM to ensure the comprehensiveness of the perceived driving information, it can avoid the repeated visualization of the same driving information perceived by SR and AVM, thereby avoiding redundancy and clutter in the perspective of driving information. This can reduce the risk of key driving information being obscured and significantly reduce the cognitive load of the driver.

[0131] In an alternative embodiment, the image rendering engine can also be configured to render consecutive frame images based on SR and maps (such as SDMAP).

[0132] In an alternative embodiment, the image rendering engine may also be configured to output the blended image to a display device in the vehicle, so that the display device can display the blended image to a user.

[0133] In an optional embodiment, the image rendering engine can also be configured to use the image rendering shader material ShaderMaterial class to overlay material effects on the current frame image rendered by the AVM module based on the material parameters of the current frame image rendered by itself.

[0134] In this embodiment, the AVM module uses its own rendered current frame image (such as multi-camera textures) as the input source for the image rendering ShaderMaterial class. This allows for dynamic combination of the image rendering material parameters (such as reflectivity and transparency) via UV mapping, thereby overlaying image rendering material effects (such as dynamic specular highlights and ambient occlusion) onto the AVM-rendered image to enhance the 3D realism of the panoramic view. Furthermore, the image rendering engine overlays material effects onto the current frame image rendered by the AVM module based on the material parameters of its own rendered current frame image, improving the integration of the AVM image and enabling seamless fusion with 3D vehicle models, HUDs, and other image rendering elements.

[0135] Please refer to Figure 18 This illustrates an architecture diagram of an image processing system provided in another embodiment of this application. The AVM module includes an AVM algorithm unit and an AVM data source. The AVM data source is communicatively connected to both the AVM algorithm unit and the image rendering engine, wherein:

[0136] The AVM algorithm unit is configured to render consecutive frame images based on AVM and transmit the rendered current frame image to the AVM data source;

[0137] The AVM data source is configured to transmit the current frame image it receives to the image rendering engine.

[0138] In practice, the AVM algorithm unit executes AVM algorithm-related logic using a pre-configured AVM algorithm library (LIB) to generate an AVM video stream (i.e., consecutive frame images). Optionally, the AVM algorithm unit also supports users integrating other related algorithms into the AVM algorithm library according to their actual needs.

[0139] In this embodiment, the AVM algorithm unit and the image rendering engine communicate loosely through the AVM data source (DataSource), thereby ensuring the system's interactive scalability.

[0140] Optionally, the AVM algorithm unit is obtained by encapsulating the AVM algorithm module as a native plugin in the image rendering engine, so that it shares the same OpenGL rendering context with the image rendering engine.

[0141] In this embodiment, the AVM algorithm module is deeply integrated as a native plugin into the image rendering engine to achieve sharing of OpenGL resources (i.e., OpenGL rendering context, also known as OpenGL context or OpenGL Context) between the AVM algorithm module and the image rendering engine. This eliminates the overhead of multiple context switching and improves the execution efficiency of the graphics pipeline. Tests show that by sharing the OpenGL context, GPU memory copy overhead can be reduced by more than 30%, and the frame rate can be increased to over 60 frames per second (FPS).

[0142] Optionally, the AVM data source is established based on the data source (DataSource) data-driven architecture of the image rendering engine;

[0143] The AVM data source can also be configured to provide a dynamic view control interface to the user, and upon receiving a view switching command from the user through the dynamic view control interface, send target information to the AVM algorithm unit and the image rendering engine. The target information is used to indicate at least one of the following: the image dimension to be rendered, the lens information for image rendering, and the view information for image rendering.

[0144] In this embodiment, the AVM data source is constructed using the data-driven architecture unique to the image rendering engine. This enables the AVM data source to provide a dynamic view control interface to the user to receive view switching commands from the user. In turn, it supports the user to control the vehicle to switch views through UI events (such as touch, voice commands, etc.) so as to achieve a smooth view transition for the fused image.

[0145] Optionally, the image rendering engine includes an Output Texture module, an OpenGL rendering context, and a memory block (Swapbuffer), wherein:

[0146] The output texture module is configured to render consecutive frame images based on SR, and to fuse the current frame image rendered by the AVM module with the current frame image rendered based on SR to obtain a fused image, and to transmit the fused image to a memory block.

[0147] The memory block is configured to submit the fused image it receives to the vehicle's display device for display.

[0148] The OpenGL rendering context is configured to support rendering operations of the AVM module and the output texture module.

[0149] In practice, the output texture module uses pre-configured SR (and map, such as SD MAP) related plugins to render images related to 2D or 3D radar perception data, 2D or 3D vehicle models, and UI interfaces.

[0150] Optionally, the image rendering engine and the AVM module can also be configured to render their current frame images sequentially according to a set rendering order to share the same rendering pipeline. This staged rendering approach avoids pipeline conflicts, ensuring that the shader, material, and lighting calculations of the image rendering engine and the AVM module are not interfered with. Furthermore, by synchronously processing AVM data using the SR rendering pipeline, the need for independent applications can be reduced, resulting in a 8-12% reduction in development time and costs.

[0151] Optionally, the image rendering engine can also be configured to transmit the current frame image rendered by itself to the FBO for storage, and then fuse the different current frame images stored in the FBO (i.e., the current frame images rendered by the image rendering engine and the AVM module respectively) to obtain the fused image.

[0152] Optionally, the image rendering engine can also be configured to render the current frame image in another pre-created FBO (i.e., another FBO different from the FBO used to store the current frame image), and to perform a frame clearing operation to clear the other FBO before rendering the current frame image.

[0153] Optionally, the AVM module can also be configured to render the current frame image in the other FBO, and to perform a clear frame operation to clear the other FBO before rendering the current frame image itself.

[0154] For example, to set the rendering order to render the SR (Simulation Rendering) screen first and then the AVM (Aspect-to-Virtual Rendering) screen, please refer to [link / reference]. Figure 19 This diagram illustrates a real-time rendering timing diagram of a system according to an embodiment of this application. The processing flow for consecutive frames may include: 1. Clear Screen; 2. SR (Screen Representation) rendering; 3. Cache to FBO (Frame-Based Object); 4. Clear Screen; 5. AVM (Aspect-Modulated Rendering); 6. Blending rendering; 7. Output frame (i.e., submit the blended image). The processing flow for each single frame (e.g., the current frame image) is as follows:

[0155] 1) Independent rendering stage of image rendering engine

[0156] The image rendering engine performs a frame clearing operation to clear the first frame blank (FBO).

[0157] The image rendering engine completes the drawing of 2D / 3D images in the first FBO according to its standard rendering pipeline to achieve SR image rendering, ensuring the correctness of its rendering logic such as materials, lighting, and shaders.

[0158] When the image rendering engine finishes rendering, the rendered current frame image is stored in the second FBO for subsequent fusion.

[0159] 2) AVM rendering stage

[0160] The AVM module performs a clear frame operation to clear the first FBO.

[0161] The AVM module executes the AVM algorithm rendering process in the first FBO to generate a panoramic view.

[0162] 3) Image blending and alignment

[0163] The image rendering engine fuses the panoramic view in the first FBO with the current frame image stored in the second FBO (i.e., the current frame image rendered by the image rendering engine) to obtain a fused image.

[0164] During the blending process, the image rendering engine can use additive blending mode, as well as independent control of color channels and alpha channels, to ensure the correct overlay of different images, thereby achieving a precise blending effect.

[0165] For example, first define an addition: chroma value: SR color value and AVM color value; transparency: SR transparency + AVM transparency. Then, define a multiplication of the color itself (overlaying the effect of the previous step): chroma value: ,transparency: ,in, This indicates the output result of the facial reflection value. This indicates the output result of transparency. Indicates the SR color value. Represents AVM color values. Indicates SR transparency. This refers to the blending effect of AVM transparency, opacity, and translucency. Figure 20 and Figure 21 As shown.

[0166] 4) Final Output

[0167] After fusion is complete, the image rendering engine submits the fused image to the display device to form a complete AVM+UI interactive interface.

[0168] Based on the above embodiments, this application provides an image processing system that integrates an image rendering engine and an AVM module to achieve fused rendering of AVM, SR (and maps, such as SD MAP), and supports functions such as multi-view switching, dynamic material fusion, and OpenGL resource sharing. Furthermore, through frame buffer object (FBO) management and multi-stage rendering, it can ensure the correct overlay display of images rendered by SR (and maps, such as SD MAP) and images rendered by AVM.

[0169] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0170] Please refer to Figure 22 This diagram illustrates a block diagram of an image processing apparatus according to an embodiment of this application. The apparatus has the functionality to implement the method example described above; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus 220 may include:

[0171] The communication module 221 is used to acquire a first image and a second image, wherein the first image is a current frame image rendered using the vehicle's surround view perception system (SR), and the second image is a current frame image rendered using the vehicle's panoramic surround view monitoring system (AVM).

[0172] Processing module 222 is used to fuse the first image and the second image to obtain a fused image. The image content of the fused image includes: the image content of the first image or the second image outside a preset display range, and the image content of the first image and the second image after fusion within the preset display range.

[0173] In summary, the technical solution provided in this application retains the image content represented by the first image or the second image outside the preset display range, and fuses the image content represented by the first image and the second image within the preset display range. Thus, while utilizing the mutual complementarity of the perception range of SR and AVM to ensure the comprehensiveness of the perceived driving information, it can avoid the repeated visualization of the same driving information perceived by SR and AVM, thereby avoiding redundancy and clutter in the perspective of driving information. This can reduce the risk of key driving information being obscured and significantly reduce the cognitive load on the driver.

[0174] Optionally, the processing module 222 is further configured to perform the following steps:

[0175] Detect whether the vehicle meets the preset activation conditions;

[0176] When the vehicle is detected to meet the preset activation conditions, the fused image is displayed through the vehicle's display device;

[0177] If the vehicle is detected to not meet the preset activation conditions, the first image or the second image is displayed on the vehicle's display device.

[0178] Optionally, the processing module 222 is further configured to perform the following steps:

[0179] Adjust the image content of the first image within the first target range and the image content outside the first target range to different transparency levels, and / or adjust the image content of the second image within the second target range and the image content outside the second target range to different transparency levels;

[0180] The first image and the second image are superimposed according to a preset superposition order.

[0181] Optionally, the image content after fusing the first image and the second image within a preset display range includes at least one of the following:

[0182] For the first driving information, the image content visualized by the first image within the preset display range; and for the second driving information, the image content visualized by the second image within the preset display range, wherein the second driving information is different from the first driving information;

[0183] For the same driving information, the first image or the second image is the image content visualized within the preset display range.

[0184] Optionally, the processing module 222 is further configured to adjust the preset display range according to the driving status of the vehicle.

[0185] Optionally, the processing module 222 is further configured to send target information to the vehicle's SR and AVM according to the vehicle's driving state, wherein the target information is used to indicate at least one of the following: the required image dimension to be rendered, the lens information for image rendering, and the viewpoint information for image rendering.

[0186] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0187] This application also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the image processing method disclosed in this application.

[0188] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the image processing method disclosed in this application.

[0189] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the image processing method disclosed in this application.

[0190] This application also provides a vehicle that includes the electronic device or the image processing system.

[0191] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0192] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0193] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, systems, devices, storage media, and program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0194] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0196] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0197] The above provides a detailed description of an image processing method, system, apparatus, medium, device, and vehicle provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a first image and a second image, the first image being a current frame image of a surrounding environment of a vehicle rendered by a surrounding display perception system SR with a first perception range of the vehicle, the first image corresponding to a SR virtual scene; the second image being a current frame image of the surrounding environment of the vehicle rendered by a surround view monitoring system AVM with a second perception range of the vehicle, the second image corresponding to an AVM real scene; the first perception range covering the second perception range; fusing the first image and the second image to obtain a fused image of the SR virtual scene and the AVM real scene, image content of the fused image comprising: image content of the first image or the second image outside a preset display range, and image content of the first image and the second image fused within the preset display range.

2. The method of claim 1, wherein, The method further comprises: detecting whether the vehicle meets a preset activation condition; in a case where it is detected that the vehicle meets the preset activation condition, displaying the fused image by a display device of the vehicle; in a case where it is detected that the vehicle does not meet the preset activation condition, displaying the first image or the second image by the display device of the vehicle.

3. The method of claim 1, wherein, The fusing of the first image and the second image comprises: adjusting image content of the first image within a first target range and image content of the first image outside the first target range to different transparencies, and / or adjusting image content of the second image within a second target range and image content of the second image outside the second target range to different transparencies; superimposing the first image and the second image according to a preset superimposition order.

4. The method according to any one of claims 1 to 3, characterized in that, The image content of the first image and the second image fused within the preset display range comprises at least one of: for first driving information, image content of the first image visualized within the preset display range, and for second driving information, image content of the second image visualized within the preset display range, the second driving information being different from the first driving information; for the same driving information, image content of the first image or the second image visualized within the preset display range.

5. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: adjusting the preset display range according to a driving state of the vehicle; or sending target information to the SR and the AVM of the vehicle according to the driving state of the vehicle, the target information being used to indicate at least one of: an image dimension required to be rendered, lens information used for image rendering, and view angle information used for image rendering.

6. An image processing system, characterized by The system comprises an AVM module, an image rendering engine, and a frame buffer object FBO, the FBO being in communication connection with the AVM module and the image rendering engine respectively, wherein: the AVM module is configured to perform rendering of a current frame image of a surrounding environment of a vehicle based on an AVM with a second perception range, and transmit the rendered current frame image to the FBO for storage, the current frame image rendered by the AVM corresponding to an AVM real scene; The image rendering engine is configured to render a continuous frame image of the vehicle surrounding environment based on the SR with the first perception range, and fuse the current frame image stored in the FBO and the current frame image rendered based on the SR to obtain a fusion image of the SR virtual scene and the AVM real scene fusion; the current frame image rendered by the SR corresponds to the SR virtual scene, and the first perception range covers the second perception range; image content of the fusion image includes image content of the current frame image rendered by the AVM module or the current frame image rendered based on the SR outside a preset display range, and image content of the current frame image rendered by the AVM module and the current frame image rendered based on the SR after fusion within the preset display range.

7. The system of claim 6, wherein, The image rendering engine and the AVM module are further configured to render the current frame image of itself in sequence to share the same rendering pipeline according to a set rendering sequence.

8. An image processing apparatus characterized by comprising: The device comprises: The communication module is configured to obtain a first image and a second image, the first image being a current frame image of a vehicle surrounding environment rendered by a surrounding display perception system (SR) with a first perception range of the vehicle, the first image corresponding to an SR virtual scene; the second image being a current frame image of the vehicle surrounding environment rendered by an all-around view monitoring system (AVM) with a second perception range of the vehicle, the second image corresponding to an AVM real scene; the first perception range covering the second perception range; The processing module is configured to fuse the first image and the second image to obtain a fusion image of the SR virtual scene and the AVM real scene fusion, image content of the fusion image including image content of the first image or the second image outside a preset display range, and image content of the first image and the second image after fusion within the preset display range.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction for being executed by the processor to implement the image processing method in any one of claims 1-5.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the image processing method in any one of claims 1-5 when executing the program.

11. A vehicle characterized by comprising: The vehicle comprises the electronic device in claim 10, or the vehicle comprises the image processing system in any one of claims 6-7.

Images