Image processing method, system and device, medium, equipment and vehicle

Abstract

The embodiment of the invention discloses an image processing method, system and device, a medium, equipment and a vehicle. The problem that driving information is redundant and disordered visually can be solved. The method comprises the steps that a first image and a second image are acquired, the first image is a current frame image rendered by using a surrounding display perception system SR of a vehicle, and the second image is a current frame image rendered by using a panoramic view monitoring system AVM of the vehicle; the first image and the second image are fused to obtain a fused image, and the image content of the fused image comprises the image content of the first image or the second image outside a preset display range and the image content of the first image and the second image fused in the preset display range.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to the Chinese patent application filed with the China Patent Office on May 9, 2025, with application number: 202510594954.8, entitled "A Method, System, Device, Medium, Equipment and Vehicle for Image Processing", the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of graphics rendering technology, and in particular to an image processing method, system, apparatus, medium, equipment and vehicle. Background Art

[0003] Surround reality technology refers to the perception and visualization of driving information such as traffic participants and road facilities around the vehicle based on the vehicle's onboard perception capabilities. It can provide a visual decision-making basis for high-level autonomous driving functions and assist the driver in observing the vehicle's surroundings, thereby improving driving safety.

[0004] In related technologies, the real-life video stream rendered by the Around View Monitor (AVM) system is mainly superimposed in the form of an independent layer on the scene rendered by the Surrounding Reality (SR) system to achieve a visual display of driving information.

[0005] However, this double-layer image processing method can easily cause redundancy and clutter in the driving information perspective, which in turn leads to the risk of blocking key driving information and increases the driver's cognitive load. Summary of the Invention

[0006] The embodiments of the present application provide an image processing method, system, apparatus, medium, device, and vehicle that can solve the problem of visual redundancy and clutter in driving information. The technical solution is as follows: In a first aspect, an image processing method is provided, the method comprising: Acquire a first image and a second image, where the first image is a current frame image rendered using a surrounding perception system (SR) of the vehicle, and the second image is a current frame image rendered using an all-around view monitoring system (AVM) of the vehicle; The first image and the second image are fused to obtain a fused image, wherein the image content of the fused image includes: the image content of the first image or the second image outside a preset display range, and the image content after the first image and the second image are fused within the preset display range.

[0007] In a second aspect, an image processing system is provided, the system comprising an AVM module, an image rendering engine, and a frame buffer object FBO, wherein the FBO is communicatively connected to the AVM module and the image rendering engine, respectively, wherein: The AVM module is configured to render continuous frame images based on AVM and transmit the rendered current frame image to the FBO for storage; The image rendering engine is configured to render continuous frame images based on SR, and to fuse the current frame image stored in the FBO with the current frame image rendered based on SR to obtain a fused image; the image content of the fused image includes: the image content of the current frame image rendered by the AVM module or the current frame image rendered based on SR outside the preset display range, and the image content after the current frame image rendered by the AVM module and the current frame image rendered based on SR are fused within the preset display range.

[0008] According to a third aspect, an image processing apparatus is provided, the apparatus comprising: a communication module configured to acquire a first image and a second image, wherein the first image is a current frame image rendered using a surrounding perception system (SR) of the vehicle, and the second image is a current frame image rendered using an surround view monitoring system (AVM) of the vehicle; A processing module is used to fuse the first image and the second image to obtain a fused image, wherein the image content of the fused image includes: the image content of the first image or the second image outside a preset display range, and the image content after the first image and the second image are fused within the preset display range.

[0009] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the image processing method as described in the first aspect.

[0010] In a fifth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image processing method as described in the first aspect when executing the program.

[0011] In a sixth aspect, a vehicle is provided, wherein the vehicle includes the electronic device as described in the fifth aspect, or the vehicle includes the image processing system as described in the second aspect.

[0012] In the image processing method provided in the embodiment of the present application, by retaining the image content represented by the first image or the second image outside the preset display range, and fusing the image content represented by the first image and the second image within the preset display range, while utilizing SR and AVM to complement each other in the perception range to ensure the comprehensiveness of the perceived driving information, it is possible to avoid repeated visual display of the same driving information perceived by SR and AVM, thereby avoiding redundancy and clutter of the driving information in the perspective, thereby reducing the risk of key driving information being obscured and significantly reducing the driver's cognitive load. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0014] Figure 1 A schematic diagram of a driving information visualization display in related technology; Figure 2 A structural block diagram of an electronic device provided in an embodiment of the present application; Figure 3 A flowchart of an image processing method provided in an embodiment of the present application; Figure 4 A schematic diagram of a close-range obstacle-triggered fusion scenario provided in an embodiment of the present application; Figure 5 A schematic diagram of an intersection turn activation fusion scenario provided in an embodiment of the present application; Figure 6 A schematic diagram of an automatic parking fusion scenario provided in an embodiment of the present application; Figure 7 A schematic diagram of a fused image under a default perspective fusion solution provided in an embodiment of the present application; Figure 8 A schematic diagram of a fused image under a top-down perspective fusion solution provided in an embodiment of the present application; Figure 9 A schematic diagram of a fused image under a forward perspective fusion solution provided in an embodiment of the present application; Figure 10 A schematic diagram of a fused image under a left-view fusion solution provided in an embodiment of the present application; Figure 11 A schematic diagram of a fused image under a right-angle perspective fusion solution provided in an embodiment of the present application; Figure 12A schematic diagram of a fused image under a near-view fusion solution provided in an embodiment of the present application; Figure 13 A schematic diagram of a fused image under a left rear perspective fusion solution provided in an embodiment of the present application; Figure 14 A schematic diagram of a fused image under a right rear view fusion solution provided in an embodiment of the present application; Figure 15 A schematic diagram of the perception range distribution of SR, AVM, and SD Map provided in an embodiment of the present application; Figure 16 A schematic diagram of the perception distance of SR, AVM and SD Map provided in an embodiment of the present application; Figure 17 An architectural diagram of an image processing system provided in an embodiment of the present application; Figure 18 An architectural diagram of another image processing system provided in an embodiment of the present application; Figure 19 A real-time rendering timing diagram of a system provided in an embodiment of the present application; Figure 20 A schematic diagram of an opaque fusion effect provided in an embodiment of the present application; Figure 21 A schematic diagram of a translucent fusion effect provided in an embodiment of the present application; Figure 22 A block diagram of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0016] Please refer to Figure 1 The schematic diagram of the driving information visualization display in the related technology shown is shown in FIG. In the surround reality technology, the real-scene video stream obtained by AVM rendering is currently superimposed on the SR rendering scene in the form of an independent layer, for example Figure 1 As shown, a small AVM window is superimposed on the full-screen SR image to achieve a visual display of the driving information perceived by the SR and AVM.

[0017] However, this dual-layer image processing method can easily lead to redundant and cluttered driving information. This can lead to confusion in the visual priority of critical driving information (such as lane markings and obstacles), creating the risk of obstruction. This can also increase the driver's cognitive load and potentially delay decision-making in emergency situations. Furthermore, the spatial coordinate system deviations caused by the dual-layer image overlay can easily lead to inconsistencies in human-computer interaction logic, resulting in interaction conflicts (for example, the predicted path displayed in the SR image is difficult to correlate with the actual AVM scene), affecting the user experience.

[0018] Based on the above analysis, in response to the problem of redundancy and clutter in the perspective of driving information in related technologies, the embodiments of the present application provide an image processing method, system, device, medium, equipment and vehicle, which perform deep data fusion of the driving information perceived by SR and AVM to avoid repeated visual display of the same driving information perceived by SR and AVM. Compared with the image processing method of double-layer screen superposition, the present application can effectively avoid redundancy and clutter in the perspective of driving information, thereby reducing the risk of key driving information being obscured and significantly reducing the driver's cognitive load.

[0019] First, to facilitate understanding of the technical solutions provided by this application, the main technical concepts involved in the embodiments of this application are briefly described below.

[0020] SR: This intelligent driving visualization system leverages advanced in-vehicle perception systems and artificial intelligence (AI) algorithms to construct and dynamically render the 360-degree traffic environment surrounding the vehicle in real time through the fusion of multiple sensors (including cameras, millimeter-wave radar, and lidar). Leveraging high-precision 3D modeling technology, the system digitizes driving information such as road infrastructure, traffic participants (vehicles, pedestrians, and non-motorized vehicles), lane markings, and traffic signals in real time. This information is intuitively displayed on the vehicle's central control screen, instrument panel, or augmented reality head-up display (AR-HUD), providing the driver with a comprehensive understanding of road conditions and providing a visual basis for decision-making for advanced autonomous driving functions.

[0021] AVM, also known as a 360° panoramic imaging system, is a panoramic vision assistance technology used in automobiles. This technology uses onboard cameras to stitch together images from multiple perspectives, providing the driver with a 360° bird's-eye view of driving information. This helps the driver more clearly observe the vehicle's surroundings, improving safety during parking, low-speed driving, and narrow road traversal.

[0022] Standard Definition Map (SD MAP): A type of map relative to High Definition Map (HD Map), usually suitable for general navigation or basic geographic information query scenarios.

[0023] Image rendering engine: such as the Kanzi engine, which is mainly used to render 2D / 3D models, textures, lighting and other data into high-fidelity images in real time / offline based on computer graphics.

[0024] Please refer to Figure 2 , which shows a block diagram of an electronic device provided by an exemplary embodiment of the present application. This electronic device can be an electronic device capable of running applications, such as an in-vehicle central control console, instrument controller, smart cockpit, or multi-screen digital cockpit. The electronic device in this application may include one or more of the following components: a processor 110, a memory 120, and input / output devices 130.

[0025] The processor 110 may include one or more processing cores. The processor 110 utilizes various interfaces and circuits to connect various components within the electronic device. It executes instructions, programs, code sets, or instruction sets stored in the memory 120, as well as accesses data stored in the memory 120, to perform various functions of the electronic device and process data. Optionally, the processor 110 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 110 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 110 and may be implemented as a separate communication chip.

[0026] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the various method embodiments described below, and the like. The operating system may be an Android system (including systems deeply developed based on the Android system), an iOS system (including systems deeply developed based on the iOS system), or other systems. The data storage area may also store data (such as audio and video data) generated by the electronic device during use.

[0027] Please refer to Figure 3 , which shows a flow chart of an image processing method provided by one embodiment of the present application. The method can be applied to the electronic device described above. The method may include the following steps: Step S101: Acquire a first image and a second image, wherein the first image is a current frame image rendered using the vehicle's surrounding display perception system SR, and the second image is a current frame image rendered using the vehicle's panoramic surround view monitoring system AVM.

[0028] During specific implementation, a frame buffer object (FBO) is created in advance, and the first image received from the SR algorithm module and the second image received from the AVM algorithm module are stored in the FBO for subsequent fusion use.

[0029] Optionally, the first image is a current frame image rendered using an image rendering engine of the vehicle, and the image rendering engine is integrated with an SR algorithm module and a map (such as SD MAP) algorithm module.

[0030] Step S102: Fusing the first image and the second image to obtain a fused image, wherein the image content of the fused image includes: the image content of the first image or the second image outside a preset display range, and the image content after the first image and the second image are fused within the preset display range.

[0031] In specific implementation, the preset display range can be flexibly adjusted according to user instructions or the driving status of the vehicle. For example, the preset display range can be adjusted to the full (or partial) perception range of SR or AVM to support different interaction needs.

[0032] The image content represented by the second image within the preset display range is fused with the image content represented by the first image within the preset display range to merge the two sets of driving information visualized using SR and AVM into one set, thereby avoiding repeated visualization of the same driving information and effectively reducing the information pressure on the driver.

[0033] For example, for the image content rendered using SR and AVM for the same driving information (i.e., the same driving information displayed visually, such as the same obstacle displayed visually), only the image content rendered by SR can be retained, or only the image content rendered by AVM can be retained.

[0034] For example, for the image content rendered using SR and AVM for the same driving information, new image content can be generated (using a generative model) based on the two image contents. This can combine the three-dimensional contours, colors, and texture features reflected in the respective image content of SR and AVM to achieve a more accurate visualization of the relevant driving information.

[0035] In the case where the image content required to be included inside and outside the preset display range comes from different images (for example, the image content required to be included outside the preset display range comes from the first image, and the image content required to be included within the preset display range comes from the second image), the corresponding image content within the preset display range and outside the preset display range is subjected to coordinate alignment and other processing to merge the two into a complete image, thereby obtaining a fused image.

[0036] Therefore, through the deep integration of SR virtual scenes and AVM real scenes (for example, the lane line prediction information visualized by SR is superimposed on the obstacles visualized by AVM), the driver's cognitive load can be reduced and an immersive experience of "what you see is what you get" can be created.

[0037] Optionally, blind spot risks (such as the risk of children suddenly running into the vehicle) can be marked in the fused image to transform the safety features of the Advanced Driving Assistance System (ADAS) into user-perceivable value points, thereby supporting an increase in vehicle pricing.

[0038] To sum up, in the technical solution provided in the embodiment of the present application, by retaining the image content represented by the first image or the second image outside the preset display range, and fusing the image content represented by the first image and the second image within the preset display range, while utilizing SR and AVM to complement each other in the perception range to ensure the comprehensiveness of the perceived driving information, the repeated visual display of the same driving information perceived by SR and AVM can be avoided, thereby avoiding redundancy and clutter of driving information in the perspective, thereby reducing the risk of key driving information being obscured and significantly reducing the driver's cognitive load.

[0039] In an optional embodiment, the method further comprises: detecting whether the vehicle meets a preset activation condition; When it is detected that the vehicle meets the preset activation condition, displaying the fused image through a display device of the vehicle; When it is detected that the vehicle does not meet the preset activation condition, the first image or the second image is displayed through a display device of the vehicle.

[0040] In this embodiment, by setting preset activation conditions, the fusion rendering functions of SR and AVM can be automatically turned on and off according to the specific scene in which the vehicle is located.

[0041] When the vehicle does not meet the preset activation conditions, the fusion rendering function of SR and AVM is automatically turned off. At this time, only the current frame image (i.e., the first image) rendered by SR (and a map, such as SD MAP) is displayed on the vehicle's display device (such as the on-board central control screen, instrument screen or AR-HUD), or the current frame image (i.e., the second image) rendered by AVM is displayed, which can reduce computing overhead and ensure display efficiency.

[0042] When the vehicle meets the preset activation conditions, the fusion rendering function of SR and AVM is automatically turned on. At this time, the fused image obtained based on the first image and the second image is displayed on the display device to ensure the comprehensiveness of driving information.

[0043] Optionally, the preset activation condition includes at least one of the following: Detecting a target object within the vehicle's preset range, such as a static obstacle, a moving pedestrian, or a low object; The vehicle's speed is less than the set speed and the turn signal is on, for example, the vehicle's speed is less than 15 km / h and the turn signal is on; The vehicle is in a parking situation.

[0044] In an optional embodiment, the image content after the first image and the second image are fused within a preset display range includes at least one of the following: For the first driving information, the image content of the first image is visualized within the preset display range, and, for the second driving information, the image content of the second image is visualized within the preset display range, and the second driving information is different from the first driving information; for example, the image content of the first image included in the preset display range of the fused image can be a three-dimensional scene rendered by the SR (and a map, such as an SD Map), and the image content of the second image included in the preset display range of the fused image can be the image content rendered by the AVM for driving information not perceived by the SR (and a map, such as an SD Map) (corresponding to the second driving information, such as static obstacles, moving objects, etc.).

[0045] For the same driving information, the first image or the second image visualizes image content within the preset display range; for example, within the preset display range, the fused image may include image content rendered by SR (and map, such as SD Map) or AVM for the same obstacle.

[0046] Optionally, the image content included in the fused image is flexibly adjusted according to the driving state of the vehicle (which can be used to characterize different scenes in which the vehicle is located).

[0047] For example, when a vehicle passes through a crosswalk, the fused image (such as the image content of the fused image within a preset display range) may include a three-dimensional road scene image (including building models and road models, etc.) rendered by SR (and a map, such as SD Map), and pedestrian images rendered by AVM, so that the fused image can accurately describe moving pedestrians on the road based on AVM, thereby improving driving safety.

[0048] For another example, when the vehicle is driving normally, the fused image (such as the image content of the fused image within a preset display range) may include image content rendered by SR (and maps, such as SD Map) for buildings and roads, so that the fused image can provide a more comprehensive description of the buildings and roads around the vehicle based on SR (and maps, such as SD Map), thereby improving the driving experience.

[0049] For another example, when a vehicle is in a parking scene, the fused image (such as the image content of the fused image within a preset display range) may include image content rendered by the AVM for the ground, so that the fused image can accurately describe the ground of the parking lot based on the AVM, thereby improving the parking experience.

[0050] Optionally, in response to detecting that the vehicle's driving state meets a first predetermined state (e.g., parking, low-speed driving, or detection of a target object within a preset range of the vehicle), the fused image includes obstacles visually displayed by the second image within the preset display range. These obstacles are obstacles not perceived by the SR (due to, for example, a lack of a pre-set relevant model), thereby ensuring an accurate depiction of obstacles around the vehicle. Optionally, the target objects include at least one of static obstacles, moving pedestrians, and low objects.

[0051] Optionally, in response to detecting that the driving state of the vehicle conforms to a second set state (such as the vehicle passing through a turning intersection or a crosswalk, etc.), the fused image includes active objects visually displayed in the second image within the preset display range to achieve enhanced feedback on active objects around the vehicle.

[0052] For example, reference Figure 4 The schematic diagram of the close-range obstacle-triggered fusion scenario shown in the figure shows that when a target object is detected within the preset range of the vehicle, such as a static obstacle is detected within 1.0 m (adjustable from 0.3 m to 1.0 m) of the vehicle, and / or a moving pedestrian is detected within 3.0 m (adjustable from 1.0 m to 3.0 m) of the vehicle, and / or a low object is detected within 0.5 m of the vehicle, the vehicle is in a close-range obstacle-triggered fusion scenario. At this time, the obstacles visualized in the second image within the preset display range are used to supplement the obstacle information visualized in the first image within the preset display range. In this way, the AVM image can be used to supplement the details of physical world obstacles that cannot be perceived by SR (and maps, such as SDMap), making it easier for users to make driving decisions.

[0053] For example, refer to Figure 5 The schematic diagram of the intersection turning activation fusion scene shown is that when the vehicle turns at an intersection (for example, if the vehicle speed is detected to be less than the set speed and the turn signal is turned on, it can be determined that it is turning at the intersection) and / or passes through a crosswalk, the active objects visually displayed by the second image within the preset display range are used to replace the active objects visually displayed by the first image within the preset display range. For example, AVM images are used to display moving pedestrians in a three-dimensional scene rendered by SR (and maps, such as SD Map), thereby enhancing feedback on active objects in the physical world by combining AVM images in the scene rendered by SR (and maps, such as SDMap) to improve driving safety.

[0054] For example, refer to Figure 6As shown in the schematic diagram of the automatic parking fusion scenario, when the vehicle is in the parking state, it can be determined that the vehicle is in the automatic parking fusion scenario. At this time, the obstacles visualized by the second image within the preset display range are used to supplement the obstacles visualized by the first image within the preset display range. In this way, the AVM image can be used to supplement the details of physical world obstacles that SR (and maps, such as SD Map) cannot perceive in parking lots with relatively complex environments, thereby enhancing the control and trust of parking and improving the user's parking experience.

[0055] In an optional embodiment, fusing the first image and the second image includes: Adjusting the image content of the first image within a first target range and the image content outside the first target range to different transparencies, and / or adjusting the image content of the second image within a second target range and the image content outside the second target range to different transparencies; The first image and the second image are superimposed according to a preset superposition order.

[0056] Optionally, at least one of the first target range, the second target range and the preset superposition order is determined according to at least one of a user instruction, a driving state of the vehicle and a preset display range.

[0057] Taking the second target range as the preset display range as an example, the transparency of the image content of the second image within the preset display range can be adjusted to 1 to fully display the image content within the preset display range; and the transparency of the image content of the second image outside the preset display range can be adjusted to 0 to make the image content outside the preset display range completely transparent; finally, the second image is superimposed on the first image to obtain a fused image, so that the fused image can include the image content of the first image outside the preset display range and the image content of the second image within the preset display range.

[0058] In this embodiment, image fusion is achieved by adjusting transparency in combination with image overlay, and by introducing clear range divisions (such as the first target range and the second target range), the degree of influence between different images within the corresponding range can be effectively controlled. For example, in the above example, interference of the AVM image outside the preset display range with the SR (and map, such as SD MAP) image can be effectively avoided.

[0059] Optionally, before superimposing the first image and the second image, the first image and / or the second image are fully distortion corrected to ensure the clarity of the image (such as the image rendered after the perspective is switched) and avoid image distortion.

[0060] Optionally, an image within a preset display range is cropped from the fused image and output to a display device for display.

[0061] In an optional embodiment, the method further comprises: According to the driving state of the vehicle (such as the road condition of the vehicle, the speed of the vehicle, etc., which can be determined based on the detection results of various vehicle-mounted sensors to reflect the scene in which the vehicle is located), the preset display range is adjusted, such as the shape, size and angle range of the preset display range; or, According to the driving state of the vehicle, target information is sent to the SR and AVM of the vehicle, where the target information is used to indicate at least one of: the image dimension required for rendering, the lens information for image rendering, and the viewing angle information for image rendering.

[0062] Among them, the image dimension can be two-dimensional or three-dimensional, etc., the lens information may include the lens position, etc., and the viewing angle information may include the viewing angle direction (such as top view, left / right view, forward view and default view, etc.), the viewing angle size (such as the multiple by which the viewing angle needs to be magnified or reduced, that is, the distance by which the viewing angle needs to be zoomed in or out), the position of the viewing angle and the position at which the viewing angle needs to be focused, etc.

[0063] For example, if the fused image displayed on the vehicle needs to be switched to a normal perspective (i.e., the default perspective), after sending the corresponding target information to the SR and AVM, the system receives a first image rendered by the SR (and a map, such as an SD Map) based on its default perspective, and also receives a second image rendered by the AVM based on its own default perspective. During the fusion phase, the original perspectives of the first and second images are maintained, and the second image is overlaid on the first image after adjusting its transparency to preserve the image content rendered by the AVM within the preset display range. This allows the AVM image to provide emphasized feedback on driving information within the preset display range.

[0064] To switch the vehicle's fused image to a bird's-eye view, after sending the corresponding target information to the SR and AVM, the system receives a first image rendered by the SR (and a map, such as an SD Map) based on the bird's-eye view, and a second (2D) image rendered by the AVM based on the same bird's-eye view. During the fusion phase, the second image can be overlaid on the first image after adjusting its transparency to preserve the image content rendered by the AVM within the preset display range, thereby emphasizing the driving information within the preset display range using the AVM's 2D real-world image.

[0065] To switch the vehicle's fused image to a forward-facing state, after sending target information to the SR and AVM, the SR (and map, such as the SD Map) zooms in and out based on the target information, quickly zooming in to the vehicle's front, and shifting its perspective to a horizontal position to produce the SR's forward view (i.e., the first image). The AVM then renders the forward AVM view based on the target information to produce the second image. During the fusion phase, the forward AVM view is fully distortion-corrected before being fused with the SR's forward view to achieve a comprehensive visualization of driving information in the area near the vehicle's front.

[0066] If the fused image displayed by the vehicle needs to be switched to a left-facing perspective, after sending target information to the SR and AVM, the SR (and map, such as the SD Map) zooms in based on the target information, synchronously rotates clockwise to the left side of the vehicle, and changes its perspective to a straight-ahead view, resulting in the SR left-facing view (i.e., the first image). The AVM renders the left-facing AVM view based on the target information to produce the second image. During the fusion phase, the left-facing AVM view is fully distortion-corrected and then fused with the SR left-facing view. The ego-vehicle model is made transparent during the fusion process, and ground information is visualized using the SR (and map, such as the SD Map). This allows for a reasonable visualization of driving information in the area near the left side of the vehicle.

[0067] If the fused image displayed by the vehicle needs to be switched to a right-facing perspective, after sending target information to the SR and AVM, the SR (and map, such as the SD Map) will magnify its own perspective based on the target information, synchronously rotate counterclockwise to the right side of the vehicle, and change its perspective to a straight-ahead view to obtain the SR right-facing view (i.e., the first image). The AVM will render the right-facing AVM view based on the target information to obtain the second image. During the fusion phase, the right-facing AVM view is fully distortion-corrected and then fused with the SR right-facing view. The ego-vehicle model is set transparent during the fusion process, and the ground information is visualized using the SR (and map, such as the SD Map). This allows for a reasonable visualization of driving information in the area near the right side of the vehicle.

[0068] If the vehicle's fused image needs to be switched to a close-up perspective, after sending target information to the SR and AVM, the SR (and map, such as the SD Map) and AVM zoom in based on the target information, focusing on the preset display range and rendering a first image and a second image (which are 3D images) that conform to their default perspectives. During the fusion phase, the second image is overlaid on the first image after adjusting its transparency to preserve the image content rendered by the AVM within the preset display range, thus emphasizing the driving information within the preset display range using the AVM's 3D real-world imagery.

[0069] When it is necessary to switch the fused image displayed on the vehicle to the left rear view, after sending the target information to the SR and AVM, the SR (and the map, such as the SD Map) will zoom out and rotate the view clockwise by a certain angle (for example, about 100 degrees) based on the target information, and move the lens to the rearview mirror position (the specific position coordinates can be flexibly calibrated) to obtain the SR left view (i.e., the first image); the AVM will render the left AVM view based on the target information to obtain the second image. During the fusion stage, the left AVM view is completely distortion-corrected and then fused with the SR left view. The image within the preset display range is cropped from the fused image and displayed on the display device. This allows for a reasonable visualization of driving information in the area near the left rear of the vehicle.

[0070] When it is necessary to switch the fused image displayed on the vehicle to the right rear view, after sending the target information to the SR and AVM, the SR (and the map, such as the SD Map) will zoom out and rotate the view counterclockwise by a certain angle (for example, about 100 degrees) according to the target information, and move the lens to the rearview mirror position (the specific position coordinates can be flexibly calibrated) to obtain the SR right view (i.e., the first image); the AVM will render the right AVM view according to the target information to obtain the second image. During the fusion stage, the right AVM view is completely distortion-corrected and then fused with the SR right view. The image within the preset display range is cropped from the fused image and displayed on the display device. This allows for a reasonable visualization of driving information in the area near the right rear of the vehicle.

[0071] Optionally, the perspective fusion scheme used by the vehicle is determined according to the driving state of the vehicle, and the perspective information of the fused image required to be displayed by the vehicle is determined accordingly, wherein the perspective information may include information such as perspective direction, perspective size and perspective position.

[0072] For example, when the vehicle uses the default perspective fusion solution (which is used to obtain a fused image under the default perspective), the preset display range is a circular area with the center point of the rear of the vehicle as the center and with a set radius. Figure 7 As shown, the value range of the set radius can be 3.13 meters to 6 meters; For another example, when the vehicle uses a bird's-eye view fusion solution (which is used to obtain a fused image under a bird's-eye view), the preset display range is an area of ​​a first set size where the vehicle is located, such as a circular area with a set radius where the vehicle is located. For example, the fused image under the bird's-eye view fusion solution can refer to Figure 8 As shown; For another example, when the vehicle uses a forward perspective fusion solution (which is used to obtain a fused image under a forward perspective), the preset display range is an area that includes the front area of ​​the vehicle and meets the second set size. For example, the fused image under the forward perspective fusion solution can refer to Figure 9 As shown; For another example, when the vehicle uses a left-view fusion solution (which is used to obtain a fused image under a left-view), the preset display range is an area that includes the left side of the vehicle and meets the third set size. For example, the fused image under the left-view fusion solution can refer to Figure 10 As shown; For another example, when the vehicle uses a right-view fusion solution (which is used to obtain a fused image under a right-view perspective), the preset display range is an area that includes the right side of the vehicle and meets the fourth set size. For example, the fused image under the right-view fusion solution can refer to Figure 11 As shown; For another example, when no new moving objects or static obstacles are detected around the vehicle (and the vehicle uses a near-field fusion solution (which is used to obtain a fused image under a near-field perspective)), the preset display range is a sector-shaped area with the center point of the vehicle body as the center and within the first angle range. For example, the fused image under the near-field fusion solution can refer to Figure 12 shown.

[0073] Optionally, when a new moving object or static obstacle is detected around the vehicle (and the vehicle uses a near-field fusion solution), the preset display range is a sector-shaped area with the center point of the vehicle body as the center and within a second angle range. The second angle range includes and is greater than the first angle range to achieve an increase in the angle range of the sector-shaped area. This can provide the user with a richer visual display of driving information when a new moving object or static obstacle appears, allowing the driver to more easily avoid the newly appeared moving object or static obstacle, thereby improving driving safety. For example, the first angle range can be 120° to 180°, the second angle range can be 120° to 360°, and the radius of the sector-shaped area can range from 3.13 meters to 6 meters. Optionally, when the vehicle uses a left rear view fusion solution (which is used to obtain a fused image under the left rear view), the preset display range is an area that includes the left rear area of ​​the vehicle and meets the fifth set size. For example, the fused image under the left rear view fusion solution can refer to Figure 13 As shown; Optionally, when the vehicle uses a right rear view fusion solution (which is used to obtain a fused image under the right rear view), the preset display range is an area that includes the right rear area of ​​the vehicle and meets the sixth set size. For example, the fused image under the right rear view fusion solution can refer to Figure 14 shown.

[0074] In an optional embodiment, the first image is a current frame image obtained by the SR placing and rendering a preset three-dimensional model according to the perception data and the standard definition map SD MAP placing and rendering a preset three-dimensional building model according to the road surrounding data; Among them, the preset three-dimensional models include at least one of the vehicle model, adjacent vehicle models (such as sedans, SUVs, trucks, ambulances, motorcycles, electric vehicles and bicycles), pedestrian models, road models (such as lanes, lane lines, parking spaces, lane signs and other related models), facility models (such as toll booths, newsstands, trash cans, cones, guardrails, lawns and trees and other common facilities related models) and traffic sign models (such as speed limit signs, traffic lights, parking lots, no parking zones, gas / charging stations and highway signs and other related models); the preset three-dimensional building models include at least one of residential building models, office building models, store building models, city landmark building models, park building models and bridge building models.

[0075] In specific implementation, SR performs real-time placement and rendering of three-dimensional models based on perception data; AVM collects real-time image data of the physical world through cameras (usually any objects in the environment surrounding the vehicle body can be collected), so as to effectively supplement the three-dimensional models preset in SR; SD Map performs real-time placement and rendering of three-dimensional building models based on data around the road. By merging and rendering the image content displayed by SR, AVM and maps (such as SD Map) in real time on the display screen of the same display device, and thereby utilizing the mutual complementation of the perception ranges of the three products, environmental perception can be made without blind spots and more intelligent, and can reduce the combined information of multiple applications (Application, APP), reducing the decision-making burden of users. For example, the perception range distribution and perception distance reference of SR, AVM and SD Map Figure 15 and Figure 16 shown.

[0076] It is understandable that under the influence of the trend of one-shot three-dimensional scenes and the combination of virtual and reality, the integrated rendering solution of SR, AVM and map (such as SD MAP) provided in the embodiment of the present application (also known as IntelliSphere Reality) can effectively enhance the visual impact of the combination of virtual and real, thereby greatly improving product competitiveness.

[0077] It should be noted that for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0078] Please refer to Figure 17 , which shows an architecture diagram of an image processing system provided by an embodiment of the present application. The system is used to implement the method embodiment of the present application. For details not disclosed in the system embodiment of the present application, please refer to the method embodiment of the present application. The system includes an AVM module, an image rendering engine, and a frame buffer object FBO. The FBO is respectively connected to the AVM module and the image rendering engine, wherein: The AVM module is configured to render continuous frame images based on AVM and transmit the rendered current frame image to the FBO for storage; The image rendering engine is configured to render continuous frame images based on SR, and to fuse the current frame image stored in the FBO with the current frame image rendered based on SR to obtain a fused image; the image content of the fused image includes: the image content of the current frame image rendered by the AVM module or the current frame image rendered based on SR outside the preset display range, and the image content after the current frame image rendered by the AVM module and the current frame image rendered based on SR are fused within the preset display range.

[0079] In the technical solution provided in the embodiment of the present application, the image rendering engine retains the image content represented by the current frame image rendered by itself or the AVM module outside the preset display range, and fuses the image content represented by itself and the current frame image rendered by the AVM module within the preset display range. Therefore, while utilizing the mutual complementation of SR and AVM to the perception range to ensure the comprehensiveness of the perceived driving information, the repeated visual display of the same driving information perceived by SR and AVM can be avoided, thereby avoiding redundancy and clutter of driving information in the perspective, thereby reducing the risk of key driving information being obscured and significantly reducing the driver's cognitive load.

[0080] In an optional embodiment, the image rendering engine may also be configured to render consecutive frame images based on SR and a map (such as SDMAP).

[0081] In an optional embodiment, the image rendering engine may also be configured to output the fused image to a display device of the vehicle, so that the display device displays the fused image to a user.

[0082] In an optional embodiment, the image rendering engine can also be configured to use the image rendering shader material ShaderMaterial class to superimpose material special effects on the current frame image rendered by the AVM module based on the material parameters of the current frame image rendered by itself.

[0083] In this embodiment, the AVM module uses the current frame image rendered by itself (such as multi-channel camera texture) as the input source of the image rendering ShaderMaterial class, so that it can be dynamically combined with the image rendering material parameters (such as reflectivity and transparency, etc.) through UV mapping, so as to superimpose the image rendering material special effects (such as dynamic highlights, ambient occlusion, etc.) on the AVM rendered picture to enhance the three-dimensional realism of the panoramic picture; and the image rendering engine superimposes the material special effects on the current frame image rendered by the AVM module according to the material parameters of the current frame image rendered by itself, which can improve the integration of the AVM picture and make it seamlessly integrated with the three-dimensional vehicle model, HUD and other image rendering elements.

[0084] Please refer to Figure 18 , which shows an architecture diagram of an image processing system provided by another embodiment of the present application, wherein the AVM module includes an AVM algorithm unit and an AVM data source, and the AVM data source is communicatively connected to the AVM algorithm unit and the image rendering engine respectively, wherein: The AVM algorithm unit is configured to render continuous frame images based on AVM and transmit the rendered current frame image to the AVM data source; The AVM data source is configured to transmit the current frame image received by itself to the image rendering engine.

[0085] In practice, the AVM algorithm unit executes the AVM algorithm-related logic with the help of a pre-configured AVM algorithm library (LIB), thereby generating an AVM video stream (i.e., continuous frame images). Optionally, the AVM algorithm unit also supports users to integrate other related algorithms into the AVM algorithm library based on actual needs.

[0086] In this embodiment, the AVM algorithm unit and the image rendering engine implement loosely coupled communication through the AVM data source (DataSource), thereby ensuring the interactive scalability of the system.

[0087] Optionally, the AVM algorithm unit is obtained by encapsulating an AVM algorithm module as a native plug-in in an image rendering engine, so that the AVM algorithm module and the image rendering engine share the same OpenGL rendering context.

[0088] In this embodiment, the present application deeply integrates the AVM algorithm module as a native plug-in into the image rendering engine, enabling the AVM algorithm module and the image rendering engine to share OpenGL resources (i.e., OpenGL rendering context, also known as OpenGL context or Opengl Context). This eliminates the overhead of multiple context switching and improves the execution efficiency of the graphics pipeline. Tests have shown that OpenGL context sharing can reduce GPU memory copy overhead by more than 30% and increase the frame rate to over 60 frames per second (FPS).

[0089] Optionally, the AVM data source is established based on a data source (DataSource) data-driven architecture of an image rendering engine; The AVM data source can also be configured to provide a dynamic perspective control interface to the user, and upon receiving a perspective switching instruction from the user through the dynamic perspective control interface, send target information to the AVM algorithm unit and the image rendering engine, wherein the target information is used to indicate at least one of: the image dimension to be rendered, the lens information used for image rendering, and the perspective information used for image rendering.

[0090] In this embodiment, the data source data-driven architecture unique to the image rendering engine is used to construct an AVM data source, so that the AVM data source can provide the user with a dynamic perspective control interface to receive perspective switching instructions from the user, and then support the user to control the vehicle to switch perspectives through UI events (such as touch, voice commands, etc.), so as to achieve a visually smooth perspective transition of the fused image.

[0091] Optionally, the image rendering engine includes an output texture module, an OpenGL rendering context, and a swap buffer, wherein: The output texture module is configured to render continuous frame images based on SR, and fuse the current frame image rendered by the AVM module with the current frame image rendered based on SR to obtain a fused image, and transmit the fused image to a memory block; The memory block is configured to submit the fused image received by the memory block to the display device of the vehicle for display; The OpenGL rendering context is configured to support rendering operations of the AVM module and the output texture module.

[0092] In specific implementations, the output texture module uses pre-configured SR (and map, such as SD MAP) related plug-ins to render related images such as 2D or 3D radar perception data, 2D or 3D vehicle models, and UI interfaces.

[0093] Optionally, the image rendering engine and the AVM module can be configured to render their current frame images sequentially according to a set rendering order, sharing the same rendering pipeline. This phased rendering approach avoids pipeline conflicts, ensuring that the shader, material, and lighting calculations of the image rendering engine and the AVM module are not interfered with. Furthermore, the SR rendering pipeline is used to synchronously process AVM data, reducing the need for independent apps and reducing development time costs by 8-12%.

[0094] Optionally, the image rendering engine can also be configured to transfer the current frame image rendered by itself to the FBO for storage, and then fuse the different current frame images stored in the FBO (i.e., the current frame images rendered by the image rendering engine and the AVM module respectively) to obtain the fused image.

[0095] Optionally, the image rendering engine may be further configured to render the current frame image in another pre-created FBO (i.e., another FBO different from the FBO used to store the current frame image), and before rendering the current frame image, perform a clear frame operation to clear the other FBO; Optionally, the AVM module may be further configured to render the current frame image in the other FBO, and before rendering the current frame image itself, perform a clear frame operation to clear the other FBO.

[0096] For example, setting the rendering order to render the SR screen first and then the AVM screen, please refer to Figure 19 , which shows a real-time rendering sequence diagram of the system provided by one embodiment of the present application. The processing flow for consecutive frames may include: 1. Clearing the frame (Clear Screen); 2. SR screen rendering; 3. Cache to FBO; 4. Clearing the frame; 5. AVM screen rendering; 6. Fusion rendering; 7. Outputting the frame (i.e., submitting the fused image). The processing flow for each single frame (such as the current frame image) is as follows: 1) Image rendering engine independent rendering stage The image rendering engine performs a clear frame operation to clear the first FBO.

[0097] The image rendering engine completes the drawing of 2D / 3D images in the first FBO according to its standard rendering pipeline to realize SR image rendering, ensuring the correctness of its rendering logic such as materials, lighting, and shaders.

[0098] When the image rendering engine finishes rendering, the rendered current frame image is stored in the second FBO for subsequent fusion.

[0099] 2) AVM rendering stage The AVM module performs a clear frame operation to clear the first FBO.

[0100] The AVM module executes the AVM algorithm rendering process in the first FBO to generate a panoramic view picture.

[0101] 3) Screen fusion and alignment The image rendering engine fuses the panoramic view picture and the current frame image stored in the second FBO (ie, the current frame image rendered by the image rendering engine) in the first FBO to obtain a fused image.

[0102] During the fusion process, the image rendering engine can use additive blending mode and independent control of color channels and alpha channels to ensure the correct superposition of different images and obtain accurate fusion effects.

[0103] For example, first define an addition, the reflection value: SR color value and AVM color value, transparency: SR transparency + AVM transparency: Then, define the multiplication of the color itself (superimposing the effect of the previous step), the reflection value: ,transparency: ,in, Indicates the output result of the facial value. Indicates the transparency output result, Indicates the SR color value, Represents the AVM color value, Indicates SR transparency, AVM transparency, opacity and translucency fusion effect reference Figure 20 and Figure 21 shown.

[0104] 4) Final output After the fusion is completed, the image rendering engine submits the fused image to the display device to form a complete AVM+UI interactive interface.

[0105] Based on the above embodiments, the present application provides an image processing system, which can realize the fusion rendering of AVM, SR (and map, such as SD MAP) by integrating the image rendering engine and the AVM module, and supports functions such as multi-perspective switching, dynamic material fusion and OpenGL resource sharing. Through frame buffer object (FBO) management and multi-stage rendering, it can ensure the correct superposition display of the image rendered by SR (and map, such as SD MAP) and the image rendered by AVM.

[0106] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0107] Please refer to Figure 22 , which shows a block diagram of an image processing device provided by an embodiment of the present application. The device has the function of implementing the above-mentioned method example, and the function can be implemented by hardware or by hardware executing corresponding software. The device 220 may include: a communication module 221 configured to acquire a first image and a second image, wherein the first image is a current frame image rendered using a surrounding perception system (SR) of the vehicle, and the second image is a current frame image rendered using an surround view monitoring system (AVM) of the vehicle; The processing module 222 is used to fuse the first image and the second image to obtain a fused image, wherein the image content of the fused image includes: the image content of the first image or the second image outside the preset display range, and the image content after the first image and the second image are fused within the preset display range.

[0108] To sum up, in the technical solution provided in the embodiment of the present application, by retaining the image content represented by the first image or the second image outside the preset display range, and fusing the image content represented by the first image and the second image within the preset display range, while utilizing SR and AVM to complement each other in the perception range to ensure the comprehensiveness of the perceived driving information, the repeated visual display of the same driving information perceived by SR and AVM can be avoided, thereby avoiding redundancy and clutter of driving information in the perspective, thereby reducing the risk of key driving information being obscured and significantly reducing the driver's cognitive load.

[0109] Optionally, the processing module 222 is further configured to perform the following steps: detecting whether the vehicle meets a preset activation condition; When it is detected that the vehicle meets the preset activation condition, displaying the fused image through a display device of the vehicle; When it is detected that the vehicle does not meet the preset activation condition, the first image or the second image is displayed through a display device of the vehicle.

[0110] Optionally, the processing module 222 is further configured to perform the following steps: Adjusting the image content of the first image within a first target range and the image content outside the first target range to different transparencies, and / or adjusting the image content of the second image within a second target range and the image content outside the second target range to different transparencies; The first image and the second image are superimposed according to a preset superposition order.

[0111] Optionally, the image content after the first image and the second image are fused within a preset display range includes at least one of the following: for first driving information, image content of the first image visualized within the preset display range, and for second driving information, image content of the second image visualized within the preset display range, the second driving information being different from the first driving information; For the same driving information, the first image or the second image is an image content that is visualized within the preset display range.

[0112] Optionally, the processing module 222 is further configured to adjust the preset display range according to the driving state of the vehicle.

[0113] Optionally, the processing module 222 is also used to send target information to the SR and AVM of the vehicle according to the driving status of the vehicle, and the target information is used to indicate: at least one of the image dimensions required for rendering, the lens information used for image rendering, and the perspective information used for image rendering.

[0114] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0115] An embodiment of the present application further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the image processing method disclosed in the embodiment of the present application is implemented.

[0116] The embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the image processing method disclosed in the embodiment of the present application.

[0117] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image processing method disclosed in the embodiment of the present application is implemented.

[0118] An embodiment of the present application further provides a vehicle, which includes the electronic device, or includes the image processing system.

[0119] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0120] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, systems, devices, storage media, and program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0124] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements that are inherent to such process, method, article, or terminal device. In the absence of further restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0125] The above is a detailed introduction to the image processing method, system, device, medium, equipment and vehicle provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first image and a second image, where the first image is a current frame image rendered using a surrounding perception system (SR) of the vehicle, and the second image is a current frame image rendered using an all-around view monitoring system (AVM) of the vehicle; The first image and the second image are fused to obtain a fused image, wherein the image content of the fused image includes: the image content of the first image or the second image outside a preset display range, and the image content after the first image and the second image are fused within the preset display range.

2. The method according to claim 1, characterized in that The method further comprises: detecting whether the vehicle meets a preset activation condition; When it is detected that the vehicle meets the preset activation condition, displaying the fused image through a display device of the vehicle; When it is detected that the vehicle does not meet the preset activation condition, the first image or the second image is displayed through a display device of the vehicle.

3. The method according to claim 1, characterized in that The fusing the first image and the second image includes: Adjusting the image content of the first image within a first target range and the image content outside the first target range to different transparencies, and / or adjusting the image content of the second image within a second target range and the image content outside the second target range to different transparencies; The first image and the second image are superimposed according to a preset superposition order.

4. The method according to any one of claims 1 to 3, characterized in that The image content after the first image and the second image are fused within a preset display range includes at least one of the following: for first driving information, image content of the first image visualized within the preset display range, and for second driving information, image content of the second image visualized within the preset display range, the second driving information being different from the first driving information; For the same driving information, the first image or the second image is an image content that is visualized within the preset display range.

5. The method according to any one of claims 1 to 3, characterized in that The method further comprises: adjusting the preset display range according to the driving state of the vehicle; or, According to the driving state of the vehicle, target information is sent to the SR and AVM of the vehicle, where the target information is used to indicate at least one of: the image dimension required for rendering, the lens information for image rendering, and the viewing angle information for image rendering.

6. An image processing system, characterized in that: The system includes an AVM module, an image rendering engine, and a frame buffer object FBO, wherein the FBO is communicatively connected to the AVM module and the image rendering engine respectively, wherein: The AVM module is configured to render continuous frame images based on AVM and transmit the rendered current frame image to the FBO for storage; The image rendering engine is configured to render continuous frame images based on SR, and to fuse the current frame image stored in the FBO with the current frame image rendered based on SR to obtain a fused image; the image content of the fused image includes: the image content of the current frame image rendered by the AVM module or the current frame image rendered based on SR outside the preset display range, and the image content after the current frame image rendered by the AVM module and the current frame image rendered based on SR are fused within the preset display range.

7. The system according to claim 6, characterized in that The image rendering engine and the AVM module are further configured to render their own current frame images in sequence according to a set rendering order so as to share the same rendering pipeline.

8. An image processing device, characterized in that: The device comprises: a communication module configured to acquire a first image and a second image, wherein the first image is a current frame image rendered using a surrounding perception system (SR) of the vehicle, and the second image is a current frame image rendered using an all-around view monitoring system (AVM) of the vehicle; A processing module is used to fuse the first image and the second image to obtain a fused image, wherein the image content of the fused image includes: the image content of the first image or the second image outside a preset display range, and the image content after the first image and the second image are fused within the preset display range.

9. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the image processing method according to any one of claims 1 to 5.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the image processing method according to any one of claims 1 to 5 is implemented.

11. A vehicle, characterized in that: The vehicle includes the electronic device according to claim 10, or the vehicle includes the image processing system according to any one of claims 6-7.

Images