A method, system, device and medium for sharing a see-through image for a vehicle

Abstract

The present application relates to a kind of car perspective image sharing method, system, equipment and medium.The method includes real-time acquisition environment image in front and / or side of the car;Image processing is carried out to the environment image, and the image processing at least includes one or more of image repair processing, multi-path image splicing processing and target marking processing, obtains the environment image after processing;The processed environment image is displayed to the rear of the car, for the driver of the vehicle located in the rear of the car observation.The method is by obtaining the environment image in front and / or side of the car, and is shown to the vehicle in the rear, so that the driver of the rear vehicle can be in advance to judge the braking or steering intention of the front vehicle, effectively reduce the rear-end collision caused by sight obstruction, improve the overall safety of queue driving.

Description

Technical Field

[0001] This invention relates to the field of automotive electronics and intelligent driving assistance technology, and in particular to a method, system, device and medium for sharing vehicle perspective images. Background Technology

[0002] With the continuous growth of car ownership, the road traffic environment is becoming increasingly complex, and rear-end collisions are frequent. A major reason for rear-end collisions is that drivers of vehicles behind cannot see the road conditions ahead of the vehicle in front.

[0003] Specifically, when multiple vehicles are traveling in a convoy, the physical obstruction of the vehicle in front completely blocks the view of the drivers in the following vehicles, preventing them from directly seeing the traffic conditions ahead. When the vehicle in front brakes suddenly due to an obstacle, pedestrian crossing, or traffic congestion, the drivers in the following vehicles often lack the time to react in time, leading to rear-end collisions. Although existing advanced driver assistance systems (ADAS) can obtain some information through radar or vehicle-to-everything (V2X) technology, these solutions are either costly or rely on complex vehicle-to-vehicle communication protocols and are not yet widespread. Currently, there is a lack of an intuitive, low-cost, and easily deployable technological solution that allows drivers in the following vehicles to directly obtain visual information about the road ahead, as if they could "see through" the vehicle in front. Summary of the Invention

[0004] The present invention provides a method, system, device and medium for sharing vehicle perspective images, in order to overcome at least one of the above-mentioned technical problems existing in the prior art.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions: In a first aspect, the present invention provides a method for sharing vehicle perspective images, comprising: Real-time acquisition of environmental images in front of and / or to the sides of the vehicle; The environmental image is processed, and the image processing includes at least one or more of image restoration processing, multi-channel image stitching processing, and target annotation processing to obtain a processed environmental image. The processed environmental image is displayed to the rear of the vehicle for observation by the driver of a vehicle located behind the vehicle.

[0006] In one possible implementation of the first aspect, the image inpainting process for the environmental image includes: Detect whether there are localized blurred or missing areas in the environmental image due to stains or occlusions; If the aforementioned locally blurred or missing regions exist, interpolation calculations are performed using the clear pixel information surrounding the locally blurred or missing regions to generate predicted pixel values. These predicted pixel values ​​are then used to fill the locally blurred or missing regions to generate a repaired environmental image.

[0007] In one possible implementation of the first aspect, after generating the repaired environmental image, the method further includes: The repaired environmental image is logically judged using a pre-trained image discrimination model. If it is determined that the repaired environmental image does not conform to the logic of the real scene, then the repaired environmental image is discarded.

[0008] In one possible implementation of the first aspect, the environmental image is a multi-channel environmental image, and the multi-channel image stitching process for the environmental image includes: Feature analysis is performed on each of the environmental images to identify and determine the overlapping areas between the environmental images. The overlapping areas in each of the environmental images are cropped and deleted, and the remaining non-overlapping image areas after cropping are edge-blended and stitched together to generate a continuous display image with a wide viewing angle.

[0009] In one possible implementation of the first aspect, the target annotation processing of the environmental image includes: The target recognition algorithm is used to identify key targets in the environmental image, and the key targets include at least one of the following: a vehicle, a pedestrian, or an obstacle. The identified key targets are marked with visual markers, including at least one of highlighted borders, color fills, or warning icons.

[0010] In one possible implementation of the first aspect, the target annotation processing of the environmental image further includes: When the identified critical target is determined to be a dangerous target with a potential collision risk, a warning instruction is generated. The warning command is sent to the vehicle's intelligent driving control system and / or cockpit control system to trigger the vehicle's warning operation or braking operation.

[0011] In one possible implementation of the first aspect, displaying the processed environmental image to the rear of the vehicle includes: The processed environmental image is displayed to the rear of the vehicle via a display device installed in the rear windshield area of ​​the vehicle. The display device is a one-way transparent LCD display or a one-way transparent OLED display.

[0012] Compared with the prior art, the present invention has at least the following beneficial effects: The vehicle perspective image sharing method provided by this invention acquires environmental images in front of and / or to the side of the vehicle and displays them to vehicles behind, enabling drivers of following vehicles to anticipate the braking or steering intentions of the vehicle in front, effectively reducing rear-end collisions caused by obstructed vision and improving the overall safety of convoy driving.

[0013] Secondly, the present invention provides a vehicle-mounted perspective image sharing system, comprising: The image acquisition module is used to acquire real-time environmental images of the front and / or sides of the vehicle; The image processing module is used to perform image processing on the environmental image, wherein the image processing includes at least one or more of image restoration processing, multi-channel image stitching processing, and target annotation processing to obtain the processed environmental image; The display module is used to display the processed environmental image to the rear of the vehicle for observation by the driver of the vehicle located behind the vehicle.

[0014] Thirdly, the present invention provides an electronic device comprising: at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors to cause the electronic device to implement the vehicle perspective image sharing method as in any implementation of the first aspect.

[0015] Fourthly, the present invention provides a storage medium having a computer-executable program stored thereon, the computer-executable program being used to cause a computer to execute the vehicle perspective image sharing method as in any implementation of the first aspect.

[0016] Understandably, the beneficial effects achieved by the system of the second aspect, the electronic device of the third aspect, and the storage medium of the fourth aspect provided above can be referred to in light of the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention; Figure 2A flowchart of a method for sharing vehicle perspective images provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an image with missing pixels in an embodiment of the present invention; Figure 4 This is a schematic diagram of the image after interpolation and repair in an embodiment of the present invention; Figure 5 This is a schematic diagram of the images to be stitched together at the same moment in an embodiment of the present invention; Figure 6 This is a schematic diagram of the splicing suture line in an embodiment of the present invention; Figure 7 This is a schematic diagram of the stitched panoramic display image in an embodiment of the present invention; Figure 8 This is a schematic diagram of enhanced visualization annotation in an embodiment of the present invention.

[0019] Figure 9 This is a structural block diagram of a vehicle perspective image sharing system provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. In the description of the present invention, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. The "or" in the present invention is merely a description of the relationship between the related objects, indicating that three relationships can exist. For example, A or B can represent: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Furthermore, in the description of the present invention, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items.

[0021] Furthermore, to facilitate a clear description of the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0022] In this embodiment of the invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this embodiment of the invention should not be construed as superior or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0023] With the continuous growth of car ownership, the road traffic environment is becoming increasingly complex, and rear-end collisions are frequent. A major reason for rear-end collisions is that drivers of vehicles behind cannot see the road conditions ahead of the vehicle in front.

[0024] Specifically, when multiple vehicles are traveling in a convoy, the physical obstruction of the vehicle in front completely blocks the view of the drivers in the following vehicles, preventing them from directly seeing the traffic conditions ahead. When the vehicle in front brakes suddenly due to an obstacle, pedestrian crossing, or traffic congestion, the drivers in the following vehicles often lack the time to react in time, leading to rear-end collisions. Although existing advanced driver assistance systems (ADAS) can obtain some information through radar or vehicle-to-everything (V2X) technology, these solutions are either costly or rely on complex vehicle-to-vehicle communication protocols and are not yet widespread. Currently, there is a lack of an intuitive, low-cost, and easily deployable technological solution that allows drivers in the following vehicles to directly obtain visual information about the road ahead, as if they could "see through" the vehicle in front.

[0025] In view of this, on the one hand, embodiments of the present invention provide a method for sharing vehicle perspective images, including: acquiring environmental images in front of and / or to the side of the vehicle in real time; performing image processing on the environmental images, the image processing including at least one or more of image restoration processing, multi-channel image stitching processing and target annotation processing, to obtain a processed environmental image; and displaying the processed environmental image to the rear of the vehicle for observation by the driver of a vehicle located behind the vehicle.

[0026] The vehicle perspective image sharing method provided in this embodiment of the invention acquires environmental images in front of and / or to the side of the vehicle and displays them to vehicles behind, enabling drivers of following vehicles to anticipate the braking or steering intentions of the vehicle in front, effectively reducing rear-end collisions caused by obstructed vision and improving the overall safety of convoy driving.

[0027] In some embodiments, the vehicle perspective image sharing method provided by the present invention can be executed by any electronic device 20 with data processing capabilities, such as a general-purpose computer, personal computer, laptop computer, switch, or tablet computer, etc. The specific implementation of the electronic device 20 is not limited here.

[0028] Figure 1A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention is shown. The electronic device 20 includes a processor 210, a memory 220, and a communication interface 230.

[0029] Processor 210 may include one or more processing cores. Processor 210 connects to various parts within electronic device 200 using various interfaces and lines, and performs various functions and processes data of electronic device 200 by running or executing instructions, programs, code sets, or instruction sets stored in memory 220, and by calling data stored in memory 220. Optionally, processor 210 may be implemented using at least one of the following hardware forms: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA).

[0030] The memory 220 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 220 may include a non-transitory computer-readable storage medium. The memory 220 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 220 may include a program storage area. This program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described above, etc.

[0031] Communication interface 230 is used to communicate with other devices, equipment or communication networks, such as data storage devices, image processing devices or Ethernet, wireless access network (RAN), wireless local area network (WLAN), etc.

[0032] In terms of physical implementation, the aforementioned devices (such as processor 210, memory 220, and communication interface 230) can each be devices within the same device (such as a laptop computer). Alternatively, at least two of these devices can be located within the same device, i.e., as different devices within the same device, similar to the deployment of devices or components in a distributed system.

[0033] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 20. In other embodiments of the present invention, the electronic device 20 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0034] The following description, in conjunction with the accompanying drawings, illustrates a method for sharing vehicle perspective images according to an embodiment of the present invention.

[0035] like Figure 2 As shown, this embodiment of the invention provides a method for sharing vehicle perspective images, applied to software-defined networks, and may include, but is not limited to: S1: Real-time acquisition of environmental images in front of and / or to the side of the vehicle.

[0036] In practical implementation, embodiments of the present invention can use a high-definition camera installed at the front of the vehicle (such as the front bumper, grille, or inside the windshield) to collect real-time images of the road conditions in front of the vehicle. Simultaneously, side-view cameras installed on the left and right sides of the vehicle can acquire broader information about the surrounding environment on both sides. It should be noted that the high-definition camera and / or the side-view camera can be factory-installed on the vehicle or added later; this is not limited here.

[0037] In specific implementation, the high-definition camera and / or the side-view camera in the embodiments of the present invention have built-in image sensors and analog-to-digital conversion circuits. The image sensor converts the collected light signals into analog electrical signals, which are then processed by the analog-to-digital conversion circuit to generate a first digital image signal that conforms to a specific format (such as RGB888, YUV422) for subsequent image processing.

[0038] S2: Perform image processing on the environmental image, the image processing including at least one or more of image restoration processing, multi-channel image stitching processing and target annotation processing, to obtain the processed environmental image.

[0039] In a specific implementation, this embodiment of the invention may embed an Electronic Control Unit (ECU) within the vehicle, dedicated to image processing of the environmental images. The ECU may include, but is not limited to, a receiving unit, an MCU processing unit, a transmitting unit, and a power supply unit, with the power supply unit providing power to the receiving unit, the MCU processing unit, and the transmitting unit. The receiving unit receives the environmental images transmitted by the high-definition camera and / or the side-view camera, i.e., the first digital image signal, via a transmission cable (e.g., a twisted-pair cable using the vehicle Ethernet protocol), and performs physical layer and data link layer protocol parsing on the first digital image signal, converting it into a parallel digital signal that the MCU processing unit can directly operate.

[0040] After receiving the environmental image data, the MCU processing unit performs image processing according to a preset algorithm logic. The image processing includes at least one or more of the following: image restoration processing, multi-channel image stitching processing, and target annotation processing, as detailed below: In one feasible implementation, the image restoration processing of the environmental image described in this embodiment of the invention may include, but is not limited to: Detect whether there are localized blurred or missing areas in the environmental image due to stains or occlusions; If the aforementioned locally blurred or missing regions exist, interpolation calculations are performed using the clear pixel information surrounding the locally blurred or missing regions to generate predicted pixel values. These predicted pixel values ​​are then used to fill the locally blurred or missing regions to generate a repaired environmental image.

[0041] In practice, when the camera lens surface is covered with water droplets, mud, dust, or other contaminants, the captured image may exhibit localized blurring, abnormal brightness, or even pixel loss. Figure 3 As shown, the image optimization module of the MCU processing unit first locates blurred or missing regions in the environmental image using an image anomaly detection algorithm (such as gradient-based edge detection or deep learning-based smudged region segmentation). Then, it performs interpolation calculations using the clear and valid pixel information surrounding the missing region (i.e., the outer contour of the smudged region). The interpolation algorithm can be traditional bilinear interpolation, bicubic interpolation, or more advanced image inpainting algorithms based on partial differential equations or deep learning-based image generation networks, etc., without limitation. Through interpolation calculations, predicted pixel values ​​are generated to fill the missing regions, thus obtaining a repaired environmental image, such as... Figure 4 As shown.

[0042] It should be noted that the various interpolation algorithms used in the embodiments of the present invention are all existing mature interpolation algorithms, and their specific interpolation calculation formulas and calculation steps are well known to those skilled in the art, and will not be repeated here.

[0043] In one feasible implementation, after generating the restored environmental image, the embodiments of the present invention may, but are not limited to, further include: The repaired environmental image is logically judged using a pre-trained image discrimination model. If it is determined that the repaired environmental image does not conform to the logic of the real scene, then the repaired environmental image is discarded.

[0044] By performing logical judgments on the repaired environmental image, this embodiment of the invention can prevent the interpolation algorithm from generating erroneous content that does not conform to the logic of the road scene, thus ensuring the accuracy of the display.

[0045] In specific implementation, the image discrimination model in this embodiment of the invention can be, but is not limited to, a semantic segmentation model based on a convolutional neural network (CNN). This model has been fully trained on a dataset containing tens of thousands of labeled images of urban and highway scenes. The model analyzes the semantic information of the repair area and its surroundings—for example, if the model identifies the surrounding area as "road surface" and the pixels of the repair area are also classified as "road surface" or "vehicle," it is determined to be "logically correct"; if the pixels of the repair area are classified as "sky" or "building," it is determined to be "illogically incorrect." Only repaired images that pass the logical verification will be adopted for subsequent processes; otherwise, the system will abandon the current repair and may choose to use historical valid data from the previous frame or request camera cleaning.

[0046] In one feasible implementation, the environmental image in this embodiment of the invention is a multi-channel environmental image, and the multi-channel image stitching process of the environmental image may include, but is not limited to: Feature analysis is performed on each of the environmental images to identify and determine the overlapping areas between the environmental images. The overlapping areas in each of the environmental images are cropped and deleted, and the remaining non-overlapping image areas after cropping are edge-blended and stitched together to generate a continuous display image with a wide viewing angle.

[0047] In specific implementation, when the vehicle is equipped with multiple cameras (such as front view + left view + right view), the image stitching module of the MCU processing unit performs stitching processing. For example... Figure 5 As shown, three environmental images are first acquired simultaneously by the front-facing camera, the left-side camera, and the right-side camera.

[0048] Subsequently, feature points are extracted and matched for each image (e.g., using SIFT, SURF, or ORB algorithms), and the overlapping area between the fields of view of two adjacent images is identified and determined through the correspondence of feature points.

[0049] Next, as Figure 6 As shown, select one or a set of optimal seam lines within the overlapping area, and crop and delete the overlapping pixel areas in each image along the seam lines.

[0050] Finally, the remaining non-overlapping image areas after cropping are edge-blended (e.g., multi-band fusion or feathering fusion) and seamlessly stitched along the stitching line, ultimately generating a wide-angle, continuous panoramic display image with a field of view far greater than that of a single camera, such as... Figure 7 As shown.

[0051] This invention stitches together multiple images acquired by multiple cameras to obtain a panoramic image with a wider field of view than a single camera, enabling drivers behind to have a more comprehensive understanding of the lateral and longitudinal traffic conditions in front of the vehicle, thereby further reducing safety hazards.

[0052] In one feasible implementation, the target annotation processing of the environmental image described in this embodiment of the invention may include, but is not limited to: The target recognition algorithm is used to identify key targets in the environmental image, and the key targets include at least one of the following: a vehicle, a pedestrian, or an obstacle. The identified key targets are marked with visual markers, including at least one of highlighted borders, color fills, or warning icons.

[0053] In specific implementation, the target recognition algorithm in this embodiment of the invention adopts existing mature target recognition algorithms, such as deep learning-based target detection networks YOLO, SSD, or Faster R-CNN, etc., and is not limited here. The target recognition algorithm is built into the target annotation module of the MCU processing unit. The target annotation module runs in parallel or serially with the image optimization module and the image stitching module, and performs real-time analysis on the received original environmental image or the image after repair / stitching to identify key targets in the image. The key targets include, but are not limited to: vehicles ahead, pedestrians, non-motorized vehicles (bicycles, electric vehicles), large animals, traffic cones, obstacles, etc.

[0054] Once the key target is identified, the MCU will perform visual enhancement annotations on that target when generating the final display image. For example... Figure 8As shown, the annotation can be done by selecting the target area with a bright rectangle (e.g., red or yellow border), filling the target area with a semi-transparent color, or overlaying a warning icon (e.g., an exclamation mark) near the target.

[0055] In one feasible implementation, the target annotation processing of the environmental image described in this embodiment of the invention may, but is not limited to, include: When the identified critical target is determined to be a dangerous target with a potential collision risk, a warning instruction is generated. The warning command is sent to the vehicle's intelligent driving control system and / or cockpit control system to trigger the vehicle's warning operation or braking operation.

[0056] In specific implementation, the MCU processing unit in this embodiment of the invention can combine the vehicle speed and acceleration information obtained from the vehicle's CAN bus, as well as the target distance and relative speed information estimated through visual ranging, to assess the threat level of the target. When it is determined that the time to collision (TTC) between a critical target (e.g., a vehicle rapidly decelerating directly in front) and the vehicle is less than a preset threshold (e.g., 5 seconds), it is determined to be a "dangerous target" with potential collision risk.

[0057] At this point, the MCU processing unit will not only highlight the warning on the display screen (e.g., flashing borders, enlarged warning icons), but also generate a warning command containing the hazard level and target information. This warning command is routed to the vehicle network via the gateway and sent to the intelligent driving control system and / or the cockpit control system. Upon receiving the command, the intelligent driving control system can perform avoidance actions such as pre-filling brake fluid pressure and active braking according to the hazard level; upon receiving the command, the cockpit control system can trigger warning operations such as flashing instrument panel warning lights, emitting a rapid warning sound, and pre-tensioning the driver's seat belt.

[0058] S3: Display the processed environmental image to the rear of the vehicle for observation by the driver of a vehicle located behind the vehicle.

[0059] In one feasible implementation, the display of the processed environmental image to the rear of the vehicle in this embodiment of the invention may include, but is not limited to: The processed environmental image is displayed to the rear of the vehicle via a display device installed in the rear windshield area of ​​the vehicle. The display device is a one-way transparent LCD display or a one-way transparent OLED display.

[0060] In specific implementation, the optical characteristics of the unidirectional transparent LCD display or the unidirectional transparent OLED display in the embodiments of the present invention are as follows: when viewed from the back (facing outwards from the vehicle), the image content can be clearly displayed; when viewed from the front (facing inwards from the vehicle), it is transparent, allowing light to pass through.

[0061] In the specific implementation process, after the MCU processing unit completes the image processing, it transmits the final generated display screen data to the sending unit. The sending unit, according to the interface protocol supported by the downstream display device, repackages and serializes the parallel image data into a second digital image signal. The transmission protocols used include, but are not limited to, LVDS (Low Voltage Differential Signaling, which has the advantages of high bandwidth, low latency, and strong anti-interference), ePB (Embedded Panel Bus), or automotive Ethernet. The packaged second digital image signal is transmitted to the display device at the rear windshield via transmission cable E.

[0062] The display device receives the second digital image signal, restores the signal to a display image through its built-in drive circuit and decoding chip, and drives the display panel to display the image. The display device clearly presents the image on the side (back) facing the rear of the vehicle, allowing drivers of following vehicles to see the road conditions in front of the vehicle intuitively, as if the vehicle body were "transparent".

[0063] Meanwhile, because the display device employs one-way transparent display technology, its side facing the interior of the vehicle (front) always maintains a high light transmittance. When the driver needs to observe the rear, looking through the center rearview mirror onto the rear windshield, their line of sight can directly penetrate the display device, clearly seeing the actual vehicles and road conditions directly behind and to the sides of the vehicle. Therefore, the display screen does not interfere with or obstruct the driver's normal rearward observation in any way.

[0064] In specific implementation, in addition to displaying the processed environmental image to the rear of the vehicle via a display device installed in the rear windshield area, this embodiment of the invention can also add a perspective correction step to improve the viewing experience for rear drivers. For example: One or more ultrasonic radars or a rear-facing wide-angle camera are installed at the rear bumper of this vehicle to detect the relative position of following vehicles, including the following distance and the lateral offset angle of the following vehicle relative to the longitudinal centerline of this vehicle. Before displaying the processed environmental image to the rear of this vehicle via a display device installed in the rear windshield area of ​​this vehicle, the MCU processing unit receives relative position data from the rear sensors. Based on the distance and lateral offset angle of the following vehicles, the MCU processing unit performs geometric transformations (e.g., perspective transformation, affine transformation, or image warping) on ​​the image to be sent to the display device. Specifically, if the following vehicle is offset to the left of this vehicle, the right half of the displayed image is appropriately stretched or the viewing angle is shifted so that the view seen by the rear driver is closer to the forward-looking view assumed to be in that position.

[0065] The perspective-corrected display image provides a stronger sense of presence and intuitiveness for drivers behind, making it easier for them to match the target location in the image with their own spatial judgment, thus further improving the accuracy of their predictions.

[0066] In specific implementation, embodiments of the present invention can also dynamically adjust image processing parameters by acquiring the vehicle's driving status information, for example: The Electronic Control Unit (ECU) obtains real-time driving status information of the vehicle via the CAN bus, including current vehicle speed, turn signal status, steering wheel angle, etc.

[0067] The MCU processing unit dynamically adjusts the image processing parameters based on this driving status information. This may include, but is not limited to, the following: When the vehicle speed exceeds the preset high-speed threshold (e.g., 80km / h), the system automatically requests the high-definition camera and / or the side-view camera to increase the image acquisition frame rate (e.g., from 30fps to 60fps) and reduce the image compression ratio. At the same time, the MCU processing unit prioritizes low processing latency and reduces or temporarily shuts down the computationally intensive image optimization module to ensure that the image seen by the driver of the following vehicle has the highest real-time performance.

[0068] When the system detects that the vehicle's turn signal is on and the steering wheel angle is greater than a preset angle (e.g., 45°), it determines that the vehicle is about to change lanes or turn. At this time, the image stitching module of the MCU processing unit automatically adjusts the stitching weights, displaying more of the side camera image on the turning side to alert drivers of following vehicles to potential risks on the side of the vehicle.

[0069] Through this dynamic adjustment mechanism, the embodiments of the present invention can provide optimal image sharing effect and system resource utilization under different driving conditions.

[0070] Compared with the prior art, the vehicle perspective image sharing method provided in this embodiment of the invention has the following beneficial effects: This method displays the road conditions ahead of the vehicle in real time and intuitively on the rear windshield, allowing drivers of following vehicles to anticipate the braking or steering intentions of the vehicle in front. It transforms the passive reaction to rear-end collisions into proactive prediction, effectively reducing traffic accidents caused by obstructed vision. It is especially suitable for high-density convoy driving scenarios.

[0071] This method uses a one-way transparent display screen as the display device, ensuring that the driver sees a real, unobstructed view of the traffic conditions behind them when observing the rear through the central rearview mirror. This fundamentally avoids the safety hazard of the driver losing their rear view due to the display screen obstructing the rear windshield, achieving an organic unity between information sharing and driving safety.

[0072] This method integrates multiple image processing algorithms, such as image restoration, multi-path stitching, and key annotation, to provide drivers behind with clear images that have been cleaned and restored, wide-view stitched images that eliminate blind spots, and enhanced annotated images with hazard warnings, greatly improving the integrity, accuracy, and readability of the information conveyed.

[0073] This method can interact with the vehicle's intelligent driving system and cockpit system. When a collision risk is detected, it can not only mark and remind the user on the display screen, but also simultaneously trigger the vehicle's active safety mechanisms (such as warning and pre-braking), forming a progressive safety protection system from visual sharing to active protection.

[0074] This method can be upgraded based on the vehicle's existing cameras, display panels, and in-vehicle networks without requiring large-scale modifications to the vehicle structure. The cost is relatively controllable, and it has good prospects for mass production applications.

[0075] Based on the vehicle perspective image sharing method provided in the first aspect, embodiments of the present invention provide a vehicle perspective image sharing system, such as... Figure 9 As shown, the vehicle-mounted perspective image sharing system includes: Image acquisition module 110 is used to acquire environmental images in front of and / or to the side of the vehicle in real time; Image processing module 120 is used to perform image processing on the environmental image, wherein the image processing includes at least one or more of image restoration processing, multi-channel image stitching processing, and target annotation processing to obtain the processed environmental image; Display module 130 is used to display the processed environmental image to the rear of the vehicle for observation by the driver of the vehicle located behind the vehicle.

[0076] Based on the vehicle perspective image sharing method provided in the first aspect, this embodiment of the invention also provides a storage medium storing a computer-executable program. The computer-executable program is used to cause a computer to execute the vehicle perspective image sharing method as described in any implementation of the first aspect. Explanations of the relevant content and descriptions of the beneficial effects of any of the computer-readable storage media provided above can be found in the corresponding embodiments described above, and will not be repeated here.

[0077] Those skilled in the art will understand that the program for implementing all or part of the steps of the above embodiments, which can be executed by a program instructing related hardware, can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a random access memory, etc. The processing unit or processor mentioned above can be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0078] This invention also provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform any of the methods described in the above embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD), etc.

[0079] It should be noted that the devices for storing computer instructions or computer programs provided in the embodiments of the present invention, such as, but not limited to, the aforementioned memory, computer-readable storage medium, and communication chip, are all non-transitory. Those skilled in the art should recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0080] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for sharing a see-through image for a vehicle, characterized by, include: Real-time acquisition of environmental images in front of and / or to the sides of the vehicle; The environmental image is processed, and the image processing includes at least one or more of image restoration processing, multi-channel image stitching processing, and target annotation processing to obtain a processed environmental image. The processed environmental image is displayed to the rear of the vehicle for observation by the driver of a vehicle located behind the vehicle.

2. The method for sharing vehicle perspective images according to claim 1, characterized in that, The image restoration process for the environmental image includes: Detect whether there are localized blurred or missing areas in the environmental image due to stains or occlusions; If the aforementioned locally blurred or missing regions exist, interpolation calculations are performed using the clear pixel information surrounding the locally blurred or missing regions to generate predicted pixel values. These predicted pixel values ​​are then used to fill the locally blurred or missing regions to generate a repaired environmental image.

3. The method for sharing vehicle perspective images according to claim 2, characterized in that, After generating the repaired environmental image, the method further includes: The repaired environmental image is logically judged using a pre-trained image discrimination model. If it is determined that the repaired environmental image does not conform to the logic of the real scene, then the repaired environmental image is discarded.

4. The method for sharing vehicle perspective images according to claim 1, characterized in that, The environmental image is a multi-channel environmental image, and the multi-channel image stitching process for the environmental image includes: Feature analysis is performed on each of the environmental images to identify and determine the overlapping areas between the environmental images. The overlapping areas in each of the environmental images are cropped and deleted, and the remaining non-overlapping image areas after cropping are edge-blended and stitched together to generate a continuous display image with a wide viewing angle.

5. The method for sharing vehicle perspective images according to claim 1, characterized in that, The target annotation process for the environmental image includes: The target recognition algorithm is used to identify key targets in the environmental image, and the key targets include at least one of the following: a vehicle, a pedestrian, or an obstacle. The identified key targets are marked with visual markers, including at least one of highlighted borders, color fills, or warning icons.

6. The method for sharing vehicle perspective images according to claim 5, characterized in that, The process of performing target annotation processing on the environmental image further includes: When the identified critical target is determined to be a dangerous target with a potential collision risk, a warning instruction is generated. The warning command is sent to the vehicle's intelligent driving control system and / or cockpit control system to trigger the vehicle's warning operation or braking operation.

7. The method for sharing vehicle perspective images according to claim 1, characterized in that, Displaying the processed environmental image to the rear of the vehicle includes: The processed environmental image is displayed to the rear of the vehicle via a display device installed in the rear windshield area of ​​the vehicle. The display device is a one-way transparent LCD display or a one-way transparent OLED display.

8. A vehicle-mounted perspective image sharing system, characterized in that, include: The image acquisition module is used to acquire real-time environmental images of the front and / or sides of the vehicle; The image processing module is used to perform image processing on the environmental image, wherein the image processing includes at least one or more of image restoration processing, multi-channel image stitching processing, and target annotation processing to obtain the processed environmental image; The display module is used to display the processed environmental image to the rear of the vehicle for observation by the driver of the vehicle located behind the vehicle.

9. An electronic device, characterized in that, include: A memory, one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the vehicle perspective image sharing method as described in any one of claims 1 to 7.

10. A storage medium storing a computer-executable program, characterized in that, The computer-executable program is used to cause the computer to perform the vehicle perspective image sharing method as described in any one of claims 1 to 7.

Images