Vehicle exterior image display method and device, vehicle and computer readable storage medium

By enhancing and removing precipitation objects on real-time outdoor images collected by the on-board camera, the problem of unclear outdoor images in dim and precipitation environments is solved, providing a clearer outdoor field of view, improving the driver's driving safety .

CN118921440APending Publication Date: 2024-11-08CHONGQING CHANGAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411159849.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In dim and precipitation environments, the vehicle lighting effect decreases, resulting in a decrease in the driver's field of view clarity, and it is difficult for the prior art to provide clear out-of-vehicle images.

Method used

By obtaining the real-time outdoor images collected by the on-board camera, we can determine whether it is a low-bright precipitation environment, and perform precipitation objects enhancement and removal operations to obtain a clear outdoor images and display them on the display screen.

Benefits of technology

In the case of insufficient light at night, improve the clarity of precipitation objects in real-time outdoor images, accurately remove precipitation objects, provide clear outdoor images, and enhance the driver's vision clarity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118921440A_ABST
    Figure CN118921440A_ABST
Patent Text Reader

Abstract

The invention provides an out-of-vehicle image display method and device, a vehicle and a computer readable storage medium, and the method comprises the steps: obtaining a real-time out-of-vehicle image collected by a vehicle-mounted camera, and determining whether the current driving environment of the vehicle is a low-brightness rainfall environment or not; if the current driving environment of the vehicle is the low-brightness precipitation environment, performing precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image; removing a precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image; and displaying the clear image outside the vehicle. The rainfall object enhancement is performed on the real-time image outside the vehicle, so that the clear degree of the rainfall object in the real-time image outside the vehicle is improved under the condition of insufficient illumination at night, the rainfall object can be removed more accurately, the clear image outside the vehicle is obtained, and the user experience is improved. And clear vehicle exterior images can be provided for a driver based on the clear vehicle exterior images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of assisted driving, and in particular to a method and device for displaying an image outside a vehicle, a vehicle, and a computer-readable storage medium. Background Art

[0002] When driving in a dim or rainy environment, the poor light and the diffuse reflection of light caused by rain will reduce the lighting effect of the vehicle and reduce the clarity of the driver's field of vision. In order to solve this problem, the prior art usually collects images of the environment outside the vehicle and displays the images on a display screen so that the driver can clearly understand the environment outside the vehicle by observing the display screen. However, in the case of precipitation, falling raindrops, snow, etc. will cause the collected images to be unclear, making it impossible to display a clear image on the display screen. Summary of the invention

[0003] The main purpose of the present invention is to propose a method, device, vehicle and computer-readable storage medium for displaying images outside the vehicle, aiming to solve the problem of how to provide the driver with clear images outside the vehicle when driving in a dim or precipitation environment in the prior art.

[0004] To achieve the above object, the present invention provides a method for displaying an image outside a vehicle, the method comprising the steps of:

[0005] Obtain real-time exterior images captured by the vehicle camera and determine whether the vehicle's current driving environment is a low-light precipitation environment;

[0006] If the current driving environment of the vehicle is a low-brightness precipitation environment, performing a precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image;

[0007] Performing a removal operation on the precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image;

[0008] The clear vehicle exterior image is displayed.

[0009] Optionally, determining whether the current driving environment of the vehicle is a low-brightness precipitation environment includes:

[0010] Acquiring precipitation data detected by a rain sensor of the vehicle and light data detected by a light sensor;

[0011] Determining whether the current driving environment of the vehicle is a precipitation environment according to the precipitation data, and determining whether the current driving environment of the vehicle is a low-light environment according to the light data;

[0012] If the current driving environment of the vehicle is a precipitation environment and a low-brightness environment, then the current driving environment of the vehicle is a low-brightness precipitation environment.

[0013] Optionally, the precipitation object includes raindrops, and performing a precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image includes:

[0014] After denoising the real-time image outside the vehicle, the contrast of the real-time image outside the vehicle is improved to obtain a first processed image corresponding to the real-time image outside the vehicle;

[0015] Performing a color enhancement operation on the first processed image to obtain a second processed image corresponding to the first processed image;

[0016] The target position of the raindrops is determined in the second processed image, and a clarity enhancement operation is performed on the pixels where the target position is located to obtain the enhanced vehicle exterior image.

[0017] Optionally, the precipitation object includes raindrops, and removing the precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image includes:

[0018] Detecting raindrop tracks in the enhanced vehicle exterior image;

[0019] determining raindrops in the enhanced vehicle exterior image based on the raindrop trajectories;

[0020] Segmenting the raindrops from the enhanced vehicle exterior image to obtain a segmented image corresponding to the enhanced vehicle exterior image that does not contain the raindrops;

[0021] The blank part in the segmented image is filled with an image to obtain the clear vehicle exterior image corresponding to the segmented image.

[0022] Optionally, displaying the clear vehicle exterior image includes:

[0023] Determining a reflective road surface area in the clear vehicle exterior image;

[0024] Softening the reflective road surface area to obtain a soft-light exterior image corresponding to the clear exterior image;

[0025] The soft light exterior vehicle image is displayed.

[0026] Optionally, acquiring a real-time image outside the vehicle collected by a vehicle-mounted camera includes:

[0027] Monitoring the control behavior of the vehicle and determining whether a turn signal switching signal is detected;

[0028] If the turn signal switching signal is detected, determining the target driving direction corresponding to the turn signal switching signal;

[0029] Matching a target camera corresponding to the target driving direction among the on-board cameras;

[0030] Acquire the real-time vehicle exterior image captured by the target camera.

[0031] Optionally, the real-time vehicle exterior image includes a real-time vehicle front image and a real-time vehicle side image, and acquiring the real-time vehicle exterior image captured by the vehicle-mounted camera includes:

[0032] Acquire the real-time front image of the vehicle, monitor the control behavior of the vehicle, and determine whether a valid turn signal is detected;

[0033] If the effective signal of the turn signal lamp is detected, determining the target turning direction corresponding to the effective signal of the turn signal lamp;

[0034] Matching a target camera corresponding to the target turning direction among the on-board cameras;

[0035] Acquire the real-time vehicle side image captured by the target camera;

[0036] The displaying of the clear vehicle exterior image comprises:

[0037] Displaying the clear vehicle exterior image corresponding to the real-time vehicle front image in full screen;

[0038] An image window is created, and the clear vehicle exterior image corresponding to the real-time vehicle side image is displayed in full screen in the image window.

[0039] To achieve the above object, the present invention further provides an external image display device, the external image display device comprising:

[0040] The first acquisition module is used to acquire the real-time vehicle exterior image captured by the vehicle-mounted camera and determine whether the current driving environment of the vehicle is a low-brightness precipitation environment;

[0041] A first enhancement module is used for performing a precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image if the current driving environment of the vehicle is a low-brightness precipitation environment;

[0042] A first removal module is used to remove the precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image;

[0043] The first display module is used to display the clear image outside the vehicle.

[0044] To achieve the above objectives, the present invention also provides a vehicle, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for displaying an external image of a vehicle as described above.

[0045] To achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the vehicle exterior image display method as described above are implemented.

[0046] The present invention proposes a method, device, vehicle and computer-readable storage medium for displaying an image outside the vehicle, which obtains a real-time image outside the vehicle captured by a vehicle-mounted camera and determines whether the current driving environment of the vehicle is a low-brightness precipitation environment; if the current driving environment of the vehicle is a low-brightness precipitation environment, the real-time image outside the vehicle is enhanced for precipitation objects to obtain an enhanced image outside the vehicle corresponding to the real-time image outside the vehicle; the precipitation objects in the enhanced image outside the vehicle are removed to obtain a clear image outside the vehicle corresponding to the enhanced image outside the vehicle; and the clear image outside the vehicle is displayed. By enhancing the precipitation objects in the real-time image outside the vehicle, the clarity of the precipitation objects in the real-time image outside the vehicle is improved in the case of insufficient light at night, so that the precipitation objects can be removed more accurately to obtain a clear image outside the vehicle, and a clear image outside the vehicle can be provided to the driver based on the clear image outside the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 It is a schematic diagram of the flow chart of the first embodiment of the vehicle exterior image display method of the present invention;

[0050] Figure 2 It is a schematic diagram of the arrangement of the vehicle-mounted camera in the vehicle exterior image display method of the present invention;

[0051] Figure 3 It is a schematic diagram of a module of the vehicle exterior image display device of the present invention;

[0052] Figure 4 It is a schematic diagram of the module structure of the vehicle of the present invention. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present application.

[0054] The present invention provides a method for displaying an image outside a vehicle, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of a method for displaying an image outside a vehicle according to the present invention. The method comprises the following steps:

[0055] Step S10, obtaining a real-time image outside the vehicle captured by the vehicle-mounted camera, and determining whether the current driving environment of the vehicle is a low-brightness precipitation environment;

[0056] The vehicle-mounted camera is a camera installed on the vehicle; it is understandable that the vehicle-mounted camera may be one or more, such as multiple vehicle-mounted cameras are respectively arranged in different directions such as the front, rear, side, and bottom of the vehicle.

[0057] The real-time outside-vehicle image is an image acquired by the on-board camera in real time; it is understandable that when the on-board camera is working, it continuously acquires images and displays them on the display screen in the form of video; the real-time outside-vehicle image in this embodiment can be a captured frame of image, and the video display is realized by performing the outside-vehicle image display method of this embodiment on each frame of image; in actual applications, the real-time outside-vehicle images acquired by multiple on-board cameras can be acquired simultaneously for subsequent operations, and the real-time outside-vehicle images acquired by a specified on-board camera can also be acquired for subsequent operations. The specific specified on-board camera can be determined based on the actual driving scene. For example, in general driving, the real-time outside-vehicle image acquired by the front camera of the vehicle is acquired; in reversing, the real-time outside-vehicle image acquired by the rear camera of the vehicle is acquired; in narrow road conditions, the real-time outside-vehicle image acquired by the side camera of the vehicle is acquired.

[0058] It is understandable that in a low-brightness precipitation environment, the ambient light is insufficient, the brightness of the real-time outside image captured by the on-board camera is low, and the image quality captured in a higher-brightness environment is poor; at the same time, the presence of precipitation reduces the driver's direct visibility, which increases the demand for the display screen to display related images; and the presence of precipitation will also reduce the clarity of the real-time outside image captured by the on-board camera; therefore, it is necessary to improve the display clarity of the image in a low-brightness precipitation environment. Therefore, in this embodiment, when the current driving environment of the vehicle is a low-brightness precipitation environment, subsequent operations are triggered; it should be noted that precipitation can include rain, snow, and hail; this embodiment and subsequent embodiments take rain as an example, and other precipitation types can be implemented by analogy.

[0059] Step S20, if the current driving environment of the vehicle is a low-brightness precipitation environment, performing a precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image;

[0060] It is understandable that in a low-light precipitation environment, falling raindrops or raindrops attached to the lens of the vehicle camera will make the real-time outside vehicle image captured by the vehicle camera have lower clarity. Therefore, in order to make the outside vehicle image displayed on the display screen clearer, the raindrops on the real-time outside vehicle image need to be processed.

[0061] The precipitation object is the target that precipitation affects the image outside the vehicle; for example, the precipitation object corresponding to rain is raindrops, and the precipitation object corresponding to snow is snow.

[0062] However, due to insufficient ambient light in a low-light environment, the brightness of the real-time image outside the vehicle collected is low, making it difficult to accurately process raindrops on the real-time image outside the vehicle, which will cause further blurring of the real-time image outside the vehicle. Therefore, in this embodiment, a precipitation object enhancement operation is performed on the real-time image outside the vehicle, so that the state of raindrops in the real-time image outside the vehicle can be highlighted, thereby improving the accuracy of subsequent processing of raindrops in the real-time image outside the vehicle.

[0063] If the current driving environment of the vehicle is not a low-brightness precipitation environment, no additional operation is performed.

[0064] It can be understood that when the vehicle's current driving environment is not a low-light precipitation environment, the real-time outside vehicle image captured by the on-board camera does not have the relevant problems in the low-light precipitation environment. Therefore, the real-time outside vehicle image can be displayed based on its actual driving environment; if the vehicle's current driving environment is a rainless day, the collected real-time outside vehicle image can be directly displayed; if the vehicle's current driving environment is a rainy day, the raindrops in the real-time outside vehicle image can be directly processed; if the vehicle's current driving environment is a rainy night, the real-time outside vehicle image can be denoised, color enhanced, contrast enhanced, locally enhanced, brightened, etc.

[0065] Step S30, removing the precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image;

[0066] The raindrops in the enhanced exterior image have undergone precipitation object enhancement operation, so the raindrops have become more prominent in the enhanced exterior image. Therefore, removing the raindrops in the enhanced exterior image at this time can achieve accurate raindrop removal, thereby providing a clear exterior image with a clear field of view.

[0067] It should be noted that the raindrop removal operation refers to processing the raindrop part in the enhanced exterior image to eliminate the influence of raindrops on the exterior image and obtain a complete and clear exterior image, rather than directly deleting the part where the raindrops are located to obtain an incomplete exterior image.

[0068] Step S40: displaying the clear image outside the vehicle.

[0069] After obtaining a clear image outside the vehicle, the clear image outside the vehicle can be displayed; at this time, the clear image outside the vehicle eliminates raindrops in the picture, and can provide the driver with a complete and clear image outside the vehicle.

[0070] When displaying a clear image outside the vehicle, the display position, method, etc. of the clear image outside the vehicle can be specifically set; for example, for the real-time image outside the vehicle captured by the front camera of the vehicle, the corresponding clear image outside the vehicle can be displayed on the central control display screen; for the real-time image outside the vehicle captured by the rear camera of the vehicle, the corresponding clear image outside the vehicle can be displayed on the rearview mirror.

[0071] This embodiment enhances the precipitation object in the real-time image outside the vehicle so that the clarity of the precipitation object in the real-time image outside the vehicle is improved in the case of insufficient light at night, thereby being able to more accurately remove the precipitation object and obtain a clear image outside the vehicle. Based on the clear image outside the vehicle, a clear image outside the vehicle can be provided to the driver.

[0072] Further, in a second embodiment of the vehicle exterior image display method of the present invention proposed based on the first embodiment of the present invention, the step S10 includes the steps of:

[0073] Step S11, obtaining precipitation data detected by a rain sensor of the vehicle and light data detected by a light sensor;

[0074] Step S12, judging whether the current driving environment of the vehicle is a precipitation environment according to the precipitation data, and judging whether the current driving environment of the vehicle is a low-light environment according to the light data;

[0075] Step S13: If the current driving environment of the vehicle is a precipitation environment and a low-brightness environment, then the current driving environment of the vehicle is a low-brightness precipitation environment.

[0076] It is understandable that the low-light precipitation environment refers to a low-light environment of rain; therefore, in this embodiment, whether the current driving environment of the vehicle is a low-light environment is determined by judging rain and night respectively.

[0077] Specifically, the rain sensor is used to detect the precipitation to obtain precipitation data. When it is not raining, the precipitation data indicates no rain, and when it is raining, the precipitation data indicates the amount of precipitation. Taking rainfall as an example, therefore, when the precipitation data indicates no rain, the current driving environment of the vehicle is determined to be a non-rain environment; when the precipitation data indicates rain, the current driving environment of the vehicle is determined to be a precipitation environment. It should be noted that in actual applications, light rain has little impact on driving. Therefore, a rainfall threshold can be set. When the rainfall indicated by the precipitation data is greater than the rainfall threshold, the current driving environment of the vehicle is considered to be a precipitation environment. When the rainfall indicated by the precipitation data is less than the rainfall threshold or indicates no rain, the current driving environment of the vehicle is considered to be a non-rain environment.

[0078] The light sensor is used to detect the ambient brightness and obtain light data; the light data indicates the brightness of the environment; a brightness threshold can be set for distinguishing between high brightness and low brightness. When the brightness indicated by the light data is less than the brightness threshold, the vehicle's current driving environment is considered to be a low-brightness environment, and when the brightness indicated by the light data reaches the brightness threshold, the vehicle's current driving environment is considered to be a high-brightness environment.

[0079] It can be understood that, generally, the ambient brightness is higher during the day and lower at night. Therefore, high-brightness and low-brightness environments can be distinguished by judging day and night. For example, the judgment can be made in combination with time, such as determining the current sunrise and sunset times, determining the day and night time periods based on the sunrise and sunset times, and determining whether the time period is day or night based on the specific driving time, and based on this result, assisting in distinguishing between high-brightness environments and low-brightness environments. That is, when the brightness indicated by the light data is less than the brightness threshold and the driving time is within the night time period, the vehicle's current driving environment is considered to be a low-brightness environment.

[0080] The specific types and settings of the rain sensor and light sensor can be selected based on actual needs.

[0081] When it is determined that the current driving environment of the vehicle is a precipitation environment and a low-brightness environment, it is determined that the current driving environment of the vehicle is a low-brightness precipitation environment.

[0082] If the current driving environment of the vehicle is not a precipitation environment, or is not a low-brightness environment, then the current driving environment of the vehicle is not a low-brightness precipitation environment.

[0083] It should be noted that the method of specifically determining the current driving environment of the vehicle can also be set based on actual needs; for example, the current driving environment of the vehicle can be determined by identifying the real-time outside image captured by the camera; or weather information can be obtained through the Internet to determine whether the current driving environment of the vehicle is a precipitation environment, and time information can be obtained to determine whether the current driving environment of the vehicle is a low-light environment, etc.

[0084] This embodiment can accurately determine the current driving environment of the vehicle through the rain sensor and the light sensor.

[0085] Furthermore, the precipitation object includes raindrops. In a third embodiment of the vehicle exterior image display method of the present invention proposed based on the first embodiment of the present invention, the step S20 includes the steps of:

[0086] Step S21, after denoising the real-time vehicle exterior image, improving the contrast of the real-time vehicle exterior image, and obtaining a first processed image corresponding to the real-time vehicle exterior image;

[0087] Step S22, performing a color enhancement operation on the first processed image to obtain a second processed image corresponding to the first processed image;

[0088] Step S23, determining the target position of the raindrops in the second processed image, and performing a clarity enhancement operation on the pixels where the target position is located to obtain the enhanced vehicle exterior image.

[0089] It is understandable that in a low-light environment, the ambient brightness is low, so there are more noise points in the real-time outside-vehicle images captured by the on-board camera. Therefore, the real-time outside-vehicle images need to be denoised; the specific image denoising method can be set based on actual needs, such as using a filtering algorithm to reduce the noise in the real-time outside-vehicle images to make the real-time outside-vehicle images clearer.

[0090] At the same time, in a low-light environment, due to the low ambient brightness, the overall brightness of the real-time outside-vehicle image captured by the on-board camera is low. Therefore, in order to make the raindrops in the image clearer, the contrast in the real-time outside-vehicle image is increased in this embodiment, thereby expanding the brightness range of the image to a larger range, making the bright parts more prominent and clarifying the location of the raindrops.

[0091] At the same time, in a low-light environment, due to the low ambient brightness, the color of the real-time image outside the vehicle captured by the on-board camera is relatively dim, which makes it difficult to distinguish the raindrops therein; therefore, in this embodiment, a color enhancement operation is performed on the real-time image outside the vehicle to make the color in the real-time image outside the vehicle more vivid, thereby facilitating the identification of raindrops; the specific color enhancement method can be set based on actual needs, such as improving the color saturation and brightness through color mapping and color balancing, and adjusting the proportion and intensity of each color in the image to make the color more vivid and vivid.

[0092] At the same time, objects at different positions in the real-time external vehicle image are affected to different degrees by the low-light environment. Therefore, it is necessary to perform fixed-point enhancement on the local content in the image, such as the ground and the sides of the road. The local details of the image are enhanced by local contrast enhancement and sharpening methods, making the details in a specific local area more clearly visible. This facilitates the subsequent removal of raindrops and content filling.

[0093] It should be noted that, in the present embodiment, image processing such as denoising, contrast enhancement, color enhancement, and local enhancement are provided in view of the specific impact types of low-light environments on raindrop processing; and in actual applications, settings can also be made based on actual image defects and the need for specific image processing.

[0094] This embodiment performs image processing such as denoising, contrast enhancement, color enhancement, and local enhancement on the real-time image outside the vehicle, so as to enhance the raindrops in the real-time image outside the vehicle.

[0095] Furthermore, the precipitation object includes raindrops. In a fourth embodiment of the vehicle exterior image display method of the present invention proposed based on the first embodiment of the present invention, the step S30 includes the steps of:

[0096] Step S31, detecting raindrop tracks in the enhanced vehicle exterior image;

[0097] Step S32, determining raindrops in the enhanced vehicle exterior image based on the raindrop trajectories;

[0098] Step S33, segmenting the raindrops from the enhanced vehicle exterior image to obtain a segmented image corresponding to the enhanced vehicle exterior image that does not contain the raindrops;

[0099] Step S34, performing image filling on the blank part in the segmented image to obtain the clear vehicle exterior image corresponding to the segmented image.

[0100] It is understandable that raindrops are photographed by the on-board camera when they fall. Due to the exposure time, the image outside the car collected by the on-board camera will record the process of raindrops falling, that is, raindrop trajectories; since the raindrop trajectories indicate the falling path of raindrops, and raindrops fall continuously, the positions of raindrops corresponding to the raindrop trajectories in the enhanced image outside the car can be determined through the raindrop trajectories. The specific detection method of raindrop trajectories can be set based on actual needs, such as identifying raindrops based on color, texture or other features.

[0101] After locating the raindrops in the enhanced exterior image, the raindrops can be segmented. It should be noted that the raindrop trajectories also constitute elements in the enhanced exterior image, which will also affect the display effect of the image and reduce the clarity of the image. Therefore, when segmenting the raindrops, the raindrop trajectories can be segmented simultaneously.

[0102] The specific raindrop segmentation method can be set based on actual conditions, such as grayscale segmentation, color segmentation, etc.; considering the authenticity of the image, color segmentation can be used for processing.

[0103] After the raindrops are segmented from the enhanced vehicle exterior image, a segmented image is obtained. It can be understood that the portion of the segmented image that originally corresponds to the raindrops is a blank portion. Therefore, for the sake of image integrity, the blank portion in the segmented image needs to be filled to obtain a complete and clear vehicle exterior image. The specific image filling method can be set based on actual needs.

[0104] In this embodiment, it is possible to remove raindrops from the enhanced vehicle exterior image, thereby obtaining a clear vehicle exterior image without raindrops and with a complete picture.

[0105] Further, in a fifth embodiment of the vehicle exterior image display method of the present invention proposed based on the first embodiment of the present invention, step S40 includes the following steps:

[0106] Step S41, determining a reflective road surface area in the clear vehicle exterior image;

[0107] Step S42, performing a softening process on the reflective road surface area to obtain a soft-light vehicle exterior image corresponding to the clear vehicle exterior image;

[0108] Step S43, displaying the soft light exterior image.

[0109] It is understandable that in low-light environments, the brightness is low, so the driver needs to turn on the headlights, and at the same time, the street lights on some sections of the road will also be turned on; in a precipitation environment, the road surface is wet due to rain; in this case, the headlights, street lights and other lights will be reflected by the water on the ground. When the on-board camera collects images of such scenes, the reflection of the water on the ground will produce glare in the image outside the car, greatly affecting the clarity of the picture. Therefore, it needs to be processed to achieve a good picture display.

[0110] Specifically, in this embodiment, the reflective road surface area is first determined in the clear image outside the vehicle; the reflective road surface area can be determined by image recognition, such as first determining the road surface area in the clear image outside the vehicle, and then determining whether there is a pixel group with brightness higher than the reflective brightness threshold in the pixels corresponding to the road surface area. If so, it can be determined that the area corresponding to the pixel group is the reflective road surface area.

[0111] After the reflective road surface area is determined, the reflective road surface area is softened to reduce glare and reflected light in the picture; the specific softening method can be set based on actual needs, such as reducing brightness.

[0112] Further, in a sixth embodiment of the vehicle exterior image display method of the present invention proposed based on the first embodiment of the present invention, the step S10 includes the steps of:

[0113] Step S14, monitoring the vehicle's control behavior and determining whether a turn signal switching signal is detected;

[0114] Step S15, if the turn signal switching signal is detected, determining the target driving direction corresponding to the turn signal switching signal;

[0115] Step S16, matching a target camera corresponding to the target driving direction in the vehicle-mounted cameras;

[0116] Step S17, obtaining the real-time vehicle exterior image captured by the target camera.

[0117] Control behavior refers to the driver's behavior of controlling the vehicle; control behaviors include accelerator, brake, steering, parking, parking, reversing, and windshield wipers. It is understandable that the driver has different needs for vision when performing different control behaviors; for example, when driving straight normally, the driver needs to observe the road conditions in front, when turning left, the driver needs to observe the road conditions on the left, and when turning right, the driver needs to observe the road conditions on the right; therefore, in this embodiment, the turn signal switching signal is monitored; the turn signal switching signal is used to indicate the change of the turn signal; the turn signal switching signal can explain the user's current need for steering, so as to further determine the target driving direction corresponding to the turn signal switching signal.

[0118] It should be noted that the turn signal switching signal in this embodiment indicates the change of the turn signal, not the state of the turn signal; for example, when the driver adjusts the turn signal to the left, the corresponding turn signal switching signal is from no turn to left turn, and the corresponding target driving direction is left; when the driver cancels the turn signal to the left, the corresponding turn signal switching signal is from left turn to no turn, and the corresponding target driving direction is forward; when the driver adjusts the turn signal to the right turn, the corresponding turn signal switching signal is from no turn to right turn, and the corresponding target driving direction is right; when the driver cancels the turn signal to the right turn, the corresponding turn signal switching signal is from right turn to no turn, and the corresponding target driving direction is forward.

[0119] In this embodiment, a plurality of vehicle-mounted cameras are provided, and different vehicle-mounted cameras are provided at different positions and correspondingly collect images outside the vehicle from different perspectives; Figure 2 The front camera collects real-time images outside the vehicle in front of the vehicle, the left camera collects real-time images outside the vehicle on the left side of the vehicle, and the right camera collects real-time images outside the vehicle on the right side of the vehicle.

[0120] After the target driving direction is determined, the target camera corresponding to the target driving direction can be determined; if the target driving direction is forward, the target camera is the front camera of the vehicle; if the target driving direction is left, the target camera is the left camera of the vehicle; if the target driving direction is right, the target camera is the right camera of the vehicle.

[0121] After determining the target camera, the real-time outside-vehicle image is collected based on the target camera, and a clear outside-vehicle image corresponding to the real-time outside-vehicle image is obtained based on subsequent operations, so as to realize the display of the perspective image corresponding to the target driving direction.

[0122] Further, in a seventh embodiment of the vehicle exterior image display method of the present invention proposed based on the first embodiment of the present invention, the real-time vehicle exterior image includes a real-time vehicle front image and a real-time vehicle side image, and the step S10 includes the steps of:

[0123] Step S18, acquiring the real-time front image of the vehicle, monitoring the control behavior of the vehicle, and determining whether a valid turn signal is detected;

[0124] Step S19, if the effective signal of the turn signal is detected, determining the target turning direction corresponding to the effective signal of the turn signal;

[0125] Step S110, matching a target camera corresponding to the target turning direction among the vehicle-mounted cameras;

[0126] Step S111, acquiring the real-time vehicle side image captured by the target camera;

[0127] The step S40 comprises:

[0128] Step S44, displaying the clear vehicle exterior image corresponding to the real-time vehicle front image in full screen;

[0129] Step S45, creating an image window, and displaying the clear vehicle exterior image corresponding to the real-time vehicle side image in full screen in the image window.

[0130] During vehicle driving, the direction directly in front of the vehicle is the direction that the driver pays most attention to. Therefore, in this embodiment, the real-time outside image directly in front of the vehicle, that is, the real-time front image of the vehicle is always collected, and the clear outside image corresponding to the real-time front image of the vehicle is displayed based on subsequent processing.

[0131] While displaying the clear image outside the vehicle corresponding to the real-time image in front of the vehicle, the effective signal of the turn signal is also monitored. It should be noted that the effective signal of the turn signal indicates the state of the turn signal; for example, when the driver adjusts the turn signal to turn left, the corresponding effective signal of the turn signal is turn left, and at this time, the corresponding target driving direction is left; when the driver cancels the turn signal to turn left, the corresponding effective signal of the turn signal is invalid, and there is no corresponding target driving direction; when the driver adjusts the turn signal to turn right, the corresponding effective signal of the turn signal is turn right, and at this time, the corresponding target driving direction is right; when the driver cancels the turn signal to turn right, the corresponding effective signal of the turn signal is invalid, and there is no corresponding target driving direction.

[0132] When the turn signal is valid, the target driving direction is determined to be left or right based on the specific state of the turn signal, and then the target camera corresponding to the target driving direction is determined, and the real-time vehicle side image is collected through the target camera.

[0133] When there is a collected real-time vehicle side image, the real-time vehicle side image is processed to obtain the corresponding clear image outside the vehicle; since the clear image outside the vehicle corresponding to the real-time vehicle front image is always displayed in full screen, in order to simultaneously display the clear image outside the vehicle corresponding to the real-time vehicle side image, in this embodiment, an image window is created on the display screen, and the clear image outside the vehicle corresponding to the real-time vehicle side image is displayed in the image window, thereby realizing the simultaneous display of the real-time vehicle front image and the clear image outside the vehicle corresponding to the real-time vehicle side image; and providing the driver with images from different perspectives.

[0134] Furthermore, in order to facilitate users to view the display screen, the display screen can be installed in the form of a sunflower screen so that the position of the display screen can be adjusted; the user can trigger the position setting of the display screen through voice commands or related operation buttons to adjust the position of the display screen to the most suitable viewing position; the specific setting method can be selected based on actual needs.

[0135] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0137] The present application also provides a vehicle exterior image display device for implementing the vehicle exterior image display method, see Figure 3 , the vehicle exterior image display device comprises:

[0138] The first acquisition module is used to acquire the real-time vehicle exterior image captured by the vehicle-mounted camera and determine whether the current driving environment of the vehicle is a low-brightness precipitation environment;

[0139] A first enhancement module is used for performing a precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image if the current driving environment of the vehicle is a low-brightness precipitation environment;

[0140] A first removal module is used to remove the precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image;

[0141] The first display module is used to display the clear image outside the vehicle.

[0142] The vehicle exterior image display device enhances the precipitation objects in the real-time vehicle exterior image so that the clarity of the precipitation objects in the real-time vehicle exterior image is improved in the case of insufficient light at night, thereby being able to more accurately remove the precipitation objects and obtain a clear vehicle exterior image. Based on the clear vehicle exterior image, a clear vehicle exterior image can be provided to the driver.

[0143] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiment of the present application, the first enhancement module in this embodiment can be used to execute step S20 in the embodiment of the present application, the first removal module in this embodiment can be used to execute step S30 in the embodiment of the present application, and the first display module in this embodiment can be used to execute step S40 in the embodiment of the present application.

[0144] Furthermore, the first acquisition module includes:

[0145] A first acquisition unit, used to acquire precipitation data detected by a rain sensor of the vehicle and light data detected by a light sensor;

[0146] a first judgment unit, configured to judge whether the current driving environment of the vehicle is a precipitation environment according to the precipitation data, and to judge whether the current driving environment of the vehicle is a low-light environment according to the light data;

[0147] The first execution unit is used to determine that the current driving environment of the vehicle is a low-brightness precipitation environment if the current driving environment of the vehicle is a precipitation environment and a low-brightness environment.

[0148] Furthermore, the precipitation object includes raindrops, and the first enhancement module includes:

[0149] A first denoising unit, configured to denoise the real-time vehicle exterior image, thereby improving the contrast of the real-time vehicle exterior image and obtaining a first processed image corresponding to the real-time vehicle exterior image;

[0150] A first enhancement unit, configured to perform a color enhancement operation on the first processed image to obtain a second processed image corresponding to the first processed image;

[0151] The second enhancement unit is used to determine the target position of the raindrops in the second processed image, and perform a clarity enhancement operation on the pixels where the target position is located to obtain the enhanced vehicle exterior image.

[0152] Furthermore, the precipitation object includes raindrops, and the first removal module includes:

[0153] A first detection unit, configured to detect raindrop tracks in the enhanced vehicle exterior image;

[0154] A first determining unit, configured to determine raindrops in the enhanced vehicle exterior image based on the raindrop trajectories;

[0155] A first segmentation unit is used to segment the raindrops from the enhanced vehicle exterior image to obtain a segmented image corresponding to the enhanced vehicle exterior image that does not contain the raindrops;

[0156] The first filling unit is used to fill the blank part in the segmented image to obtain the clear vehicle exterior image corresponding to the segmented image.

[0157] Furthermore, the first display module includes:

[0158] A second determining unit is used to determine a reflective road surface area in the clear vehicle exterior image;

[0159] A first softening unit, configured to perform a softening process on the reflective road surface area to obtain a soft-light vehicle exterior image corresponding to the clear vehicle exterior image;

[0160] The first display unit is used to display the soft light exterior image of the vehicle.

[0161] Furthermore, the first acquisition module includes:

[0162] A first monitoring unit is used to monitor the control behavior of the vehicle and determine whether a turn signal switching signal is detected;

[0163] a third determining unit, configured to determine a target driving direction corresponding to the turn signal switching signal if the turn signal switching signal is detected;

[0164] A fourth determination unit, configured to match a target camera corresponding to the target driving direction among the vehicle-mounted cameras;

[0165] The second acquisition unit is used to acquire the real-time vehicle exterior image captured by the target camera.

[0166] Furthermore, the real-time vehicle exterior image includes a real-time vehicle front image and a real-time vehicle side image, and the first acquisition module includes:

[0167] A third acquisition unit is used to acquire the real-time front image of the vehicle, monitor the control behavior of the vehicle, and determine whether a valid turn signal is detected;

[0168] a fifth determining unit, configured to determine a target turning direction corresponding to the effective signal of the turn signal if the effective signal of the turn signal is detected;

[0169] A first matching unit, used for matching a target camera corresponding to the target turning direction in the vehicle-mounted camera;

[0170] A fourth acquisition unit, configured to acquire the real-time vehicle side image captured by the target camera;

[0171] The first display module comprises:

[0172] A second display unit, used for displaying the clear vehicle exterior image corresponding to the real-time vehicle front image in full screen;

[0173] The first creating unit is used to create an image window, and to display the clear vehicle exterior image corresponding to the real-time vehicle side image in the image window in full screen.

[0174] Reference Figure 4 In terms of hardware structure, the vehicle may include components such as a communication module 10, a memory 20, and a processor 30. In the vehicle, the processor 30 is connected to the memory 20 and the communication module 10 respectively, and a computer program is stored in the memory 20. The computer program is executed by the processor 30 at the same time, and the steps of the above method embodiment are implemented when the computer program is executed.

[0175] The communication module 10 can be connected to an external communication device through a network. The communication module 10 can receive requests from an external communication device, and can also send requests, instructions and information to the external communication device, which can be other vehicles, servers or IoT devices, such as televisions, etc.

[0176] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as obtaining real-time images outside the vehicle collected by a vehicle-mounted camera), etc.; the data storage area can include a database, and the data storage area can store data or information created according to the use of the system, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0177] The processor 30 is the control center of the vehicle. It uses various interfaces and lines to connect various parts of the entire vehicle. It executes various functions of the vehicle and processes data by running or executing software programs and / or modules stored in the memory 20, and calling data stored in the memory 20, so as to monitor the vehicle as a whole. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, driver interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 30.

[0178] although Figure 4 Although not shown, the above vehicle may further include a circuit control module, which is used to connect to a power source to ensure the normal operation of other components. Figure 4 The vehicle structure shown in the figure does not constitute a limitation on the vehicle, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0179] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium may be Figure 4 The memory 20 in the vehicle may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a TV, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0180] In the present invention, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0181] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0182] Although the embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify and replace the above embodiments within the scope of the present invention, and these changes, modifications and replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for displaying an image outside a vehicle, characterized in that: The vehicle exterior image display method comprises: Obtain real-time images outside the vehicle collected by the on-board camera, and determine whether the current driving environment of the vehicle is a low-light precipitation environment; If the current driving environment of the vehicle is a low-brightness precipitation environment, performing a precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image; Performing a removal operation on the precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image; The clear vehicle exterior image is displayed.

2. The method for displaying an image outside a vehicle according to claim 1, wherein: Determining whether the current driving environment of the vehicle is a low-brightness precipitation environment includes: Acquiring precipitation data detected by a rain sensor of the vehicle and light data detected by a light sensor; Determining whether the current driving environment of the vehicle is a precipitation environment according to the precipitation data, and determining whether the current driving environment of the vehicle is a low-light environment according to the light data; If the current driving environment of the vehicle is a precipitation environment and a low-brightness environment, then the current driving environment of the vehicle is a low-brightness precipitation environment.

3. The method for displaying an image outside a vehicle according to claim 1, wherein: The precipitation object includes raindrops, and the performing of the precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image includes: After denoising the real-time image outside the vehicle, the contrast of the real-time image outside the vehicle is improved to obtain a first processed image corresponding to the real-time image outside the vehicle; Performing a color enhancement operation on the first processed image to obtain a second processed image corresponding to the first processed image; The target position of the raindrops is determined in the second processed image, and a clarity enhancement operation is performed on the pixels where the target position is located to obtain the enhanced vehicle exterior image.

4. The method for displaying an image outside a vehicle according to claim 1, wherein: The precipitation object includes raindrops, and the removing operation of the precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image includes: Detecting raindrop tracks in the enhanced vehicle exterior image; determining raindrops in the enhanced vehicle exterior image based on the raindrop trajectories; Segmenting the raindrops from the enhanced vehicle exterior image to obtain a segmented image corresponding to the enhanced vehicle exterior image that does not contain the raindrops; The blank part in the segmented image is filled with an image to obtain the clear vehicle exterior image corresponding to the segmented image.

5. The method for displaying an image outside a vehicle according to claim 1, wherein: The displaying of the clear vehicle exterior image comprises: Determining a reflective road surface area in the clear vehicle exterior image; Softening the reflective road surface area to obtain a soft-light exterior image corresponding to the clear exterior image; The soft light exterior vehicle image is displayed.

6. The vehicle exterior image display method according to claim 1, characterized in that: The acquisition of the real-time vehicle exterior image collected by the vehicle-mounted camera comprises: Monitoring the control behavior of the vehicle and determining whether a turn signal switching signal is detected; If the turn signal switching signal is detected, determining the target driving direction corresponding to the turn signal switching signal; Matching a target camera corresponding to the target driving direction among the on-board cameras; Acquire the real-time vehicle exterior image captured by the target camera.

7. The method for displaying an image outside a vehicle according to claim 1, wherein: The real-time vehicle exterior image includes a real-time vehicle front image and a real-time vehicle side image, and the real-time vehicle exterior image acquired by the vehicle-mounted camera includes: Acquire the real-time front image of the vehicle, monitor the control behavior of the vehicle, and determine whether a valid turn signal is detected; If the effective signal of the turn signal lamp is detected, determining the target turning direction corresponding to the effective signal of the turn signal lamp; Matching a target camera corresponding to the target turning direction among the on-board cameras; Acquire the real-time vehicle side image captured by the target camera; The displaying of the clear vehicle exterior image comprises: Displaying the clear vehicle exterior image corresponding to the real-time vehicle front image in full screen; An image window is created, and the clear vehicle exterior image corresponding to the real-time vehicle side image is displayed in full screen in the image window.

8. An external image display device, characterized in that: The vehicle exterior image display device comprises: The first acquisition module is used to acquire the real-time vehicle exterior image captured by the vehicle-mounted camera and determine whether the current driving environment of the vehicle is a low-brightness precipitation environment; A first enhancement module is used for performing a precipitation object enhancement operation on the real-time vehicle exterior image to obtain an enhanced vehicle exterior image corresponding to the real-time vehicle exterior image if the current driving environment of the vehicle is a low-brightness precipitation environment; A first removal module is used to remove the precipitation object in the enhanced vehicle exterior image to obtain a clear vehicle exterior image corresponding to the enhanced vehicle exterior image; The first display module is used to display the clear image outside the vehicle.

9. A vehicle, characterized in that: The vehicle includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the vehicle exterior image display method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle exterior image display method according to any one of claims 1 to 7 are implemented.