Forward view blind area assisting method, system and device and storage medium

Abstract

The invention discloses a forward view blind area assisting method, which comprises the following steps of: requesting to start a forward fisheye camera when recognizing that a view blind area appears in front according to the shooting content of the forward camera; the forward fisheye camera obtains a fisheye image of the forward view blind area to form to-be-displayed content; transmitting the to-be-displayed content to a display end of the vehicle for displaying; and after it is detected that the forward view is recovered, requesting to turn off the forward fisheye camera. The invention further discloses a corresponding system and device and a storage medium. By implementing the method and the device, the safety of the driving function can be improved when a forward view blind area appears.

Description

Technical Field

[0001] The present invention relates to the field of assisted driving of intelligent network-connected vehicles, and in particular to a forward visual field blind spot assistance method, system, device and storage medium. Background Art

[0002] With the widespread popularity of automobiles, the vehicle usage environment has become more complex. Setting up an intelligent surround view system during vehicle driving can bring great convenience to user driving.

[0003] With the continuous development of intelligent driving technology, today's vehicles are equipped with more and more visual sensors and domain controllers for integrated driving and parking control, which enables cars to have stronger perception capabilities to handle more complex scenarios. The integrated driving and parking controller makes it easier to share sensor data between the driving domain and the parking domain. Generally, in this usage mode, the front-view camera is the front-view driving camera, and the forward fisheye camera is the parking camera; among them, the parking fisheye camera is generally used in low-speed conditions when the vehicle is parking.

[0004] However, during the driving process, the vehicle may encounter situations such as direct sunlight, reflections from wet ground, or even the front hood being flipped, which may affect the driver's forward vision and the image collection of the driving front camera, thus forming a forward vision blind spot. When a forward vision blind spot occurs, it will affect driving safety, and there is no good solution in the prior art. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the present invention proposes a forward blind spot auxiliary method, system, device and storage medium, which can improve driving safety when a forward blind spot occurs.

[0006] As one aspect of the present invention, a forward visual field blind spot assistance method is provided, which at least comprises the following steps:

[0007] When a blind spot is detected in front of the camera based on the content captured by the front camera, a request is made to turn on the front fisheye camera;

[0008] The forward fisheye camera obtains the fisheye image of the forward blind area to form the content to be displayed;

[0009] Transmitting the content to be displayed to a display terminal of the vehicle for display;

[0010] After detecting that the forward field of view is restored, request to close the forward fisheye camera.

[0011] When a blind spot is detected in front of the camera according to the content captured by the front camera, a request is made to turn on the front fisheye camera, including:

[0012] Real-time acquisition of the front view image captured by the front view camera installed on the front windshield of the vehicle;

[0013] Checking the output quality of the forward-view image, and performing a hood-lifted view, blur, contrast detection, and brightness change detection according to the output quality, and determining that a blind spot in the field of vision appears in front when a strong backlight or reflective scene is identified or the front hood is flipped over;

[0014] Sends a request to the vehicle's parking assistance system to light up the forward-facing fisheye camera.

[0015] The forward fisheye camera obtains a fisheye image of the forward blind area to form content to be displayed, further comprising:

[0016] Turn on the forward-facing fisheye camera installed on the front bumper of the vehicle to obtain a fisheye image of the front area;

[0017] The fisheye image is subjected to distortion correction processing to obtain content to be displayed.

[0018] Among them, further include:

[0019] The display information of the forward fisheye camera is transmitted to the driving assistance module and participates in the driving fusion processing as a secondary sensor data.

[0020] Wherein, after detecting that the forward field of view is restored, requesting to close the forward fisheye camera and canceling display of the to-be-displayed content further includes:

[0021] Acquire the front view image captured by the front view camera in real time;

[0022] Checking the output quality of the forward-view image, and performing a hood-lifted view, blur, contrast detection, and brightness change detection based on the output quality to detect whether the forward visual blind spot has returned to normal;

[0023] After recognizing that the front blind spot has been restored, a shutdown request is sent to the vehicle's parking assistance driving system, requesting that the forward fisheye camera be shut down.

[0024] Accordingly, as another aspect of the present invention, a forward visual field blind spot assistance system is also provided, comprising:

[0025] A blind spot detection processing unit is used to request to turn on the forward-facing fisheye camera when a blind spot in the front field of view is identified based on the content captured by the forward-facing camera;

[0026] A fisheye content acquisition unit, used to acquire a fisheye image of a forward visual field blind area through a forward fisheye camera to form content to be displayed;

[0027] A display unit, used for transmitting the content to be displayed to a display terminal of the vehicle for display;

[0028] The blind spot recovery processing unit is used to request to turn off the forward fisheye camera after detecting that the forward field of view is restored.

[0029] Wherein, the blind spot detection processing unit includes:

[0030] A front view image acquisition unit acquires in real time a front view image taken by a front view camera installed on the front windshield of the vehicle;

[0031] a blind spot judgment unit, used to check the output quality of the front view image, and perform a hood-lifted view, blur, contrast detection, and brightness change detection according to the output quality, and determine that a blind spot appears in front when a strong backlight or reflective scene appears in front or the front hood is flipped over;

[0032] The activation request unit is used to send an activation request to the parking assistance driving system of the vehicle, requesting to light up the forward fisheye camera.

[0033] Wherein, the fisheye content acquisition unit further comprises:

[0034] A fisheye camera shooting unit is used to turn on a forward-facing fisheye camera disposed on the front bumper of the vehicle to obtain a fisheye image of the front area;

[0035] The distortion correction processing unit is used to perform distortion correction processing on the fisheye image to obtain content to be displayed.

[0036] Among them, further include:

[0037] The fusion processing unit is used to transmit the display information of the forward fisheye camera to the driving assistance module, and participate in the driving fusion processing as the auxiliary sensor data.

[0038] Wherein, the blind area recovery processing unit further includes:

[0039] A recovery detection unit is used to check the image quality of the front view image captured by the front view camera in real time, and perform a hood lift view, blur, contrast detection and brightness change detection according to the image quality to detect whether the front blind area has returned to normal;

[0040] The shutdown request unit is used to send a shutdown request to the parking assistance driving system of the vehicle to shut down the front fisheye camera after recognizing that the front blind spot has been restored.

[0041] As another aspect of the present invention, an intelligent driving processing device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the computer program.

[0042] As another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned method are implemented.

[0043] As another aspect of the present invention, a computer program product is provided, comprising computer instructions, wherein the computer instructions instruct a computer device to perform operations corresponding to the aforementioned method.

[0044] Implementing the embodiments of the present invention has the following beneficial effects:

[0045] The present invention provides a forward visual field blind spot assistance method, system, device and storage medium. When the front-view camera installed on the windshield identifies the visual field blind spots such as strong backlight, reflection or windshield fogging, the intelligent driving system can send a request to the parking assistance driving system to light up the forward fisheye camera, and transmit the display information of the forward fisheye camera to the display end of the vehicle. After the driving forward-view camera detects that the forward field of view is restored, the display of the forward fisheye camera is canceled. Since the forward fisheye camera of the parking system is installed at the front bumper, its position is relatively low, and there is basically no reflection and the windshield is fogged due to the temperature difference between the inside and outside of the car, which affects the image quality. Therefore, auxiliary display is performed when a blind spot appears in the forward-view camera.

[0046] At the same time, when there is a blind spot in the forward field of view, the forward fisheye camera can be dedistorted so that the image transmitted to the driver should be a normal view rather than a fisheye view. At the same time, in the scene where the image quality of the driving camera is poor and the driving assisted driving function is turned on, the forward fisheye camera can be used as a secondary sensor to participate in the sensor fusion decision of the driving system, realizing the complementary advantages of each sensor to the greatest extent, and ultimately improving the safety of driving functions;

[0047] The present invention does not add hardware modules such as sensors to existing vehicles. It only needs to use a fisheye camera triggered by specific conditions and dedistort and restore it to a normal view to supplement the driver's front close-range blind spot perspective when the windshield vision is blurred. There is no need to increase too much cost, thereby improving the robustness and safety of the assisted driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, without paying creative labor, other drawings obtained based on these drawings still belong to the scope of the present invention.

[0049] Figure 1 A schematic diagram of the main process of an embodiment of a forward visual field blind spot assistance method provided by the present invention;

[0050] Figure 2 for Figure 1 A more detailed flow chart of

[0051] Figure 3 A schematic structural diagram of an embodiment of a forward visual field blind spot auxiliary display system provided by the present invention;

[0052] Figure 4 for Figure 3 A structural diagram of a blind spot detection processing unit;

[0053] Figure 5 for Figure 3 A structural diagram of a fisheye content acquisition unit;

[0054] Figure 6 for Figure 3 Schematic diagram of the structure of the blind area recovery processing unit. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0056] like Figure 1 FIG. 1 is a schematic diagram showing a main flow chart of an embodiment of a forward visual field blind spot assistance method provided by the present invention; and FIG. Figure 2 As shown, in this embodiment, the forward visual field blind spot assistance method includes at least the following steps:

[0057] Step S10, when a blind spot in the front is identified according to the content captured by the front-view camera, requesting to turn on the front-facing fisheye camera;

[0058] In a specific example, the step S10 further includes:

[0059] Step S100, acquiring in real time a front view image captured by a front view camera disposed on a front windshield of the vehicle;

[0060] Step S101, checking the output quality of the front view image, performing a hood-lifted view, blur, contrast detection, and brightness change detection according to the output quality, and determining that a blind spot appears in front when a strong backlight or reflective scene appears in front or the front hood is flipped over;

[0061] Step S102, sending a request to light up the forward fisheye camera to the parking assistance driving system of the vehicle. It can be understood that the forward fisheye camera is generally used in the parking system, and at this time it serves as an auxiliary display module when a forward blind spot occurs.

[0062] More specifically, in step S101, the blur detection involved is mainly to detect whether the collected forward image is blurry, so as to determine whether it is in a strong light state; its contrast and brightness detection is used to detect whether there is a strong backlight environment.

[0063] More specifically, there are two methods for detecting the hood being lifted: one is to detect whether the CAN message signal of the hood in the D gear state is in the open state, or to determine whether the hood is in the lifted state through the view collected by the front-view camera installed on the windshield. For the second method, it is necessary to store pre-marked pictures of the hood being lifted in the driving system, so that when the same state is encountered during driving, a comparison can be made to achieve immediate recognition.

[0064] For blur detection, it can be achieved by estimating the local contrast of the image. A common method is to use the gradient information of the image to calculate the gradient standard deviation of the image, which represents the clarity of the image. The gradient standard deviation of a blurred image is usually small. It can be expressed using the following formula:

[0065]

[0066] in, represents the gradient vector modulus of the i-th pixel of image I, and N is the total number of pixels in the image.

[0067] The gradient vector can be defined by calculating the gray value change of the pixels around the point. The method adopted by the present invention is to use the Sobel operator, which is a convolution-based operator for detecting edges in images. The Sobel operator applies a 3×3 convolution kernel to each pixel of the image to calculate the gradient value of the pixel. Specifically, for the horizontal gradient, the Sobel operator is as follows:

[0068]

[0069] For the vertical gradient, the Sobel operator is as follows:

[0070]

[0071] These two operators can then be applied to the image I to compute the gradients in the horizontal and vertical directions respectively:

[0072] G x =I*G x

[0073] G y =I*G y

[0074] Among them, ** represents the convolution operation. Finally, the horizontal and vertical gradients can be combined into a gradient vector:

[0075]

[0076] When the image gradient standard deviation is less than a certain threshold Trigger_dim_0 (for example, 0.5 or 1.0), the image can be considered relatively blurred. However, it needs to be adjusted and determined according to the specific situation. In actual vehicle applications, the parameters of the visual sensor vary, so the specific threshold can be determined through experiments and tests to achieve the best image quality evaluation effect.

[0077] For strong backlight detection, when the calculated reflection intensity (Brightness) is greater than Trigger_lum_0, it is judged that the ambient light in the current environment is too strong and the driver may be dazzled, and the forward blind spot assist function needs to be activated.

[0078] The reflection intensity can be represented by the brightness value of the reflection area in the image. The reflection area ROI is detected by the image processing algorithm (the ROI of ground reflection is roughly the lower half of the image, and the ROI of direct sunlight backlight is roughly the upper half of the icon), and the pixel value of the area is obtained. The reflection intensity can be simply expressed as the average or sum of the pixel values ​​of the reflection area. The mean reflection intensity can be calculated according to the formula, that is,

[0079]

[0080] Among them, the brightness value Ii of the i-th pixel can usually be obtained by converting the color image into a grayscale image. Each pixel value in the grayscale image represents the grayscale level of the corresponding position, that is, the brightness of the image. The pixel value in the grayscale image is usually between 0 and 255, where 0 represents black (darkest) and 255 represents white (brightest). Therefore, the brightness value Ii of the i-th pixel can be calculated using the following formula:

[0081] Ii=grayscale image (xi,yi),

[0082] Among them, (xi,yi) is the coordinate of the i-th pixel, and the grayscale image (xi,yi) represents the pixel value of the corresponding position in the grayscale image.

[0083] The acquisition of grayscale images is usually a preprocessing step for image processing. In practical applications, different grayscale methods can be used to calculate the grayscale value of pixels, such as the following formula based on the weighted average method:

[0084] Gray value = 0.299 × red channel + 0.587 × green channel + 0.114 × blue channel

[0085] Among them, the values ​​of the red, green and blue channels respectively represent the brightness values ​​of the red, green and blue channels of the color image at the corresponding positions.

[0086] Step S11, the forward fisheye camera acquires a fisheye image of the forward blind area to form content to be displayed;

[0087] In a specific example, the step S11 further includes:

[0088] Step S110, turning on a forward-facing fisheye camera disposed on the front bumper of the vehicle to obtain a fisheye image of the front area;

[0089] Step S111, performing distortion correction processing on the fisheye image to obtain content to be displayed.

[0090] In this embodiment, the intrinsic parameters of the camera (such as focal length, optical center coordinates, etc.), radial distortion parameters (such as k1, k2, k3, etc.) and tangential distortion parameters (such as p1, p2, etc.) are known, and the focal lengths of the camera are fx and fy. For the coordinates (xd, yd) of each pixel point (x, y) in the original fisheye image, in an example, the corresponding ideal projection coordinates (xu, yu) can be calculated by the following formula:

[0091] xn=xd / fx / / normalization

[0092] yn=yd / fy

[0093] r2=xn^2+yn^2

[0094] xun=xn*(1+k1*r2+k2*r2^2+k3*r2^3) / / dedistortion

[0095] yun=yn*(1+k1*r2+k2*r2^2+k3*r2^3)

[0096] xu=xun*fx / / denormalization

[0097] yu=yun*fy

[0098] Through the distortion processing step, an ideal normal view after dedistortion can be obtained. It can be understood that the above example is only for example and not for limitation; in other examples, similar processing methods can be used, and the main process is to firstly process the dedistortion through the distortion module (including radial distortion and tangential distortion), and then obtain the dedistortion effect through coordinate transformation.

[0099] Step S12, transmitting the content to be displayed to a display terminal of the vehicle for display, and the display terminal may be a central control instrument;

[0100] Step S13: after detecting that the forward field of view is restored, request to close the forward fisheye camera.

[0101] In a specific example, the step S13 includes:

[0102] Step S130, acquiring a front view image captured by a front view camera in real time;

[0103] Step S131, checking the output quality of the front view image, and performing a hood-lifting view, blur, contrast detection, and brightness change detection according to the output quality to detect whether the front blind area has returned to normal;

[0104] Step S132, after recognizing that the front blind spot has been restored, a shutdown request is sent to the parking assistance driving system of the vehicle, requesting to shut down the forward fisheye camera.

[0105] It can be understood that, in an embodiment of the present invention, when the driving assistance driving function is turned on, the display information of the forward fisheye camera can be transmitted to the driving assistance driving module to participate in the driving fusion processing as the secondary sensor data.

[0106] It is understandable that in some preferred solutions, when a forward-facing fisheye camera is needed, the fisheye camera can be rotated upward in the vertical direction (Z axis) by a certain angle (e.g., 3°-8°) to obtain a better forward field of view image; generally, the forward-facing fisheye camera is installed horizontally or at a slightly negative angle in the horizontal direction. If the forward-facing fisheye camera supports rotation, a better field of view effect can be obtained. Figure 2 As shown, a more detailed flow chart of the method provided by the present invention is shown. In an embodiment, as shown in the figure, the process occurs after the vehicle is started and shifted into the D gear, and the main interaction units are the driving system (the right branch of the main process) and the parking system (the left branch of the main process) in the intelligent assisted driving system and the display module of the cockpit;

[0107] The parking forward fish-eye camera (FF) is a surround-view fish-eye camera installed in the area near the front bumper; the driving front-view camera is a front-view camera (FR) installed on the windshield, and the number of them can be greater than 1;

[0108] Use the function module for image self-check to perform quality self-check on the images collected by the driving front-view camera. This module depends on the data collected by the driving front-view camera and the intelligent driving computing unit's driving system to check the image quality. The parameters for image inspection include the hood-up view, blur, contrast detection, and brightness change detection; the blur detection mainly detects whether the forward image is blurred in a strong light environment, and the contrast and brightness detections mainly detect whether there is a strong backlight environment. The specific detection process can refer to the description of Figure 2 Step S11 above.

[0109] It can be understood that in this process, weather condition inspection is also required. The Tbox connects to the network to obtain the weather information of the current location, and the current location is provided by the GPS. The purpose of weather verification is to filter out the situation where the reason for the blurred image may be that the wipers are in the fast gear due to heavy fog or heavy rain at that time, resulting in a blurred image captured by the camera, and only perform blind spot vision assistance decision-making for the windshield fogging caused by the temperature difference inside and outside the vehicle or the forward vision blind spot caused by strong light.

[0110] In this embodiment, after each trigger of the forward blind spot vision display request, it is continuously displayed for 6 seconds, and the image of the front-view camera is recollected to the image self-check module starting from the 5th second of the display; in the image self-check module, if 6 consecutive frames (calculated at a frame rate of 30fps, 6 frames are 200ms) meet the self-check Trigger_dim_0 or Trigger_lum_0, the next process in the flowchart is triggered; if the corresponding blur value is less than Trigger_dim_1 and the backlight intensity value is less than Trigger_lum_1, the process exits and returns to the image self-check step. Among them, 0 < Trigger_dim_0 < Trigger_dim_1 (Trigger_dim_0 must be greater than zero. If it is equal to zero, it may be that the camera itself has a fault or is completely blocked); Trigger_lum_0 > Trigger_lum_1; their values need to be calibrated according to the internal and external parameters of the camera.

[0111] After the forward fish-eye camera is turned on, it is necessary to perform de-distortion processing on the images taken by the forward fish-eye camera to obtain an ideal normal view after de-distortion.

[0112] In this process, the normal view is obtained after dedistortion based on the fisheye view, which ensures that the camera's internal parameters are not modified again, so that the parking system can still be used after switching back to the fisheye view;

[0113] When the driving assistance function is activated in this scenario, the front camera fails. As a backup, the forward fisheye camera can transmit the 30-80m forward field of view to the driving system parameter sensor fusion. However, due to the limitation of the installation position of the forward fisheye camera, it cannot provide a good bird's-eye view (BEV) and can only participate in the perception module of the driving system as a secondary sensor. This is a temporary redundant measure based on the complete failure of the forward image taken by the forward camera, providing a buffer for the driver to take over.

[0114] After requesting to activate the parking forward fisheye camera and dedistorting the process, the dedistorted view can be transmitted to the cockpit module to be displayed in the central control instrument.

[0115] It can be understood that the present invention provides a forward blind spot vision assistance method, the core of which is based on the fact that when the front-view camera installed on the windshield recognizes strong backlight, reflective scenes or the front hood is flipped, the intelligent driving assistance system processing module sends a request to light up the forward fisheye camera to the parking assistance system processing module, and transmits the display information of the forward fisheye camera to the driving lane display end, and cancels the parking front-view camera display request after the driving front-view camera detects that the forward field of view is restored. Because the front-view camera of the parking system is installed at the front bumper, its installation position is relatively low, and there is basically no reflection and the windshield is fogged due to the temperature difference between the inside and outside of the car, which affects the image quality.

[0116] In this embodiment, a process of dedistorting the forward fisheye camera is also provided, so that the image transmitted to the driver should be a normal view rather than a fisheye view. In addition, in the scenario where the image quality of the driving camera is poor and the driving assisted driving function is turned on, the parking fisheye camera can be used as a secondary sensor to participate in the sensor fusion decision of the driving system, so as to achieve the complementary advantages of each sensor to the greatest extent, and ultimately improve the safety of the driving function.

[0117] like Figure 3 FIG. 1 is a schematic diagram showing a structure of an embodiment of a forward vision blind spot assistance system provided by the present invention. Figures 4 to 6 As shown, in this embodiment, the forward field of vision blind spot assistance system 1 includes:

[0118] The blind spot detection processing unit 10 is used to request to start the forward fisheye camera when a blind spot in the field of vision is identified in front according to the content captured by the forward-looking camera;

[0119] A fisheye content acquisition unit 11, used to acquire a fisheye image of a forward visual field blind area through a forward fisheye camera to form content to be displayed;

[0120] A display unit 12, used for transmitting the content to be displayed to a display terminal of the vehicle for display, and the display terminal may be a central control instrument;

[0121] A blind spot recovery processing unit 13, configured to request to turn off the forward fisheye camera after detecting that the forward field of view is restored;

[0122] The fusion processing unit 14 is used to transmit the display information of the forward fisheye camera to the driving assistance module, and participate in the driving fusion processing as the auxiliary sensor data.

[0123] like Figure 4 As shown, in a specific example, the blind spot detection processing unit 10 further includes:

[0124] A front view image acquisition unit 100 acquires in real time a front view image taken by a front view camera disposed on a front windshield of the vehicle;

[0125] The blind spot judgment unit 101 is used to check the image quality of the front view image, and perform the hood-lifted view, blur, contrast detection and brightness change detection according to the image quality, and determine that a blind spot appears in front when a strong backlight or reflective scene appears in front or the front hood is flipped over;

[0126] The activation request unit 102 is used to send an activation request to the parking assistance driving system of the vehicle, requesting to light up the front fisheye camera.

[0127] like Figure 5 As shown, in a specific example, the fisheye content acquisition unit 11 further includes:

[0128] The fisheye camera shooting unit 110 is used to start the forward-facing fisheye camera disposed on the front bumper of the vehicle to obtain a fisheye image of the front area;

[0129] The distortion correction processing unit 111 is used to perform distortion correction processing on the fisheye image to obtain content to be displayed.

[0130] like Figure 6 As shown, in a specific example, the blind area recovery processing unit 13 further includes:

[0131] The recovery detection unit 130 is used to check the image quality of the front view image captured by the front view camera in real time, and perform a hood lifting view, blur, contrast detection and brightness change detection according to the image quality to detect whether the front blind area has returned to normal;

[0132] The closing request unit 131 is used to send a closing request to the parking assistance driving system of the vehicle to request closing the forward fisheye camera after recognizing that the front blind spot is restored.

[0133] For more details, please refer to and combine the above Figure 1 to Figure 2 The description is not repeated here.

[0134] As another aspect of the present invention, there is also provided an intelligent driving processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the above-mentioned Figure 1 and Figure 2 For more details, please refer to and combine with the above Figure 1 to Figure 2 The description is not repeated here.

[0135] As another aspect of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 2 For more details, please refer to and combine with the above Figure 1 to Figure 2 The description is not repeated here.

[0136] As another aspect of the present invention, a computer program product is also provided, comprising computer instructions, wherein the computer instructions instruct a computer device to execute the above Figure 1 to Figure 2 For more details, please refer to and combine the above Figure 1 to Figure 2 The description is not repeated here.

[0137] Implementing the embodiments of the present invention has the following beneficial effects:

[0138] The present invention provides a forward visual field blind spot assistance method, system, device and storage medium. When the front-view camera installed on the windshield identifies the visual field blind spots such as strong backlight, reflection or windshield fogging, the intelligent driving system can send a request to the parking assistance driving system to light up the forward fisheye camera, and transmit the display information of the forward fisheye camera to the display end of the vehicle. After the driving forward-view camera detects that the forward field of view is restored, the display of the forward fisheye camera is canceled. Since the forward fisheye camera of the parking system is installed at the front bumper, its position is relatively low, and there is basically no reflection and the windshield is fogged due to the temperature difference between the inside and outside of the car, which affects the image quality. Therefore, auxiliary display is performed when a blind spot appears in the forward-view camera.

[0139] At the same time, when there is a blind spot in the forward field of view, the forward fisheye camera can be dedistorted so that the image transmitted to the driver should be a normal view rather than a fisheye view. At the same time, in the scene where the image quality of the driving camera is poor and the driving assisted driving function is turned on, the forward fisheye camera can be used as a secondary sensor to participate in the sensor fusion decision of the driving system, realizing the complementary advantages of each sensor to the greatest extent, and ultimately improving the safety of driving functions;

[0140] The present invention does not add hardware modules such as sensors to existing vehicles. It only needs to use a fisheye camera triggered by specific conditions and dedistort and restore it to a normal view to supplement the driver's front close-range blind spot perspective when the windshield vision is blurred. There is no need to increase too much cost, thereby improving the robustness and safety of the assisted driving system.

[0141] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] The above disclosure is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A forward visual field blind spot assistance method, characterized in that: At least the following steps are included: When a blind spot is detected in front of the camera based on the content captured by the front camera, a request is made to turn on the front fisheye camera; The forward fisheye camera obtains the fisheye image of the forward blind area to form the content to be displayed; Transmitting the content to be displayed to a display terminal of the vehicle for display; After detecting that the forward field of view is restored, request to close the forward fisheye camera.

2. The method according to claim 1, characterized in that When a blind spot is detected in front of the camera based on the content captured by the front camera, a request is made to turn on the front fisheye camera, including: Real-time acquisition of the front view image captured by the front view camera installed on the front windshield of the vehicle; Checking the output quality of the forward-view image, and performing a hood-lifted view, blur, contrast detection, and brightness change detection according to the output quality, and determining that a blind spot in the field of vision appears in front when a strong backlight or reflective scene is identified or the front hood is lifted; Sends a request to the vehicle's parking assistance system to light up the forward-facing fisheye camera.

3. The method according to claim 2, characterized in that The forward fisheye camera obtains the fisheye image of the forward blind area to form the content to be displayed, further including: Turn on the forward-facing fisheye camera installed on the front bumper of the vehicle to obtain a fisheye image of the front area; The fisheye image is subjected to distortion correction processing to obtain content to be displayed.

4. The method according to claim 3, characterized in that Further including: The display information of the forward fisheye camera is transmitted to the driving assistance module and participates in the driving fusion processing as a secondary sensor data.

5. The method according to any one of claims 1 to 4, characterized in that: After detecting that the forward field of view is restored, requesting to close the forward fisheye camera and canceling display of the to-be-displayed content, further includes: Acquire the front view image captured by the front view camera in real time; Checking the output quality of the forward-view image, and performing a hood-lifted view, blur, contrast detection, and brightness change detection based on the output quality to detect whether the forward visual blind spot has returned to normal; After recognizing that the front blind spot has been restored, a shutdown request is sent to the vehicle's parking assistance driving system, requesting that the forward fisheye camera be shut down.

6. A forward vision blind spot assistance system, characterized in that: include: A blind spot detection processing unit is used to request to turn on the forward-facing fisheye camera when a blind spot in the front field of view is identified based on the content captured by the forward-facing camera; A fisheye content acquisition unit, used to acquire a fisheye image of a forward visual field blind area through a forward fisheye camera to form content to be displayed; A display unit, used for transmitting the content to be displayed to a display terminal of the vehicle for display; The blind spot recovery processing unit is used to request to turn off the forward fisheye camera after detecting that the forward field of view is restored.

7. The system according to claim 6, characterized in that The blind spot detection processing unit comprises: A front view image acquisition unit acquires in real time a front view image taken by a front view camera installed on the front windshield of the vehicle; a blind spot judgment unit, used to check the output quality of the front view image, and perform a hood-lifted view, blur, contrast detection, and brightness change detection according to the output quality, and determine that a blind spot appears in front when a strong backlight or reflective scene appears in front or the front hood is flipped over; The activation request unit is used to send an activation request to the parking assistance driving system of the vehicle, requesting to light up the forward fisheye camera.

8. The system according to claim 7, characterized in that The fisheye content acquisition unit further comprises: A fisheye camera shooting unit is used to turn on a forward-facing fisheye camera disposed on the front bumper of the vehicle to obtain a fisheye image of the front area; The distortion correction processing unit is used to perform distortion correction processing on the fisheye image to obtain content to be displayed.

9. The system according to claim 8, characterized in that Further including: The fusion processing unit is used to transmit the display information of the forward fisheye camera to the driving assistance module, and participate in the driving fusion processing as the auxiliary sensor data.

10. The system according to any one of claims 6 to 9, characterized in that The blind area recovery processing unit further comprises: A recovery detection unit is used to check the image quality of the front view image captured by the front view camera in real time, and perform a hood lift view, blur, contrast detection and brightness change detection according to the image quality to detect whether the front blind area has returned to normal; The shutdown request unit is used to send a shutdown request to the parking assistance driving system of the vehicle to shut down the front fisheye camera after recognizing that the front blind spot has been restored.

11. An intelligent driving processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

13. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computer device to perform operations corresponding to the method according to any one of claims 1 to 5.

Images