Auxiliary driving method and system based on rearview mirror camera and vehicle

Abstract

The invention discloses an auxiliary driving method and system based on a rearview mirror camera and a vehicle, and the method comprises the steps: obtaining a front vehicle image collected by a front-view camera, and obtaining the spatial position of a front vehicle according to the front vehicle image; if the spatial position of the front vehicle is located in the lane, two rearview mirror cameras are controlled to collect blind area images of the front vehicle, distance measurement is carried out on the blind area images of the front vehicle, and a distance measurement result is obtained; and if the distance measurement result is smaller than a preset distance, sending the blind area image of the front vehicle to a display screen for window display so as to assist a driver in observing the road condition of the front blind area, and performing driving pre-judgment according to the road condition of the front blind area. By monitoring the conditions of the front vehicle and the blind area in real time, the driving safety is improved, the accident risk is reduced, the environment perception ability of a driver is enhanced, and the driving decision is optimized.

Description

Technical Field

[0001] The present invention relates to the field of image processing and safe driving technology, and in particular to an assisted driving method, system, vehicle and computer-readable storage medium based on a rearview mirror camera. Background Art

[0002] When a driver is driving a vehicle, if there is a vehicle within a certain distance ahead in the same lane, it will block the driver's line of sight, making it impossible for the driver to check the road conditions ahead and make predictive driving actions.

[0003] Therefore, when the driver's vision is blocked by the vehicle in front or the A-pillar, the driver cannot expand his field of vision and know the road conditions ahead. The driver cannot judge the next operation, such as whether the driver can overtake safely or whether the brakes are needed, etc., which makes the driver unable to drive safely and is more likely to cause accidents.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide an assisted driving method, system, device and computer-readable storage medium based on a rearview mirror camera, aiming to solve the problem in the prior art that the driver cannot obtain more observation fields to observe the front field when the front vehicle blocks the line of sight or the A-pillar blocks the line of sight.

[0006] To achieve the above object, the present invention provides an assisted driving method based on a rearview mirror camera, and the assisted driving method based on a rearview mirror camera comprises the following steps:

[0007] Obtaining a front vehicle image captured by a front-view camera, and obtaining a spatial position of the front vehicle based on the front vehicle image;

[0008] If the spatial position of the front vehicle is in the lane, the two rearview mirror cameras are controlled to collect the blind spot image of the front vehicle, and the distance of the blind spot image of the front vehicle is measured to obtain the distance measurement result;

[0009] If the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display to assist the driver in observing the road conditions in the blind spot in front and making driving predictions based on the road conditions in the blind spot in front.

[0010] Optionally, the assisted driving method based on rearview mirror cameras, wherein if the spatial position of the front vehicle is located in the lane, the two rearview mirror cameras are controlled to collect the blind spot image of the front vehicle, and the distance of the blind spot image of the front vehicle is measured to obtain the distance measurement result, specifically including:

[0011] If the spatial position of the vehicle in front is in the lane, the blind spot image of the vehicle in front is collected by the left rearview mirror camera and the right rearview mirror camera;

[0012] The distance of the blind spot image of the front vehicle is measured according to the front vehicle distance detection algorithm to obtain a distance measurement result.

[0013] Optionally, the assisted driving method based on a rearview mirror camera, wherein if the distance measurement result is less than a preset distance, the blind spot image of the vehicle in front is sent to a display screen for window display to assist the driver in observing the road condition in the blind spot in front, and making driving predictions based on the road condition in the blind spot in front, specifically includes:

[0014] Determine whether the distance measurement result is less than a preset distance;

[0015] If the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display based on the transmission protocol of the communication channel to assist the driver in observing the road conditions in the blind spot in front. If there is an obstacle in the blind spot image of the vehicle in front, the driver is reminded to avoid it.

[0016] Optionally, in the rearview mirror camera-based assisted driving method, the two rearview mirror cameras are respectively arranged on the outer edges of the rearview mirror.

[0017] Optionally, in the rearview mirror camera-based assisted driving method, the two rearview mirror cameras are respectively arranged at the top ends of both sides of the rearview mirror.

[0018] Optionally, in the rearview mirror camera-based assisted driving method, the rearview mirror camera adopts a camera with high luminous flux and large aperture.

[0019] Optionally, in the rearview mirror camera-based assisted driving method, the window display adopts picture-in-picture or left-right screen display.

[0020] Optionally, in the rearview mirror camera-based assisted driving method, the rearview mirror camera-based assisted driving system comprises:

[0021] A front vehicle position acquisition module is used to acquire the front vehicle image captured by the front-view camera, and acquire the spatial position of the front vehicle based on the front vehicle image;

[0022] The distance measurement result calculation module is used to control the two rearview mirror cameras to collect the blind spot image of the front vehicle if the spatial position of the front vehicle is in the lane, and to perform distance measurement on the blind spot image of the front vehicle to obtain the distance measurement result;

[0023] The blind spot video display module is used to send the blind spot image of the vehicle in front to the display screen for window display if the distance measurement result is less than the preset distance, so as to assist the driver in observing the road conditions in the blind spot in front and make driving predictions based on the road conditions in the blind spot in front.

[0024] In addition, to achieve the above-mentioned purpose, the present invention also provides a vehicle, wherein the vehicle includes: a memory, a processor, and an assisted driving program based on a rearview mirror camera stored in the memory and executable on the processor, wherein the assisted driving program based on the rearview mirror camera, when executed by the processor, implements the steps of the assisted driving method based on the rearview mirror camera as described above.

[0025] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an assisted driving program based on a rearview mirror camera, and when the assisted driving program based on a rearview mirror camera is executed by a processor, the steps of the assisted driving method based on a rearview mirror camera as described above are implemented.

[0026] In the present invention, an image of a vehicle in front captured by a front-view camera is obtained, and a spatial position of the vehicle in front is obtained according to the image of the vehicle in front; if the spatial position of the vehicle in front is located in the lane, two rearview mirror cameras are controlled to capture a blind spot image of the vehicle in front, and a distance is measured for the blind spot image of the vehicle in front to obtain a distance measurement result; if the distance measurement result is less than a preset distance, the blind spot image of the vehicle in front is sent to a display screen for window display, so as to assist the driver in observing the road condition in the blind spot in front, and a driving prediction is performed according to the road condition in the blind spot in front, so as to improve driving safety, reduce accident risks, enhance the driver's environmental perception ability, and optimize driving decisions by real-time monitoring of the vehicle in front and the blind spot. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of a preferred embodiment of the assisted driving method based on a rearview mirror camera in the present invention;

[0028] Figure 2 It is a structural diagram of a preferred embodiment of the auxiliary driving system based on the rearview mirror camera in the present invention;

[0029] Figure 3 It is a schematic diagram of a preferred embodiment of the assisted driving system based on a rearview mirror camera in the invention;

[0030] Figure 4 It is a structural diagram of a preferred embodiment of the vehicle of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0032] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0033] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0034] The assisted driving method based on the rearview mirror camera described in the preferred embodiment of the present invention is as follows: Figure 1 and Figure 2 As shown, the rearview mirror camera-based assisted driving method includes the following steps:

[0035] Step S10: Acquire a front vehicle image captured by a front-view camera, and acquire a spatial position of the front vehicle based on the front vehicle image.

[0036] Specifically, in an embodiment of the present invention, the spatial position of the vehicle in front is judged based on the lane lines in the image of the vehicle in front captured by the front-view camera: if the vehicle in front is between two lane lines, the target object is located in this lane; if the vehicle in front is on the left side of the left lane line, the target object is located on the left side of this lane; if the vehicle in front is on the right side of the right lane line, the vehicle in front is located on the right side of this lane (the front-view camera starts video acquisition when the ignition is turned on, and the first image is road information, a real-time video of the road; it is used to judge whether there are vehicles, pedestrians, obstacles, etc. in the front field of view).

[0037] Step S20: If the spatial position of the front vehicle is located in the lane, control the two rearview mirror cameras to collect the blind spot image of the front vehicle, perform distance measurement on the blind spot image of the front vehicle, and obtain the distance measurement result.

[0038] The step S20 comprises:

[0039] Step S21: If the spatial position of the front vehicle is located in the lane, the blind spot image of the front vehicle is collected by the left rearview mirror camera and the right rearview mirror camera;

[0040] The distance of the blind spot image of the front vehicle is measured according to the front vehicle distance detection algorithm to obtain a distance measurement result.

[0041] Specifically, in an embodiment of the present invention, if the spatial position of the front vehicle is located in the lane, the blind spot image of the front vehicle is collected by the left rearview mirror camera and the right rearview mirror camera; the distance of the blind spot image of the front vehicle is measured according to the front vehicle distance detection algorithm to obtain a distance measurement result, that is, the distance measurement result calculated by collecting the vehicle blind spot image by the left rearview mirror camera and the right rearview mirror camera in the present invention is more accurate (the front vehicle distance detection algorithm includes but is not limited to radar-based, vision-based, and ultrasonic-based distance detection algorithms, such as using a camera to capture images, using computer vision technology to analyze feature points in the image, and further, estimating the distance through triangulation or a deep learning model or a machine learning-based distance detection algorithm using a deep learning model to train a large amount of image and radar data, estimating the distance by identifying feature points and patterns, and inputting the feature points into the deep learning model for distance estimation to obtain the distance between the vehicle and the front vehicle).

[0042] Furthermore, the two rearview mirror cameras are respectively arranged at the outer edge of the rearview mirror (installed on the left exterior rearview mirror of the vehicle, collecting the road conditions ahead, and transmitting the real-time video to the display device; the camera is installed as far away from the driver as possible, installed on the right exterior rearview mirror of the vehicle, collecting the road conditions ahead, and transmitting the real-time video to the display device; the camera is installed as far away from the driver as possible); the two rearview mirror cameras are respectively arranged at the top of both sides of the rearview mirror; the rearview mirror camera adopts a high-luminous flux and large-aperture camera (the lens adopts a high-luminous flux and large-aperture lens, the sensor adopts a high-sensitivity sensor, and a camera with low-light and starlight performance is adopted, and it can collect images when powered on real-time video); the window display adopts picture-in-picture or left-right screen display (there are two display screen display modes, one is to use two display screens, one display screen displays the left front camera video, and the other displays the right front camera video, and the other mode is to use one display screen to display the left and right front camera videos at the same time; left and right window mode, the left window displays the left front camera video, and the right window displays the right front camera video; upper and lower two window mode, the upper window displays the left front camera video, the lower window displays the right front camera video, or vice versa; picture-in-picture mode, the main screen displays the left front camera video, the small window displays the right front camera video, or vice versa; the user can operate to switch).

[0043] Step S30: If the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display to assist the driver in observing the road condition in the blind spot in front and making driving predictions based on the road condition in the blind spot in front.

[0044] The step 30 comprises:

[0045] Step S31, determining whether the distance measurement result is less than a preset distance;

[0046] Step S32: If the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display based on the transmission protocol of the communication channel to assist the driver in observing the road conditions in the blind spot in front. If there is an obstacle in the blind spot image of the vehicle in front, the driver is reminded to avoid it.

[0047] Specifically, in an embodiment of the present invention, it is determined whether the ranging result is less than a preset distance; if the ranging result is less than the preset distance, based on the transmission protocol of the communication channel, the blind spot image of the vehicle in front is sent to the display screen for window display to assist the driver in observing the road conditions in the blind spot in front, and if there is an obstacle in the blind spot image of the vehicle in front, the driver is reminded to avoid it (real-time video of the road; whether there are cars, people, obstacles, whether it is safe to drive, etc., for example: when a vehicle is coming from the left or right side of the vehicle in front, the driver is reminded to avoid it through the display screen); that is, the communication channel of the communication module is controlled to obtain the real-time video of the left and right cameras, and the real-time video is displayed on the display screen through the display module; at the same time, the storage module is controlled to store the video and other configuration parameters, and other configuration parameters include but are not limited to: a preset distance (front vehicle distance threshold), parameters required by the distance detection method, and display switch setting parameters.

[0048] Furthermore, if Figure 3 As shown, based on the above-mentioned assisted driving method based on a rearview mirror camera, the present invention also provides an assisted driving system based on a rearview mirror camera, and the assisted driving system based on a rearview mirror camera includes:

[0049] The front vehicle position acquisition module 51 is used to acquire the front vehicle image captured by the front-view camera, and acquire the spatial position of the front vehicle according to the front vehicle image;

[0050] The distance measurement result calculation module 52 is used to control the two rearview mirror cameras to collect the blind spot image of the front vehicle if the spatial position of the front vehicle is located in the lane, and to perform distance measurement on the blind spot image of the front vehicle to obtain the distance measurement result;

[0051] The blind spot video display module 53 is used to send the blind spot image of the vehicle ahead to the display screen for window display if the distance measurement result is less than the preset distance, so as to assist the driver in observing the road condition in the blind spot ahead and make driving predictions based on the road condition in the blind spot ahead.

[0052] Furthermore, if Figure 4 As shown, based on the above-mentioned assisted driving method based on a rearview mirror camera, the present invention also provides a vehicle accordingly, and the vehicle includes a processor 10, a memory 20 and a display 30. Figure 4 Only some of the components of the vehicle are shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0053] The memory 20 may be an internal storage unit of the vehicle in some embodiments, such as a hard disk or memory of the vehicle. The memory 20 may also be an external storage device of the vehicle in other embodiments, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the vehicle. Further, the memory 20 may also include both an internal storage unit of the vehicle and an external storage device. The memory 20 is used to store application software and various types of data installed in the vehicle, such as the program code of the installed vehicle. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, an assisted driving program 40 based on a rearview mirror camera is stored on the memory 20, and the assisted driving program 40 based on a rearview mirror camera can be executed by the processor 10, thereby realizing the assisted driving method based on a rearview mirror camera in the present application.

[0054] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the rearview mirror camera-based assisted driving method.

[0055] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, a HUD display, a projection display, an aerial imaging display, and an OLED (Organic Light-Emitting Diode) touch device. The display 30 is used to display information on the vehicle and to display a visual user interface. The components 10-30 of the vehicle communicate with each other via a system bus.

[0056] In one embodiment, when the processor 10 executes the program 40 for assisted driving based on a rearview mirror camera in the memory 20, the following steps are implemented:

[0057] Obtaining a front vehicle image captured by a front-view camera, and obtaining a spatial position of the front vehicle based on the front vehicle image;

[0058] If the spatial position of the front vehicle is in the lane, the two rearview mirror cameras are controlled to collect the blind spot image of the front vehicle, and the distance of the blind spot image of the front vehicle is measured to obtain the distance measurement result;

[0059] If the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display to assist the driver in observing the road conditions in the blind spot in front and making driving predictions based on the road conditions in the blind spot in front.

[0060] If the spatial position of the front vehicle is located in the lane, the two rearview mirror cameras are controlled to collect the blind spot image of the front vehicle, and the distance of the blind spot image of the front vehicle is measured to obtain the distance measurement result, which specifically includes:

[0061] If the spatial position of the vehicle in front is in the lane, the blind spot image of the vehicle in front is collected by the left rearview mirror camera and the right rearview mirror camera;

[0062] The distance of the blind spot image of the front vehicle is measured according to the front vehicle distance detection algorithm to obtain a distance measurement result.

[0063] Wherein, if the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display to assist the driver in observing the road condition in the blind spot in front and making driving predictions based on the road condition in the blind spot in front, specifically including:

[0064] Determine whether the distance measurement result is less than a preset distance;

[0065] If the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display based on the transmission protocol of the communication channel to assist the driver in observing the road conditions in the blind spot in front. If there is an obstacle in the blind spot image of the vehicle in front, the driver is reminded to avoid it.

[0066] Wherein, the two rearview mirror cameras are respectively arranged on the outer edges of the rearview mirror.

[0067] Wherein, the two rearview mirror cameras are respectively arranged at the top ends of both sides of the rearview mirror.

[0068] Wherein, the rearview mirror camera adopts a camera with high luminous flux and large aperture.

[0069] The window display adopts picture-in-picture or left-right screen display.

[0070] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an assisted driving program based on a rearview mirror camera, and when the assisted driving program based on a rearview mirror camera is executed by a processor, the steps of the assisted driving method based on a rearview mirror camera as described above are implemented.

[0071] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or vehicle including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or vehicle. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or vehicle including the element.

[0072] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0073] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An assisted driving method based on a rearview mirror camera, characterized in that: The rearview mirror camera-based assisted driving method comprises: Obtaining a front vehicle image captured by a front-view camera, and obtaining a spatial position of the front vehicle based on the front vehicle image; If the spatial position of the front vehicle is in the lane, the two rearview mirror cameras are controlled to collect the blind spot image of the front vehicle, and the distance of the blind spot image of the front vehicle is measured to obtain the distance measurement result; If the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display to assist the driver in observing the road conditions in the blind spot in front and making driving predictions based on the road conditions in the blind spot in front.

2. The rearview mirror camera-based assisted driving method according to claim 1, characterized in that: If the spatial position of the front vehicle is located in the lane, the two rearview mirror cameras are controlled to collect the blind spot image of the front vehicle, and the distance of the blind spot image of the front vehicle is measured to obtain the distance measurement result, which specifically includes: If the spatial position of the vehicle in front is in the lane, the blind spot image of the vehicle in front is collected by the left rearview mirror camera and the right rearview mirror camera; The distance of the blind spot image of the front vehicle is measured according to the front vehicle distance detection algorithm to obtain a distance measurement result.

3. The rearview mirror camera-based assisted driving method according to claim 2, characterized in that: If the distance measurement result is less than the preset distance, the blind spot image of the vehicle ahead is sent to a display screen for window display to assist the driver in observing the road condition in the blind spot ahead and making driving predictions based on the road condition in the blind spot ahead, specifically including: Determine whether the distance measurement result is less than a preset distance; If the distance measurement result is less than the preset distance, the blind spot image of the vehicle in front is sent to the display screen for window display based on the transmission protocol of the communication channel to assist the driver in observing the road conditions in the blind spot in front. If there is an obstacle in the blind spot image of the vehicle in front, the driver is reminded to avoid it.

4. The rearview mirror camera-based assisted driving method according to claim 1, characterized in that: The two rearview mirror cameras are respectively arranged on the outer edges of the rearview mirror.

5. The rearview mirror camera-based assisted driving method according to claim 1, characterized in that: The two rearview mirror cameras are respectively arranged at the top ends of both sides of the rearview mirror.

6. The rearview mirror camera-based assisted driving method according to claim 1, characterized in that: The rearview mirror camera adopts a camera with high luminous flux and large aperture.

7. The rearview mirror camera-based assisted driving method according to claim 1, characterized in that: The window display adopts picture-in-picture or left-right screen display.

8. An assisted driving system based on a rearview mirror camera, characterized in that: The rearview mirror camera-based assisted driving system includes: A front vehicle position acquisition module is used to acquire the front vehicle image captured by the front-view camera, and acquire the spatial position of the front vehicle based on the front vehicle image; The distance measurement result calculation module is used to control the two rearview mirror cameras to collect the blind spot image of the front vehicle if the spatial position of the front vehicle is in the lane, and to perform distance measurement on the blind spot image of the front vehicle to obtain the distance measurement result; The blind spot video display module is used to send the blind spot image of the vehicle in front to the display screen for window display if the distance measurement result is less than the preset distance, so as to assist the driver in observing the road conditions in the blind spot in front and make driving predictions based on the road conditions in the blind spot in front.

9. A vehicle, characterized in that: The vehicle includes a memory, a processor, and an assisted driving program based on a rearview mirror camera stored in the memory and executable on the processor. When the assisted driving program based on the rearview mirror camera is executed by the processor, the steps of the assisted driving method based on the rearview mirror camera as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer-readable storage medium stores an assisted driving program based on a rearview mirror camera. When the assisted driving program based on a rearview mirror camera is executed by a processor, the steps of the assisted driving method based on a rearview mirror camera as described in any one of claims 1 to 7 are implemented.

Images