Automobile A-pillar blind spot display system and method

Abstract

This application discloses a vehicle A-pillar blind spot display system and method. It acquires a real-time image of the driver's front view from inside the vehicle using an in-vehicle binocular camera, and a data processing module performs eye recognition and positioning on the driver's front view to achieve real-time eye position acquisition. It also acquires real-time images of the outside environment using an external camera, and the data processing module, combined with wide-angle perspective transformation technology, adaptively crops and adjusts the images based on eye positioning. This allows for the acquisition of the A-pillar blind spot image inside the vehicle, where the external image is obscured by the A-pillar. The A-pillar display screen then displays this blind spot image in real-time, achieving complete integration of the blind spot image inside the vehicle with the external scenery.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and in particular to a blind spot display system and method for automotive A-pillars. Background Technology

[0002] The A-pillar of a car refers to the connecting pillar between the roof and the front compartment on the left and right front sides, located between the engine compartment and the passenger compartment, above the left and right rearview mirrors. During vehicle operation, the A-pillar will more or less obstruct the driver's view, with a greater impact during cornering, creating blind spots. These blind spots caused by the A-pillar are one of the causes of traffic accidents.

[0003] Currently, to prevent traffic accidents caused by A-pillar blind spots when driving on curves, V2X (vehicle-to-everything) is mainly used to obtain real-time information about the outside world. However, due to inconvenient management and technological limitations, V2X is not yet widely adopted. Furthermore, while existing A-pillar blind spot displays can show information from outside the vehicle, they cannot fully integrate the displayed content with the surrounding scenery.

[0004] Therefore, existing technologies for eliminating the interference of the A-pillar blind spot on a car during driving have the problem of not being able to fully integrate the displayed content with the external scenery. Summary of the Invention

[0005] In view of this, it is necessary to provide a vehicle A-pillar blind spot display system and method to solve the problem in the prior art that the displayed content cannot be completely integrated with the external scenery when eliminating the interference of the vehicle A-pillar blind spot on the vehicle's driving process.

[0006] To address the above problems, the present invention provides a blind spot display system for automotive A-pillars, comprising:

[0007] In-vehicle binocular camera, used to acquire a frontal image of the driver inside the vehicle;

[0008] External camera, used to acquire images of the outside of the vehicle;

[0009] The data processing module is connected to the signals of the in-vehicle binocular camera and the external camera respectively; it is used to determine the driver's eye position based on the in-vehicle frontal image of the driver, and to determine the A-pillar blind spot image based on the eye position and the external image, combined with wide-angle perspective transformation technology.

[0010] The A-pillar display screen is connected to the data processing module and is used to display the blind spot image of the A-pillar inside the vehicle.

[0011] To address the above problems, this invention provides a method for displaying blind spots in an automotive A-pillar, comprising:

[0012] Acquire a frontal image of the driver inside the vehicle using an in-vehicle binocular camera;

[0013] Acquire images of the vehicle's exterior using an external camera;

[0014] Based on the data processing module, the driver's eye position is determined according to the frontal image of the driver inside the vehicle. Based on the eye position and the image outside the vehicle, combined with wide-angle perspective transformation technology, the blind spot image of the A-pillar inside the vehicle is determined.

[0015] The A-pillar display screen shows the blind spot image inside the vehicle.

[0016] Furthermore, based on the in-vehicle binocular camera, a frontal image of the driver inside the vehicle is acquired, including:

[0017] The in-vehicle binocular camera was calibrated to determine the relative positional relationship between the first and second binocular cameras.

[0018] The vehicle's binocular camera captured a frontal image of the driver inside the vehicle.

[0019] The driver's frontal image inside the vehicle includes a first driver's frontal image inside the vehicle and a second driver's frontal image inside the vehicle. The first driver's frontal image inside the vehicle is captured by a first camera, and the second driver's frontal image inside the vehicle is captured by a second camera. The first and second cameras have the same focal length.

[0020] Furthermore, the driver's eye position is determined based on the frontal image of the driver inside the vehicle, including:

[0021] Based on the relative positional relationship between the two cameras, the in-vehicle binocular camera is calibrated and transformed to construct an ideal binocular ranging model;

[0022] Based on the ideal binocular ranging model, stereo matching is performed on the frontal images of the driver inside the first vehicle and the frontal images of the driver inside the second vehicle according to the color space conversion function to determine the two-dimensional coordinates of the first and second eyes.

[0023] Based on an ideal binocular ranging model, the parallax of the first and second cameras is determined.

[0024] Based on the two-dimensional coordinates of the first human eye, the two-dimensional coordinates of the second human eye, and the parallax, the human eye positioning is determined using the object distance calculation formula of the binocular camera.

[0025] Furthermore, the formula for calculating the distance of an object to the stereo camera is:

[0026]

[0027] Where Z is the distance from the human eye to the camera, B is the baseline length of the first and second eye cameras, and X is the distance from the human eye to the camera. cX is the column coordinate of the driver's frontal image from inside the first vehicle. R Let f be the column coordinates of the driver's frontal image from inside the second vehicle, f be the focal length of the first and second cameras, and D be the parallax.

[0028] Furthermore, based on human eye positioning and external vehicle images, combined with wide-angle perspective transformation technology, the blind spot image of the A-pillar inside the vehicle is determined, including:

[0029] Obtain the coordinates of the first center point and the boundary points of the A-pillar display screen, wherein the boundary points include at least three;

[0030] Based on the human eye positioning, the coordinates of the first center point, and the coordinates of the boundary point, the first distance between the human eye and the coordinates of the first center point, the second distance between the human eye and the boundary point, and the third distance between the coordinates of the first center point and the coordinates of the boundary point are determined respectively.

[0031] Based on the first distance, the second distance, and the third distance, determine the first angle between the human eye and the boundary point;

[0032] Based on the first angle and the image outside the vehicle, the cropping points in the image outside the vehicle are determined by wide-angle perspective transformation technology, wherein each cropping point corresponds one-to-one with a boundary point;

[0033] The exterior image is cropped based on the capture point to determine the blind spot image of the A-pillar inside the vehicle.

[0034] Furthermore, based on the first distance, the second distance, and the third distance, the first angle between the human eye and the boundary point is determined, including:

[0035] Based on the first distance, the second distance, and the third distance, the first angle between the human eye and the boundary point is determined using the angle conversion formula;

[0036] The formula for calculating the angle conversion is:

[0037]

[0038] Where θ is the first angle, and D eyes For the first distance, D hy For the second distance, D A This is the third distance.

[0039] Furthermore, based on the first angle and the external image, the cropping point in the external image is determined using wide-angle perspective transformation technology, including:

[0040] Acquire the wide-angle shot of the external camera and the second center point of the external image;

[0041] Based on the shooting wide-angle, the first angle, and the second center point, the boundary point coordinates are determined to correspond to the intercept point in the vehicle exterior image using the formula for calculating the ratio of angle to length.

[0042] Furthermore, the formula for calculating the ratio of angle to length is:

[0043]

[0044] Where θ is the first angle, W is the shooting wide angle, D is the parallax, O is the second center point, and A' is the capture point.

[0045] Furthermore, the interception points include at least three; based on the A-pillar display screen, an image of the blind spot inside the vehicle's A-pillar is displayed, including:

[0046] The interception point is matched with the boundary point, and the blind spot image of the A-pillar inside the vehicle is displayed on the A-pillar display screen.

[0047] The beneficial effects of the above embodiments are as follows: The present invention provides a vehicle A-pillar blind spot display system and method, which acquires the driver's frontal image in real time through an in-vehicle binocular camera, and performs human eye recognition and positioning on the driver's frontal image in the in-vehicle by a data processing module, thereby realizing the real-time acquisition of the position of the human eye; acquires the external image in real time through an external camera, and the data processing module, combined with wide-angle perspective transformation technology, adaptively crops and adjusts the external image based on the human eye positioning, which can acquire the in-vehicle A-pillar blind spot image where the external image is blocked by the A-pillar, and displays the in-vehicle A-pillar blind spot image in real time on the A-pillar display screen, which can achieve complete integration of the in-vehicle A-pillar blind spot image with the external scenery. Attached Figure Description

[0048] Figure 1 This is a structural block diagram of an embodiment of the automotive A-pillar blind spot display system provided by the present invention;

[0049] Figure 2 This is a flowchart illustrating an embodiment of the blind spot display method for automobile A-pillars provided by the present invention;

[0050] Figure 3 A flowchart illustrating an embodiment of the present invention for determining the driver's eye position;

[0051] Figure 4 This is a schematic diagram showing the results of an embodiment of the non-ideal binocular ranging model provided by the present invention.

[0052] Figure 5 This is a schematic diagram showing the results of an embodiment of the ideal binocular ranging model provided by the present invention;

[0053] Figure 6 This is a flowchart illustrating an embodiment of the present invention for determining the blind spot image of the A-pillar inside a vehicle;

[0054] Figure 7 A schematic diagram showing the results of an embodiment of the boundary point A related data provided by the present invention;

[0055] Figure 8 This is a schematic diagram showing the result of an embodiment of perspective transformation provided by the present invention. Detailed Implementation

[0056] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0057] Before describing the embodiments, let's first explain column A:

[0058] The A-pillar (or A-pillar in the context of automobiles) is the connecting pillar between the roof and the front compartment of a car, located between the engine compartment and the passenger compartment, above the side mirrors. During driving, the A-pillar will more or less obstruct the driver's view, with a greater impact during turns, creating blind spots. These blind spots are a contributing factor to traffic accidents.

[0059] Currently, to prevent traffic accidents caused by A-pillar blind spots when driving on curves, V2X (vehicle-to-everything) is mainly used to obtain real-time information about the outside world. However, due to inconvenient management and technological limitations, V2X is not yet widely adopted. Furthermore, while existing A-pillar blind spot displays can show information from outside the vehicle, they cannot fully integrate the displayed content with the surrounding scenery.

[0060] Therefore, existing technologies for eliminating the interference of the A-pillar blind spot on a car during driving have the problem of not being able to fully integrate the displayed content with the external scenery.

[0061] To address the aforementioned problems, this invention provides a blind spot display system and method for automotive A-pillars, which will be described in detail below.

[0062] like Figure 1 As shown, Figure 1 This is a structural block diagram of an embodiment of the automotive A-pillar blind spot display system provided by the present invention. The automotive A-pillar blind spot display system 100 includes:

[0063] The in-vehicle binocular camera 101 is used to acquire a frontal image of the driver inside the vehicle.

[0064] External camera 102 is used to acquire images outside the vehicle.

[0065] The data processing module 103 is connected to the in-vehicle binocular camera 101 and the external camera 102 respectively; it is used to determine the driver's eye position based on the in-vehicle driver's frontal image, and to determine the in-vehicle A-pillar blind spot image based on the in-vehicle eye position and the external image, combined with wide-angle perspective transformation technology.

[0066] The A-pillar display screen 104 is connected to the data processing module 103 and is used to display the blind spot image of the A-pillar inside the vehicle.

[0067] In this embodiment, firstly, the in-vehicle binocular camera 101 acquires a frontal image of the driver inside the vehicle, and the external camera 102 acquires an external image. Then, the data processing module 103 is connected to the in-vehicle binocular camera 101 and the external camera 102 respectively. First, the driver's eye position is determined based on the frontal image of the driver inside the vehicle. Then, the driver's eye position is determined based on the frontal image of the driver inside the vehicle. Based on the eye position and the external image, combined with wide-angle perspective transformation technology, the blind spot image of the A-pillar inside the vehicle is determined. Finally, the A-pillar display screen 104 is connected to the data processing module 103 to display the blind spot image of the A-pillar inside the vehicle in real time.

[0068] In this embodiment, the in-vehicle binocular camera 101 acquires a real-time image of the driver's front view, and the data processing module 103 performs eye recognition and positioning on the in-vehicle driver's front view to achieve real-time acquisition of the driver's eye position; the external camera 102 acquires an external image in real-time, and the data processing module 103, in conjunction with wide-angle perspective transformation technology, adaptively crops and adjusts the external image based on the driver's eye positioning, which can acquire the internal A-pillar blind spot image where the external image is obscured by the A-pillar, and the internal A-pillar blind spot image is displayed in real-time on the A-pillar display screen 104, which can achieve complete integration of the internal A-pillar blind spot image with the external scenery.

[0069] In a preferred embodiment, the in-vehicle binocular camera 101 is positioned above and to the side of the car seat, near the steering wheel, and is capable of acquiring a real-time frontal image of the driver inside the vehicle, especially the eyes.

[0070] In a preferred embodiment, the exterior camera 102 is positioned on the edge of the vehicle's windshield, near the driver's seat.

[0071] In other embodiments, the exterior camera 102 can also be positioned on the outside of the A-pillar; it can also be positioned in other locations as needed.

[0072] In a preferred embodiment, the data processing module 103 is also connected to an external signal, which can convert the human eye positioning to the world coordinate system in real time. That is, it can obtain the unique position of the human eye in the world coordinate system in real time, thereby facilitating subsequent data processing.

[0073] In a preferred embodiment, the A-pillar display screen 104 is disposed on the A-pillar inside the car, and the vertex of the A-pillar display screen 104 corresponds and coincides with the vertex of the A-pillar inside the car, so that the A-pillar display screen 104 completely covers the A-pillar inside the car.

[0074] To address the aforementioned problems, this invention also provides a method for displaying blind spots in automotive A-pillars, such as... Figure 2 As shown, Figure 2 A flowchart illustrating an embodiment of the automotive A-pillar blind spot display method provided by the present invention includes:

[0075] Step S101: Acquire a frontal image of the driver inside the vehicle using the in-vehicle binocular camera;

[0076] Step S102: Acquire images of the exterior of the vehicle using the exterior camera;

[0077] Step S103: Based on the data processing module, determine the driver's eye position according to the frontal image of the driver inside the vehicle, and determine the blind spot image of the A-pillar inside the vehicle according to the eye position and the image outside the vehicle, combined with wide-angle perspective transformation technology.

[0078] Step S104: Display the blind spot image of the A-pillar inside the vehicle based on the A-pillar display screen.

[0079] In this embodiment, firstly, an in-vehicle binocular camera is used to acquire a frontal image of the driver inside the vehicle, and an external camera is used to acquire an external image. Then, based on the data processing module, the driver's eye position is determined according to the in-vehicle frontal image, and based on the eye position and the external image, combined with wide-angle perspective transformation technology, the blind spot image of the A-pillar inside the vehicle is determined. Finally, the blind spot image of the A-pillar inside the vehicle is displayed on the A-pillar display screen.

[0080] In this embodiment, a binocular camera inside the vehicle acquires a real-time image of the driver's front view, and a data processing module performs eye recognition and positioning on the driver's front view image to achieve real-time acquisition of the driver's eye position. An external camera acquires real-time images of the outside of the vehicle, and the data processing module, combined with wide-angle perspective transformation technology, adaptively crops and adjusts the images of the outside of the vehicle based on the driver's eye positioning. This allows for the acquisition of images of the blind spot inside the vehicle caused by the A-pillar obstructing the external image. The A-pillar display screen then displays the images of the blind spot inside the vehicle in real time, achieving complete integration of the images of the blind spot inside the vehicle with the external scenery.

[0081] In a preferred embodiment, in step S101, since the in-vehicle binocular camera has two cameras, the in-vehicle driver frontal image includes a first in-vehicle driver frontal image and a second in-vehicle driver frontal image, wherein the first camera captures the first in-vehicle driver frontal image and the second camera captures the second in-vehicle driver frontal image.

[0082] To improve the reliability of the acquired first and second in-vehicle driver frontal images, it is necessary to calibrate the in-vehicle binocular cameras to determine the relative positional relationship between the first and second binocular cameras; then, the in-vehicle driver frontal images are acquired based on the calibrated in-vehicle binocular cameras.

[0083] It should be noted that during the calibration of the in-vehicle binocular cameras, the intrinsic parameters, extrinsic parameters, and distortion coefficients of the first and second cameras were also obtained.

[0084] The intrinsic parameters include the camera's fx, fy, cx, and cy; the extrinsic parameters include the rotation matrix and translation vector of the first camera relative to the second camera; and the distortion coefficients include radial distortion coefficients (k1, k2, k3) and tangential distortion coefficients (p1, p2).

[0085] In a preferred embodiment, in step S102, after acquiring the exterior image, to improve image quality, a grayscale transformation method is used to enable the image to display more details and improve image contrast. The grayscale transformation method includes linear grayscale transformation and nonlinear grayscale transformation.

[0086] In one specific embodiment, the logarithmic transform and power-law transform in grayscale nonlinear transformation are selected, wherein the formula for the logarithmic transform is: ：

[0087] f(x) = c * log(1 + g(x))

[0088] Where c is the scaling factor, f(x) is the input image, and g(x) is the output image after logarithmic transformation.

[0089] Power-law transform can correct images, refining overly bleached or overly dark images and enhancing contrast. The basic expression for power-law transform is:

[0090] y = cx r +b

[0091] Both c and r are positive numbers.

[0092] Similar to the logarithmic transform, the power-law transform maps a portion of the grayscale region onto a wider region.

[0093] Furthermore, due to the imperfections of imaging systems, digital images are often contaminated by various noises during their formation, transmission, and recording. These noises often manifest as isolated pixels in the image, affecting image quality. Therefore, the mean filtering method is adopted. For the current pixel (x, y) to be processed, a template composed of several neighboring pixels is selected, the mean value of all pixels in the template is calculated, and this mean value is assigned to the current pixel (x, y) as the gray level g(x, y) of the processed image at that point. That is, g(x, y) = ∑f(x, y) / mm, where g(x, y) is the total number of pixels in the template, including the current pixel.

[0094] By smoothing the image, noise in the target image is suppressed while preserving as much detail as possible, making the image smoother, sharper, and with enhanced boundaries. This results in clearer image contours.

[0095] In a preferred embodiment, in step S103, in order to determine the driver's eye position, such as... Figure 3 As shown, Figure 3 A flowchart illustrating an embodiment of the present invention for determining the driver's eye position includes:

[0096] Step S131: Based on the relative positional relationship of the binoculars, perform calibration transformation on the in-vehicle binocular camera to construct an ideal binocular ranging model;

[0097] Step S132: Based on the ideal binocular ranging model, according to the color space conversion function, perform stereo matching on the first driver's frontal image and the second driver's frontal image inside the vehicle to determine the two-dimensional coordinates of the first eye and the second eye.

[0098] Step S133: Based on the ideal binocular ranging model, determine the parallax of the first camera and the second camera;

[0099] Step S134: Based on the two-dimensional coordinates of the first human eye, the two-dimensional coordinates of the second human eye, and the parallax, determine the human eye positioning using the calculation formula for the distance between the object and the binocular camera.

[0100] In this embodiment, firstly, the in-vehicle binocular camera is calibrated and transformed according to the relative positional relationship of the two eyes to obtain an ideal binocular ranging model; secondly, based on the ideal binocular ranging model, stereo matching is performed on the first and second in-vehicle driver frontal images according to the color space conversion function to determine the two-dimensional coordinates of the first and second eyes; then, based on the ideal binocular ranging model, the parallax of the first and second cameras is determined; finally, based on the two-dimensional coordinates of the first and second eyes and the parallax, the eye positioning is determined using the formula for calculating the distance of an object to the binocular camera.

[0101] In this embodiment, by correcting and transforming the first and second frontal images of the driver inside the vehicle, the two imaging planes are made parallel to the baseline and the same point is located in the same row in the left and right images, thus realizing the construction of an ideal binocular ranging model, so as to facilitate the application of triangulation principles to calculate the distance.

[0102] It should be noted that, generally, the optical centers of the first and second cameras are not on the same horizontal line. This is called a non-ideal binocular ranging model, which is detrimental to subsequent disparity and distance calculations. Examples of non-ideal binocular ranging models include... Figure 4 As shown, Figure 4 This is a schematic diagram showing the results of an embodiment of the non-ideal binocular ranging model provided by the present invention.

[0103] In a preferred embodiment, in step S131, to achieve the accuracy of binocular ranging, the binocular camera needs to be calibrated so that the two imaging planes are parallel to the baseline and the same point is located in the same row in both the left and right images; after the calibration transformation, the left and right imaging planes are located on the same horizontal plane, such as... Figure 5 As shown, Figure 5 This is a schematic diagram showing the results of an embodiment of the ideal binocular ranging model provided by the present invention.

[0104] Where C1 and C2 are the optical centers of the left and right cameras, P is the target object, f is the camera focal length, B is the camera baseline, and X... L Let X be the column coordinates of the point of object P on the imaging plane of the first camera. R Let D be the column coordinates of the point P on the imaging plane of the second camera, D be the parallax between the two corresponding points, and Z be the distance from the target object point P to the stereo camera.

[0105] In a preferred embodiment, in step S132, in order to determine the two-dimensional coordinates of the first human eye and the second human eye, the get_frontal_face_detector() function of the dlib library is used to detect the faces in the first and second frontal images of the driver inside the vehicle, respectively, thereby determining the two-dimensional coordinates of the human eyes in the first and second cameras, that is, the two-dimensional coordinates of the first human eye and the second human eye.

[0106] In a preferred embodiment, in step S133, the parallax of the human eye coordinates in the first and second cameras is calculated according to the formula:

[0107] D = disparity = X L -X R .

[0108] Parallax refers to the change and difference in the position of an object in the field of vision when we observe the same object from two different positions.

[0109] In a preferred embodiment, in step S134, in order to determine the human eye location, it is necessary to calculate the distance between the object and the binocular camera using the formula for calculating the distance between the object and the binocular camera. The formula for calculating the distance between the object and the binocular camera is as follows:

[0110]

[0111] Where Z is the distance from the human eye to the camera, B is the baseline length of the first and second eye cameras, and X is the distance from the human eye to the camera. c X is the column coordinate of the driver's frontal image from inside the first vehicle. R Let f be the column coordinates of the driver's frontal image from inside the second vehicle, f be the focal length of the first and second cameras, and D be the parallax.

[0112] After determining the human eye localization, we used the Weighted Least Squares (WLS) method to filter the disparity map to improve accuracy and reduce noise. Disparity map filtering transforms sparse disparities into dense disparities, thus improving the visual effect of the disparity map. Then, after filtering the disparity map, we used the StereoRectify() function from the OpenCV library to calculate pixel depth. This function returns a reprojection matrix Q, a 4x4 mapping matrix from the disparity map to the depth map. Next, we used the Q matrix and the reprojectImageTo3D() function from the OpenCV library to convert the pixel coordinates into 3D coordinates. This function returns a 3-channel matrix storing the X, Y, and Z coordinates (based on the left camera coordinate system). Finally, through simple 3D mathematical calculations, we obtained the distances of the left and right eyes from the stereo camera, thus obtaining their accurate world 3D coordinates.

[0113] In a preferred embodiment, after determining the location of the human eye, it is also necessary to determine the blind spot image of the A-pillar inside the vehicle, such as... Figure 6 As shown, Figure 6 A flowchart illustrating an embodiment of the present invention for determining a blind spot image of the A-pillar inside a vehicle includes:

[0114] Step S231: Obtain the coordinates of the first center point and the boundary point of the A-pillar display screen, wherein the boundary point includes at least three points;

[0115] Step S232: Based on the human eye positioning, the coordinates of the first center point, and the coordinates of the boundary point, determine the first distance between the human eye and the coordinates of the first center point, the second distance between the human eye and the boundary point, and the third distance between the coordinates of the first center point and the coordinates of the boundary point, respectively.

[0116] Step S233: Determine the first angle between the human eye and the boundary point based on the first distance, the second distance, and the third distance;

[0117] Step S234: Based on the first angle and the image outside the vehicle, determine the cropping points in the image outside the vehicle using wide-angle perspective transformation technology, wherein each cropping point corresponds one-to-one with a boundary point;

[0118] Step S235: Capture the exterior image based on the capture point to determine the blind spot image of the A-pillar inside the vehicle.

[0119] In this embodiment, firstly, the coordinates of the first center point and the coordinates of the boundary points of the A-pillar display screen are obtained, wherein the boundary points include at least three. Secondly, based on the human eye positioning, the coordinates of the first center point, and the coordinates of the boundary points, the first distance between the human eye and the coordinates of the first center point, the second distance between the human eye and the boundary points, and the third distance between the coordinates of the first center point and the coordinates of the boundary points are determined respectively. Based on the first distance, the second distance, and the third distance, the first angle between the human eye and the boundary points is determined. Then, based on the first angle and the external image, the cropping point in the external image is determined by wide-angle perspective transformation technology, wherein the cropping point corresponds one-to-one with the boundary point. Finally, the external image is cropped based on the cropping point to determine the blind spot image of the A-pillar inside the vehicle.

[0120] In this embodiment, by analyzing the data of human eye positioning and points on the A-pillar display screen, the viewing angle between the human eye and the A-pillar display screen is realized, thereby determining the required part in the external image. Through angle transformation and image cropping, the blind spot image of the A-pillar inside the vehicle is determined, effectively linking the human eye with the A-pillar display screen and the external image to facilitate the determination of the final required blind spot image of the A-pillar inside the vehicle.

[0121] As a preferred embodiment, in step S231, in order to reduce the complexity of data processing while ensuring the accuracy of image processing, considering that a rectangle is a simple geometric shape, easy to process and calculate, and can effectively cover the area of ​​the blind spot of the A-pillar, a rectangle is used for cropping and deformation, and four boundary points are selected.

[0122] In one specific embodiment, the A-pillar display screen is 18cm long and 4cm wide.

[0123] Based on the positioning function in the data processing module, the coordinates of the first center point and the coordinates of the four boundary points of the A-pillar display screen are obtained in real time.

[0124] In a preferred embodiment, in step S232, based on the determination of the human eye location, and combining the coordinates of the first center point and the coordinates of the four boundary points, the first distance between the human eye and the coordinates of the first center point, the second distance between the human eye and the boundary points, and the third distance between the coordinates of the first center point and the coordinates of the boundary points can be determined by distance calculation.

[0125] Furthermore, in step S233, in order to determine the first angle between the human eye and the boundary point, it is determined using an angle conversion formula. The calculation formula for the angle conversion formula is as follows:

[0126]

[0127] Where θ is the first angle, and D eyes For the first distance, D hy For the second distance, D A This is the third distance.

[0128] In one specific embodiment, to clearly illustrate the relationship between the various data points, taking boundary point A as an example, as follows: Figure 7 As shown, Figure 7 This is a schematic diagram showing the results of an embodiment of the boundary point A related data provided by the present invention.

[0129] Among them, point e is the human eye, and D eyes For the first distance, D h For the second distance, D A The third distance is O, the center point of the A-pillar display screen, O', the center point of the external image, A, the boundary point of the A-pillar, A', the corresponding point of the boundary point A on the external image after mapping, and θ, the first angle.

[0130] Using the above method, the corresponding points of the other three boundary points on the vehicle exterior image can be determined.

[0131] In a preferred embodiment, in step S234, in order to determine the capture point in the external image, firstly, the shooting wide-angle of the external camera and the second center point of the external image are obtained; then, based on the shooting wide-angle, the first angle and the second center point, the capture point corresponding to the boundary point coordinates in the external image is determined by the formula for calculating the ratio of angle to length.

[0132] The formula for calculating the ratio of angle to length is:

[0133]

[0134] Where θ is the first angle, W is the shooting wide angle, D is the parallax, O is the second center point, and A' is the capture point.

[0135] In one specific embodiment, the `getPerspectiveTransform()` function of OpenCV is called, passing in an array of coordinates of four corresponding points and parameters from the A-pillar display screen, to perform perspective transformation and obtain an image of the blind spot inside the vehicle caused by the A-pillar. For example... Figure 8 As shown, Figure 8 This is a schematic diagram showing the result of an embodiment of perspective transformation provided by the present invention.

[0136] Where O represents the position of the human eye, A'B'C'D' represents the A-pillar display screen, and ABCD represents the portion of the external image that is blocked by the A-pillar.

[0137] In a preferred embodiment, in step S105, in order to display the blind spot image of the A-pillar inside the vehicle, the capture point and the boundary point are matched in position, and the blind spot image of the A-pillar inside the vehicle is displayed on the A-pillar display screen.

[0138] The above method allows for the real-time acquisition of a frontal image of the driver inside the vehicle using an in-vehicle binocular camera. The data processing module then performs eye recognition and positioning on the driver's frontal image to achieve real-time eye position acquisition. Simultaneously, an external camera acquires real-time images of the outside environment. The data processing module, combined with wide-angle perspective transformation technology, adaptively crops and adjusts these images based on eye positioning. This enables the acquisition of blind spots inside the vehicle, such as those obscured by the A-pillar, which are displayed in real-time on the A-pillar screen. This allows for complete integration of the blind spots inside the vehicle with the external scenery.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0140] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An automobile A-pillar blind area display method, characterized by, The method comprises the following steps: Based on the binocular camera in the vehicle, the front image of the driver in the vehicle is obtained, including: calibrating the binocular camera in the vehicle to determine the binocular relative position relationship of the first camera and the second camera; the binocular camera in the vehicle is photographed to obtain the front image of the driver in the vehicle; wherein the front image of the driver in the vehicle includes the first front image of the driver in the vehicle and the second front image of the driver in the vehicle, the first front image of the driver in the vehicle is photographed based on the first camera, the second front image of the driver in the vehicle is photographed based on the second camera, and the focal length of the first camera and the second camera is the same; Based on the camera outside the vehicle, the image outside the vehicle is obtained; Based on the data processing module, the position of the driver's eyes is determined according to the front image of the driver in the vehicle, and the image of the A-pillar blind area in the vehicle is determined according to the position of the eyes and the image outside the vehicle, combined with the wide-angle perspective transformation technology; Based on the A-pillar display screen, the image of the A-pillar blind area in the vehicle is displayed; The position of the driver's eyes is determined according to the front image of the driver in the vehicle, including: According to the binocular relative position relationship, the binocular camera in the vehicle is corrected and transformed to construct an ideal binocular distance measurement model; Based on the ideal binocular distance measurement model, the first front image of the driver in the vehicle and the second front image of the driver in the vehicle are stereoscopically matched according to the color space conversion function to determine the first two-dimensional coordinates of the eyes and the second two-dimensional coordinates of the eyes; Based on the ideal binocular distance measurement model, the parallax of the first camera and the second camera is determined; According to the first two-dimensional coordinates of the eyes, the second two-dimensional coordinates of the eyes and the parallax, the position of the eyes is determined through the object distance binocular camera calculation formula; According to the position of the eyes and the image outside the vehicle, combined with the wide-angle perspective transformation technology, the image of the A-pillar blind area in the vehicle is determined, including: Obtain the first center point coordinates and boundary point coordinates of the A-pillar display screen, wherein the boundary point includes at least three; According to the position of the eyes, the first center point coordinates and the boundary point coordinates, the first distance between the eyes and the first center point coordinates, the second distance between the eyes and the boundary points, and the third distance between the first center point coordinates and the boundary point coordinates are determined respectively; According to the first distance, the second distance and the third distance, the first angle between the eyes and the boundary points is determined; According to the first angle and the image outside the vehicle, the intercepting point in the image outside the vehicle is determined through the wide-angle perspective transformation technology, wherein the intercepting point corresponds to the boundary point one by one; According to the intercepting point, the image outside the vehicle is intercepted to determine the image of the A-pillar blind area in the vehicle.

2. The automobile A-pillar blind spot display method according to claim 1, characterized by, The object distance binocular camera calculation formula is: wherein, Z is a distance from the human eye to the camera, B is a baseline length of the first eye camera and the second eye camera, X c is a column coordinate of the human eye in the first in-vehicle driver frontal image, X R is a column coordinate of the human eye in the second in-vehicle driver frontal image, f is a focal length of the first eye camera and the second eye camera, D is the parallax.

3. The automotive A-pillar blind-spot display method of claim 1, wherein, According to the first distance, the second distance and the third distance, the first angle between the eyes and the boundary points is determined through the angle conversion formula; The calculation formula of the angle conversion formula is: ​ wherein is the first angle, D eyes is the first distance, D hy is the second distance, D A is the third distance.

4. The automotive A-pillar blind-spot display method of claim 1, wherein, The method comprises the following steps: According to the first angle and the image outside the vehicle, the intercepting point in the image outside the vehicle is determined by wide-angle perspective transformation technology, comprising: Obtaining the shooting wide-angle of the camera outside the vehicle and the second center point of the image outside the vehicle; 5. The automotive A-pillar blind-spot display method according to claim 4, characterized in that, According to the shooting wide-angle, the first angle and the second center point, the intercepting point corresponding to the boundary point coordinate in the image outside the vehicle is determined by the calculation formula of the ratio of angle and length. wherein, is the first angle, W is the shot wide angle, D is the parallax, O is the second center point, A’ is the intercept point.

6. The automotive A-pillar blind-spot display method of claim 1, wherein, The calculation formula of the ratio of angle and length is: The intercepting point includes at least three; based on the A-pillar display screen, the A-pillar blind area image inside the vehicle is displayed, comprising: The intercepting point is positionally corresponding to the boundary point, and the A-pillar blind area image inside the vehicle is displayed on the A-pillar display screen.

Images