An automobile A-pillar blind area image generation method, device, equipment and storage medium

Abstract

The application relates to an automobile A-pillar blind area image generation method, device, equipment and storage medium, which comprises the following steps: determining a driver line-of-sight direction and driver head posture information according to collected driver facial contour information; processing a collected real-time image outside the vehicle to determine a feature vector of the real-time image outside the vehicle; determining an initial blind area image in the driver line-of-sight direction according to the feature vector; and fitting the head posture information with the initial blind area image to obtain a target blind area image. The application provides an automobile A-pillar blind area image generation method, device, equipment and storage medium, which analyzes a driver facial contour to determine a driver line-of-sight direction and driver head posture information, processes a real-time image outside the vehicle to obtain an initial blind area image in the driver line-of-sight direction, and combines the head posture information to determine an accurate blind area image conforming to a driver visual angle, thereby eliminating a safety hidden danger in driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobile safety monitoring, and in particular to an automobile A-pillar blind area image generation method, device, equipment and storage medium. BACKGROUND

[0002] The automobile A-pillar refers to the column connecting the two front doors and the engine compartment on both sides of the front windshield of the automobile. In the event of an accident, the existence of the A-pillar can effectively reduce personnel injuries, but at the same time, the A-pillar will block the driver's view, causing a blind area. Since the driver's seat is located on the left side in China, the blind area caused by the left A-pillar is larger. Therefore, how to reduce or eliminate the A-pillar blind area when turning left is an important aspect of improving traffic safety.

[0003] At present, the method for eliminating the A-pillar blind area in the prior art generally uses a camera outside the automobile to collect image information outside the automobile, further extracts the A-pillar blind area image of the automobile from the image information outside the automobile, and then displays the image in the A-pillar blind area through a display screen installed in the vehicle.

[0004] However, the method for generating the A-pillar blind area in the prior art does not consider adjusting the range of the A-pillar blind area according to the facial contour information of the driver, and cannot accurately display the blind area range of the driver. The driver needs to further identify the accurate blind area range, and long-term fixation on the screen image will also cause the driver to be dazzled, which exists a driving safety hazard. SUMMARY

[0005] Therefore, it is necessary to provide an automobile A-pillar blind area image generation method, device, equipment and storage medium to solve the problem that the prior art cannot generate an accurate blind area range according to the facial contour information of the driver when generating the A-pillar blind area image of the automobile, the driver needs to further identify the blind area range, and long-term fixation on the screen image will cause the driver to be dazzled, which exists a driving safety hazard.

[0006] To achieve the above technical purposes, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides an automobile A-pillar blind area image generation method, comprising:

[0008] determining the driver's line of sight direction and the driver's head posture information according to the collected facial contour information of the driver;

[0009] processing the collected real-time image outside the vehicle to determine the feature vector of the real-time image outside the vehicle;

[0010] determining the initial blind area image in the driver's line of sight direction according to the feature vector;

[0011] The head pose information is fitted with the initial blind area image to obtain a target blind area image.

[0012] In some possible implementation manners, the collected real-time image outside the vehicle is processed to determine a feature vector of the real-time image outside the vehicle, including:

[0013] The real-time image outside the vehicle is preprocessed to determine a candidate feature point;

[0014] The candidate feature point is screened to determine a target feature point;

[0015] A corresponding feature vector is calculated according to the target feature point.

[0016] In some possible implementation manners, the real-time image outside the vehicle is preprocessed to determine a candidate feature point, including:

[0017] The real-time image outside the vehicle is denoised by a preset denoising method;

[0018] The real-time image outside the vehicle after denoising is processed by a plurality of times of Gaussian filtering and down-sampling until a preset threshold requirement is met, to obtain a plurality of groups of Gaussian images;

[0019] A Gaussian difference image is obtained according to two adjacent groups of Gaussian images;

[0020] Sampling points of the Gaussian difference image are compared to obtain a candidate feature point meeting a preset pixel requirement.

[0021] In some possible implementation manners, the candidate feature point is screened to determine a target feature point, including:

[0022] The candidate feature point is filtered;

[0023] The Gaussian difference image is fitted to determine a Taylor expansion of the Gaussian difference image of the filtered candidate feature point;

[0024] A Hessian matrix is determined according to the Taylor expansion;

[0025] The target feature point is determined according to the Hessian matrix and a preset screening threshold.

[0026] In some possible implementation manners, a corresponding feature vector is calculated according to the target feature point, including:

[0027] An image gradient amplitude is calculated according to the target feature point;

[0028] An image gradient direction is calculated according to the target feature point;

[0029] The feature vector is calculated according to the image gradient amplitude and the image gradient direction.

[0030] In some possible implementation manners, determining the initial blind area image in the driver's line-of-sight direction according to the feature vector comprises:

[0031] calculating a similarity measure between the feature vector of the real-time image outside the vehicle and the feature vector of the Gaussian difference image;

[0032] determining the alternative blind area image according to the similarity measure;

[0033] converting the alternative blind area image to the driver's line-of-sight direction to determine the initial blind area image.

[0034] In some possible implementation manners, fitting the head posture information with the initial blind area image to obtain the target blind area image comprises:

[0035] determining a driver head posture transformation condition according to the head posture information;

[0036] performing coordinate transformation on the initial blind area image according to the driver head posture transformation condition to obtain the target blind area image.

[0037] In a second aspect, the present application further provides an automobile A-pillar blind area image generation device, comprising:

[0038] an information processing module configured to determine a driver line-of-sight direction and driver head posture information according to collected driver facial contour information;

[0039] a feature vector module configured to process the collected real-time image outside the vehicle to determine a feature vector of the real-time image outside the vehicle;

[0040] a blind area generation module configured to determine an initial blind area image in the driver's line-of-sight direction according to the feature vector;

[0041] a fitting module configured to fit the head posture information with the initial blind area image to obtain a target blind area image.

[0042] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein:

[0043] the memory is configured to store a program;

[0044] the processor is coupled to the memory and is configured to execute the program stored in the memory to implement the steps in the automobile A-pillar blind area image generation method in any of the above implementation manners.

[0045] In a fourth aspect, the present application further provides a computer readable storage medium configured to store computer readable programs or instructions, which, when executed by a processor, can implement the steps in the automobile A-pillar blind area image generation method in any of the above implementation manners.

[0046] The beneficial effects of the above embodiment are: the application relates to an automobile A-pillar blind area image generation method, device, equipment and storage medium, the method comprises the following steps: determining the driver's line-of-sight direction and the driver's head posture information according to the collected facial contour information of the driver; processing the collected real-time image outside the vehicle to determine the feature vector of the real-time image outside the vehicle; determining the initial blind area image in the driver's line-of-sight direction according to the feature vector; and fitting the head posture information with the initial blind area image to obtain a target blind area image. The automobile A-pillar blind area image generation method, device, equipment and storage medium provided by the application can analyze the facial contour of the driver to determine the driver's line-of-sight direction and the driver's head posture information, then process the real-time image outside the vehicle to determine the initial blind area image in the driver's line-of-sight direction, and then combine the head posture information to determine the accurate blind area image that meets the driver's visual angle, so that the accurate blind area position under the current head posture of the driver can be directly determined according to the head posture of the driver during driving, the accurate blind area image that meets the driver's visual angle can be accurately displayed, the driver does not need to identify the range of the blind area again, the problem that the driver is dazzled by the long-time gaze at the screen image during driving is avoided, and finally the safety hazard during driving is eliminated, and the safety of driving is improved. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A flowchart of an embodiment of the automobile A-pillar blind area image generation method provided by the application is shown in the figure;

[0048] Figure 2 An embodiment of the automobile A-pillar blind area image generation device provided by the application is shown in the figure; Figure 1 A flowchart of an embodiment of step S102 in the method is shown in the figure;

[0049] Figure 3 An embodiment of the automobile A-pillar blind area image generation device provided by the application is shown in the figure; Figure 2 A flowchart of an embodiment of step S201 in the method is shown in the figure;

[0050] Figure 4 An embodiment of the automobile A-pillar blind area image generation device provided by the application is shown in the figure; Figure 2 A flowchart of an embodiment of step S202 in the method is shown in the figure;

[0051] Figure 5 An embodiment of the automobile A-pillar blind area image generation device provided by the application is shown in the figure; Figure 2 A flowchart of an embodiment of step S203 in the method is shown in the figure;

[0052] Figure 6 An embodiment of the automobile A-pillar blind area image generation device provided by the application is shown in the figure; Figure 1 A flowchart of an embodiment of step S103 in the method is shown in the figure;

[0053] Figure 7 A structural diagram of an embodiment of the automobile A-pillar blind area image generation device provided by the application is shown in the figure;

[0054] Figure 8 The structural schematic diagram of the electronic device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0055] The preferred embodiments of the present application will be described in detail below with reference to the drawings, which form a part of this application. The drawings and the associated descriptions are provided to illustrate the embodiments of the present application and to explain the principles of the present application, but are not intended to limit the scope of the present application.

[0056] In the description of the present application, the meaning of “a plurality of” is two or more, unless otherwise explicitly and specifically limited.

[0057] Reference herein to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive or alternative embodiments. It is expressly understood that the embodiments described herein are combinable.

[0058] The present application provides a car A-pillar blind area image generation method, device, equipment and storage medium, which are described below respectively.

[0059] Please refer to Figure 1 , Figure 1 The flowchart of an embodiment of the car A-pillar blind area image generation method provided by the present application is shown. A specific embodiment of the present application discloses a car A-pillar blind area image generation method, which comprises:

[0060] S101, determining the driver's line of sight direction and the driver's head posture information according to the collected facial contour information of the driver;

[0061] S102, processing the collected real-time image outside the vehicle to determine the feature vector of the real-time image outside the vehicle;

[0062] S103, determining the initial blind area image in the driver's line of sight direction according to the feature vector;

[0063] S104, fitting the head posture information with the initial blind area image to obtain the target blind area image.

[0064] In the above embodiment, the facial contour information of the driver is collected by the built-in binocular camera (placed on the instrument panel) and the driver's perspective simulation camera (placed near the driver's head), the real-time image outside the vehicle is collected by the external wide-angle camera (placed outside the A-pillar), and the generated image is displayed on the small round mirror display screen on the left side mirror.

[0065] The wide-angle camera outside the vehicle monitors the road condition outside the vehicle in real time. When the vehicle is always on a straight road, the small mirror display screen enlarges the field of view behind the vehicle as an ordinary small mirror. When the wide-angle camera outside the vehicle monitors that the vehicle is about to enter a curve, the real-time image outside the vehicle and the facial contour information of the driver are collected to provide the blind area image for the driver.

[0066] The face feature of the driver needs to be aligned first. As a preferred embodiment, a regression tree integration algorithm is used for face alignment. The algorithm is fast and can realize face alignment in milliseconds. The relative position relationship of the head in the three-dimensional image relative to the camera is inferred from the two-dimensional image. A general head model is used, that is, the coordinates of the tip of the nose in the world coordinate system are (0, 0, 0), the coordinates of the chin are (0, -330, -65), the coordinates of the left eye corner are (-225, -170, -135), the coordinates of the right eye corner are (225, 170, -135), the coordinates of the left corner of the mouth are (-150, -150, -125), and the coordinates of the right corner of the mouth are (150, -150, -125). By comparing the corresponding mark points obtained from the image, as a preferred embodiment, the head posture information is calculated by using the EPnP solution to solve the PnP problem. It should be noted that the head posture information of the driver can also be determined by using other methods, and the present application does not make any further limitation in this regard.

[0067] Compared with the prior art, the automobile A-pillar blind area image generation method provided by the embodiment comprises the following steps: determining the driver's line of sight direction and the driver's head posture information according to the collected facial contour information of the driver; processing the collected real-time image outside the vehicle to determine the feature vector of the real-time image outside the vehicle; determining the initial blind area image in the driver's line of sight direction according to the feature vector; and fitting the head posture information with the initial blind area image to obtain the target blind area image. The automobile A-pillar blind area image generation method, device, equipment and storage medium provided by the present application analyze the facial contour of the driver to determine the driver's line of sight direction and the driver's head posture information, then process the real-time image outside the vehicle to determine the initial blind area image in the driver's line of sight direction, and then combine the head posture information to determine the accurate blind area image that conforms to the driver's visual angle. The accurate blind area image that conforms to the driver's visual angle can be directly determined according to the driver's head posture during driving, the accurate blind area image that conforms to the driver's visual angle can be accurately displayed, the driver does not need to identify the range of the blind area, the problem that the driver's vision is dazzled due to the long-time gaze at the screen image during driving is avoided, the safety hazard during driving is finally eliminated, and the safety of driving is improved.

[0068] Please refer to Figure 2 , Figure 2 to provide Figure 1A flowchart of an embodiment of step S102, in some embodiments of the present application, the collected real-time image outside the vehicle is processed to determine the feature vector of the real-time image outside the vehicle, comprising:

[0069] S201, performing a preset processing on the real-time image outside the vehicle to determine the candidate feature points;

[0070] S202, screening the candidate feature points to determine the target feature points;

[0071] S203, calculating the corresponding feature vector according to the target feature points.

[0072] In the above embodiment, the real-time image outside the vehicle is the original image, and the preset processing on the real-time image outside the vehicle includes segmenting the real-time image outside the vehicle to determine the candidate feature points of each part, and the feature points refer to the points where the image gray value changes sharply or the points with large curvature on the image edge (i.e. the intersection of two edges). The image feature points play a very important role in the image matching algorithm based on feature points. The image feature points can reflect the essential characteristics of the image and can identify the target objects in the image. The matching of the feature points can complete the matching of the image.

[0073] The obtained candidate feature points are screened to eliminate the feature points that do not meet the requirements to obtain the target feature points, and the corresponding feature vector of the image is further calculated and determined through the target feature points, which can be converted through the feature vector to finally determine the image of the blind area.

[0074] Please refer to Figure 3 , Figure 3 to provide Figure 2 A flowchart of an embodiment of step S201, in some embodiments of the present application, the real-time image outside the vehicle is processed to determine the candidate feature points, comprising:

[0075] S301, performing denoising processing on the real-time image outside the vehicle through a preset denoising method;

[0076] S302, performing several times of Gaussian filtering processing and down-sampling processing on the denoised real-time image outside the vehicle until the preset threshold requirement is met to obtain several groups of Gaussian images;

[0077] S303, obtaining a Gaussian difference image according to two adjacent groups of Gaussian images;

[0078] S304, comparing the sampling points of the Gaussian difference image to obtain the candidate feature points that meet the preset pixel requirement.

[0079] In the above embodiment, the preset denoising method is the BM3D algorithm, and the real-time image outside the vehicle is denoised by the BM3D algorithm, and it can be understood that the BM3D algorithm can be directly used for denoising, and the BM3D algorithm belongs to the prior art, and does not need to be described in detail.

[0080] The original image is subjected to multiple Gaussian filtering, and then the image scale group is down-sampled from the bottom to the top, the even rows and even columns of the current image scale function group are deleted, so that a down-sampled image with a size of one fourth of the original image is obtained, and then Gaussian filtering is used again to obtain a second group of image scales, and then the above operation is continuously performed until a preset threshold requirement (the preset is one sixteenth of the original image) is reached, and the Gaussian difference image is obtained by subtracting two adjacent groups of Gaussian images.

[0081] Each sampling point in the Gaussian difference image and its 8-direction field points in the same layer and 9 field points in the upper and lower layers are compared respectively. If the pixel value is greater than or less than the pixel values of all adjacent 26 pixel points, the column is selected as a candidate feature point, and the position and scale information are recorded.

[0082] The specific calculation process is as follows:

[0083] The Gaussian filtering formula is used:

[0084]

[0085] Wherein, sigma represents the standard deviation of the Gaussian kernel function.

[0086] Suppose that the original image is represented as I(x,y), and the scale space formula is represented as:

[0087] L(x,y,σ)=G(x,y,σ)*I(x,y) (2);

[0088] Wherein, I(x,y) represents the pixel value at point (x,y).

[0089] The calculation formula of the Gaussian difference image is:

[0090] D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))*I(x,y)=L(x,y,kσ)-L(x,y,σ) (3);

[0091] Wherein, k is a constant.

[0092] Please refer to Figure 4 , Figure 4 to provide Figure 2A flowchart of an embodiment of step S202 is shown in FIG. 2. In some embodiments of the present application, the target feature point is determined by screening the alternative feature points, including:

[0093] S401, filtering the alternative feature points;

[0094] S402, fitting the Gaussian difference image to determine the Taylor expansion of the Gaussian difference image of the filtered alternative feature points;

[0095] S403, determining the Hessian matrix according to the Taylor expansion;

[0096] S404, determining the target feature point according to the Hessian matrix and a preset screening threshold.

[0097] In the above embodiment, the unstable feature points and the feature points with interference are removed by filtering the alternative feature points. It can be understood that the filtering of the feature points can be achieved by the prior art, and the filtering is not the focus of the present application. Therefore, the present application will not be described in detail.

[0098] The Gaussian difference image is fitted to derive the Taylor expansion of the Gaussian difference image of the alternative feature points, and the corresponding Hessian matrix is determined according to the Taylor expansion to screen the filtered alternative feature points, thereby obtaining the target feature point meeting the requirements.

[0099] The Taylor expansion is:

[0100]

[0101] Derive the above formula and set the derivative to 0:

[0102]

[0103] Substitute it into formula (4) and take the first two terms:

[0104]

[0105] When the offset value of a certain alternative feature point in any dimension is greater than 0.5, the distance between the extreme point and other points is closer, and the new alternative feature point needs to be selected for solving. Repeat the above steps until the offset value is less than 0.5.

[0106] However, the edge response value in the above method is low, and needs to be selectively removed by using the Hessian matrix. The Hessian matrix formula is represented as:

[0107]

[0108] Wherein Dxx(x,y), Dxy(x,y), Dyy(x,y) are obtained by the pixel difference of the field point of the candidate feature point.

[0109] The determinant and trace formula of the matrix are as follows:

[0110] Det(H)=D xx D yy -(D xy ) 2 =α·β

[0111]

[0112] Wherein alpha and beta represent the maximum value and minimum value of the eigenvalue respectively, and gamma is the ratio of the two, then:

[0113]

[0114] The preset screening threshold gamma is set to 0.3, and the points with small edge response values are removed, that is, the retained feature points need to meet:

[0115]

[0116] The edge response value of the feature point obtained by the Hessian matrix and the preset screening threshold gamma is larger, so the target feature point obtained by screening can better represent the characteristics of the Gaussian difference image.

[0117] Please refer to Figure 5 , Figure 5 provide Figure 2 The flowchart of an embodiment of step S203 in the application, in some embodiments of the application, the corresponding feature vector is calculated according to the target feature point, which includes:

[0118] S501, calculating the image gradient amplitude according to the target feature point;

[0119] S502, calculating the image gradient direction according to the target feature point;

[0120] S503, calculating the feature vector according to the image gradient amplitude and the image gradient direction.

[0121] In the above embodiment, the gradient amplitude formula of the Gaussian difference image corresponding to the target feature point is:

[0122]

[0123] The image gradient direction formula in the feature point sub-region is:

[0124]

[0125] In the 16*16 window centered on the target feature point, the 16*16 window is divided into 16 4*4 regions, each of which represents a pixel point, each pixel point represents its gradient amplitude and gradient direction, and each pixel point is assigned a weight according to its distance from the feature point, so that a feature vector representing the feature point can be obtained.

[0126] It can be understood that the weight assigned to the pixel point according to its distance from the feature point can be adjusted according to actual conditions, and the present application does not make further limitations.

[0127] Please refer to Figure 6 , Figure 6 The present application provides Figure 1 The flowchart of an embodiment of step S103 in the present application, in some embodiments of the present application, the initial blind area image in the driver's line of sight direction is determined according to the feature vector, which includes:

[0128] S601, calculating the similarity measure between the feature vector of the real-time image outside the vehicle and the feature vector of the Gaussian difference image;

[0129] S602, determining the candidate blind area image according to the similarity measure;

[0130] S603, converting the candidate blind area image to the driver's line of sight direction to determine the initial blind area image.

[0131] In the above embodiment, the feature vector of the real-time image outside the vehicle is represented by X, and the feature vector of the Gaussian difference image is represented by Y, then the similarity measure between X and Y is calculated according to the following formula:

[0132]

[0133] The similarity measure is calculated to screen the Gaussian difference image, and the Gaussian difference image that meets the similarity measure requirement is selected to obtain the candidate blind area image.

[0134] The blind area region is perspective transformed to be converted to the driver's perspective. The coordinates of a point on the selected blind area image are p(x, y), and the point p'(x', y') is obtained after perspective transformation, and the corresponding relationship between the two points is:

[0135]

[0136] Where θ is the rotation angle, and (tx, ty)T represents the translation distance.

[0137] In some embodiments of the present application, the head pose information is fitted with the initial blind area image to obtain the target blind area image, which includes:

[0138] The driver head posture transformation condition is determined according to the head posture information.

[0139] The initial blind area image is subjected to coordinate transformation to obtain a target blind area image according to the driver head posture transformation condition.

[0140] In the above embodiment, it is assumed that O is the camera optical center, OD is the display screen plane, the distance OH between the outside scene and the external wide-angle camera is fixed as t, when the driver head is at point B, the scene M corresponding to the right side boundary point of the A-pillar blind area on the external wide-angle camera image is point m. When the head moves from point B to point A, the boundary point of the blind area on the external wide-angle camera image moves from m to n. After the blind area boundary point coordinate of the driver head at point B is obtained through the image registration technology, the blind area pixel coordinate of the driver head at point A can be calculated. AG is parallel to MN, and intersects BM at point G, and DK is the perpendicular line of AGD. According to the triangle similarity principle, DK / OH=AG / MN. It is assumed that the three-dimensional coordinates of point M in the external wide-angle camera coordinate system are (X1, Y1, Z1), and the corresponding pixel coordinates on the external wide-angle camera image are (u1, v1), and the three-dimensional coordinates of point N in the external wide-angle camera coordinate system are (X2, Y2, Z2), and the corresponding pixel coordinates on the external wide-angle camera image are (u2, v2). According to the camera calibration principle, the following formulae are obtained:

[0141]

[0142] Because the distance between the scene and the external wide-angle camera is fixed as t, Z1=Z2=t. The above two formulae are subtracted:

[0143]

[0144] Therefore, the horizontal moving distance of the blind area boundary pixel on the external wide-angle camera image is:

[0145]

[0146] The blind area boundary in the vertical direction is not changed by default.

[0147] The embodiment provided by the application also has the functions of ultrasonic speed measurement and warning indicator, the data obtained by the ultrasonic speed measurement is transmitted to the warning indicator, and the warning indicator judges:

[0148] The single-chip microcomputer sends a trigger pulse to the ultrasonic sensor, and the ultrasonic sensor generates a 40 kHz pulse electrical signal after being triggered by the pulse, the electrical signal is converted into mechanical vibration energy after being processed by the excitation transducer, thereby forming ultrasonic waves and emitting outward, the ultrasonic waves emitted outward are immediately reflected after encountering an obstacle, the receiver receives the reflected ultrasonic wave signal, converts the ultrasonic wave into weak electrical oscillation through internal conversion, amplifies the signal, obtains the required pulse signal, and returns the pulse signal to the single-chip microcomputer. The pulse width obtained in the process corresponds to the ultrasonic wave propagation time, and the distance between the vehicle and the obstacle can be calculated according to the ultrasonic wave speed and the propagation time, and distance judgment is performed:

[0149] 1) When the distance of the obstacle is in a relatively safe range (more than 30 m), the warning device emits a relatively gentle prompt sound;

[0150] 2) When the distance of the obstacle is less than 30 m, the warning device emits a prompt sound to remind the driver to slow down.

[0151] In order to better implement the method for generating an A-pillar blind area image of a vehicle in the embodiment of the present application, on the basis of the method for generating an A-pillar blind area image of a vehicle, please refer to Figure 7 , Figure 7 An embodiment of a structure diagram of a device for generating an A-pillar blind area image of a vehicle provided by the present application, the embodiment of the present application provides a device 700 for generating an A-pillar blind area image of a vehicle, comprising:

[0152] An information processing module 710 is configured to determine a driver's line-of-sight direction and driver's head posture information according to the collected facial contour information of the driver;

[0153] A feature vector module 720 is configured to process the collected real-time image outside the vehicle to determine a feature vector of the real-time image outside the vehicle;

[0154] A blind area generation module 730 is configured to determine an initial blind area image in the driver's line-of-sight direction according to the feature vector;

[0155] A fitting module 740 is configured to fit the head posture information with the initial blind area image to obtain a target blind area image.

[0156] It should be noted that the device 700 provided in the above embodiment can implement the technical solutions described in the above method embodiments, and the principles of the implementation of the above modules or units can be referred to the corresponding content in the above method embodiments, which will not be described here.

[0157] Please refer to Figure 8 , Figure 8A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown. Based on the above-mentioned method for generating an image of an A-pillar blind area of a vehicle, the present application also provides a device for generating an image of an A-pillar blind area of a vehicle. The device for generating an image of an A-pillar blind area of a vehicle can be a mobile terminal, a desktop computer, a notebook, a palm computer, a server, or the like. The device for generating an image of an A-pillar blind area of a vehicle includes a processor 810, a memory 820, and a display 830. Figure 8 Only some components of the electronic device are shown, but it should be understood that all the shown components are not required, and more or less components can be implemented instead.

[0158] The memory 820 can be an internal storage unit of the device for generating an image of an A-pillar blind area of a vehicle in some embodiments, such as a hard disk or a memory of the device for generating an image of an A-pillar blind area of a vehicle. The memory 820 can also be an external storage device of the device for generating an image of an A-pillar blind area of a vehicle in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like. Further, the memory 820 can include both an internal storage unit and an external storage device of the device for generating an image of an A-pillar blind area of a vehicle. The memory 820 is used to store application software installed on the device for generating an image of an A-pillar blind area of a vehicle and various types of data, such as program codes of the device for generating an image of an A-pillar blind area of a vehicle. The memory 820 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 820 stores a program for generating an image of an A-pillar blind area of a vehicle 840, which can be executed by the processor 810 to implement the method for generating an image of an A-pillar blind area of a vehicle according to the embodiments of the present application.

[0159] The processor 810 can be a central processing unit (CPU), a microprocessor, or other data processing chip in some embodiments, which is used to run program codes stored in the memory 820 or process data, such as executing the method for generating an image of an A-pillar blind area of a vehicle.

[0160] The display 830 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like in some embodiments. The display 830 is used to display information of the device for generating an image of an A-pillar blind area of a vehicle and to display a visualized user interface. The components 810-830 of the device for generating an image of an A-pillar blind area of a vehicle communicate with each other through a system bus.

[0161] In an embodiment, the above-mentioned steps of the method for generating an A-pillar blind area image of a vehicle are implemented when the processor 810 executes the generating program 840 of the A-pillar blind area image of a vehicle in the memory 820.

[0162] The embodiment also provides a computer-readable storage medium having stored thereon a generating program of an A-pillar blind area image of a vehicle, which, when executed by a processor, implements the following steps:

[0163] determining a driver's line-of-sight direction and head posture information according to the collected facial contour information of the driver;

[0164] processing the collected real-time image outside the vehicle to determine a feature vector of the real-time image outside the vehicle;

[0165] determining an initial blind area image in the driver's line-of-sight direction according to the feature vector;

[0166] fiting the head posture information with the initial blind area image to obtain a target blind area image.

[0167] In summary, the embodiment provides a method, device, equipment and storage medium for generating an A-pillar blind area image of a vehicle, which includes: determining a driver's line-of-sight direction and head posture information according to the collected facial contour information of the driver; processing the collected real-time image outside the vehicle to determine a feature vector of the real-time image outside the vehicle; determining an initial blind area image in the driver's line-of-sight direction according to the feature vector; and fitting the head posture information with the initial blind area image to obtain a target blind area image. The method, device, equipment and storage medium provided by the present application can determine a driver's line-of-sight direction and head posture information by analyzing the facial contour of the driver, then process a real-time image outside the vehicle to determine an initial blind area image in the driver's line-of-sight direction, and then determine an accurate blind area image in accordance with the driver's visual angle by combining the head posture information, so as to directly determine the accurate blind area position under the current head posture of the driver, and to accurately display the blind area image in the driver's visual angle, without the need for the driver to identify the range of the blind area, thus avoiding the problem of dizziness caused by the driver's long-time gaze at the screen image during driving, and ultimately eliminating the safety hazards during driving and improving the safety of driving.

[0168] The above describes only the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements within the technical range disclosed by the present application can be easily thought of by those skilled in the art, which should be covered within the protection scope of the present application.

Claims

1. A method for generating an image of an automobile A-pillar blind spot, characterized in that: include: Determine the driver's sight direction and head posture information based on the collected facial profile information of the driver; Processing the collected real-time image outside the vehicle to determine a feature vector of the real-time image outside the vehicle, including: performing preset processing on the real-time image outside the vehicle to determine candidate feature points; screening the candidate feature points to determine target feature points; calculating an image gradient amplitude based on the target feature points; calculating an image gradient direction based on the target feature points; and calculating a feature vector based on the image gradient amplitude and the image gradient direction; Determining an initial blind spot image in the driver's sight line direction based on the feature vector, including: calculating a similarity measure between a feature vector of a real-time image outside the vehicle and a feature vector of a Gaussian difference image; determining an alternative blind spot image based on the similarity measure; and converting the alternative blind spot image to the driver's sight line direction to determine an initial blind spot image; The head posture information is fitted with the initial blind spot image to obtain a target blind spot image.

2. The method for generating an image of an automobile A-pillar blind spot according to claim 1, characterized in that: The performing preset processing on the real-time image outside the vehicle to determine candidate feature points includes: Performing denoising processing on the real-time image outside the vehicle by using a preset denoising method; The denoised real-time image outside the vehicle is subjected to several Gaussian filtering and downsampling processes until it meets the preset threshold requirements, thereby obtaining several sets of Gaussian images; Obtaining a Gaussian difference image based on the two adjacent groups of Gaussian images; The sampling points of the Gaussian difference image are compared to obtain candidate feature points that meet preset pixel requirements.

3. The method for generating an image of an automobile A-pillar blind spot according to claim 2, characterized in that: The screening of the candidate feature points to determine the target feature points includes: Performing filtering on the candidate feature points; Performing fitting processing on the Gaussian difference image to determine a Taylor expansion of the Gaussian difference image of the filtered candidate feature points; Determine a Hessian matrix according to the Taylor expansion; Target feature points are determined according to the Hessian matrix and a preset screening threshold.

4. The method for generating an image of an automobile A-pillar blind spot according to claim 1, characterized in that: The step of fitting the head posture information with the initial blind spot image to obtain a target blind spot image includes: determining a change in the driver's head posture according to the head posture information; The initial blind spot image is coordinate-transformed according to the driver's head posture change to obtain a target blind spot image.

5. A vehicle A-pillar blind spot image generation device, characterized in that: include: An information processing module is used to determine the driver's sight direction and head posture information based on the collected facial contour information of the driver; A feature vector module is configured to process the collected real-time image outside the vehicle and determine a feature vector of the real-time image outside the vehicle, including: performing preset processing on the real-time image outside the vehicle to determine candidate feature points; screening the candidate feature points to determine target feature points; calculating an image gradient amplitude based on the target feature points; calculating an image gradient direction based on the target feature points; and calculating a feature vector based on the image gradient amplitude and the image gradient direction. a blind spot generation module, configured to determine an initial blind spot image in the driver's sight line direction based on the feature vector, comprising: calculating a similarity measure between a feature vector of a real-time image outside the vehicle and a feature vector of a Gaussian difference image; determining an alternative blind spot image based on the similarity measure; and converting the alternative blind spot image to the driver's sight line direction to determine an initial blind spot image; A fitting module is used to fit the head posture information with the initial blind spot image to obtain a target blind spot image.

6. An electronic device, characterized in that: comprising a memory and a processor, wherein, The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the method for generating an image of a blind spot of an automobile A-pillar as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the method for generating an image of the blind spot of the automobile A-pillar as described in any one of claims 1 to 4.

Images