A vehicle turning guidance method, device, electronic equipment and storage medium

Abstract

The application provides a vehicle turning guiding method and device, electronic equipment and storage medium, the method comprises the following steps: obtaining the head image of the driver at the current time, and the environment image outside the A column at the current time; determining the center of the driver's eyes according to the head image, and determining the coordinate range of the A column blind area image under the current visual angle according to the center of the eyes; according to the coordinate range of the A column blind area image, the corresponding A column blind area image is intercepted from the environment image outside the A column, the perspective transformation is carried out on the A column blind area image, and the perspective transformed A column blind area image is projected on the corresponding A column inside the vehicle. Considering the difference between the images observed by the driver's eyes when the vehicle is running, the coordinate range of the A column blind area image is determined by identifying the center of the eyes, the parallax problem of left and right eye observation is solved, the A column blind area image is projected after perspective transformation, the depth feeling is stronger, and the inconsistency between the projection of the fixed mode and the real field of view is eliminated.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of automobile auxiliary driving, and in particular to a vehicle turning guiding method and device, an electronic device and a storage medium. BACKGROUND

[0002] The A-pillar is above the side mirrors between the engine compartment and the driver's cabin, which will block part of the turning field of view, that is, the field of view blind area. The angle of the driver's line of sight will be more or less blocked. Generally, the overlapping angle of the driver's line of sight through the A-pillar is 5-6 degrees. If the driver's comfort angle is considered, the smaller the overlapping angle is, the better the A-pillar is, but at the same time, the safety of the vehicle is a problem. Obviously, the high rigidity of the A-pillar must be ensured to reduce the risk of safety, otherwise, while reducing the field of view blind area, the safety hazard of the vehicle is brought.

[0003] The prior art projects the image obtained by the camera outside the A-pillar through a projector, or reflects the scene outside the A-pillar through a reflection lens, or displays the scene obtained by the camera outside the A-pillar through a display. However, the method in the prior art is a fixed projection, which cannot adapt to the up-down and left-right position movement of the human eye, resulting in misalignment between the observation of the human eye and the influence of camera collection, and increasing the visual error. SUMMARY

[0004] Therefore, it is necessary to provide a vehicle turning guiding method, device, electronic device and storage medium to solve the problem that the turning guiding method based on fixed projection in the prior art cannot adapt to the up-down and left-right position movement of the human eye, resulting in misalignment between the observation of the human eye and the influence of camera collection, and increasing the visual error.

[0005] To solve the above technical problems, in a first aspect, embodiments of the present application provide a vehicle turning guiding method, comprising:

[0006] obtaining a head image of a driver at a current time and an environment image outside an A-pillar at the current time;

[0007] determining a center of the driver's eyes according to the head image, and determining a coordinate range of an A-pillar blind area image under a current view angle according to the center of the driver's eyes;

[0008] cutting a corresponding A-pillar blind area image from the environment image outside the A-pillar according to the coordinate range of the A-pillar blind area image, performing perspective transformation on the A-pillar blind area image, and projecting the A-pillar blind area image after perspective transformation onto a corresponding A-pillar inside the vehicle.

[0009] In some possible implementation manners, the head image includes a left head image and a right head image; the left head image is a head image captured from the left side of the driver, and the right head image is a head image captured from the right side of the driver; and the left head image and the right head image each include at least the left eye and the right eye of the driver.

[0010] In some possible implementation manners, the determining the center of the two eyes of the driver according to the head image includes:

[0011] extracting a face region in the left head image to obtain a left face image, performing eye edge detection on the left face image according to a Sobel edge detection operator, extracting eye edge information, and determining first two-dimensional coordinates of the center of the two eyes according to the eye edge information;

[0012] performing horizontal gray integral projection on the right head image to obtain a right face image, performing vertical gray integral projection on the right face image to obtain second two-dimensional coordinates of the center of the two eyes;

[0013] determining a parallax of the left head image and the right head image, and determining three-dimensional coordinates of the center of the two eyes according to the parallax, the first two-dimensional coordinates and the second two-dimensional coordinates.

[0014] In some possible implementation manners, the determining the coordinate range of the A-pillar blind area image under the current visual angle according to the center of the two eyes includes:

[0015] determining the current visual angle of the driver according to the center of the two eyes, and determining the coordinate range of the A-pillar blind area image under the current visual angle according to a pre-stored corresponding relationship between the visual angle of the driver and the coordinate range of the A-pillar blind area image.

[0016] In some possible implementation manners, the obtaining the head image of the driver at the current moment specifically includes:

[0017] obtaining a rotation angle of a steering wheel in real time, and if it is determined that the vehicle is turning according to the rotation angle, obtaining the head image of the driver at the current moment from both sides of the driver.

[0018] In some possible implementation manners, after the determining the coordinate range of the A-pillar blind area image under the current visual angle according to the center of the two eyes, the method further includes:

[0019] obtaining an obstacle target in the coordinate range of the A-pillar blind area image, comparing a trajectory of the obstacle target with a current trajectory of the vehicle according to a pre-established vehicle-pedestrian trajectory prediction model, and if it is determined that the obstacle target and the vehicle will collide according to a comparison result of the trajectory of the obstacle target and the current trajectory of the vehicle, issuing an alarm voice to remind.

[0020] In some possible implementation manners, the comparing the trajectory of the obstacle target and the current trajectory of the vehicle according to the pre-established vehicle-pedestrian trajectory prediction model comprises:

[0021] predicting second trajectory data of the obstacle target in a second time period according to first trajectory data of the obstacle target in a first time period;

[0022] if it is determined that the braking deceleration of the vehicle in the second time period is greater than a preset braking deceleration threshold, or it is determined that the moving speed of the obstacle target in the second time period increases according to the second trajectory data, it is determined that the obstacle target and the vehicle will collide.

[0023] In a second aspect, an embodiment of the present application further provides a vehicle turning guiding device, comprising:

[0024] an image acquisition module configured to acquire a head image of a driver at a current time and an environment image outside an A-pillar at the current time;

[0025] an image processing module configured to determine a center of two eyes of the driver according to the head image, and determine a coordinate range of an A-pillar blind area image under a current visual angle according to the center of the two eyes;

[0026] an image projection module configured to intercept a corresponding A-pillar blind area image from the environment image outside the A-pillar according to the coordinate range of the A-pillar blind area image, perform perspective transformation on the A-pillar blind area image, and project the A-pillar blind area image after the perspective transformation onto a corresponding A-pillar inside the vehicle.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the vehicle turning guiding method according to the first aspect of the present application.

[0028] In a fourth aspect, an embodiment of the present application provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the vehicle turning guiding method according to the first aspect of the present application.

[0029] The beneficial effects of the above embodiment are that the vehicle turning guiding method provided by the application comprehensively considers the difference between the images observed by the driver's two eyes when the vehicle is driving, determines the head posture of the driver by recognizing the centers of the driver's two eyes, determines the coordinate range of the A-pillar blind area image, solves the parallax problem of left and right eye observation, and after perspective transformation of the A-pillar blind area image, the projection is performed again, the depth feeling is stronger, the inconsistency feeling of the driver caused by the fixed mode projection with the real field of view is eliminated, the vehicle A-pillar blind area caused by the vehicle turning driving problem is efficiently and accurately reduced, and the driver is provided with the A-pillar blind area situation in the turning process, and timely effective decision is made. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0031] Figure 1 An embodiment flow diagram of the vehicle turning guiding method provided by the application;

[0032] Figure 2 An embodiment flow diagram of the method for determining the centers of the driver's two eyes provided by the application;

[0033] Figure 3 An embodiment structure diagram of the vehicle turning guiding device provided by the application;

[0034] Figure 4 An embodiment structure diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0036] Some block diagrams shown in the drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.

[0037] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described herein can be combinable with other embodiments.

[0038] The prior art research on the problem of the A-pillar blind area of a vehicle usually only carries out monocular eye recognition, image display and a single warning mode, without considering the comprehensive factors such as the difference between the images observed by the driver's two eyes when the vehicle is running and the distance conversion of the external obstacles. In addition, the driver cannot be warned when turning.

[0039] Therefore, the embodiment of the application provides a vehicle turning guiding method and device, which determines the coordinate range of the A-pillar blind area image by identifying the centers of the driver's two eyes, solves the parallax problem of the left and right eyes, and projects the A-pillar blind area image after perspective transformation, so that the depth sense is stronger, and the inconsistency with the real field of view caused by the fixed projection to the driver is eliminated. The following will be described and introduced through multiple embodiments.

[0040] Figure 1 An embodiment flowchart of the vehicle turning guiding method provided by the application is shown in FIG. 1, which includes the following steps. Figure 1 As shown in FIG. 1, the vehicle turning guiding method includes:

[0041] Step S1, acquiring a head image of a driver at a current time and an environment image outside an A-pillar at the current time;

[0042] Step S2, determining the centers of the driver's two eyes according to the head image, and determining the coordinate range of the A-pillar blind area image under the current view angle according to the centers of the two eyes;

[0043] Step S3, cutting the corresponding A-pillar blind area image from the environment image outside the A-pillar according to the coordinate range of the A-pillar blind area image, performing perspective transformation on the A-pillar blind area image, and projecting the A-pillar blind area image after the perspective transformation to the corresponding A-pillar inside the vehicle.

[0044] It can be understood that the head posture in the embodiment mainly refers to the horizontal angle of the head of the driver, which determines the visual angle range of the driver, the eye centers include the left eye center and the right eye center, and the head posture of the driver is determined through the three coordinates of the left and right eyes to further determine the current visual angle range of the driver. The inclined columns on both sides of the front windshield of the car are called A pillars. The A pillars in the embodiment include a left A pillar and a right A pillar, and the A pillar blind area refers to the visual blind area during driving of the vehicle. Whenever the driver's field of view is partially blocked by the A pillars when the vehicle is turning or entering a curve, a blind area in the field of view is caused.

[0045] The perspective transformation refers to a transformation that, by using the collinearity condition of a perspective center, an image point and a target point, rotates a shadow surface (a perspective surface) around a trace line (a perspective axis) by a certain angle according to the perspective rotation law, destroys the original projection light beam, and still maintains the unchanged projection geometric figure on the shadow surface. In the embodiment, the perspective center refers to the A pillar of the vehicle, the image point refers to the human eye, and the target point refers to the A pillar blind area image. By tracking the line of sight direction of the driver, the image of the scene displayed by the display is dynamically adjusted by using the perspective transformation algorithm, and the transformed scene is further adjusted and cropped, so as to realize the effect of making the A pillar transparent as much as possible, and to provide visual guidance for the vehicle turning.

[0046] Compared with the prior art, the method in the embodiment comprehensively considers the difference between the images observed by the left and right eyes of the driver during driving of the vehicle, determines the coordinate range of the A pillar blind area image by identifying the head posture and the eye centers of the driver, solves the parallax problem of the left and right eye observation, and further projects the A pillar blind area image after perspective transformation, so that the depth feeling is stronger, the inconsistency feeling of the driver caused by the fixed projection mode with the real visual field is eliminated, the problem of the A pillar blind area of the vehicle caused by the turning driving is efficiently and accurately reduced, and the driver is provided with the situation of the A pillar blind area during turning, so as to make effective decisions in time.

[0047] As a preferred implementation manner, the head image includes a left head image and a right head image; the left head image is a head image photographed from the left side of the driver, and the right head image is a head image photographed from the right side of the driver; and the left head image and the right head image each include at least the left eye and the right eye of the driver.

[0048] In the embodiment, the head images of the driver are acquired from the left side and the right side of the driver respectively by using the binocular vision positioning method, so as to eliminate the difference between the images observed by the left and right eyes of the driver.

[0049] After the head images are acquired, the images need to be preprocessed, so that the images become smooth, sharpened and boundary enhanced for comparison by using the gray scale change method and the filtering method.

[0050] On the basis of the above-mentioned embodiments, as a preferred embodiment, as shown in Figure 2 determining the centers of the driver's two eyes according to the head image comprises:

[0051] In step S211, the face region in the left head image is extracted to obtain a left face image, the left face image is subjected to eye edge detection according to a Sobel edge detection operator, eye edge information is extracted, and the two-dimensional coordinates of the center of the left eye are determined according to the eye edge information.

[0052] In this step, the Haar feature is first used to describe the face features. In the face, the eye region is darker than the cheek region, the lip region is also darker than the surrounding region, but the nose region is brighter than the cheek regions on both sides, and the regions can be distinguished by the different pixels.

[0053] Further, the integral image is established, and the integral image is used to quickly obtain several different rectangular features. For any point in the integral image, the integral image value of the point is equal to the sum of all pixels located at the top left corner of the point.

[0054]

[0055] In the above formula, (x, y) represents the position of any point, f(x', y') represents the pixel point at the relevant position in the input image, s(x, y) represents the integral image value of point (x, y), and (x', y') represents the point at the top left corner of point (x, y).

[0056] The Adaboost algorithm is used for training, and the function is to select the feature that is the largest for detecting the face and the non-face from a plurality of features. The Adaboost is a strong classifier that combines a plurality of weak classifiers.

[0057]

[0058]

[0059] In the above formula, h(k) represents the strong classifier, M represents the number of weak classifiers, j represents the jth weak classifier, h j (k) is a weak classifier, is the coefficient of the jth weak classifier. s j ∈{-1,1};θ j is the coefficient threshold value of the jth weak classifier, and f j is a set value.

[0060] Further, the hierarchical classifier is established, and the role of the cascade classifier is to reduce the calculation as much as possible, and the obvious non-face area is removed by using a small amount of features, and then the remaining area is trained. The trained model can realize the recognition of the face area.

[0061] The eye positioning algorithm based on Sobel operator is used to obtain the accurate two-dimensional pixel coordinates of the center points of the left and right eyes of the driver. Let c represent the center, d i represent the unit vector of the gradient composed of the two components of the Sobel operator at x i , d i represents the unit vector of the direction from point p j to point x i ; s i =1, N represents the number of points.

[0062]

[0063] ||s i ||=1

[0064] Step S212, performing horizontal gray integral projection on the right head image to obtain a right face image, and performing vertical gray integral projection on the right face image to obtain the two-dimensional coordinates of the right eye center.

[0065] For the right head image, since the calculation amount of the algorithm in the above step S211 is large, the embodiment uses an eye positioning algorithm based on integral projection with high precision and low calculation cost. First, the face area is segmented according to the skin color, and then the eye coordinates are located by using the integral projection function. The eye positioning algorithm based on integral projection is as follows:

[0066]

[0067]

[0068] In the above formula, I(x,y) represents the pixel gray value at point (x,y), s h (y) represents the horizontal gray integral projection function in the interval [x1,x2], s y (x) represents the vertical gray integral projection function in the interval [y1,y2]. And the average integral projection function is:

[0069]

[0070]

[0071] Step S213, determining the parallax of the left head image and the right head image, determining the three-dimensional coordinates of the left eye center and the three-dimensional coordinates of the right eye center according to the parallax, the two-dimensional coordinates of the left eye center and the two-dimensional coordinates of the right eye center.

[0072] In the embodiment, the binocular center coordinates of the two images are obtained by averaging, the parallax of the left and right images is obtained by calculation, and the three-dimensional coordinates of the binocular center of the driver are obtained by the camera imaging principle.

[0073] Based on the above embodiment, as a preferred embodiment, the coordinate range of the A-pillar blind area image under the current viewing angle determined according to the binocular center includes:

[0074] The current viewing angle of the driver is determined according to the binocular center, and the coordinate range of the A-pillar blind area image under the current viewing angle is determined according to the corresponding relationship between the viewing angle of the driver and the coordinate range of the A-pillar blind area image stored in advance.

[0075] The image blocked by the A-pillar is determined by the condition that the perspective center, the image point and the target point are collinear, and the essence of perspective transformation is to project the image to a new view plane, and the general transformation formula is:

[0076]

[0077] (x,y) is the original image coordinate, (x'=X / Z, y'=Y / Z) is the image pixel coordinate after transformation.

[0078] X=m11*x+m12*y+m13

[0079] Y=m21*x+m22*y+m23

[0080] Z=m31*x+m32*y+m33

[0081]

[0082]

[0083] In the above formula, (x', y') is the transformed coordinate; m11, m12, m21, m22, m31, m32 are rotation quantities, and m13, m23, m33 are translation quantities.

[0084] In this embodiment, the image of the A-pillar outside being occluded is determined by the condition that the three points of the perspective center, the image point and the target point are collinear. The image of the scene displayed by the display is dynamically adjusted by using the perspective change algorithm, and the transformed scene is further adjusted and cropped to realize the effect of "transparency" of the A-pillar as much as possible, which can provide a visual guide for the vehicle turning.

[0085] On the basis of the above-mentioned embodiments, as a preferred embodiment, the obtaining of the head image of the driver at the current time specifically includes:

[0086] The rotation angle of the steering wheel is obtained in real time, and if it is judged that the vehicle is turning according to the rotation angle, the head image of the driver at the current time is obtained from both sides of the driver.

[0087] The rotation angle of the steering wheel is obtained by the rotation angle sensor of the steering wheel to judge whether the driver is about to perform a turning operation, and if it is judged that the vehicle is about to turn or is turning, the driver's visual angle is determined, the A-pillar blind area image is collected and matched, and the A-pillar blind area image is displayed to provide a visual reference for the driver.

[0088] On the basis of the above-mentioned embodiments, as a preferred embodiment, after the coordinate range of the A-pillar blind area image under the current visual angle is determined according to the centers of the two eyes, the method further includes:

[0089] Obtain the obstacle target in the coordinate range of the A-pillar blind area image, compare the trajectory of the obstacle target with the current trajectory of the vehicle according to the pre-established vehicle-pedestrian trajectory prediction model, and if it is judged that the obstacle target and the vehicle will collide according to the comparison result of the trajectory of the obstacle target and the current trajectory of the vehicle, an alarm voice is issued to remind.

[0090] Obtain the blind area obstacle information by emitting signals of different wave bands, and judge whether a collision will occur by comparing the trajectory of the obstacle with the current trajectory of the vehicle, and remind the driver to slow down, accelerate and other operations by issuing an alarm sound.

[0091] On the basis of the above-mentioned embodiments, as a preferred embodiment, the comparison of the trajectory of the obstacle target with the current trajectory of the vehicle according to the pre-established vehicle-pedestrian trajectory prediction model includes:

[0092] According to the first trajectory data of the obstacle target in the first time period, the second trajectory data of the obstacle target in the second time period is predicted;

[0093] If it is judged that the deceleration of the vehicle is greater than the preset deceleration threshold in the second time period, or the moving speed of the obstacle target rises according to the second trajectory data in the second time period, it is judged that the obstacle target and the vehicle will collide.

[0094] The vehicle-pedestrian trajectory prediction model in the embodiment is a vehicle-pedestrian trajectory prediction model based on a hidden Markov model, and combines the Viterbi algorithm with a state transition probability matrix and a divergence probability matrix. The trajectory data in the next 0.48 s is predicted by using the target trajectory data in the previous 1.44 s, so as to determine whether a traffic conflict will occur between the vehicle and the pedestrian at a future time. When the vehicle actively gives way, when the braking deceleration of the vehicle in the next 0.48 s is greater than 3.0 m / s 2 or the crossing speed of the pedestrian in the next 0.48 s is significantly increased, it is considered that a vehicle-pedestrian conflict exists at present.

[0095] In order to better implement the vehicle turning guidance method in the embodiment of the present application, on the basis of the vehicle turning guidance method, the present application also provides a vehicle turning guidance device corresponding thereto, as shown in Figure 3 The vehicle turning guidance device 300 comprises:

[0096] An image acquisition module 310 acquires the head image of the driver at the current time and the environmental image outside the A-pillar at the current time. Specifically, the image acquisition module 310 comprises a first camera 311 arranged outside the A-pillar and used for acquiring the obstacle, a second camera 312 arranged above the inside of the A-pillar and used for the head image of the driver, and an image transmission line connected between the two cameras.

[0097] An image processing module 320 determines the centers of the eyes of the driver according to the head image, and determines the coordinate range of the A-pillar blind area image under the current visual angle according to the centers of the eyes.

[0098] The image processing module 320 comprises an image preprocessing unit 321, a human eye positioning unit 322, and an image matching unit 323. The image preprocessing unit 321 is configured to preprocess the head image, and make the image smooth, sharpened, and boundary enhanced by a gray scale variation method and a filtering method to facilitate contrast. The human eye positioning unit 322 is configured to extract a human face region in the left head image to obtain a left human face image, perform human eye edge detection on the left human face image according to a Sobel edge detection operator, extract eye edge information, determine two-dimensional coordinates of a left eye center according to the eye edge information, perform horizontal gray scale integral projection on the right head image to obtain a right human face image, perform vertical gray scale integral projection on the right human face image to obtain two-dimensional coordinates of a right eye center, determine a disparity between the left head image and the right head image, and determine three-dimensional coordinates of the left eye center and the right eye center according to the disparity, the two-dimensional coordinates of the left eye center, and the two-dimensional coordinates of the right eye center. The image matching unit 323 is configured to determine a current viewing angle of the driver according to the two eye centers, and determine a coordinate range of an A-pillar blind area image under the current viewing angle according to a correspondence between the viewing angle of the driver and the coordinate range of the A-pillar blind area image stored in advance.

[0099] The image projection module 330 is configured to cut a corresponding A-pillar blind area image from an environment image outside the A pillar according to the coordinate range of the A-pillar blind area image, perform perspective transformation on the A-pillar blind area image, and project the perspective-transformed A-pillar blind area image onto a corresponding A pillar inside the vehicle.

[0100] The image projection module 330 is configured to cut a corresponding A-pillar blind area image from an environment image outside the A pillar according to the coordinate range of the A-pillar blind area image, perform perspective transformation on the A-pillar blind area image, and project the perspective-transformed A-pillar blind area image onto a corresponding A pillar inside the vehicle.

[0101] The image projection module 330 comprises a corner judgment starting unit 331, a projection unit 332, and an A-pillar curtain unit 333. The corner judgment starting unit 331 obtains a steering wheel rotation angle by using a steering wheel rotation sensor, and judges whether the driver is going to perform a turning operation. The projection unit 332 projects and displays the blind area image determined by the image matching unit 323 on the A-pillar curtain unit 333.

[0102] The vehicle turning guidance device 300 provided by the above embodiment can implement the technical solutions described in the vehicle turning guidance method embodiments. The principles of implementation of the above modules or units can be found in the corresponding content in the vehicle turning guidance method embodiments, which will not be described herein again.

[0103] As Figure 4As shown, the present application also correspondingly provides an electronic device 400. The electronic device 400 comprises a processor 401, a memory 402 and a display 403. Figure 4 Only part of the components of the electronic device 400 are shown, but it should be understood that all the shown components are not required to be implemented, and more or less components can be alternatively implemented.

[0104] The memory 402 can be an internal storage unit of the electronic device 400 in some embodiments, such as a hard disk or a memory of the electronic device 400. The memory 402 can also be an external storage device of the electronic device 400 in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 400.

[0105] Further, the memory 402 can include both the internal storage unit and the external storage device of the electronic device 400. The memory 402 is used to store application software and various data installed on the electronic device 400.

[0106] The processor 401 can be a central processing unit (CPU), a microprocessor or other data processing chip in some embodiments, used to run program codes or process data stored in the memory 402, such as the vehicle turning guidance method in the present application.

[0107] The display 403 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 403 is used to display information of the electronic device 400 and to display visualized user interfaces. The components 401-403 of the electronic device 400 communicate with each other through a system bus.

[0108] In some embodiments of the present application, when the processor 401 executes the vehicle turning guidance program in the memory 402, the following steps can be implemented:

[0109] Obtaining a head image of the driver at the current time and an environment image outside the A-pillar at the current time;

[0110] Determining a center of the driver's eyes according to the head image, and determining a coordinate range of the A-pillar blind area image under the current visual angle according to the center of the eyes;

[0111] According to the coordinate range of the A-pillar blind area image, a corresponding A-pillar blind area image is intercepted from the environment image outside the A-pillar, perspective transformation is performed on the A-pillar blind area image, and the perspective-transformed A-pillar blind area image is projected onto the corresponding A-pillar inside the vehicle.

[0112] It should be understood that, in addition to the above functions, the processor 401 can also implement other functions when executing the vehicle turning guidance program in the memory 402, and specific descriptions can be made with reference to the descriptions of the corresponding method embodiments.

[0113] Further, the type of the electronic device 400 is not specifically limited, and the electronic device 400 can be a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop, or the like. Exemplary embodiments of the electronic device include, but are not limited to, electronic devices running IOS, android, microsoft, or other operating systems. The electronic device can also be other electronic devices, such as a laptop having a touch-sensitive surface (e.g., a touch panel). It should also be understood that, in some other embodiments of the present application, the electronic device 400 can also be a desktop computer having a touch-sensitive surface (e.g., a touch panel).

[0114] Correspondingly, the present application also provides a computer-readable storage medium for storing computer-readable programs or instructions, which, when executed by a processor, can implement the vehicle turning guidance method steps or functions provided by the above method embodiments.

[0115] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiments can be completed by a computer program instructing related hardware (such as a processor, a controller, etc.) to complete. The computer program can be stored in a computer-readable storage medium. The computer-readable storage medium includes a magnetic disk, an optical disk, a read-only memory, or a random access memory, etc.

[0116] The vehicle turning guidance method and device provided by the present application are described in detail above, and specific examples are applied to describe the principles and implementation modes of the present application. The above descriptions of the embodiments are only used to help understand the method and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation modes and application ranges can be changed, and the above descriptions should not be understood as limiting the present application.

Claims

1. A vehicle turn guidance method characterized by, The method comprises the following steps: acquiring a head image of the driver at the current time, and an environment image outside the A-pillar at the current time, the head image comprising a left head image and a right head image; the left head image is a head image taken from the left side of the driver, and the right head image is a head image taken from the right side of the driver; at least the left eye and the right eye of the driver are included in the left head image and the right head image; determining the center of the eyes of the driver according to the head image, and determining the coordinate range of the A-pillar blind area image under the current visual angle according to the center of the eyes; cutting the corresponding A-pillar blind area image from the environment image outside the A-pillar according to the coordinate range of the A-pillar blind area image, performing perspective transformation on the A-pillar blind area image, and projecting the perspective-transformed A-pillar blind area image onto the corresponding A-pillar inside the vehicle; the step of determining the center of the eyes of the driver according to the head image comprises: extracting a face region in the left head image to obtain a left face image, performing eye edge detection on the left face image according to a Sobel edge detection operator, extracting eye edge information, and determining the first two-dimensional coordinates of the center of the eyes according to the eye edge information; performing horizontal gray integral projection on the right head image to obtain a right face image, performing vertical gray integral projection on the right face image to obtain the second two-dimensional coordinates of the center of the eyes; determining the parallax of the left head image and the right head image, and determining the three-dimensional coordinates of the center of the eyes according to the parallax, the first two-dimensional coordinates and the second two-dimensional coordinates.

2. The vehicle turn guidance method according to claim 1, characterized by, the step of determining the coordinate range of the A-pillar blind area image under the current visual angle according to the center of the eyes comprises: determining the current visual angle of the driver according to the center of the eyes, and determining the coordinate range of the A-pillar blind area image under the current visual angle according to a corresponding relationship between the visual angle of the driver and the coordinate range of the A-pillar blind area image which is stored in advance.

3. The vehicle turn guidance method according to claim 1, characterized by, the step of acquiring the head image of the driver at the current time comprises: acquiring the rotation angle of the steering wheel in real time, and acquiring the head image of the driver at the current time from both sides of the driver if it is determined that the vehicle is turning according to the rotation angle.

4. The vehicle turn guidance method according to claim 1, characterized by, after the step of determining the coordinate range of the A-pillar blind area image under the current visual angle according to the center of the eyes, the method further comprises: acquiring an obstacle target in the coordinate range of the A-pillar blind area image, comparing the trajectory of the obstacle target with the current trajectory of the vehicle according to a vehicle-pedestrian trajectory prediction model established in advance, and issuing an alarm voice to remind if it is determined that the obstacle target and the vehicle will collide according to the comparison result of the trajectory of the obstacle target and the current trajectory of the vehicle.

5. The vehicle turn guidance method according to claim 4, characterized by, the step of comparing the trajectory of the obstacle target with the current trajectory of the vehicle according to the vehicle-pedestrian trajectory prediction model established in advance comprises: predicting the second trajectory data of the obstacle target in a second time period according to the first trajectory data of the obstacle target in a first time period; determining that the obstacle target and the vehicle will collide if it is determined that the braking deceleration of the vehicle is greater than a preset braking deceleration threshold in the second time period, or the moving speed of the obstacle target rises in the second time period according to the second trajectory data.

6. A vehicle turning guidance device characterized by comprising: ​ The image acquisition module acquires the driver's head image at the current moment, as well as the environmental image outside the A-pillar at the current moment. The head image includes a left head image and a right head image; the left head image is a head image taken from the driver's left side, and the right head image is a head image taken from the driver's right side. Both the left-side head image and the right-side head image include at least the driver's left and right eyes; The image processing module determines the center of the driver's eyes based on the head image, and determines the coordinate range of the A-pillar blind spot image from the current viewpoint based on the center of the eyes. The image projection module extracts the corresponding A-pillar blind spot image from the environmental image outside the A-pillar according to the coordinate range of the A-pillar blind spot image, performs perspective transformation on the A-pillar blind spot image, and projects the perspective-transformed A-pillar blind spot image onto the corresponding A-pillar inside the vehicle. Determining the center of the driver's eyes based on the head image includes: Extract the face region from the left head image to obtain the left face image. Perform eye edge detection on the left face image using the Sobel edge detection operator to extract eye edge information. Determine the first two-dimensional coordinates of the center of both eyes based on the eye edge information. A horizontal grayscale integral projection is performed on the right head image to obtain a right face image. A vertical grayscale integral projection is performed on the right face image to obtain the second two-dimensional coordinates of the center of the eyes. Determine the disparity between the left head image and the right head image, and determine the three-dimensional coordinates of the eye center based on the disparity, the first two-dimensional coordinates, and the second two-dimensional coordinates.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the vehicle turning guidance method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the vehicle turning guidance method as described in any one of claims 1 to 5.

Images