Driving safety display method, device, equipment, medium and program product

Abstract

The invention relates to a driving safety display method and device, display equipment, a computer readable storage medium and a computer program product. The method comprises the steps that according to a driving mode of a vehicle, a driving display area of display equipment of the vehicle is determined, and the driving display area is a display area located on a column A or all display areas formed by the display equipment; acquiring an out-of-vehicle image acquired by an out-of-vehicle camera sensor corresponding to the driving display area; controlling a front-facing camera to collect human eye tracking information of the driver, and determining visual field blind area information of the driver according to the human eye tracking information; and based on the view blind area information, cutting the outside image to obtain a current blind area image, and controlling a display device to display the current blind area image in a driving display area. By adopting the method, the driving safety can be improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a driving safety display method, device, display equipment, computer-readable storage medium and computer program product. Background Art

[0002] The A-pillar of a vehicle is the first pillar connecting the front windshield and the roof of the vehicle, usually located on both sides of the driver and the front passenger, just above the left and right rearview mirrors. It is also an important part to ensure vehicle safety, but the existence of the A-pillar of the vehicle blocks the driver's line of sight, bringing great safety hazards. The blind spot of the A-pillar is an important factor leading to traffic accidents.

[0003] In the related art, a camera is installed outside the vehicle body and the image of the blind spot outside the A-pillar is displayed on the central control screen. This method will cause the driver to divert his attention to the central control screen, which is not conducive to safe driving. Summary of the invention

[0004] Based on this, it is necessary to provide a driving safety display method, device, display equipment, computer-readable storage medium and computer program product that can improve driving safety in response to the above technical problems.

[0005] In a first aspect, the present application provides a driving safety display method, wherein a vehicle comprises an external camera sensor disposed on the vehicle body, a display device, and a front camera disposed inside the vehicle; the display device is disposed at a vehicle pillar position in the cabin; the method comprises

[0006] Determining a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle;

[0007] Acquire an image outside the vehicle captured by an outside camera sensor corresponding to the driving display area;

[0008] Controlling the front camera to collect eye tracking information of the driver, and determining the blind spot information of the driver's field of vision according to the eye tracking information;

[0009] Based on the blind spot information, the image outside the vehicle is segmented to obtain a current blind spot image;

[0010] The display device is controlled to display the current blind spot image in the driving display area.

[0011] In one embodiment, determining the driving display area of ​​the display device of the vehicle according to the driving mode of the vehicle includes:

[0012] The display device is formed with a display area located on the A-pillar, a display area located on the B-pillar, and a display area located on the C-pillar;

[0013] According to the driving mode of the vehicle, a driving display area of ​​the vehicle is determined, and the driving display area is a display area located on the A-pillar or all display areas formed by the display device.

[0014] In one embodiment, determining the driving display area of ​​the vehicle according to the driving mode of the vehicle includes:

[0015] When the driving mode is a starting mode or a parking mode, determining the driving display area to be all display areas formed by the display device;

[0016] Preferably, when the driving mode is a forward mode or a reverse mode, obtaining a steering wheel angle;

[0017] When the steering wheel angle is greater than or equal to a preset angle value, determining that the driving display area is all display areas formed by the display device;

[0018] When the steering wheel angle is less than or equal to a preset angle value, the driving display area is determined to be a display area on the A-pillar.

[0019] In one embodiment, controlling the front camera to collect eye tracking information of the driver, and determining the blind spot information of the driver according to the eye tracking information includes:

[0020] Controlling the front camera to collect eye tracking information of the driver, and collecting distance information between the driver's eyes and the driving display area;

[0021] According to the eye tracking information, obtain the corresponding eye movement trajectory;

[0022] Corresponding visual field blind area information is determined according to the motion trajectory and the distance information.

[0023] In one embodiment, the blind spot information includes a boundary of the blind spot; and the segmentation process of the vehicle exterior image based on the blind spot information to obtain the current blind spot image includes:

[0024] Based on the preset visual field expansion information, the boundary of the visual field blind area is expanded to obtain updated visual field blind area information;

[0025] Based on the updated blind spot information, the image outside the vehicle is cut to obtain a current blind spot image.

[0026] In one embodiment, the display screen of the display device is a flexible display screen; and controlling the display device to display the current blind spot image in the driving display area includes:

[0027] According to the distortion correction model and the bending correction model of the display screen corresponding to the driving display area, the current blind spot image is subjected to distortion correction processing and bending correction processing to obtain the blind spot image information after the correction processing, wherein the distortion correction model is generated by the shape of the display screen, and the bending correction model is generated by the bending information of the display screen;

[0028] Preferably, obtaining pixel arrangement data of a display screen corresponding to the driving display area;

[0029] Based on the pixel arrangement data, a mapping relationship between the corrected blind spot image information and pixels in the corresponding display screen of the driving display area is established;

[0030] Based on the mapping relationship and the blind spot image information, generating display information of the driving display area;

[0031] According to the display information, the display device is controlled to display the current blind spot image in the driving display area.

[0032] In one embodiment, the method further comprises:

[0033] Controlling the external camera sensor to identify obstacles in a preset area;

[0034] According to the information of the obstacle, life features of the obstacle in the preset area are identified to obtain the obstacle type;

[0035] Obtaining a relative speed of an obstacle in a preset area, wherein the relative speed is determined based on a distance between the obstacle and the vehicle in a previous time frame and a current time frame;

[0036] When the relative speed is greater than or equal to a relative speed threshold corresponding to the obstacle type, the display device is controlled to display an alarm reminder box on the boundary of the driving display area.

[0037] In a second aspect, the present application also provides a driving safety display device,

[0038] The device comprises: an external camera sensor placed on the vehicle body, a display device and a front camera located inside the vehicle; the display device is located at a vehicle pillar position in the cockpit;

[0039] A determination module, used to determine a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle;

[0040] An acquisition module, used for acquiring an image outside the vehicle collected by an outside camera sensor corresponding to the driving display area;

[0041] An acquisition module, used for controlling the front camera to collect the driver's eye tracking information, and determining the driver's visual field blind spot information according to the eye tracking information;

[0042] The display module is used to perform segmentation processing on the vehicle exterior image based on the visual field blind spot information to obtain a current blind spot image, and control the display device to display the current blind spot image in the driving display area.

[0043] In a third aspect, the present application further provides a display device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0044] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0045] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0046] The above-mentioned driving safety display method, device, display device, computer-readable storage medium and computer program product determine the driving display area of ​​the vehicle according to the driving mode of the vehicle, and the driving display area is the display area located on the A-pillar or all display areas formed by the display device; obtain the external image of the vehicle collected by the external camera sensor corresponding to the driving display area; based on the front camera, the driver's eyes are tracked to obtain the corresponding blind spot information; based on the blind spot information, the external image of the vehicle is cut and processed to obtain the current blind spot image, and the current blind spot image is displayed in the driving display area. By adopting this method, by real-time displaying the blind spot image matching the driving mode, the driver can have a more comprehensive understanding of the vehicle's surrounding environment, so as to make more accurate driving decisions and improve driving safety. By using the collaborative work of multiple display areas, the driver can obtain the required vehicle surrounding information without frequently adjusting the line of sight or turning the head, which improves the convenience of driving. In addition, through the human eye tracking function of the front camera, the driver's blind spot can be intelligently identified, and the displayed image content can be adjusted accordingly, realizing a personalized driving assistance method, which is more in line with the actual needs of the driver and realizes the intelligence of driving assistance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of a flow chart of a driving safety display method in one embodiment;

[0049] Figure 2 A schematic diagram of a trip safety display process in a starting mode in an embodiment;

[0050] Figure 3 A schematic diagram of a safe display process of a parking mode in an embodiment;

[0051] Figure 4 It is a schematic diagram of a process flow of displaying travel safety in a driving mode in an embodiment;

[0052] Figure 5 A schematic diagram of a travel safety display process in a reverse mode in one embodiment;

[0053] Figure 6 A schematic diagram of determining a blind area of ​​vision in one embodiment;

[0054] Figure 7 It is a schematic diagram of a process of displaying a current blind area image in one embodiment;

[0055] Figure 8 A structural block diagram of displaying an image on a driving display area in one embodiment;

[0056] Fig. 9 A schematic diagram of a flow chart of a driving safety display method in another embodiment

[0057] Fig.10 is a structural block diagram of a driving safety display device in one embodiment;

[0058] Fig.11 The figure is a diagram showing the internal structure of a device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] With the development of science and technology, smart travel and smart cockpit technologies are developing rapidly. The intelligent cockpit aims to integrate a variety of IT and artificial intelligence technologies to create a new integrated digital platform in the car, provide drivers with an intelligent experience, and promote driving safety.

[0061] The A-pillar of a vehicle is the first pillar connecting the front windshield and the roof of the vehicle, usually located on both sides of the driver and the front passenger, just above the left and right rearview mirrors. It is also an important part to ensure vehicle safety, but the existence of the A-pillar of the vehicle blocks the driver's line of sight, bringing great safety hazards. The blind spot of the A-pillar is an important factor leading to traffic accidents.

[0062] In the related art, a camera is installed outside the vehicle body and the image of the blind spot outside the A-pillar is displayed on the central control screen. This method will cause the driver to divert his attention to the central control screen, which is not conducive to safe driving.

[0063] In an exemplary embodiment, Figure 1 As shown, a driving safety display method is provided, which is applied to a vehicle, wherein the vehicle includes a plurality of external camera sensors placed on the vehicle body, a display device, and a front camera located inside the vehicle; the display device is located at the vehicle pillar position in the cockpit, and the pillar position includes the A pillar, the B pillar, and the C pillar; the display device is formed with a display area located on the A pillar, a display area located on the B pillar, and a display area located on the C pillar, and includes the following steps 102 to 108. Among them:

[0064] Step 102: Determine a driving display area of ​​a display device of the vehicle according to the driving mode of the vehicle.

[0065] It should be noted that this embodiment is applied to a vehicle, and the vehicle is equipped with multiple external camera sensors, a front camera located inside the vehicle, and a display device including display areas on the A-pillar, B-pillar, and C-pillar. When the driver starts the vehicle and starts driving, the driving display area is determined based on the driving mode of the vehicle. The driving display area is the display area located on the A-pillar, or all display areas formed by the display device (including the display areas on the A-pillar, B-pillar, and C-pillar).

[0066] Among them, the driving mode may include a starting mode, a parking mode, a forward mode and a reverse mode. Specifically, the driving mode of the current vehicle can be identified based on the current state of the vehicle (for example, vehicle speed, gear position, turn signal, etc.). Among them, the forward mode usually occurs when the vehicle starts to accelerate from a stationary state and is ready to enter the driving state, that is, the vehicle gear enters the D gear. The parking mode occurs when the vehicle approaches a parking space or needs to be parked. The forward mode refers to the state of the vehicle in normal driving, that is, the vehicle's gear is the normal D gear. The reverse mode usually occurs when the vehicle needs to back up, such as reversing into a warehouse, reversing to avoid, etc., that is, the vehicle enters the R gear.

[0067] The driving display area can be used to display the all-round view around the vehicle, effectively filling the driver's natural blind spots, helping the driver to better understand the road conditions and obstacles around the vehicle and improve driving safety.

[0068] It should be noted that when the vehicle is started, it will perform a self-check to ensure that the vehicle's external camera sensors, display devices and front camera are in normal working condition.

[0069] Step 104 , obtaining an image outside the vehicle captured by an outside camera sensor corresponding to the driving display area.

[0070] The driving display area is equipped with a corresponding external camera sensor. The external camera sensor may be a wide-angle camera sensor. In practice, after determining the driving display area, the external camera sensor may control the external camera sensor corresponding to the driving display area to start working, so as to collect real-time images around the vehicle. Alternatively, all external camera sensors may be started when the vehicle is started.

[0071] The exterior camera sensors can be distributed around the vehicle body according to their positions, such as the front, rear, and sides of the vehicle, to ensure that a full range of vision can be obtained. In the vehicle, the A-pillar may include a left A-pillar and a right A-pillar. The exterior camera sensor corresponding to the display area on the left A-pillar may be located on the vehicle outer side of the left A-pillar, and the exterior camera sensor corresponding to the display area on the right A-pillar may be located on the vehicle outer side of the right A-pillar.

[0072] Step 106, controlling the front camera to collect the driver's eye tracking information, and determining the driver's blind spot information according to the eye tracking information.

[0073] The front camera may be a binocular camera. In a vehicle, there may be multiple front cameras, one mounted on the inner side of the left A-pillar and the other mounted on the inner side of the right A-pillar; in other embodiments, the front camera may also be located directly in front of the driver.

[0074] Specifically, the driver's image is obtained through the front camera, the driver's eyes are tracked, and the driver's eye movements are tracked in real time. By analyzing the driver's gaze direction and eye movement trajectory, the driver's current blind spot can be determined.

[0075] Step 108 , based on the blind spot information, the image outside the vehicle is cut and processed to obtain the current blind spot image, and the display device is controlled to display the current blind spot image in the driving display area.

[0076] Specifically, based on the blind spot information, the collected images outside the vehicle are cut and processed to extract the image portion corresponding to the blind spot. The cut blind spot image is transmitted in real time to the previously determined driving display area for display. In this way, the driver can view the real-time situation of the blind spot through the display device without adjusting the line of sight. During driving, the driver's blind spot is monitored in real time through the front camera, and the displayed blind spot image is adjusted in real time as needed. At the same time, the displayed area and the content displayed on the driving display area will be adjusted in time according to the changes in the driving mode.

[0077] In the above-mentioned driving safety display method, according to the driving mode of the vehicle, the driving display area of ​​the vehicle is determined, and the driving display area is the display area located on the A-pillar or all display areas formed by the display device; the external image of the vehicle collected by the external camera sensor corresponding to the driving display area is obtained; based on the front camera, the driver's eyes are tracked to obtain the corresponding blind spot information; based on the blind spot information, the external image of the vehicle is cut and processed to obtain the current blind spot image, and the current blind spot image is displayed in the driving display area. By adopting this method, by real-time displaying the blind spot image matching the driving mode, the driver can have a more comprehensive understanding of the surrounding environment of the vehicle, so as to make more accurate driving decisions and improve driving safety. By using the collaborative work of multiple display areas, the driver can obtain the required information around the vehicle without frequently adjusting the line of sight or turning the head, which improves the convenience of driving. In addition, through the human eye tracking function of the front camera, the driver's blind spot can be intelligently identified, and the displayed image content can be adjusted accordingly, realizing a personalized driving assistance method, which is more in line with the actual needs of the driver and realizes the intelligence of driving assistance.

[0078] In an exemplary embodiment, step 102 includes:

[0079] like Figure 2 and Figure 3 As shown, when the driving mode is the starting mode or the parking mode, the driving display area is determined to be all display areas formed by the display device.

[0080] like Figure 4 and Figure 5 As shown, when the driving mode is the forward mode or the reverse mode, the steering wheel angle is obtained; when the steering wheel angle is greater than or equal to the preset angle value, the driving display area is determined to be all display areas formed by the display device; when the steering wheel angle is less than or equal to the preset angle value, the driving display area is determined to be the display area on the A-pillar.

[0081] Specifically, when the vehicle is in starting mode or parking mode, it is necessary to have a more comprehensive understanding of the conditions around the vehicle, and the driving display area is determined to be all display areas formed by the display device, including the display areas of the A-pillar, B-pillar, and C-pillar. The driver can view real-time images from multiple angles at once to ensure safe starting or parking.

[0082] When the vehicle is in forward mode or reverse mode, the steering wheel angle information, i.e., the steering wheel rotation angle, is obtained through the vehicle's steering wheel angle sensor. When the steering wheel angle is greater than or equal to the preset angle value, it means that the vehicle is making a large turn or adjusting the driving direction. At this time, in order to ensure that the driver can fully observe the road conditions when turning or adjusting the direction, the driving display area is determined as all display areas of the display device.

[0083] If the steering wheel angle does not exceed the preset value, it means that the vehicle is driving in a straight line or making a small direction adjustment. In this case, the driver is mainly concerned about the road conditions ahead, and the driving display area is determined to be the display area on the A-pillar to display the real-time image ahead. Specifically, the preset value of the steering angle is usually determined according to the vehicle model and the user's driving habits.

[0084] In this embodiment, the driving display area is dynamically adjusted according to the driving mode and steering wheel angle, which can ensure that the driver obtains key road condition information, help reduce potential risks caused by blind spots or insufficient information, and thus significantly improve driving safety. In addition, the display content is adjusted according to the actual needs of the driver, making the information display during driving more intelligent and personalized.

[0085] In an exemplary embodiment, step 106 includes:

[0086] Step 1062, controlling the front camera to collect the driver's eye tracking information, and collect the distance information between the driver's eyes and the driving display area.

[0087] Step 1064, obtaining the corresponding eye movement trajectory according to the eye tracking information.

[0088] Step 1066, determining corresponding blind spot information according to the motion trajectory and distance information.

[0089] Specifically, the driver's eyes are tracked using a front camera located in the vehicle. By capturing the movement of the eyeball and the position change of the pupil, the movement trajectory of the eye can be accurately obtained. The movement trajectory of the eye may include the position information of the pupil. It should be noted that after the vehicle is started, the front camera will start and perform self-calibration to ensure that the parameters such as the angle and focal length of the front camera are in the best state so as to accurately capture the driver's eye dynamics. Using image recognition technology, the driver's eyes are detected in real time in the video stream captured by the front camera. Once the eyes are detected, the tracking algorithm is started to continuously track the position changes of the eyes. During the tracking process, by analyzing the position data of the eyes in the video frame, the precise position information of the eyes relative to the camera coordinate system is extracted, thereby forming the movement trajectory of the eyes. The precise position information of the eyes relative to the camera coordinate system can usually include the horizontal and vertical coordinates of the eyes, etc.

[0090] The depth sensor working in conjunction with the front camera can be used to measure the physical distance between the driver's eyes and the driving display area in real time, which can include the distance between the eyes and the display screen located on the A-pillar, the distance between the eyes and the display screen located on the B-pillar, and the distance between the eyes and the display screen located on the C-pillar. It should be noted that the depth sensor working in conjunction with the front camera can be an infrared sensor, an ultrasonic sensor, or a laser radar, etc., which varies depending on the vehicle model and configuration. The depth sensor working in conjunction with the front camera is used to measure the distance between the driver's eyes and the driving display area.

[0091] In order to ensure the accuracy and real-time performance of the distance measurement, the eye position in each frame of video is synchronized and matched with the corresponding distance information.

[0092] Exemplarily, the driver's current blind spot information is determined based on the motion trajectory, distance information, and the size of the display screen corresponding to the driving display area. Figure 6 , shows a schematic diagram of determining a blind spot in an embodiment. Taking the A-pillar as an example, Figure 6 In the figure, X1 and X2 are the sight-shielded areas blocked by the A-pillar in the horizontal direction. Y1 and Y2 are the sight-shielded areas blocked by the A-pillar in the vertical direction. X1, X2, Y1 and Y2 constitute the sight-shielded area on one side of the A-pillar.

[0093] In this embodiment, by tracking the driver's eyes and determining the blind spot of the visual field, a personalized service can be provided to the driver. By displaying the blind spot image on the driving display area, it helps the driver to understand the state of the vehicle's surroundings more comprehensively, thereby improving driving safety.

[0094] In an exemplary embodiment, the blind spot information includes the boundary of the blind spot. Step 106 includes: based on the preset blind spot expansion information, expanding the boundary of the blind spot to obtain updated blind spot information; based on the updated blind spot information, cutting the image outside the vehicle to obtain the current blind spot image.

[0095] The preset field of view expansion information is determined based on factors such as vehicle design, driver habits, and safety standards. The preset field of view expansion information may include parameters such as the angle and distance of the field of view expansion, for example, expanding the business boundary outward by 10 degrees or 20 centimeters. The preset field of view expansion information may be pre-stored in the vehicle's control system or a related auxiliary driving system. The updated field of view blind spot information includes the boundary of the field of view blind spot.

[0096] Exemplarily, after determining the initial blind spot information, the boundary of the blind spot is expanded according to the preset visual field expansion information, and the updated blind spot information is obtained. Specifically, the boundary of the blind spot is expanded according to the preset visual field expansion information, which can be achieved by a preset algorithm, and the boundary of the blind spot is adjusted accordingly according to the preset angle or distance parameter, so as to obtain the updated blind spot information.

[0097] For example, in Figure 6 In the example, if the preset field of view expansion information is 10 degrees, the horizontal blind spot angle can be expanded by 10 degrees; the vertical blind spot angle can be expanded by 10 degrees. Alternatively, the left and right sides of the horizontal blind spot angle can be expanded by 10 degrees, and the upper and lower sides of the vertical blind spot angle can be expanded by 10 degrees. Of course, the preset field of view expansion information can include four angle information to expand different areas respectively. Specifically, the horizontal angle between X1X2 is In the case of, the adjustment of the horizontal direction to expand the field of view by presetting the field of view expansion information can be ; The vertical angle between Y1Y2 is In the case of, the adjustment of the vertical direction to expand the field of view by presetting the field of view expansion information can be .

[0098] In practice, the preset field of view expansion information can also be adjusted according to different driving scenarios. For example, in complex urban road conditions, a larger expansion range can be set to deal with more potential risks; while in relatively simple road conditions such as highways, a smaller expansion range can be set to reduce unnecessary interference.

[0099] The blind spot information may include the boundary of the current blind spot. Specifically, according to the updated blind spot boundary, the image of the corresponding area is cut out from the original image outside the vehicle. This part of the cut-out image is the current blind spot image, which represents the vehicle surroundings that the driver cannot directly observe under the current circumstances. The current blind spot image obtained by the cutting process will be displayed in real time on the vehicle's driving display area. In this way, even if the driver cannot directly observe the situation in the blind spot through his line of sight, he can still understand the relevant information through the display device, so as to make safer driving decisions.

[0100] In this embodiment, by expanding the boundaries of the blind spot and cutting the image outside the vehicle, more blind spot information can be provided to the driver, so that the driver can have a more comprehensive understanding of the environment around the vehicle, reduce potential risks caused by the blind spot, and thus improve driving safety.

[0101] In an exemplary embodiment, the display screen of the display device is a flexible display screen; Figure 7 , is a schematic diagram of a process of displaying a current blind spot image in an embodiment. Controlling a display device to display the current blind spot image in a driving display area includes:

[0102] Step 702, according to the distortion correction model and bending correction model of the display screen corresponding to the driving display area, the current blind spot image is subjected to distortion correction processing and bending correction processing to obtain a corrected blind spot image; the distortion correction model is generated based on the shape of the display screen; the bending correction model is generated based on the bending information of the display screen.

[0103] Step 704, obtaining pixel arrangement data of the display screen corresponding to the driving display area.

[0104] Step 706 , based on the pixel arrangement data, establish a mapping relationship between the corrected blind spot image information and the pixels in the corresponding display screen of the driving display area.

[0105] Step 708, generating display information of the driving display area based on the mapping relationship and the blind spot image information;

[0106] Step 710: Control the display device to display the current blind spot image in the driving display area.

[0107] The display screen of the display device is a flexible display screen. The flexible display screen is highly bendable and can adapt to installation environments of different shapes and curvatures. Taking the A-pillar as an example, the flexible display screen is installed on the inner decorative panel of the left A-pillar and the inner decorative panel of the right A-pillar, and the width is equal to the width of the A-pillar.

[0108] Specifically, during the manufacturing and use of the display screen, a corresponding distortion correction model and a bending correction model are established for the shape and bending information of each display screen. The distortion correction model is generated based on the shape of the display screen. The bending correction model is generated based on the bending information of the display screen.

[0109] Among them, the generation process of the distortion correction model is: obtaining information such as the physical size and pixel layout of the display screen. Through high-precision image sampling equipment, the image on the display screen is captured under various conditions (such as different viewing angles, brightness, contrast, etc.), and the known calibration pattern (such as a checkerboard) is used to obtain the precise geometric information of the display screen, so as to use image processing technology and mathematical algorithms to extract the distortion parameters of the display screen, and establish the corresponding distortion correction model based on the distortion parameters. Specifically, since the structure of the display screens on the A-pillar, B-pillar, and C-pillar on the vehicle is related to the structure of the vehicle itself. Therefore, when determining the distortion correction model, the corresponding distortion correction model can be further improved from the perspective of the driver to maximize the display effect when the driver views the driving display area.

[0110] It should be noted that different types and specifications of display screens require different distortion correction models. Therefore, different distortion correction models are also required for the display screens on the A-pillar, B-pillar, and C-pillar of the vehicle.

[0111] The generation process of the bending correction model is to test the bending characteristics of the flexible display screen to obtain the shape change data of the display screen under different bending states; in different bending states of the display screen, a high-precision image acquisition device is used to capture the image on the display screen. At the same time, a known calibration pattern is used to obtain the geometric information of the display screen under the bending state. Based on the collected images and calibration data, image processing technology and mathematical algorithms are used to extract the bending parameters of the display screen, and a bending correction model is established based on the extracted bending parameters.

[0112] The pixel arrangement data of the display screen is used to describe the position and color information of each pixel on the display screen.

[0113] For example, according to the current shape and bending state of the display screen, the pre-established distortion correction model and bending correction model are used to correct the current blind spot image to eliminate the image distortion and distortion caused by the deformation of the display screen, ensuring that the image can be accurately and clearly displayed on the display screen. Based on the acquired pixel arrangement data, a mapping relationship between the corrected blind spot image and the pixels in the display screen can be established. This mapping relationship ensures that each pixel in the image can accurately correspond to the correct position on the display screen, so that the current blind spot image is displayed in the driving display area according to the mapping relationship.

[0114] In this embodiment, the distortion and bend correction processing can eliminate the image distortion and twisting caused by the deformation of the display screen, thereby significantly improving the image quality, allowing the driver to view the blind spot image more clearly and improving driving safety. A flexible display screen is used, which can adapt to installation environments of different shapes and curvatures, and can ensure accurate display of images through corresponding correction processing regardless of whether it is a flat or curved display.

[0115] In order to better understand the process of displaying images on the driving display area in this solution, an example is used for explanation. Figure 8 , shows a structural block diagram of displaying an image on a driving display area in one embodiment. It includes a signal filtering and noise reduction module 802, a signal preprocessing module 804, a human eye tracking and positioning module 806, a blind spot confirmation module 808 and a video cutting module 810. Among them,

[0116] The signal filtering and denoising module 802 and the signal preprocessing module 804 are used to collect images around the vehicle using the external camera sensor, and perform signal filtering and denoising processing and signal preprocessing on the collected images to obtain images outside the vehicle. Among them, the signal filtering and denoising processing is mainly used to remove the noise components in the image. Noise may come from a variety of factors, such as the noise of the sensor itself, external environmental interference, etc. Through the filtering algorithm, the impact of noise on image quality can be effectively reduced, making the image clearer. Signal preprocessing is a series of operations on the image to improve its quality and processability. The preprocessing steps may include: digital filtering, contrast adjustment, color correction, etc.

[0117] The eye tracking and positioning module 806 is used to control the front camera to collect the driver's eye tracking information and the distance information between the driver's eyes and the driving display area; according to the eye tracking information, the corresponding eye movement trajectory is obtained.

[0118] The blind spot confirmation module 808 is used to determine the corresponding visual field blind spot information according to the motion trajectory and the distance information between the driver's eyes and the driving display area.

[0119] The video cutting module 810 is used to expand the boundary of the blind spot based on the preset field of view expansion information to obtain updated blind spot information; based on the updated blind spot information, the image outside the vehicle is cut to obtain the current blind spot image.

[0120] In an exemplary embodiment, the vehicle includes a plurality of distance sensors disposed on the vehicle body, and the method further includes: obtaining information of obstacles in a preset area identified by an external camera sensor; performing life feature recognition on the obstacles in the preset area based on the information of the obstacles to obtain the obstacle type; obtaining a relative speed of the obstacles in the preset area, the relative speed being determined based on the distance between the obstacle and the vehicle in a previous time frame and a current time frame; and when the relative speed is greater than or equal to a relative speed threshold corresponding to the obstacle type, controlling the display device to display an alarm reminder box on the boundary of the driving display area.

[0121] The vehicle is equipped with multiple distance sensors (such as laser radar, millimeter wave radar, etc.) and external camera sensors placed on the vehicle body. These sensors collect environmental data around the vehicle in real time, including the distance to obstacles, image information of obstacles, etc.

[0122] Specifically, the vehicle exterior camera sensor can be controlled to identify information of obstacles in a preset area; according to the obstacle information, the obstacle in the preset area is identified as a living body, and the obstacle type is obtained. The obstacle type may include a living body type obstacle and a non-living body obstacle type. The preset area may be centered on the vehicle and have a preset length as a radius.

[0123] The relative speed of obstacles in a preset area is obtained through distance sensors. The relative speed is determined based on the change in distance between the obstacle and the vehicle in the previous time frame and the current time frame. By continuously monitoring these distance changes, the speed and direction of the obstacle can be calculated in real time.

[0124] After calculating the relative speed of the obstacle, it will be compared with the relative speed threshold of the corresponding obstacle type. The relative speed threshold of the obstacle type is set according to different types of obstacles and the safety standards of vehicle driving. If the relative speed is greater than or equal to the corresponding threshold, the display device will be controlled to display an alarm reminder box on the boundary of the driving display area to warn the driver to pay attention to avoid it.

[0125] In this embodiment, by identifying obstacles in real time, determining obstacle types, and monitoring the relative speed of obstacles, early warnings can be issued in a timely manner in potential dangerous situations to help drivers make correct driving decisions, thereby significantly improving driving safety. In addition, by displaying an alarm reminder box in the driving display area, intuitive visual feedback can be provided to the driver, helping the driver to respond quickly.

[0126] In order to better understand the process of driving safety display in the present invention, an example is used for explanation. Fig. 9 , shows a flow chart of a driving safety display method in another embodiment. The method comprises the following steps:

[0127] When the driving mode is the start mode or the parking mode, the driving display area is determined to be all display areas formed by the display device. When the driving mode is the forward mode or the reverse mode, the steering wheel angle is obtained; when the steering wheel angle is greater than or equal to the preset angle value, the driving display area is determined to be all display areas formed by the display device; when the steering wheel angle is less than or equal to the preset angle value, the driving display area is determined to be the display area on the A-pillar.

[0128] Step 902: Acquire an image outside the vehicle captured by an outside camera sensor corresponding to the driving display area.

[0129] Step 904, control the front camera to collect the driver's eye tracking information, and collect the distance information between the driver's eyes and the driving display area; obtain the corresponding eye movement trajectory based on the eye tracking information; determine the corresponding field of view blind spot information based on the movement trajectory and distance information.

[0130] Step 906, based on the preset field of view expansion information, the boundary of the field of view blind spot is expanded to obtain updated field of view blind spot information; based on the updated field of view blind spot information, the image outside the vehicle is cut to obtain the current blind spot image.

[0131] Step 908, according to the distortion correction model and the bending correction model of the display screen corresponding to the driving display area, perform distortion correction processing and bending correction processing on the current blind spot image to obtain the blind spot image information after the correction processing; obtain the pixel arrangement data of the display screen corresponding to the driving display area; based on the pixel arrangement data, establish a mapping relationship between the blind spot image after the correction processing and the pixels in the display screen corresponding to the driving display area; based on the mapping relationship and the blind spot image information, generate the display information of the driving display area; according to the display information, control the display device to display the current blind spot image in the driving display area.

[0132] Step 910, control the camera sensor outside the vehicle to identify the information of obstacles in the preset area; perform life feature recognition on the obstacles in the preset area according to the information of the obstacles to obtain the obstacle type; obtain the relative speed of the obstacles in the preset area, the relative speed is determined based on the distance between the obstacle and the vehicle in the previous time frame and the current time frame; when the relative speed is greater than or equal to the relative speed threshold corresponding to the obstacle type, control the display device to display an alarm reminder box on the boundary of the driving display area.

[0133] In the above embodiment, according to the driving mode of the vehicle, the driving display area of ​​the vehicle is determined, and the driving display area is the display area located on the A-pillar or all display areas formed by the display device; the vehicle exterior image collected by the vehicle exterior camera sensor corresponding to the driving display area is obtained; based on the front camera, the driver's eyes are tracked to obtain the corresponding visual blind spot information; based on the visual blind spot information, the vehicle exterior image is cut and processed to obtain the current blind spot image, and the current blind spot image is displayed in the driving display area. By adopting this method, by real-time displaying the blind spot image matching the driving mode, the driver can have a more comprehensive understanding of the vehicle's surrounding environment, thereby making more accurate driving decisions and improving driving safety. By using the collaborative work of multiple display areas, the driver can obtain the required vehicle surrounding information without frequently adjusting the line of sight or turning the head, thereby improving the convenience of driving. In addition, through the human eye tracking function of the front camera, the driver's visual blind spot can be intelligently identified, and the displayed image content can be adjusted accordingly, realizing a personalized driving assistance method, which is more in line with the actual needs of the driver and realizes the intelligence of driving assistance.

[0134] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0135] Based on the same inventive concept, the embodiment of the present application also provides a driving safety display device for implementing the driving safety display method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more driving safety display device embodiments provided below can refer to the limitations of the driving safety display method above, and will not be repeated here.

[0136] In an exemplary embodiment, Fig.10As shown, a driving safety display device is provided, which is applied to a vehicle, wherein the vehicle comprises a plurality of external camera sensors disposed on the vehicle body, a display device and a front camera disposed inside the vehicle; the display device is disposed at a vehicle pillar position in the cockpit; the display device is formed with a display area disposed on the A pillar, a display area disposed on the B pillar and a display area disposed on the C pillar; the device comprises: a determination module 1002, an acquisition module 1004, an acquisition module 1006 and a display module 1008, wherein:

[0137] A determination module 1002 is used to determine a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle, where the driving display area is a display area located on an A-pillar or all display areas formed by the display device;

[0138] The acquisition module 1004 is used to acquire the vehicle exterior image captured by the vehicle exterior camera sensor corresponding to the vehicle driving display area;

[0139] An acquisition module 1006 is used to control the front camera to collect the driver's eye tracking information, and determine the driver's visual field blind spot information according to the eye tracking information;

[0140] The display module 1008 is used to cut and process the image outside the vehicle based on the blind spot information to obtain the current blind spot image, and control the display device to display the current blind spot image in the driving display area.

[0141] In some embodiments, the determination module 1002 is also used to: when the driving mode is the starting mode or the parking mode, determine that the driving display area is all display areas formed by the display device; when the driving mode is the forward mode or the reverse mode, obtain the steering wheel angle; when the steering wheel angle is greater than or equal to the preset angle value, determine that the driving display area is all display areas formed by the display device; when the steering wheel angle is less than or equal to the preset angle value, determine that the driving display area is the display area on the A-pillar.

[0142] The acquisition module 1006 is also used to: control the front camera to collect the driver's eye tracking information, and collect the distance information between the driver's eyes and the driving display area; obtain the corresponding eye movement trajectory based on the eye tracking information; and determine the corresponding blind spot information based on the movement trajectory and distance information.

[0143] In some embodiments, the blind spot information includes the boundary of the blind spot; the display module 1008 is also used to: expand the boundary of the blind spot based on the preset field of view expansion information to obtain updated blind spot information; based on the updated blind spot information, cut the image outside the vehicle to obtain the current blind spot image.

[0144] In some embodiments, the display screen of the display device is a flexible display screen; the display module 1008 is also used to: perform distortion correction processing and bending correction processing on the current blind spot image according to the distortion correction model and bending correction model of the display screen corresponding to the driving display area, and obtain the blind spot image information after the correction processing; the distortion correction model is generated by the shape of the display screen; the bending correction model is generated by the bending information of the display screen; obtain the pixel arrangement data of the display screen corresponding to the driving display area; based on the pixel arrangement data, establish a mapping relationship between the blind spot image information after the correction processing and the pixels in the display screen corresponding to the driving display area; based on the mapping relationship and the blind spot image information, generate the display information of the driving display area; according to the display information, control the display device to display the current blind spot image in the driving display area.

[0145] In some embodiments, the vehicle includes a plurality of distance sensors disposed on the vehicle body, and the apparatus further includes:

[0146] The obstacle recognition module is used to control the camera sensor outside the vehicle to recognize the information of obstacles in the preset area;

[0147] A type recognition module is used to identify the life features of obstacles in a preset area based on the information of the obstacles and obtain the obstacle type;

[0148] A speed acquisition module, used to acquire the relative speed of an obstacle in a preset area, the relative speed being determined based on the distance between the obstacle and the vehicle in a previous time frame and a current time frame;

[0149] The reminder display module is used to control the display device to display an alarm reminder box on the boundary of the driving display area when the relative speed is greater than or equal to the relative speed threshold corresponding to the obstacle type.

[0150] Each module in the above-mentioned driving safety display device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the display device in the form of hardware, or can be stored in the memory in the display device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0151] In an exemplary embodiment, a display device is provided. The display device may be a server, and its internal structure diagram may be as shown in FIG. Fig.11As shown. The display device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the display device is used to provide computing and control capabilities. The memory of the display device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the display device is used to store the distortion correction model and the bending correction model of the display screen, as well as the relative speed threshold corresponding to the obstacle type. The input / output interface of the display device is used to exchange information between the processor and the external device. The communication interface of the display device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a driving safety display method is implemented.

[0152] The display unit of the display device is used to form a visually visible picture, and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the display device can be a touch layer covered on the display screen, or a key, trackball, or touchpad provided on the display device housing, or an external keyboard, touchpad, or mouse, etc.

[0153] Those skilled in the art will understand that Fig.11 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the display device to which the scheme of the present application is applied. The specific display device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0154] In an exemplary embodiment, a display device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the above method are implemented:

[0155] Determining a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle;

[0156] Acquire the vehicle exterior image captured by the vehicle exterior camera sensor corresponding to the driving display area;

[0157] Control the front camera to collect the driver's eye tracking information, and determine the driver's blind spot information based on the eye tracking information;

[0158] Based on the blind spot information, the image outside the vehicle is cut and processed to obtain the current blind spot image;

[0159] The display device is controlled to display the current blind spot image in the driving display area.

[0160] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0161] Determining a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle;

[0162] Acquire the vehicle exterior image captured by the vehicle exterior camera sensor corresponding to the driving display area;

[0163] Control the front camera to collect the driver's eye tracking information, and determine the driver's blind spot information based on the eye tracking information;

[0164] Based on the blind spot information, the image outside the vehicle is cut and processed to obtain the current blind spot image;

[0165] The display device is controlled to display the current blind spot image in the driving display area.

[0166] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0167] Determining a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle;

[0168] Acquire the vehicle exterior image captured by the vehicle exterior camera sensor corresponding to the driving display area;

[0169] Control the front camera to collect the driver's eye tracking information, and determine the driver's blind spot information based on the eye tracking information;

[0170] Based on the blind spot information, the image outside the vehicle is cut and processed to obtain the current blind spot image;

[0171] The display device is controlled to display the current blind spot image in the driving display area.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0173] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0174] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0175] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A driving safety display method, characterized in that: The vehicle includes an external camera sensor disposed on the vehicle body, a display device, and a front camera located inside the vehicle; the display device is located at a vehicle pillar position in the cabin; the method includes: Determining a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle; Acquire an image outside the vehicle captured by an outside camera sensor corresponding to the driving display area; Controlling the front camera to collect eye tracking information of the driver, and determining the blind spot information of the driver's field of vision according to the eye tracking information; Based on the blind spot information, the image outside the vehicle is segmented to obtain a current blind spot image; The driving display area is controlled to display the current blind spot image.

2. The method according to claim 1, characterized in that The step of determining a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle includes: The display device is formed with a display area located on the A-pillar, a display area located on the B-pillar, and a display area located on the C-pillar; According to the driving mode of the vehicle, a driving display area of ​​the vehicle is determined, and the driving display area is a display area located on the A-pillar or all display areas formed by the display device.

3. The method according to claim 2, characterized in that Determining the driving display area of ​​the vehicle according to the driving mode of the vehicle includes: When the driving mode is a starting mode or a parking mode, determining the driving display area to be all display areas formed by the display device; Preferably, when the driving mode is a forward mode or a reverse mode, obtaining a steering wheel angle; When the steering wheel angle is greater than or equal to a preset angle value, determining that the driving display area is all display areas formed by the display device; When the steering wheel angle is less than or equal to a preset angle value, the driving display area is determined to be a display area on the A-pillar.

4. The method according to claim 1, characterized in that: The controlling the front camera to collect the driver's eye tracking information, and determining the driver's visual field blind spot information according to the eye tracking information, includes: Controlling the front camera to collect eye tracking information of the driver, and collecting distance information between the driver's eyes and the driving display area; According to the eye tracking information, obtain the corresponding eye movement trajectory; Corresponding visual field blind area information is determined according to the motion trajectory and the distance information.

5. The method according to claim 1, characterized in that The blind area information includes the boundary of the blind area; The step of performing a cutting process on the vehicle exterior image based on the visual field blind spot information to obtain a current blind spot image includes: Based on the preset visual field expansion information, the boundary of the visual field blind area is expanded to obtain updated visual field blind area information; Based on the updated blind spot information, the image outside the vehicle is cut to obtain a current blind spot image.

6. The method according to claim 1, characterized in that The display screen of the display device is a flexible display screen; and controlling the display device to display the current blind spot image in the driving display area includes: According to the distortion correction model and the bending correction model of the display screen corresponding to the driving display area, the current blind spot image is subjected to distortion correction processing and bending correction processing to obtain the blind spot image information after the correction processing, wherein the distortion correction model is generated by the shape of the display screen, and the bending correction model is generated by the bending information of the display screen; Preferably, obtaining pixel arrangement data of a display screen corresponding to the driving display area; Based on the pixel arrangement data, a mapping relationship between the corrected blind spot image information and pixels in the corresponding display screen of the driving display area is established; Based on the mapping relationship and the blind spot image information, generating display information of the driving display area; According to the display information, the display device is controlled to display the current blind spot image in the driving display area.

7. The method according to claim 1, characterized in that The method further comprises: Controlling the external camera sensor to identify obstacles in a preset area; According to the information of the obstacle, life features of the obstacle in the preset area are identified to obtain the obstacle type; Obtaining a relative speed of an obstacle in a preset area, wherein the relative speed is determined based on a distance between the obstacle and the vehicle in a previous time frame and a current time frame; When the relative speed is greater than or equal to a relative speed threshold corresponding to the obstacle type, the display device is controlled to display an alarm reminder box on the boundary of the driving display area.

8. A driving safety display device, characterized in that: The device comprises: an external camera sensor placed on the vehicle body, a display device and a front camera located inside the vehicle; the display device is located at a vehicle pillar position in the cockpit; A determination module, used to determine a driving display area of ​​a display device of the vehicle according to a driving mode of the vehicle; An acquisition module, used for acquiring an image outside the vehicle collected by an outside camera sensor corresponding to the driving display area; An acquisition module, used for controlling the front camera to collect the driver's eye tracking information, and determining the driver's visual field blind spot information according to the eye tracking information; The display module is used to perform segmentation processing on the vehicle exterior image based on the visual field blind spot information to obtain a current blind spot image, and control the display device to display the current blind spot image in the driving display area.

9. A display device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images