Display picture processing method and device, electronic equipment and vehicle

Abstract

The invention relates to a display picture processing method and device, electronic equipment and a vehicle, and relates to the technical field of image processing, and the method comprises the steps: obtaining the first ambient brightness outside the vehicle, the second ambient brightness inside the vehicle, and a target image containing the face area of a driver; determining the driving sight line direction of the driver by analyzing facial features contained in the target image; adjusting the screen brightness of a target display screen to the first ambient brightness in response to the change of the driving sight line direction from the area outside the vehicle to the display area of the target display screen; and in response to the change of the driving sight line direction from the display area of the target display screen to the area outside the vehicle, adjusting the screen brightness of the target display screen to the second ambient brightness. According to the method, automatic compensation of the brightness of the display screen is realized through detection of the sight direction of the driver and automatic sensing of the environment inside and outside the vehicle, and the visual zoom feeling of the driver can be reduced in the dynamic environment of the vehicle, so that the driving experience of a user using the electronic rearview mirror is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic equipment and vehicle for processing a display image. Background Art

[0002] Electronic rearview mirrors are products installed outside the car to replace traditional glass rearview mirrors. Unlike traditional rearview mirrors, electronic rearview mirrors are "display screens installed inside the car". Drivers use electronic imaging systems to achieve the effect of rearview mirrors through these two screens. Compared with traditional rearview mirrors, electronic exterior rearview mirrors are not only more aerodynamically designed, but also have significant improvements in viewing angle and night tolerance.

[0003] However, in the use scenario of electronic rearview mirrors, the driver's eyes need to focus and switch back and forth between the road surface and the display screen. Because of the difference in brightness and depth of field between the road surface in front and the display screen in the dynamic environment of the vehicle, it will be obvious that the focus is difficult, causing obvious discomfort and dizziness. How to reduce the discomfort of users using electronic rearview mirrors and improve the user experience of using electronic rearview mirrors is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the present application provides a display screen processing method, device, electronic device and vehicle, which can reduce the discomfort of users when using electronic rearview mirrors and enhance the user's experience of using electronic rearview mirrors.

[0005] According to a first aspect of the present application, a method for processing a display screen is provided, comprising:

[0006] Acquire a first ambient brightness outside the vehicle, a second ambient brightness inside the vehicle, and a target image including a driver's facial area;

[0007] Determining the driving sight direction of the driver by analyzing facial features contained in the target image;

[0008] In response to the driving sight line direction changing from the area outside the vehicle to the display area of ​​the target display screen, adjusting the screen brightness of the target display screen to the first ambient brightness;

[0009] In response to the driving sight line direction changing from the display area of ​​the target display screen to the area outside the vehicle, the screen brightness of the target display screen is adjusted to the second ambient brightness.

[0010] According to a second aspect of the present application, a display screen processing device is provided, comprising:

[0011] An acquisition module, used to acquire a first ambient brightness outside the vehicle, a second ambient brightness inside the vehicle, and a target image including a driver's facial area;

[0012] An analysis module, configured to determine the driving sight direction of the driver by analyzing facial features contained in the target image;

[0013] A first adjustment module is used for adjusting the screen brightness of the target display screen to the first ambient brightness in response to the driving sight line direction changing from the area outside the vehicle to the display area of ​​the target display screen;

[0014] The second adjustment module is used to adjust the screen brightness of the target display screen to the second ambient brightness in response to the driving sight line direction changing from the display area of ​​the target display screen to the area outside the vehicle.

[0015] According to a third aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for processing a display screen described in the first aspect is implemented.

[0016] According to a fourth aspect of the present application, an electronic device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein when the processor executes the computer program, the display screen processing method described in the first aspect is implemented.

[0017] According to a fifth aspect of the present application, a vehicle is provided, comprising: the electronic device as described in the fourth aspect.

[0018] By means of the above technical scheme, the present application provides a display screen processing method, device, electronic device and vehicle, which can first obtain the first ambient brightness outside the vehicle, the second ambient brightness inside the vehicle and the target image containing the driver's facial area; then determine the driver's driving sight direction by analyzing the facial features contained in the target image; when the driving sight direction changes from the area outside the vehicle to the display area of ​​the target display screen, adjust the screen brightness of the target display screen to the first ambient brightness; when the driving sight direction changes from the display area of ​​the target display screen to the area outside the vehicle, adjust the screen brightness of the target display screen to the second ambient brightness. The present application realizes automatic compensation of the display screen brightness by detecting the driver's sight direction and automatically sensing the environment inside and outside the vehicle, which can reduce the driver's visual zoom feeling in the dynamic environment of the vehicle, thereby improving the user's driving experience when using an electronic rearview mirror.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 A flowchart of a method for processing a display image provided by an embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram of a principle flow of display picture processing provided by an embodiment of the present disclosure;

[0024] Figure 3 A flowchart of a method for processing a display image provided by another embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of an example of depth of field enhancement processing provided by an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of the structure of a display image processing device provided by an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of the structure of a display image processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] The following describes a display screen processing method, device, electronic device, and vehicle according to an embodiment of the present disclosure with reference to the accompanying drawings.

[0030] In the use scenario of electronic rearview mirrors, the driver's eyes need to focus and switch back and forth between the road surface and the display screen. Because of the difference in brightness and depth of field between the road surface in front and the display screen in a dynamic vehicle environment, it will be difficult to focus and cause obvious discomfort (looking at a shaking screen for a long time and high-speed vehicle speeds will cause dizziness in the eyes). How to reduce the discomfort of users using electronic rearview mirrors and improve the user experience of using electronic rearview mirrors is a problem that needs to be solved urgently.

[0031] In order to solve the above technical problems, the present disclosure provides a display screen processing method, device, electronic device and vehicle, which can automatically compensate for the brightness of the display screen through the driver's line of sight direction detection and automatic perception of the environment inside and outside the vehicle, and can reduce the driver's visual zoom feeling in a dynamic vehicle environment, thereby improving the user's driving experience when using an electronic rearview mirror.

[0032] like Figure 1 , an embodiment of the present disclosure provides a method for processing a display screen, comprising:

[0033] Step 101: Acquire a first ambient brightness outside a vehicle, a second ambient brightness inside the vehicle, and a target image including a driver's facial area.

[0034] In specific application scenarios, environmental information collection devices such as cameras and / or sensors can be deployed inside and outside the vehicle respectively. The cameras and sensors deployed outside the vehicle can be used to collect real-time environmental information outside the vehicle, such as external environmental images, temperature, airflow, air pressure, weather, light, radiation, etc., and through further analysis of the external environmental information of the vehicle, the first environmental brightness outside the vehicle can be obtained; the cameras and sensors deployed inside the vehicle can be used to collect real-time environmental information inside the vehicle, such as internal environmental images, temperature, airflow, air pressure, light, etc., and through further analysis of the internal environmental information of the vehicle, the second environmental brightness inside the vehicle can be obtained; in addition, a camera capable of collecting images of the driver's facial area can be set inside the vehicle, and the camera can be used to collect real-time images of the driver's facial area during the driver's driving process, so as to determine the driver's driving line of sight direction through comparative learning of the facial area images.

[0035] like Figure 2As shown, the execution subject of the present disclosure may be a vehicle host, which is connected to an external camera and an internal camera respectively. The vehicle host can obtain the external environment information of the vehicle in real time based on the external camera, and obtain the first environment brightness of the external vehicle by analyzing the external environment information of the vehicle; the vehicle host can also obtain the internal environment information of the vehicle in real time based on the internal camera, and obtain the second environment brightness of the internal vehicle by analyzing the internal environment information of the vehicle; in addition, the vehicle host can also collect the driver's facial area image in real time based on the internal camera, and determine the driver's driving line of sight direction by comparing and learning the facial area image; then the vehicle host can adjust the brightness of the electronic rearview mirror corresponding to the in-vehicle display screen according to the driving line of sight direction, the first environment brightness or the second environment brightness. By adaptively compensating the brightness of the display screen, the discomfort of the user using the electronic rearview mirror can be reduced, and the user's experience of using the electronic rearview mirror can be improved.

[0036] Step 102: Determine the driver's driving sight direction by analyzing facial features contained in the target image.

[0037] Among them, facial features may include head features, eye features and other human biological features that can be used to determine the direction of the driver's line of sight, which are not exhaustively listed here. In the following embodiment steps of the present disclosure, the facial features include head features and eye features as an example to illustrate the technical solution in the present disclosure, but it does not constitute a specific limitation.

[0038] For the disclosed embodiment, the head features and eye features contained in the target image can be first extracted, and the driver's driving sight direction can be determined by further comprehensive analysis of the head features and eye features. Among them, the head features can include the offset direction of the head relative to the front of the body, the position coordinates of the head movement, etc., and the eye features can be the center line of the left and right pupils, the corneal reflection image of the left or right eye is in a line-symmetrical position, the three-dimensional position of the left and right pupil centers, the radius of the eyeball, etc. When analyzing the head features and eye features to determine the driver's driving sight direction, as a possible implementation method, in the facial feature detection stage, by analyzing the distribution of the driver's facial features, the relative position of the driver's pupil corresponding to the face when the driver looks at different directions can be determined respectively, and the driver's facial orientation can be further determined by analyzing the driver's head position in the target image, and the driver's sight direction can be determined by combining the judgment of the relative position of the eye pupil. In addition, in order to avoid the problem of easy judgment errors when frequently judging the driving sight direction, as a possible implementation method, the trigger condition judgment of the driving sight direction can be first performed, and when it is determined that the trigger condition is met, the driver's driving sight direction can be further determined by analyzing the head features and eye features. Among them, the trigger conditions may be that the driver's gaze time in the same direction reaches a preset time threshold, the cumulative time the driver maintains the same action posture reaches a preset time threshold, the time period for judging the driving line of sight direction is reached at the current moment, etc. The trigger conditions are not specifically limited here.

[0039] Step 103: In response to the driver's line of sight changing from the area outside the vehicle to the display area of ​​the target display screen, the screen brightness of the target display screen is adjusted to the first ambient brightness.

[0040] In a specific application scenario, the electronic rearview mirror is configured with an in-car screen (i.e., in-car display) in the car, and the rear environment is observed through the display of the in-car screen. The in-car screen may include one or more screens, such as left and right display screens, and the target display screen is the in-car screen that the driver is looking at among the one or more in-car screens.

[0041] For the disclosed embodiment, when it is determined that the driving sight direction is changed from the area outside the vehicle to the display area of ​​the target display screen, that is, the driver's driving sight is transferred from the physical object outside the vehicle (such as the road ahead, etc.) to the display screen, the screen brightness of the target display screen can be adjusted to the brightness perceived outside the vehicle (the same as the brightness of the physical object outside the vehicle) through the high-definition display and precise brightness control of the screen, which can visually reduce the sense of zoom, thereby reducing the user's discomfort when looking at the electronic screen. Specifically, the screen brightness of the target display screen can be adjusted according to the first ambient brightness outside the vehicle. As a possible implementation method, the first ambient brightness can be directly used as the target brightness value for screen brightness adjustment, so that the adjusted screen brightness is equal to the first ambient brightness; as a possible implementation method, the brightness adjustment interval of the screen brightness can be set according to the first ambient brightness, and the adjusted screen brightness is within the brightness adjustment interval by increasing or decreasing the screen brightness, wherein the brightness adjustment interval contains multiple brightness adjustment values ​​that are close to the first ambient brightness and are not easy for human eyes to identify. The specific values ​​of the brightness adjustment values ​​can be determined according to the actual application scenario, and are not specifically limited here.

[0042] Step 104: in response to the driver's line of sight changing from the display area of ​​the target display screen to the area outside the vehicle, adjusting the screen brightness of the target display screen to a second ambient brightness.

[0043] For the disclosed embodiment, when it is determined that the driving sight direction is changed from the display area of ​​the target display screen to the area outside the vehicle, that is, the driver's driving sight is transferred from the display screen to the physical object outside the vehicle (such as the road ahead, etc.), the screen brightness can be reduced to a brightness close to the brightness of the interior panel through the high-definition display and precise brightness control of the screen, thereby reducing the interference caused by the motion picture, visually reducing the sense of zoom, and not affecting the driver's driving. Specifically, the screen brightness of the target display screen can be adjusted according to the second ambient brightness inside the vehicle. As a possible implementation method, the second ambient brightness can be directly used as the target brightness value for screen brightness adjustment, so that the adjusted screen brightness is equal to the second ambient brightness; as a possible implementation method, the brightness adjustment interval of the screen brightness can be set according to the second ambient brightness, and the adjusted screen brightness is within the brightness adjustment interval by increasing or decreasing the screen brightness, wherein the brightness adjustment interval contains multiple brightness adjustment values ​​that are close to the second ambient brightness and are not easy for human eyes to identify. The specific values ​​of the brightness adjustment values ​​can be determined according to the actual application scenario, and are not specifically limited here.

[0044] In summary, according to the display screen processing method provided by the present application, the first ambient brightness outside the vehicle, the second ambient brightness inside the vehicle, and the target image containing the driver's facial area can be first obtained; then, the driver's driving sight direction is determined by analyzing the facial features contained in the target image; when the driving sight direction is from the area outside the vehicle to the display area of ​​the target display screen, the screen brightness of the target display screen is adjusted to the first ambient brightness; when the driving sight direction is from the display area of ​​the target display screen to the area outside the vehicle, the screen brightness of the target display screen is adjusted to the second ambient brightness. The present application realizes automatic compensation of the display screen brightness through the detection of the driver's sight direction and the automatic perception of the environment inside and outside the vehicle, which can reduce the driver's visual zoom feeling in the dynamic environment of the vehicle, thereby improving the user's driving experience when using the electronic rearview mirror.

[0045] Further, as a refinement and extension of the above embodiment, in order to fully illustrate the specific implementation process of the method of this embodiment, this embodiment provides the following Figure 3 The specific method shown includes:

[0046] Step 201: Acquire a first ambient brightness outside a vehicle, a second ambient brightness inside the vehicle, and a target image including a driver's facial area.

[0047] For the embodiment of the present disclosure, the specific implementation process can refer to the relevant description in step 101 of the embodiment, which will not be repeated here.

[0048] Step 202: extract the driver's head area image and eye area image from the target image by detecting key points of the face.

[0049] For the embodiments of the present disclosure, face detection technology can be used to select a head area frame in the target image, and the target image can be cropped according to the selected head area frame to obtain a head area image. The present disclosure is not limited to the above-mentioned face area detection technology, such as face detection neural network, etc. The head area image can be a face map that only contains the face area in the target image after cropping, which retains all the original information of the target image to ensure the accuracy and integrity of subsequent data processing. If the interference of light and noise is obvious, the face map can only include a face shape map of face contour feature points, and the above-mentioned face contour feature points include but are not limited to: face contour points, eye contour points, nose contour points, eyebrow contour points, forehead contour points and lip contour points. The face shape map can assist in estimating the angle. When there is interference information such as light and noise, the face shape map can better represent the face angle information compared to the face area map. In addition, facial key point detection is performed on the target image, and the target image is cropped according to the distribution of left and right eye feature points, respectively, to obtain left eye image and right eye image, collectively referred to as eye area images, wherein the facial key points may include facial key point data corresponding to various face areas, and various face areas include left eyebrow area, right eyebrow area, left eye area, right eye area, nose area and mouth area, etc. The present disclosure does not limit the facial key point detection technology.

[0050] Step 203: input the head area image and the eye area image into the trained sight line detection model, use the sight line detection model to perform sight line detection, and determine the driving sight line direction of the driver.

[0051] For the embodiments of the present disclosure, the sight line detection model can be any computing model that can realize the driving sight line direction detection function, such as a neural network model, a deep learning model, etc. Exemplarily, the sight line detection model in the present disclosure can be selected as a hidden Markov model, a conditional random field and a neural network model, etc., which are not specifically limited here. In a specific application scenario, before executing the steps of this embodiment, the sight line detection model can also be trained for sight line direction detection tasks based on a certain number of sight line detection samples through supervised learning, or unsupervised learning, semi-supervised learning, and reinforcement learning. The corresponding training method is used until it is determined that the sight line detection model has reached a convergence state and the training is completed. Then the trained sight line detection model can be directly put into the specific application of sight line detection, that is, the head area image and the eye area image are input into the trained sight line detection model, and the sight line detection model will directly output the driver's driving sight line direction, such as looking at the road surface, looking at the left and right electronic rearview mirror screens, etc.

[0052] Among them, taking the gaze detection model as a neural network model as an example, when training the gaze detection model, a training data set can be first constructed, the training data set including various types of head area images and eye area images, and preset gaze direction labels matching the head area images and eye area images; the training data set with the configured gaze direction labels is input into the gaze detection model, the head area images and eye area images in the training data set are used as input features, and the preset gaze direction labels corresponding to the head area images and eye area images are used as label data to train the gaze detection model; the predicted gaze direction output by the gaze detection model is obtained, and the loss function of the gaze detection model is calculated based on the preset gaze direction label and the predicted gaze direction; if it is determined that the loss function is less than a preset threshold, it is determined that the training of the gaze detection model is completed; if it is determined that the loss function is greater than or equal to the preset threshold, the model parameters of the gaze detection model are iteratively updated using the training data set until the loss function of the gaze detection model is less than the preset threshold, and it is determined that the training of the gaze detection model is completed.

[0053] Correspondingly, the trained sight line detection model may include a feature extraction layer and a linear classification layer. For the disclosed embodiment, when using the sight line detection model to perform sight line detection and determine the driver's driving sight line direction, the embodiment steps may include: using the feature extraction layer to perform feature extraction on the head area image and the eye area image respectively to obtain the driver's facial feature vector and eye feature vector; using the linear classification layer to splice the facial feature vector and the eye feature vector, and performing linear classification processing of the sight line direction on the obtained spliced ​​vector to determine the driver's driving sight line direction. Among them, the facial feature vector and the eye feature vector can be spliced ​​according to a preset vector splicing method, such as splicing the facial feature vector to the preset position of the eye feature vector, or splicing the eye feature vector to the preset position of the facial feature vector, or performing vector fusion on the facial feature vector and the eye feature vector, etc. The splicing method is not specifically limited here.

[0054] Step 204: in response to the driver's line of sight changing from the area outside the vehicle to the display area of ​​the target display screen, the screen brightness of the target display screen is adjusted to the first ambient brightness.

[0055] For the disclosed embodiment, when it is determined that the driving sight direction is changed from the area outside the vehicle to the display area of ​​the target display screen, that is, when the driver's driving sight is transferred from the physical object outside the vehicle (such as the road ahead, etc.) to the display screen, the vehicle host can adjust the screen brightness of the target display screen to the brightness perceived outside the vehicle (the same as the brightness of the physical object outside the vehicle) through the brightness control of the screen, which can visually reduce the sense of zoom, thereby reducing the user's discomfort when looking at the electronic screen. As a possible application scenario, the embodiment steps may include: determining the current screen brightness of the target display screen; calculating the first brightness compensation value of the current screen brightness relative to the first ambient brightness; adjusting the screen brightness of the target display screen based on the first brightness compensation value, so that the brightness difference between the adjusted screen brightness and the first ambient brightness is less than the first preset threshold. Among them, the first preset threshold can be the maximum brightness difference that cannot be perceived and identified by the human eye. The specific value can be determined according to the actual application scenario, and no specific limitation is made here.

[0056] Step 205: In response to the driver's line of sight changing from the display area of ​​the target display screen to the area outside the vehicle, the screen brightness of the target display screen is adjusted to a second ambient brightness.

[0057] For the embodiments of the present disclosure, when it is determined that the driving sight direction is changed from the display area of ​​the target display screen to the area outside the vehicle, that is, the driver's driving sight is transferred from the display screen to the physical object outside the vehicle (such as the road ahead, etc.), the screen brightness can be controlled to reduce the screen brightness to a brightness close to the brightness of the interior panel, thereby reducing the interference caused by the moving picture and visually reducing the sense of zoom, so as not to affect the driver's driving. As a possible application scenario, the embodiment steps may include: determining the current screen brightness of the target display screen; calculating the second brightness compensation value of the current screen brightness relative to the second ambient brightness; adjusting the screen brightness of the target display screen based on the second brightness compensation value, so that the brightness difference between the adjusted screen brightness and the second ambient brightness is less than the first preset threshold. Among them, the first preset threshold can be the maximum brightness difference that cannot be perceived and identified by the human eye. The specific value can be determined according to the actual application scenario, and no specific limitation is made here.

[0058] Step 206: Based on the driving line of sight, the display image of the target display screen is subjected to depth-of-field sharpening processing.

[0059] Among them, depth of field can be understood as the depth of scenery. A large depth of field means a large depth of scenery and contains more background information; a small depth of field, that is, a shallow depth of field, contains less background information, which will produce a blurred image effect.

[0060] In specific application scenarios, in order to avoid the driver's eyes focusing and switching back and forth between the road and the display screen, which will cause a noticeable sense of zooming in the dynamic environment of the vehicle due to the difference in depth of field, the difference in the depth of field of the display screen can be automatically compensated through automatic perception of the environment inside and outside the car. Figure 4 As shown, specifically, when the driving sight line direction is changed from the area outside the vehicle to the display area of ​​the target display screen, the depth of field of the target display screen can be enhanced (the image is processed as near real and far virtual and near bright and far dark, and the contrast between n2 and n3 in the diagram is enhanced) to reduce the time for the user's eyes to judge the depth of field of the display screen, so that the front and back real and virtual distance relationship of the three-dimensional image can be judged in a short time, and the time caused by the recalculation of the virtual image of the plane image in the user's brain can be reduced, so that the user can more easily obtain the depth of field image and improve the user's viewing experience; when the driving sight line direction is changed from the display area of ​​the target display screen to the area outside the vehicle, the depth of field of the target display screen can be weakened (the contrast between n2 and n3 in the diagram is reduced) to prevent the display screen from being too prominent and avoid distracting the driver's attention.

[0061] For the disclosed embodiment, as a possible implementation method, in response to the driver's line of sight changing from the area outside the vehicle to the display area of ​​the target display screen, the display screen of the target display screen is subjected to a first depth of field sharpening process, and the first depth of field sharpening process is used to increase the depth of field value of the display screen, so that the depth of field difference between the display screen and the in-vehicle environment is greater than a second preset threshold. Among them, the second preset threshold is the minimum depth of field difference that the human eye can perceive and identify, and the specific value can be determined according to the actual application scenario, and is not specifically limited here. In this way, the image in the display screen of the observed side will be enhanced to obtain a better and clearer observation effect, which can ensure that the user can easily obtain an image with enhanced depth of field.

[0062] As a possible implementation method, in response to the driver's line of sight changing from the display area of ​​the target display screen to the area outside the vehicle, the display screen of the target display screen is subjected to a second depth of field sharpening process, and the second depth of field sharpening process is used to reduce the depth of field value of the display screen, so that the depth of field difference between the display screen and the in-vehicle environment is less than a third preset threshold. Among them, the third preset threshold is the maximum depth of field difference that cannot be perceived and identified by the human eye, and the third preset threshold is less than the second preset threshold. The specific value can be determined according to the actual application scenario and is not specifically limited here. In this way, the interference of the moving picture to the driver can be reduced and the driving experience can be improved.

[0063] In summary, according to the display screen processing method provided by the present application, the first ambient brightness outside the vehicle, the second ambient brightness inside the vehicle, and the target image containing the driver's facial area can be first obtained; then, the driver's driving sight direction can be determined by analyzing the facial features contained in the target image; when the driving sight direction changes from the area outside the vehicle to the display area of ​​the target display screen, the screen brightness of the target display screen is adjusted to the first ambient brightness; when the driving sight direction changes from the display area of ​​the target display screen to the area outside the vehicle, the screen brightness of the target display screen is adjusted to the second ambient brightness. In addition, the display screen of the target display screen can also be sharpened based on the driving sight direction. The present application realizes automatic compensation for the difference in display screen brightness and depth of field through the detection of the driver's sight direction and the automatic perception of the environment inside and outside the vehicle, which can reduce the driver's visual zoom feeling in the dynamic environment of the vehicle, reduce the user's eye judgment time for the depth of field of the display screen, and can judge the front and rear virtual and real distance relationship of the stereoscopic picture in a short time, reduce the time brought by the virtual image of the plane picture to be recalculated in the user's brain, so that the user can more easily obtain the depth of field image and improve the user's driving experience when using the electronic rearview mirror.

[0064] Based on the above Figure 1 , Figure 3 The specific implementation of the method shown in the embodiment provides a processing device for displaying a screen, such as Figure 5 As shown, the device comprises:

[0065] An acquisition module 31, which can be used to acquire a first ambient brightness outside the vehicle, a second ambient brightness inside the vehicle, and a target image including a driver's facial area;

[0066] The analysis module 32 may be used to determine the driving sight direction of the driver by analyzing facial features contained in the target image;

[0067] The first adjustment module 33 may be used to adjust the screen brightness of the target display screen to the first ambient brightness in response to the driver's sight line direction changing from the area outside the vehicle to the display area of ​​the target display screen;

[0068] The second adjustment module 34 may be used to adjust the screen brightness of the target display screen to a second ambient brightness in response to the driver's line of sight changing from the display area of ​​the target display screen to the area outside the vehicle.

[0069] In a specific application scenario, the analysis module 32 can be used to extract the driver's head area image and eye area image in the target image through facial key point detection; the head area image and eye area image are input into the trained line of sight detection model, and the line of sight detection is performed using the line of sight detection model to determine the driver's driving line of sight direction.

[0070] In a specific application scenario, the trained line of sight detection model includes a feature extraction layer and a linear classification layer. The analysis module 32 can be used to use the feature extraction layer to perform feature extraction on the head area image and the eye area image respectively to obtain the driver's head feature vector and eye feature vector; use the linear classification layer to splice the head feature vector and the eye feature vector, and perform linear classification processing on the obtained spliced ​​vector for the line of sight direction to determine the driver's driving line of sight direction.

[0071] In a specific application scenario, the first adjustment module 33 can be used to determine the current screen brightness of the target display screen; calculate a first brightness compensation value of the current screen brightness relative to the first ambient brightness; adjust the screen brightness of the target display screen based on the first brightness compensation value so that the brightness difference between the adjusted screen brightness and the first ambient brightness is less than a first preset threshold; the second adjustment module 34 can be used to calculate a second brightness compensation value of the current screen brightness relative to the second ambient brightness; adjust the screen brightness of the target display screen based on the second brightness compensation value so that the brightness difference between the adjusted screen brightness and the second ambient brightness is less than the first preset threshold.

[0072] In specific application scenarios, such as Figure 6 As shown, the device further includes: a processing module 35;

[0073] The processing module 35 can be used to perform depth-of-field sharpening processing on the display image of the target display screen based on the driving line of sight direction.

[0074] In a specific application scenario, the processing module 35 can be used to perform a first depth-of-field sharpening process on the display image of the target display screen in response to the driver's line of sight changing from the area outside the vehicle to the display area of ​​the target display screen. The first depth-of-field sharpening process is used to increase the depth-of-field value of the display image so that the depth-of-field difference between the display image and the interior environment of the vehicle is greater than a second preset threshold.

[0075] In a specific application scenario, the processing module 35 can be used to perform a second depth of field sharpening process on the display image of the target display screen in response to the driver's line of sight changing from the display area of ​​the target display screen to the area outside the vehicle. The second depth of field sharpening process is used to reduce the depth of field value of the display image so that the depth of field difference between the display image and the interior environment of the vehicle is less than a third preset threshold, wherein the second preset threshold is greater than the third preset threshold.

[0076] It should be noted that for other corresponding descriptions of the functional units involved in the display screen processing device provided in this embodiment, reference can be made to Figure 1 and Figure 3 The corresponding description in will not be repeated here.

[0077] Based on the above Figure 1 and Figure 3The method shown in the embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 3 The method shown.

[0078] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0079] Based on the above Figure 1 and Figure 3 The method shown, and Figure 5 and Figure 6 In order to achieve the above-mentioned purpose, the embodiment of the present application also provides an electronic device that can be configured on the vehicle (such as an electric vehicle) side, and the device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 3 The method shown.

[0080] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0081] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0082] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0083] Based on the above electronic device, the embodiment of the present application further provides a vehicle, which may specifically include: the above electronic device. The vehicle may specifically be an electric car, etc.

[0084] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. The technical solution disclosed in the present invention can first obtain the first ambient brightness outside the vehicle, the second ambient brightness inside the vehicle, and the target image containing the driver's facial area; then determine the driver's driving line of sight direction by analyzing the facial features contained in the target image; when the driving line of sight direction changes from the area outside the vehicle to the display area of ​​the target display screen, adjust the screen brightness of the target display screen to the first ambient brightness; when the driving line of sight direction changes from the display area of ​​the target display screen to the area outside the vehicle, adjust the screen brightness of the target display screen to the second ambient brightness. In addition, the display screen of the target display screen can also be subjected to depth of field sharpening processing based on the driving line of sight direction. The present application realizes automatic compensation for the brightness and depth of field differences of the display screen through the detection of the driver's line of sight direction and the automatic perception of the environment inside and outside the vehicle. It can reduce the driver's visual zoom feeling in a dynamic vehicle environment and reduce the time it takes for the user's eyes to judge the depth of field of the display screen. It can judge the front and back virtual and real distance relationship of the stereoscopic image in a short time and reduce the time taken for the user's brain to recalculate the virtual image of the flat image, so that the user can more easily obtain the depth of field image and enhance the user's driving experience when using the electronic rearview mirror.

[0085] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0086] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

Claims

1. A method for processing a display screen, characterized in that: include: Acquire a first ambient brightness outside the vehicle, a second ambient brightness inside the vehicle, and a target image including a driver's facial area; Determining the driving sight direction of the driver by analyzing facial features contained in the target image; In response to the driving sight line direction changing from the area outside the vehicle to the display area of ​​the target display screen, adjusting the screen brightness of the target display screen to the first ambient brightness; In response to the driving sight line direction changing from the display area of ​​the target display screen to the area outside the vehicle, the screen brightness of the target display screen is adjusted to the second ambient brightness.

2. The method according to claim 1, characterized in that The step of determining the driving sight direction of the driver by analyzing the facial features contained in the target image information includes: By detecting key points of the face, extracting the driver's head area image and eye area image from the target image; The head area image and the eye area image are input into a trained sight line detection model, and the sight line detection model is used to perform sight line detection to determine the driving sight line direction of the driver.

3. The method according to claim 2, characterized in that The trained sight line detection model includes a feature extraction layer and a linear classification layer. The sight line detection is performed by using the sight line detection model to determine the driving sight line direction of the driver, including: Using the feature extraction layer to perform feature extraction on the head region image and the eye region image respectively, to obtain a head feature vector and an eye feature vector of the driver; The head feature vector and the eye feature vector are concatenated using the linear classification layer, and the obtained concatenated vector is subjected to linear classification processing of the sight line direction to determine the driving sight line direction of the driver.

4. The method according to claim 1, characterized in that: The step of adjusting the screen brightness of the target display screen to the first ambient brightness includes: Determining the current screen brightness of the target display screen; Calculating a first brightness compensation value of the current screen brightness relative to the first ambient brightness; adjusting the screen brightness of the target display screen based on the first brightness compensation value so that a brightness difference between the adjusted screen brightness and the first ambient brightness is less than a first preset threshold; The step of adjusting the screen brightness of the target display screen to the second ambient brightness includes: Calculating a second brightness compensation value of the current screen brightness relative to the second ambient brightness; The screen brightness of the target display screen is adjusted based on the second brightness compensation value so that the brightness difference between the adjusted screen brightness and the second ambient brightness is less than the first preset threshold.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Based on the driving sight line direction, a depth-of-field sharpening process is performed on the display image of the target display screen.

6. The method according to claim 5, characterized in that The step of performing depth of field sharpening processing on the display image of the target display screen based on the driving sight line direction includes: In response to the driving sight line direction changing from the area outside the vehicle to the display area of ​​the target display screen, a first depth of field sharpening process is performed on the display screen of the target display screen, wherein the first depth of field sharpening process is used to increase the depth of field value of the display screen so that the depth of field difference between the display screen and the vehicle interior environment is greater than a second preset threshold value; In response to the driving line of sight changing from the display area of ​​the target display screen to the area outside the vehicle, a second depth of field sharpening process is performed on the display image of the target display screen. The second depth of field sharpening process is used to reduce the depth of field value of the display image so that the depth of field difference between the display image and the interior environment of the vehicle is less than a third preset threshold, wherein the second preset threshold is greater than the third preset threshold.

7. A display screen processing device, characterized in that: include: An acquisition module, used to acquire a first ambient brightness outside the vehicle, a second ambient brightness inside the vehicle, and a target image including a driver's facial area; An analysis module, configured to determine the driving sight direction of the driver by analyzing facial features contained in the target image; A first adjustment module is used for adjusting the screen brightness of the target display screen to the first ambient brightness in response to the driving sight line direction changing from the area outside the vehicle to the display area of ​​the target display screen; The second adjustment module is used to adjust the screen brightness of the target display screen to the second ambient brightness in response to the driving sight line direction changing from the display area of ​​the target display screen to the area outside the vehicle.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A vehicle, characterized in that: include: The electronic device as claimed in claim 9.

Images