Control method and device of electronic rearview mirror, vehicle and storage medium

Abstract

The invention provides a control method and device of an electronic rearview mirror, a vehicle and a storage medium, and belongs to the technical field of vehicles. The method comprises the steps that head information of a driver is determined through an image sensor arranged in a vehicle, and the head information comprises the head observation direction; eyeball information of the driver is determined through a camera arranged in the vehicle, and the eyeball information comprises the position of an eyeball fixation point; and controlling the angle and focal length of a lens on the electronic rearview mirror based on the head information and the eyeball information. The visual effect is optimized, and the driver can be effectively assisted in driving, so that the driving safety is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a control method, device, vehicle, and storage medium for an electronic rearview mirror. Background Art

[0002] The rapid development of the modern automotive industry has driven continuous innovation in vehicle safety technologies, particularly in the design and implementation of vehicle assistance systems. As an integral component of driving safety, rearview mirrors have evolved from traditional manually adjustable mirrors to electronic mirrors.

[0003] Although electronic rearview mirrors have many advantages, they still have limitations that may affect the driver's safe driving. Summary of the Invention

[0004] The embodiments of the present application provide a control method, device, vehicle, and storage medium for an electronic rearview mirror, which optimize visual effects and can effectively assist the driver in driving, thereby improving driving safety. The technical solution is as follows:

[0005] According to one aspect of an embodiment of the present application, a method for controlling an electronic rearview mirror is provided, the method comprising:

[0006] Determining the driver's head information, including the head viewing direction, through an image sensor configured in the vehicle;

[0007] Determining the driver's eye information by using a camera configured in the vehicle, the eye information including the position of the eye's gaze point;

[0008] Based on the head information and the eyeball information, the angle and focal length of the lens on the electronic rearview mirror are controlled.

[0009] According to another aspect of an embodiment of the present application, a control device for an electronic rearview mirror is provided. The control device is configured on a vehicle and includes:

[0010] a head tracking module, configured to determine the driver's head information using an image sensor configured in the vehicle, the head information including the head viewing direction;

[0011] an eye tracking module, configured to determine the driver's eye information through a camera configured in the vehicle, the eye information including the position of the eye's gaze point;

[0012] A rearview mirror adjustment module is used to control the angle and focal length of the lens on the electronic rearview mirror based on the head information and the eye information.

[0013] Optionally, the head tracking module is used to determine the driver's head position and head movement direction through the image sensor; and determine the driver's head viewing direction based on the head position and the head movement direction.

[0014] Optionally, the eye tracking module is used to capture the driver's eye image through the camera; determine the driver's eye gaze direction based on the eye gaze direction and the position of the camera, and determine the position of the driver's eye gaze point.

[0015] Optionally, the eye tracking module is used to identify the position of the pupil center and the position of the corneal reflection point in the eye based on the eye image; and determine the eye gaze direction based on the relative relationship between the position of the pupil center and the position of the corneal reflection point.

[0016] Optionally, the rearview mirror adjustment module is used to adjust the angle of the lens so that the image captured by the lens after adjustment is consistent with the observation direction of the head; and adjust the focal length of the lens so that the focus position of the lens after adjustment is consistent with the position of the eye gaze point.

[0017] Optionally, the rearview mirror adjustment module is used to determine parameter information of the lens based on the head information and the eye information, the parameter information including a target angle and a target focal length; adjust the lens to the target angle, and adjust the focal length of the lens to the target focal length.

[0018] Optionally, the control device further includes:

[0019] An initialization module is used to determine target parameter information based on multiple sets of parameter information used by the electronic rearview mirror; and determine the target parameter information as initialization parameter information, and the initialization parameter information is used to initialize the electronic rearview mirror when the vehicle is started.

[0020] According to another aspect of an embodiment of the present application, a vehicle is provided, comprising: an electronic rearview mirror, a display, and a control device for the electronic rearview mirror as described in the above aspect;

[0021] The electronic rearview mirror is provided with a lens, and the display is used to display the images captured by the lens.

[0022] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor of a vehicle to implement the control method of the electronic rearview mirror described in the above aspect.

[0023] According to another aspect of an embodiment of the present application, a computer program product is provided, which includes a computer program code, and the computer program code is stored in a computer-readable storage medium. A vehicle processor reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code to implement the control method of the electronic rearview mirror as described in the above aspects.

[0024] The solution provided in the embodiment of the present application can determine the driver's head information and eye information, and control the angle and focal length of the lens on the electronic rearview mirror based on the driver's head information and eye information, so that the picture captured by the lens after adjustment includes the area the driver is observing, so that the area the driver is observing can be clearly displayed on the display. The user can determine the specific situation of the area by watching the display, which optimizes the visual effect and can effectively assist the driver in driving, thereby improving driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 This is a flow chart of a method for controlling an electronic rearview mirror provided in an embodiment of the present application;

[0027] Figure 2 is a flow chart of another method for controlling an electronic rearview mirror provided in an embodiment of the present application;

[0028] Figure 3 This is a schematic diagram of a system workflow provided by an embodiment of the present application;

[0029] Figure 4 This is a data flow diagram of various modules in a control device for an electronic rearview mirror provided in an embodiment of the present application;

[0030] Figure 5 1 is a schematic structural diagram of a control device for an electronic rearview mirror provided in an embodiment of the present application;

[0031] Figure 6 This is a schematic structural diagram of a vehicle provided in an embodiment of the present application;

[0032] Figure 7 This is a schematic diagram of a system usage process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0034] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another.

[0035] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0036] It should be noted that the information (including but not limited to information used for processing, storage, and display) and data (including but not limited to data used for processing, storage, and display) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the interactive information involved in this application is obtained with full authorization.

[0037] Figure 1 This is a flow chart of a method for controlling an electronic rearview mirror provided in an embodiment of the present application. The embodiment of the present application is performed by a vehicle. The vehicle is internally equipped with an image sensor, a camera, and a display, and externally equipped with an electronic rearview mirror. The electronic rearview mirror is equipped with a lens connected to the display, which displays images captured by the lens in real time.

[0038] See also Figure 1 , the method comprises the following steps:

[0039] 101. The driver's head information is determined by the image sensor configured in the vehicle, and the head information includes the head viewing direction.

[0040] The image sensor can capture the three-dimensional position information of the driver's head in real time, and then determine the head viewing direction based on the three-dimensional position information of the head. The head viewing direction is the direction the driver's head is facing, so it can be considered that the driver is observing the area in this direction.

[0041] The image sensor can be configured in front of the driver's seat in the vehicle, such as on the A-pillar in front of the driver's seat, or above the instrument panel. The image sensor faces the driver's seat and can capture the three-dimensional position information of the driver's head.

[0042] 102. The driver's eye information is determined by a camera installed in the vehicle, and the eye information includes the position of the eye's gaze point.

[0043] Among them, the camera can capture the driver's eye image in real time, and thus determine the position of the eye's gaze point based on the eye image. Therefore, it can be considered that the driver is looking at the position of the eye's gaze point and observing the object at that position.

[0044] The camera can be placed in a position in the vehicle where the driver's eyes can be clearly seen. For example, the camera can be installed on the top of the vehicle and hung slightly above and in front of the driver's eyes. This will not affect the driver's view of the front, but can also capture the driver's eye image.

[0045] 103. Based on the head information and eye information, control the angle and focal length of the lens on the electronic rearview mirror.

[0046] The electronic rearview mirror's lens captures images and displays them on a display inside the vehicle for the driver to see. The vehicle controls the lens' angle and focal length based on head and eye information, ensuring the image captured by the lens includes the area the driver is viewing, allowing the display to clearly show the area the driver is looking at.

[0047] For example, when the driver's head turns to the left, the angle of the left electronic rearview mirror will automatically adjust to provide a wider left rear view.

[0048] The method provided in the embodiment of the present application can determine the driver's head information and eye information, and control the angle and focal length of the lens on the electronic rearview mirror based on the driver's head information and eye information, so that the picture captured by the lens after adjustment includes the area the driver is observing, so that the area the driver is observing can be clearly displayed on the display. The user can determine the specific situation of the area by watching the display, which optimizes the visual effect and can effectively assist the driver in driving, thereby improving driving safety.

[0049] Figure 2 This is a flow chart of a method for controlling an electronic rearview mirror provided in an embodiment of the present application. The embodiment of the present application is performed by a vehicle. The vehicle is internally equipped with an image sensor, a camera, and a display, and externally equipped with an electronic rearview mirror. The electronic rearview mirror is equipped with a lens connected to a display for displaying images captured by the lens.

[0050] See also Figure 2 , the method comprises the following steps:

[0051] 201. The driver's head position and head movement direction are determined by the image sensor configured in the vehicle.

[0052] 202. Determine the driver's head viewing direction based on the head position and the head movement direction.

[0053] The image sensor can capture the driver's head's three-dimensional position information in real time, thereby determining the driver's head position and movement direction based on the three-dimensional position information. The head movement direction represents the head's movement trend, and the head's observation direction can be determined based on the head position and movement direction. The head observation direction is the direction the driver's head is facing, so it can be assumed that the driver is observing the area in that direction.

[0054] Optionally, a direction having the head position as a vertex, being perpendicular to the head movement direction and facing forward of the head is determined as the head observation direction.

[0055] Optionally, considering that there may be a certain delay in the process of the vehicle controlling the electronic rearview mirror, if the electronic rearview mirror is adjusted according to the user's current head viewing direction, the user's head position may have changed when the electronic rearview mirror adjustment is completed, which may cause the user to be dissatisfied with the adjustment effect of the electronic rearview mirror. Therefore, at the current moment, after determining the driver's head position and head movement direction through the image sensor, the head position and head movement direction at the next moment can be predicted. Based on the head position and head movement direction at the next moment, the head viewing direction at the next moment can be determined. Subsequently, the electronic rearview mirror can be adjusted according to the head viewing direction at the next moment, so that after the adjustment is completed, the image captured by the electronic rearview mirror includes the area that the driver is viewing at the next moment.

[0056] The interval between any two moments can be determined by the vehicle based on the delay required to control the electronic rearview mirror. The vehicle can set a typical head movement distance for a user within the interval. Based on the current head position and direction, the vehicle can determine the head position and direction after moving the head that distance in the direction. This will be the predicted head position and direction at the next moment.

[0057] Optionally, step 201 may be performed in real time or at predetermined intervals, wherein the predetermined interval may be 0.5 seconds or 1 second, etc., and may be set by the vehicle by default or by the driver in the vehicle, and this embodiment of the application does not limit this.

[0058] 203. Use a camera configured in the vehicle to capture an image of the driver's eyeballs, and determine the driver's eye gaze direction based on the eyeball image.

[0059] The eye gaze direction is the driver's line of sight, indicating the direction the driver is observing.

[0060] Among them, the camera can be a high-precision infrared camera, which can better capture the contrast between the pupil and the white of the eye, while avoiding interference from ambient light. It can be applied to a variety of driving conditions and will not be affected by external environments such as weather and light conditions.

[0061] Optionally, determining the driver's eye gaze direction based on an eye image includes: identifying the position of the pupil center and the corneal reflection point in the eye based on the eye image; and determining the eye gaze direction based on the relative relationship between the pupil center and the corneal reflection point. An image processing algorithm can be used to detect the outline of the pupil edge, thereby identifying the position of the pupil center. Furthermore, the infrared light source emitted by the infrared camera forms a reflection point on the cornea, namely the corneal reflection point. The infrared camera can capture the position of the corneal reflection point as a reference point. By calculating the relative positional relationship between the pupil center and the corneal reflection point, combined with a known eye model, the eye gaze direction can be inferred.

[0062] 204. Determine the position of the driver's eye gaze point based on the eye gaze direction and the position of the camera.

[0063] Since the determined eye gaze direction is determined relative to the position of the pupil center, the vehicle can also determine the actual eye gaze direction based on the position of the camera and the relative eye gaze direction, thereby determining the position of the eye gaze point in the display in the actual eye gaze direction.

[0064] For example, a ray with the center of the pupil in the eye as the vertex and along the eye's gaze direction is determined, and the intersection of the ray and the display is determined as the position of the eye's gaze point, indicating that the driver is observing an object in the display located at the eye's gaze point.

[0065] For example, when the vehicle starts, the eye-tracking camera begins capturing the driver's eye images. The system then undergoes initial calibration, and the camera continuously captures eye images at a high frame rate to ensure real-time gaze detection. An image processing algorithm is used to extract the position of the pupil center and the corneal reflection point from the real-time captured eye image. The driver's eye gaze direction, or line of sight, is calculated based on these positions. The line of sight and the camera's position are then mapped to the electronic rearview mirror's display, predicting the eye gaze point's position on the display. This position is then output to the central control unit, which then provides a basis for subsequent lens focal length adjustments.

[0066] 205. Adjust the angle of the lens so that the image captured by the lens after adjustment is consistent with the viewing direction of the head, and adjust the focal length of the lens so that the focus position of the lens after adjustment is consistent with the position of the eye's gaze point.

[0067] After determining the head's viewing direction and the location of the driver's eye gaze point, at least one of the lens's angle and focal length can be adjusted so that the image captured by the lens aligns with the head's viewing direction and the lens's focal point aligns with the driver's eye gaze point. This ensures that when the display shows the image captured by the lens, it captures the image in the head's viewing direction, with the image's focal point being the driver's eye gaze point. This allows the driver to view the location of the driver's eye gaze point and the surrounding area in detail by viewing the display.

[0068] The lens angle is adjusted via an electric motor or stepper motor, ensuring that the electronic rearview mirror's viewing angle dynamically adapts to the driver's needs. After determining the driver's eye gaze point, the required focal length is calculated based on the distance information. This information is then sent to the lens controller, which drives a motor or other actuator within the lens to move the lens assembly, dynamically adjusting the lens's focal length. This ensures that the object at the driver's eye gaze is clearly imaged on the focal plane of the image sensor in the lens, ensuring that the object the driver is looking at is clearly visible on the display connected to the electronic rearview mirror. The image clarity on the display can then be monitored to ensure that the desired effect of the focal length adjustment has been achieved.

[0069] By adjusting the optical focal length of the electronic rearview mirror's lens, the depth of field between objects at different distances can be enhanced, making it easier for the driver to judge the actual distance of objects. For example, when the driver is looking at a distant object, the system will adjust the focus to enhance the clarity of the distant object and provide better distance perception. Moreover, the focal length adjustment is dynamic and continuous, constantly adjusting the lens's focus according to the driver's line of sight to ensure that objects at different distances are in clear focus and can be clearly displayed on the display.

[0070] Optionally, based on the head information and eye information, the parameter information of the lens is determined, the parameter information including the target angle and the target focal length, the lens is adjusted to the target angle, and the focal length of the lens is adjusted to the target focal length. The target angle refers to the angle at which the image captured by the lens is consistent with the head's observation direction, and the target focal length refers to the focal length that makes the focal position of the lens consistent with the position of the eye's gaze point when the lens is shooting at the target angle. By adjusting the lens to the target angle and adjusting the focal length of the lens to the target focal length, the image captured by the lens after adjustment can be consistent with the head's observation direction, and the focal position of the lens after adjustment can be consistent with the position of the eye's gaze point.

[0071] Optionally, the method further includes: determining target parameter information based on multiple sets of parameter information used by the electronic rearview mirror; determining the target parameter information as initialization parameter information, and the initialization parameter information is used to initialize the electronic rearview mirror when the vehicle is started.

[0072] Each time the electronic rearview mirror is adjusted, a set of parameter information is used, and each set of parameter information includes a target angle and a target focal length of the lens. Multiple sets of parameter information used by the electronic rearview mirror can be collected, and target parameter information can be determined based on the multiple sets of parameter information. The target parameter information can be the average parameter information of the multiple sets of parameter information, i.e., the average lens angle and the average lens focal length. Alternatively, the target parameter information can also be the parameter information that is used the most times among the multiple sets of parameter information, or the target parameter information can also be the parameter information that has been modified by the driver among the multiple sets of parameter information, etc., which is not limited in the embodiments of the present application. The target parameter information can be considered as the parameter information that the driver most wants to use, so the target parameter information is determined as the initialization parameter information, so that the electronic rearview mirror can be initialized according to the initialization parameter information when the vehicle is started next time, so that the initial angle of the lens when the vehicle is started next time is the lens angle in the initialization parameter information, and the initial focal length of the lens is the lens focal length in the initialization parameter information.

[0073] Among them, the process of determining the initialization parameter information can be executed each time the vehicle is turned off, or after a certain period of time, which can be 12 hours or 24 hours, etc., and the embodiment of the present application does not limit this.

[0074] The embodiments of the present application record the driver's operating behaviors in different scenarios, including head movements and eye movement patterns, and continuously learn and optimize the parameter information of the electronic rearview mirror lens, so that it can adapt to the driver's personal habits. For example, some drivers prefer a wider field of view, while some drivers pay more attention to details. The parameter settings are dynamically adjusted according to the driver's personal habits to ensure that the image displayed on the display meets the driving needs to the greatest extent, providing a more personalized user experience.

[0075] Optionally, the vehicle is equipped with a head tracking module, an eye tracking module, a central control unit, and a rearview mirror adjustment module. The head tracking module includes an image sensor that captures three-dimensional head position information and transmits it to the central control unit. The central control unit determines the head position and direction of movement based on the three-dimensional head position information, thereby determining the head's viewing direction. The eye tracking module includes a camera and a processor. The camera captures an image of the driver's eyes and transmits it to the processor. The processor determines the driver's eye gaze direction based on the eye image and transmits it to the central control unit. The central control unit then determines the driver's eye gaze point based on the eye gaze direction and objects in the surrounding area, and determines lens parameter information, including a target angle and a target focal length. The central control unit transmits this parameter information to the rearview mirror adjustment module. Based on this parameter information, the rearview mirror adjustment module adjusts the electronic rearview mirror's lens to the target angle and focal length. The lens captures images according to the adjusted target angle and focal length and transmits them to the display, where the captured images are displayed.

[0076] Exemplarily, the rearview mirror adjustment module includes an angle adjustment unit and a focal length adjustment unit. The central control unit sends a target angle to the angle adjustment unit, and the angle adjustment unit adjusts the angle of the lens of the electronic rearview mirror to the target angle. The central control unit sends a target focal length to the focal length adjustment unit, and the focal length adjustment unit adjusts the focal length of the lens of the electronic rearview mirror to the target focal length.

[0077] Exemplarily, the vehicle is also equipped with a feedback and learning module, which can record multiple sets of parameter information used, and determine the target parameter information corresponding to the multiple sets of parameter information as initialization parameter information. The initialization parameter information is used to initialize the electronic rearview mirror when the vehicle is started.

[0078] Figure 3 This is a schematic diagram of a system workflow provided by an embodiment of the present application. Figure 4 This is a data flow diagram of each module in a control device for an electronic rearview mirror provided in an embodiment of the present application, see Figure 3 and Figure 4The workflow includes: the head tracking module collects head tracking data, including the driver's head position and head movement direction, and sends it to the central control unit. The eye tracking module collects eye tracking data, including the position of the eye gaze point, and sends it to the central control unit. The central control unit integrates and makes decisions based on the head tracking data and eye tracking data, determines the target angle and target focal length of the electronic rearview mirror lens, sends the target angle to the angle adjustment unit, and sends the target focal length to the focal length adjustment unit. The angle adjustment unit adjusts the angle of the lens to the target angle, and the focal length adjustment unit adjusts the focal length of the lens to the target focal length. Afterwards, the image captured by the lens according to the adjusted target angle and target focal length is displayed on the monitor.

[0079] Figure 5 This is a schematic diagram of the structure of a control device for an electronic rearview mirror provided in an embodiment of the present application, see Figure 5 The control device is configured on a vehicle and includes:

[0080] The head tracking module 501 is used to determine the driver's head information through the image sensor configured in the vehicle, and the head information includes the head viewing direction;

[0081] An eye tracking module 502 is configured to determine the driver's eye information using a camera configured in the vehicle, where the eye information includes the location of the eye's gaze point;

[0082] The rearview mirror adjustment module 503 is used to control the angle and focal length of the lens on the electronic rearview mirror based on the head information and the eye information.

[0083] Optionally, the head tracking module 501 is used to determine the driver's head position and head movement direction through an image sensor; and determine the driver's head viewing direction based on the head position and head movement direction.

[0084] Optionally, the eye tracking module 502 is used to capture an eye image of the driver through a camera; determine the driver's eye gaze direction based on the eye image; and determine the position of the driver's eye gaze point based on the eye gaze direction and the position of the camera.

[0085] Optionally, the eye tracking module 502 is used to identify the position of the pupil center and the position of the corneal reflection point in the eye based on the eye image; and determine the eye gaze direction based on the relative relationship between the position of the pupil center and the position of the corneal reflection point.

[0086] Optionally, the rearview mirror adjustment module 503 is used to adjust the angle of the lens so that the image captured by the lens after adjustment is consistent with the head observation direction; adjust the focal length of the lens so that the focus position of the lens after adjustment is consistent with the position of the eye gaze point.

[0087] Optionally, the rearview mirror adjustment module 503 is used to determine lens parameter information based on the head information and eye information, the parameter information including target angle and target focal length; adjust the lens to the target angle, and adjust the focal length of the lens to the target focal length.

[0088] Optionally, the control device further includes:

[0089] The initialization module is used to determine target parameter information based on multiple sets of parameter information used by the electronic rearview mirror; the target parameter information is determined as initialization parameter information, and the initialization parameter information is used to initialize the electronic rearview mirror when the vehicle is started.

[0090] The device provided in the embodiment of the present application can determine the driver's head information and eye information, and control the angle and focal length of the lens on the electronic rearview mirror based on the driver's head information and eye information, so that the picture captured by the lens after adjustment includes the area the driver is observing, so that the area the driver is observing can be clearly displayed on the display. The user can determine the specific situation of the area by watching the display, which optimizes the visual effect and can effectively assist the driver in driving, thereby improving driving safety.

[0091] Figure 6 This is a schematic diagram of the structure of a vehicle provided in an embodiment of the present application, see Figure 6 The vehicle includes an electronic rearview mirror 601, a display 602, and a control device for the electronic rearview mirror as shown in the above embodiment. The electronic rearview mirror 601 is equipped with a lens 611, and the display 602 is used to display the image captured by the lens 611. The control device includes an image sensor 613 and a camera 623.

[0092] Electronic rearview mirrors 601 are mounted on either side of the vehicle, similar to the placement of traditional rearview mirrors. Displays 602 are mounted inside the vehicle, in front of the driver's seat. One or more displays 602 can be configured on the vehicle. For example, if a vehicle has two electronic rearview mirrors, two displays can be configured inside the vehicle, one for each electronic rearview mirror, to display the image captured by the electronic rearview mirror's lens.

[0093] Image sensor 613 is mounted on the A-pillar or above the dashboard of the vehicle's cab. Image sensor 613 can be integrated with a DMS (Driver Monitor System). Camera 623 is mounted on the roof of the vehicle, in front of the cab. Alternatively, it can be mounted above display 602. When the driver is viewing display 602, camera 623 can capture the driver's eye information.

[0094] Figure 7This is a schematic diagram of a system usage process provided by an embodiment of the present application, see Figure 7 After the driver gets in the vehicle and starts it, the system checks whether the driver is a registered driver. If so, the system uses the full functionality of the CMS (Camera Monitor System), including the safe driving features provided by the embodiments of the present application. If not, the system determines whether the electronic rearview mirror needs to be calibrated.

[0095] If calibration is required, the electronic rearview mirror is system-aligned and calibrated, i.e., the electronic rearview mirror's parameter information is calibrated, including information such as the angle and focal length of the electronic rearview mirror's lens. After successful calibration, the full functionality of the CMS system can also be used. If calibration is not required, the basic functionality of the CMS system is used, excluding the safe driving functions provided by the embodiments of this application.

[0096] Additionally, an initial calibration is required before first use. This process typically involves the following steps: the driver adjusts the driver's seat and electronic rearview mirror according to the system's instructions, and a preliminary calibration is performed based on the driver's head and eye position data. After calibration is complete, the data is saved and used as a reference for subsequent adjustments. The calibration system should be customized for each vehicle model to ensure the accuracy and reliability of the calibration data.

[0097] By adopting the above method, the system functions can be used flexibly for different users. The full functions of the system are used by default for registered drivers, and for other drivers, the full functions of the system can be used after the system is aligned and calibrated. Without the system alignment and calibration, the basic functions of the vehicle can be used. This improves flexibility, is suitable for a variety of driving scenarios, and expands the scope of use.

[0098] Among related technologies, electronic rearview mirrors improve the convenience of adjustment, but there are still some key technical limitations and challenges, especially in adapting to the visual needs of individual drivers.

[0099] Fixed focus: Traditional rearview mirrors naturally convey a sense of relative distance (i.e., depth of field) by reflecting light from the physical world. Electronic rearview mirrors, on the other hand, capture images and display them on the vehicle's display. However, electronic rearview mirror displays are typically flat, lacking depth of field and thus unable to clearly distinguish between near and far objects. This can make it difficult for the driver to judge the actual distance of objects. For example, pedestrians, other vehicles, or obstacles behind the vehicle may appear farther or closer, increasing the risk of misjudgment.

[0100] Fixed Perspective: While electronic rearview mirrors offer a fixed perspective, which improves visual stability to a certain extent, they overlook the critical factor of the driver's variability in their field of vision. Minor body movements or seat adjustments during driving often require the driver to readjust the rearview mirror for optimal viewing angles, a requirement often difficult to meet with traditional electronic rearview mirror systems. Consequently, the fixed nature of these systems can lead to visual discomfort or blind spots, significantly compromising driving safety, particularly in emergency avoidance or complex driving situations.

[0101] Personalized Adaptation: Electronic rearview mirrors typically lack personalized adaptation to the driver's visual preferences. For example, different drivers have different requirements for the clarity and depth of focus of the rearview mirror image. These requirements may vary from person to person and may also change with changes in the external environment (such as weather and lighting conditions). Existing electronic rearview mirror technology fails to fully account for these factors and lacks dynamic response to changes in the driver's line of sight. It usually only provides a standardized view output, resulting in visual information that may not fully meet the driver's current observation needs and lacks the necessary flexibility and adaptability.

[0102] These technical deficiencies highlight the limitations of electronic rearview mirrors in dynamic visual adaptation. These limitations can lead to insufficient driver information during high-speed driving, frequent lane changes, nighttime driving, or in adverse weather conditions, hindering the timeliness and accuracy of driver decision-making. Therefore, to improve driving safety and enhance the driving experience, there is an urgent need to develop an electronic rearview mirror system that can automatically adjust the lens angle and focal length based on the driver's head position and gaze direction.

[0103] The embodiments of the present application utilize head tracking technology and eye tracking technology to dynamically adjust the lens angle and focal length of the electronic rearview mirror, achieving the following technical effects:

[0104] 1. Enhanced depth of field perception: By adjusting the focal length of the electronic rearview mirror in real time, the depth of field effect can be dynamically optimized according to the driver's line of sight. This allows the driver to more clearly perceive the distance and spatial position of objects behind them when observing them on the display, reducing misjudgments and visual illusions. This significantly improves driving safety, especially in complex driving scenarios (such as at night and in bad weather).

[0105] 2. Dynamic Viewing Angle Adjustment: The electronic rearview mirror automatically adjusts its lens angle based on the driver's head position, ensuring that the mirror's display aligns with the driver's actual viewing needs. This dynamic adjustment significantly improves the driver's field of view and reduces blind spots caused by viewing angle limitations, facilitating safer maneuvers such as merging, overtaking, and reversing. For example, when reversing, if the driver's head moves to see a lower view of the tires, the electronic rearview mirror's lens angle will automatically adjust, optimizing the user experience.

[0106] 3. Improved User Convenience: Through the adaptive learning module, the system continuously optimizes and adjusts its strategies, gradually adapting to the driver's individual habits and providing a more user-friendly experience. For example, the system can automatically adjust the lens angle and focal length based on the driver's usage habits, eliminating the need for frequent manual adjustments and improving convenience and comfort during driving.

[0107] 4. Reduce driver fatigue: Through precise angle and focal length adjustment, the driver can obtain the required visual field information without frequently turning his head or eyes. This not only reduces the driver's operating burden, but also reduces the fatigue caused by long-term driving.

[0108] The present application also provides a computer-readable storage medium storing at least one program code, which is loaded and executed by a vehicle processor to implement the electronic rearview mirror control method described in the above embodiment. The computer-readable storage medium can be a memory. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage terminal, etc.

[0109] An embodiment of the present application also provides a computer program product, which includes computer program code, the computer program code is stored in a computer-readable storage medium, the vehicle's processor reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code to implement the control method of the electronic rearview mirror as in the above embodiment.

[0110] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0111] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for controlling an electronic rearview mirror, characterized in that: The method comprises: Determining the driver's head information, including the head viewing direction, through an image sensor configured in the vehicle; Determining the driver's eye information by using a camera configured in the vehicle, the eye information including the position of the eye's gaze point; Based on the head information and the eyeball information, the angle and focal length of the lens on the electronic rearview mirror are controlled.

2. The method according to claim 1, characterized in that The driver's head information is determined by an image sensor configured in the vehicle, and the head information includes the head viewing direction, including: determining the driver's head position and head movement direction by using the image sensor; A head viewing direction of the driver is determined based on the head position and the head movement direction.

3. The method according to claim 1, characterized in that The driver's eye information is determined by a camera configured in the vehicle, wherein the eye information includes the position of the eye gaze point, including: capturing an eye image of the driver via the camera; determining the driver's eye gaze direction based on the eye image; The position of the driver's eye gaze point is determined based on the eye gaze direction and the position of the camera.

4. The method according to claim 3, characterized in that The determining the driver's eye gaze direction based on the eye image includes: Based on the eyeball image, identifying the position of the pupil center and the position of the corneal reflection point in the eyeball; The eye gaze direction is determined based on the relative relationship between the position of the pupil center and the position of the corneal reflection point.

5. The method according to any one of claims 1 to 4, characterized in that The controlling the angle and focal length of the lens on the electronic rearview mirror based on the head information and the eye information includes: Adjusting the angle of the lens so that the image captured by the lens after adjustment is consistent with the viewing direction of the head; The focal length of the lens is adjusted so that the focus position of the lens after adjustment is consistent with the position of the eyeball's gaze point.

6. The method according to any one of claims 1 to 4, characterized in that The controlling the angle and focal length of the lens on the electronic rearview mirror based on the head information and the eye information includes: Determining parameter information of the lens based on the head information and the eye information, the parameter information including a target angle and a target focal length; The lens is adjusted to the target angle, and the focal length of the lens is adjusted to the target focal length.

7. The method according to claim 6, characterized in that The method further comprises: determining target parameter information based on multiple sets of parameter information used by the electronic rearview mirror; The target parameter information is determined as initialization parameter information, and the initialization parameter information is used to initialize the electronic rearview mirror when the vehicle is started.

8. A control device for an electronic rearview mirror, characterized in that: The control device is configured on a vehicle, and includes: a head tracking module, configured to determine the driver's head information using an image sensor configured in the vehicle, the head information including the head viewing direction; an eye tracking module, configured to determine the driver's eye information through a camera configured in the vehicle, the eye information including the position of the eye's gaze point; A rearview mirror adjustment module is used to control the angle and focal length of the lens on the electronic rearview mirror based on the head information and the eye information.

9. A vehicle, characterized in that: The vehicle comprises: an electronic rearview mirror, a display, and a control device for the electronic rearview mirror according to claim 8; The electronic rearview mirror is provided with a lens, and the display is used to display the images captured by the lens.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by a processor of a vehicle to implement the control method of the electronic rearview mirror according to any one of claims 1 to 7.

Images