A method of rearview mirror control and related apparatus

Abstract

The embodiment of the application provides a rearview mirror system control method, belongs to the intelligent cabin field, is used for adjusting a central rearview mirror to the optimal observation angle of a driver when determining that a target object has the intention of viewing the rearview mirror, and can be applied to intelligent vehicles, automatic driving vehicles, new energy vehicles and the like. The embodiment of the application comprises the following steps: acquiring facial feature information of the target object; if the facial feature information satisfies an instruction triggering condition, generating a control instruction for the central rearview mirror; and in response to the control instruction, adjusting the central rearview mirror according to the relative position of the central rearview mirror and the target object.

Description

[0001] This application is a divisional application. The original application has the application number 2020800043142 and the original application date is December 16, 2020. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence, specifically to a method and related equipment for controlling a rearview mirror. The method determines in a timely manner whether the driver intends to look at the central rearview mirror based on facial feature information, and adjusts the central rearview mirror according to the relative position of the central rearview mirror and the target object. Background Technology

[0003] Car rearview mirrors include interior and exterior mirrors. Exterior mirrors include the mirrors on the left and right sides of the front of the car, while the interior mirror is located in the center of the front of the car and is also known as the center rearview mirror. While driving, the driver can effectively judge the distance of vehicles behind by looking at the image in the center rearview mirror. The angle between the center rearview mirror and the driver's eyes needs to be adjusted to obtain the best field of vision and angle. Adjusting the center rearview mirror while driving requires the driver to take one hand off the steering wheel and keep their eyes on the mirror, causing the driver's attention to be diverted from the road, posing a traffic safety hazard. Currently, center rearview mirrors can be equipped with automatic adjustment mechanisms to improve driving safety.

[0004] In existing technologies, the automatic adjustment method for the central rearview mirror generally uses voice interaction commands to trigger it; that is, when the driver issues a voice command to the central rearview mirror, the mirror begins to adjust. However, when passengers in the car are talking, it can affect the accuracy of the driver's voice command input, and the voice recognition process during voice command triggering is relatively long, which cannot meet the driver's needs for adjusting the angle of the central rearview mirror in a timely manner. Summary of the Invention

[0005] This application provides a method and related equipment for controlling a rearview mirror, which is used to adjust the central rearview mirror to the driver's optimal viewing angle in a timely manner after determining that the driver intends to view the central rearview mirror based on the driver's facial feature information.

[0006] The first aspect of this application provides a method for controlling a rearview mirror:

[0007] The rearview mirror system is pre-installed inside the driver's cab. When the rearview mirror system is turned on, it continuously acquires facial feature information of the target object located in the driver's seat.

[0008] The program in the rearview mirror system will preset trigger conditions. If the facial feature information of the driver in the driver's seat obtained by the rearview mirror system meets the command trigger conditions, the rearview mirror system will generate control commands for the central rearview mirror.

[0009] The rearview mirror system responds to control commands generated after triggering conditions are met. These control commands are used to adjust the central rearview mirror based on the relative position of the central rearview mirror and the target object.

[0010] Understandably, the rearview mirror system can promptly determine the driver's intention to check the central rearview mirror based on facial feature information, and adjust the central rearview mirror according to the relative position of the central rearview mirror and the target object, thus solving the problem of insufficient accuracy and timeliness when voice commands are triggered, and providing users with a more efficient experience.

[0011] Based on the first aspect, the embodiments of this application provide a first implementation of the first aspect:

[0012] The rearview mirror system continuously acquires the facial posture of the target object located in the driver's seat. When the target object is facing the central rearview mirror, the rearview mirror system acquires the effective facial posture indicating that the target object's face is facing the central rearview mirror.

[0013] When the duration of the effective facial posture of the target object facing the central rearview mirror is greater than or equal to a preset duration threshold, the rearview mirror system determines that the facial feature information of the target object meets the triggering conditions for the command to adjust the central rearview mirror.

[0014] Understandably, the rearview mirror system determines whether a target has the intention to look at the central rearview mirror based on the duration of the target's facial posture facing the rearview mirror, which can more directly and accurately determine whether the target has the intention to look at the central rearview mirror.

[0015] Based on the first aspect and the first implementation of the first aspect, this application provides a second implementation of the first aspect:

[0016] The rearview mirror system first acquires a sequence of facial images of the target object. The facial image sequence is a sequence of images in which the single-frame static images acquired by the rearview mirror system are arranged in chronological order.

[0017] When the rearview mirror system detects that the facial pose of each face image in the face image sequence is facing the central rearview mirror, it obtains the effective facial pose of the target object representing the target object's face facing the central rearview mirror.

[0018] Based on any one of the first to second embodiments of the first aspect, this application provides a third embodiment of the first aspect:

[0019] When the rearview mirror system detects that the left and right turning angles and pitch angles of the target object's face in the face image relative to the frontal view are less than a preset angle threshold, it determines that the face in the face image is facing the central rearview mirror.

[0020] Based on any one of the first to third embodiments of the first aspect, this application provides a fourth embodiment of the first aspect:

[0021] The rearview mirror system acquires a sequence of driver's seat images representing the target object being in the driver's seat. The driver's seat image sequence is a sequence of single-frame static images representing the target object being in the driver's seat, arranged in chronological order of acquisition time.

[0022] The rearview mirror system uses an automatic background removal algorithm to remove the background from each driver's seat image in the driver's seat image sequence, capturing only the face image. The processed face image corresponds to the driver's seat image, generating a face image sequence.

[0023] Understandably, the rearview mirror system obtains a face image by using an automatic image matting algorithm to process the driver's seat image, which represents the target object being in the driver's seat. Based on the face image, it determines whether the target object intends to look at the central rearview mirror, thus avoiding interference from other objects in the driver's seat image and improving the accuracy of face recognition.

[0024] Based on any one of the first to fourth embodiments of the first aspect, this application provides a fifth embodiment of the first aspect:

[0025] The rearview mirror system can directly obtain the face image sequence of the target object from the driver's seat image sequence, or it can use a contour estimation engine to remove the background from the driver's seat image sequence, obtain the contour of the seat, and generate a seat image sequence.

[0026] The rearview mirror system uses an automatic background removal algorithm to remove the background and obtain the face image sequence from the seat image sequence.

[0027] Understandably, the rearview mirror system first acquires the seat image from the driver's seat image, and then acquires the face image from the seat image. This avoids interference from the face recognition of the target object caused by the angle of the rearview mirror system's image acquisition device being deflected, thus improving the accuracy of face image acquisition.

[0028] Based on any one of the first to fifth embodiments of the first aspect, this application provides a sixth embodiment of the first aspect:

[0029] The camera in the rearview mirror system can be installed in various ways. If the installation position allows the camera to capture a larger image range, the rearview mirror system can capture an image of the front row of the vehicle's driver's cab.

[0030] The rearview mirror system uses an automatic image matting algorithm and contour estimation to obtain images of the left seat, right seat, and steering wheel from the front row images of the driver's cab.

[0031] The rearview mirror system identifies the seat on the same side as the steering wheel as the driver's seat. If the left seat and steering wheel are on the same side, then the left seat is the driver's seat.

[0032] After determining the driver's seat, the rearview mirror system acquires a sequence of images of the target object's driver's seat.

[0033] Understandably, the rearview mirror system acquires images of the front row of the driver's cab. Based on the images of the left seat, right seat, and steering wheel in the front row of the driver's cab, it determines the driver's position and acquires a sequence of driver's seat images. This can be applied to both left-hand drive and right-hand drive vehicles, expanding its application scope and enabling new functions.

[0034] Based on any one of the first to sixth embodiments of the first aspect, this application provides a seventh embodiment of the first aspect:

[0035] When the rearview mirror system determines that the target object intends to look in the rearview mirror, it will trigger the adjustment of the central rearview mirror angle. First, the geometric point of the human eye position is determined based on the effective facial posture of the face facing the central rearview mirror.

[0036] The rearview mirror system determines the target line connecting the position of the human eye and any target position on the central rearview mirror.

[0037] The rearview mirror system calculates the first projection of the target line onto the vertical normal plane of the central rearview mirror.

[0038] The rearview mirror system acquires a second projection of the target line onto the mirror surface of the central rearview mirror.

[0039] The rearview mirror system calculates the first angle between the first projection and the vertical line of the center rearview mirror.

[0040] The rearview mirror system determines the second angle between the second projection and the mirror normal of the center rearview mirror.

[0041] The rearview mirror system adjusts the first angle to the first preset value and the second angle to the second preset value.

[0042] Based on the first aspect, this application provides an eighth implementation of the first aspect:

[0043] Facial feature information can be the facial posture of the target object or the eye state information of the target object. The rearview mirror system includes an infrared emitter. Different parts of the human eye have different reflectivities to infrared light. The pupil and iris have significantly different reflectivities to infrared light. Under infrared light, a clear infrared image of the pupil can be obtained, and the pupil will produce a bright reflected spot.

[0044] The rearview mirror system acquires an infrared image of the target object's pupil and the reflected light spot from the target object's pupil.

[0045] The rearview mirror system calculates the relative position between the infrared image of the pupil and the reflected light spot of the pupil;

[0046] If the relative position between the infrared image of the pupil and the reflective plate of the pupil meets the preset triggering conditions of the central rearview mirror, then the control command of the central rearview mirror is triggered.

[0047] Understandably, using the target's eye state information to determine whether the target intends to look at the rearview mirror can be faster and more accurate.

[0048] A second aspect of this application provides a rearview mirror system that performs the functions of the rearview mirror system described in the first aspect. These functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions.

[0049] A third aspect of this application provides a rearview mirror system, including a processor, a memory, an input / output device, and a bus;

[0050] The processor, memory, input / output devices are connected to the bus;

[0051] The processor is used to execute the method described in any one of the first aspects above.

[0052] A fourth aspect of this application provides a computer storage medium that stores a program, which, when executed by the computer, performs the method described in any of the first aspects above.

[0053] The fifth aspect of this application provides a computer program product that, when executed on a computer, performs the method described in any one of the first aspects.

[0054] The sixth aspect of this application provides a vehicle that includes a rearview mirror system provided in any of the second or third aspects.

[0055] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: They acquire facial feature information of the target object; when the facial feature information meets the command triggering conditions, they generate a control command for the central rearview mirror and adjust the central rearview mirror according to the relative position of the central rearview mirror and the target object in response to the control command. By promptly determining the driver's intention to check the central rearview mirror based on facial feature information and adjusting the central rearview mirror according to the relative position of the central rearview mirror and the target object, the accuracy and timeliness issues of voice command triggering are solved, providing users with a more efficient experience. Attached Figure Description

[0056] Figure 1 This is a hardware structure diagram of the rearview mirror system;

[0057] Figure 2 This is an interactive diagram of the rearview mirror system modules;

[0058] Figure 3 This is a schematic diagram of one embodiment of the rearview mirror control method in this application.

[0059] Figure 4 This is a schematic diagram of another embodiment of the rearview mirror control method in this application.

[0060] Figure 5 This is a schematic diagram of another embodiment of the rearview mirror control method in this application.

[0061] Figure 6(A) is a schematic diagram of another embodiment of the rearview mirror control method in this application;

[0062] Figure 6(B) is a schematic diagram of another embodiment of the rearview mirror control method in this application;

[0063] Figure 6(C) is a schematic diagram of another embodiment of the rearview mirror control method in this application;

[0064] Figure 6(D) is a schematic diagram of another embodiment of the rearview mirror control method in this application;

[0065] Figure 7(A) is a schematic diagram of another embodiment of the rearview mirror control method in this application;

[0066] Figure 7(B) is a schematic diagram of another embodiment of the rearview mirror control method in this application;

[0067] Figure 7(C) is a schematic diagram of another embodiment of the rearview mirror control method in this application;

[0068] Figure 8 This is a structural diagram of a rearview mirror system in an embodiment of this application;

[0069] Figure 9This is another structural diagram of the rearview mirror system in an embodiment of this application;

[0070] Figure 10 This is another structural diagram of the rearview mirror system in an embodiment of this application. Detailed Implementation

[0071] This application provides a method for controlling a rearview mirror, which can promptly determine the driver's intention to view the central rearview mirror based on facial feature information, and adjust the central rearview mirror according to the relative position of the central rearview mirror and the target object.

[0072] Car rearview mirrors are located on the left and right sides of the front of the car, and also in the front of the interior. They reflect the situation behind, to the sides, and below the car, allowing the driver to see these areas clearly and expanding their field of vision. The center rearview mirror is used to observe road conditions behind, observe rear passengers, or judge distance to vehicles behind.

[0073] With the development of technology, electronic rearview mirrors are becoming a trend. While driving, the driver's head position changes, and the rearview mirror's orientation may not match the driver's viewing angle. Adjusting the rearview mirror requires the driver to take one hand off the steering wheel and keep their eyes on the mirror, causing their attention to wander and posing a traffic safety hazard. Reflections from surrounding objects are also reflected on the electronic rearview mirror's screen, affecting the driver's ability to see the content. Therefore, the trend is towards electronic rearview mirrors that automatically adjust their orientation based on the driver's posture and head position.

[0074] The embodiments of this application can be applied as follows: Figure 1 The vehicle rearview mirror system equipment architecture shown includes a driver monitoring system (DMS) camera 101, a cockpit domain controller (CDC) 102, an electronic rearview mirror 103, and a telematics box (T-BOX) 104.

[0075] The functions of each device in the vehicle's rearview mirror system are described below:

[0076] Driver monitoring system camera 101: The infrared camera installed in front of the driver monitors the driver's driving status in real time, captures the driver's facial image, calculates the facial orientation, and enables the electric drive unit to adjust the rearview mirror to a position that is convenient for the driver to see based on the calculated facial orientation.

[0077] Intelligent Cockpit Domain Controller 102: The intelligent cockpit domain controller is equivalent to the central processing unit of the cockpit. All components in the cockpit, such as the central control, seats, and window regulators, are controlled by the intelligent cockpit domain controller. The rearview mirror in this application is also managed by the intelligent cockpit domain controller.

[0078] Electronic rearview mirror 103: Includes an interior electronic rearview mirror and an exterior electronic rearview mirror. Used to display images captured by an exterior rearview (surround view) camera.

[0079] It should be noted that the term "electronic rearview mirror" generally refers to an electronic screen used to display rearview cameras, and is not entirely equivalent to a traditional rearview mirror.

[0080] Communication Box 104: The communication box is used to connect the car to the internet. The facial recognition and orientation recognition technologies mentioned in this invention can be implemented locally or in the cloud. If implemented in the cloud, the communication box is required to enable network access.

[0081] The driver monitoring system camera 101, intelligent cockpit domain controller 102, electronic rearview mirror 103 and communication box 104 are connected via a controller area network (CAN) bus.

[0082] The embodiments of this application can be applied as follows: Figure 2 The functional architecture of the vehicle rearview mirror system shown includes a camera 201, an electronic control unit (ECU) 203, an electronic rearview mirror 205, a steering motor 204, and a processor 202. The processor 202 includes a rearview mirror viewing intention detection module 2021 and a rearview mirror deflection angle calculation module 2022.

[0083] The functions of each part in the functional architecture of this vehicle's rearview mirror system are described below:

[0084] Camera 201: Used to capture facial images and transmit them to the processor or computing unit for rearview mirror viewing intent detection.

[0085] Electronic control unit 203: Used for electronic control of the vehicle.

[0086] Electronic rearview mirror 205: Electronic rearview mirrors include interior electronic rearview mirrors and exterior electronic rearview mirrors.

[0087] Steering motor 204: Used to adjust the angle of the electronic rearview mirror after receiving control commands.

[0088] Rearview mirror deflection angle calculation module 2022: used to estimate the position of the driver's eyes and calculate the adjustment angle of the rearview mirror after confirming that the driver intends to check the rearview mirror.

[0089] Rearview mirror viewing intent detection module 2021: used to perform target and target contour detection, face detection and face pose estimation on the face image captured by camera 201.

[0090] Based on the above description, the following describes an embodiment of the rearview mirror control method in this application:

[0091] A camera can be pre-installed at a fixed position on the vehicle body to directly acquire the facial image of the target object. Alternatively, a camera fixed to the rearview mirror can capture a larger area of ​​imagery, which can then be processed by an algorithm to obtain the facial image of the target object. These methods are explained below:

[0092] 1. Directly obtain the face image of the target object:

[0093] In this embodiment, the camera in the rearview mirror system is fixed to the vehicle body, facing the driver's face, and directly captures the facial image of the target object. Please refer to [link to relevant documentation]. Figure 3 .

[0094] 301. Obtain the driver's seat image sequence of the target object.

[0095] The rearview mirror system acquires a sequence of driver's seat images over a period of time, and each driver's seat image in the sequence is a single-frame static image.

[0096] 302. Obtain the face image sequence based on the driver's seat image sequence.

[0097] The rearview mirror system detects the acquired driver's seat image sequence, removes the background using an automatic image matting algorithm, and then obtains a face image sequence. Each face image in the face image sequence corresponds to each driver's seat image in the driver's seat image sequence, and each face image is a single-frame static image.

[0098] 303. Detect the left and right turning angles and pitch angles of the face in a face image.

[0099] After acquiring the face image sequence, the rearview mirror system starts the face pose estimation algorithm engine to obtain the face pose of each single frame of static face image in the face image, and obtains the left and right turning angles and pitch angles of the face pose of the face image relative to the face pose in the frontal state.

[0100] In this embodiment, the facial pose in the frontal view indicates that the driver is currently looking in the rearview mirror. If the rearview mirror system detects that the left and right turning angles and the pitch angle of the facial pose in the face image are less than a preset angle threshold, step 304 is executed.

[0101] Understandably, the facial posture in a frontal view has different meanings depending on the camera's location. When the camera is on the steering wheel, the frontal facial posture indicates that the driver is driving attentively; when the camera is on the rearview mirror, the frontal facial posture can also indicate that the driver is looking in the rearview mirror. The specific meaning of the frontal facial posture is not limited.

[0102] It should be noted that when the meaning of facial posture in a frontal view is different, the judgment criteria for left and right turning angles and pitch angles should be adjusted accordingly, but the specific adjustment methods are not limited.

[0103] 304. If the left and right turning angles and pitch angles of the face pose of each face image are less than the angle threshold, then the effective face pose corresponding to the target object is obtained.

[0104] When the rearview mirror system detects that the left and right turning angles and pitch angles of a face image are less than a preset angle threshold, it means that the driver's face is almost directly facing the rearview mirror, and the driver may have the intention to check the rearview mirror. Therefore, the rearview mirror system marks the face posture with left and right turning angles and pitch angles less than the angle threshold as a valid face posture.

[0105] 305. Determine whether the duration of maintaining an effective facial pose is greater than or equal to the duration threshold.

[0106] When the rearview mirror system acquires a valid facial pose, a timer starts counting. If the detected facial pose of a single frame of face image is valid after the timer starts counting, the timer continues counting. If an invalid facial pose is detected, the timer resets to zero, indicating that the target object's facial pose was not facing the rearview mirror for long enough, and the target object's intention to look in the rearview mirror was not valid. When the timer's duration exceeds a preset value, it means that the target object's valid facial pose has been maintained for a period of time, and the target object's intention to look in the rearview mirror is valid, thus triggering step 307.

[0107] 306. If the duration of the valid facial expression is less than the duration threshold, the timer is reset to zero. After the rearview mirror system acquires a valid facial expression indicating that the driver may be looking at the rearview mirror, it determines the duration of the valid facial expression to further determine whether the driver intends to look at the rearview mirror. If the duration of the valid facial expression is less than the duration threshold, the timer is reset to zero, indicating that the rearview mirror determines that the driver's face is turned towards the rearview mirror not for the purpose of looking at the rearview mirror.

[0108] 307. If the duration of maintaining an effective facial pose is greater than or equal to the duration threshold, a control command is triggered.

[0109] When the rearview mirror system determines that the duration of the effective facial posture is maintained exceeds the duration threshold, the rearview mirror system will trigger a control command to prepare for adjusting the rearview mirror angle.

[0110] 308. Determine the position of the human eye based on the effective facial posture.

[0111] The rearview mirror system determines the position of the human eye on the effective facial pose based on the acquired effective facial pose.

[0112] It should be noted that the human eye position point can be a geometric point generated at the center of the human eyeball, or it can be one or more points generated at any position on the human eye, such as a geometric point generated at the edge of the human eye. The specific location is not limited here.

[0113] 309. Determine the target line based on the position of the human eye and the target position on the central rearview mirror.

[0114] The rearview mirror system determines the line connecting the two points based on the already determined position of the human eye and the target position on the central rearview mirror.

[0115] It should be noted that the target line can be a direct line between a single eye position point and the target position point, or it can be a line between two eye position points and the target position point. For example, the line connecting the midpoint of the line connecting two eye position points and the target position point. No specific limitation is made here.

[0116] It should be noted that the target position point on the central rearview mirror can be a geometric point on the vertical line of the central rearview mirror surface, or it can be a geometric point at any position on the central rearview mirror surface. For example, the target position point can be the center of the central rearview mirror surface. There is no specific limitation here.

[0117] 310. Obtain the first projection of the target line onto the normal plane perpendicular to the mirror surface of the central rearview mirror.

[0118] After determining the target line, the rearview mirror system obtains the first projection of the target line onto the vertical normal plane of the central rearview mirror.

[0119] 311. Obtain the second projection of the target line onto the mirror surface of the central rearview mirror.

[0120] After determining the target line, the rearview mirror system obtains a second projection of the target line onto the mirror surface of the central rearview mirror.

[0121] 312. Determine the first included angle between the first projection and the vertical line of the center rearview mirror.

[0122] The rearview mirror system determines the first angle between the first projection and the vertical line of the center rearview mirror.

[0123] 313. Determine the second included angle between the second projection and the mirror normal of the central rearview mirror.

[0124] The rearview mirror system determines the second angle between the second projection and the vertical line of the center rearview mirror.

[0125] 314. Adjust the first angle of the central rearview mirror to the first preset value.

[0126] Based on the actual confirmation of the human eye position, target position, and target connection line in actual use, a first preset value is determined, and the first angle of the central rearview mirror is adjusted to the first preset value.

[0127] 315. Adjust the second angle of the central rearview mirror to the second preset value.

[0128] Based on the actual confirmation of the human eye position, target position, and target connection line in actual use, a second preset value is determined, and the second included angle of the central rearview mirror is adjusted to the second preset value.

[0129] In this embodiment, the rearview mirror system directly acquires the facial image of the target object through a camera at a fixed position. Based on the facial image, when the facial posture obtained from the facial image meets the command triggering conditions, a control command is generated for the central rearview mirror. Responding to the control command, the central rearview mirror is adjusted according to the relative position of the central rearview mirror and the target object. By promptly determining the driver's intention to check the central rearview mirror based on facial feature information and adjusting the central rearview mirror according to the relative position of the central rearview mirror and the target object, the system solves the problems of insufficient accuracy and timeliness when triggering voice commands.

[0130] II. Obtain the face image of the target object by processing the acquired images using algorithms:

[0131] Please see Figure 4 The camera is located on the central rearview mirror and uses image processing algorithms to obtain facial images of the target object.

[0132] 401. Obtain the front view of the driver's cab.

[0133] The rearview mirror system acquires an image of the front row of the driver's cab, which is obtained by a camera capturing the image of the front seat of the driver's cab.

[0134] 402. Obtain the left seat image, right seat image, and steering wheel image based on the front row image of the driver's cab.

[0135] The rearview mirror system performs background removal and contour estimation on the acquired front row images of the driver's cab to obtain images of the left seat, right seat, and steering wheel.

[0136] 403. Determine the driver's seat based on the left seat image, right seat image, and steering wheel image.

[0137] The rearview mirror system determines whether the left seat image or the right seat image is on the same side as the steering wheel image. If the left seat image and the steering wheel image are on the same side, then the left seat is determined to be the driver's seat. If the right seat image and the steering wheel image are on the same side, then the right seat is determined to be the driver's seat.

[0138] 404. Obtain the driver's seat image sequence based on the driver's seat position.

[0139] After determining the driver's position, the rearview mirror system acquires the corresponding driver's position image sequence.

[0140] 405. Obtain the seat image sequence based on the driver's seat image sequence.

[0141] The driver's seat image is a rectangular image with the driver's seat as the main subject. After the rearview mirror system estimates the contour of the driver's seat image sequence, it obtains the seat image sequence, which is an image of the actual shape of the seat.

[0142] 406. Obtain the face image sequence based on the seat image sequence.

[0143] The rearview mirror system detects faces in the seat image, and if a face is found, it acquires a sequence of face images.

[0144] 407. Detect the left and right turning angles and pitch angles of the face in each face image in a face image sequence.

[0145] 408. Obtain the effective facial pose corresponding to the target object.

[0146] 409. Determine whether the duration of maintaining an effective facial pose is greater than or equal to the duration threshold.

[0147] 410. If the duration of maintaining an effective facial pose is less than the duration threshold, the timer is reset to zero.

[0148] 411. If the duration of maintaining an effective facial pose is greater than or equal to the duration threshold, a control command is triggered.

[0149] 412. Determine the position of the human eye based on the effective facial posture.

[0150] 413. Determine the target line based on the position of the human eye and the target position on the central rearview mirror.

[0151] 414. Obtain the first projection of the target line onto the normal plane perpendicular to the mirror surface of the central rearview mirror.

[0152] 415. Obtain the second projection of the target line onto the mirror surface of the central rearview mirror.

[0153] 416. Determine the first included angle between the first projection and the vertical line of the center rearview mirror.

[0154] 417. Determine the second included angle between the second projection and the mirror normal of the central rearview mirror.

[0155] 418. Adjust the first angle of the central rearview mirror to the first preset value.

[0156] 419. Adjust the second angle of the central rearview mirror to the second preset value.

[0157] Steps 407 to 419 in this embodiment are the same as those described above. Figure 3 Steps 303 to 315 in the illustrated embodiment are similar and will not be repeated here.

[0158] based on Figure 3 The rearview mirror control method of the illustrated embodiment is described below as an application scenario of the rearview mirror control method of this application embodiment:

[0159] Please see Figure 5 In this method, the driver's seat image acquisition device uses a custom-designed narrow-angle lens. The upper and left and right edges of the image captured by this lens are close to the edge of the seat, and the image typically only includes the seat image. The device is fixed to the vehicle body, and the fixing position meets the following requirements:

[0160] (1) Use the vertical line in the center of the rearview mirror;

[0161] (2) The camera is pointed towards the driver's head area;

[0162] (3) Make the top edge of the captured image close to the top contour of the driver's seat, and the left edge (left-hand drive vehicle) or right edge (right-hand drive vehicle) close to the driver's seat.

[0163] based on Figure 4 The rearview mirror control method of the illustrated embodiment is described below as another application scenario of the rearview mirror control method of this application embodiment:

[0164] Please refer to Figures 6(A) and 6(B). The driver's seat image acquisition device uses a customized standard lens or wide-angle lens. The image captured by this lens must be able to cover the entire front of the driver's cab, including images of the left and right seats and the steering wheel. The device is fixed to the rearview mirror, and the fixing position meets the following requirements:

[0165] (1) Use the vertical line in the center of the rearview mirror;

[0166] (2) The camera is pointed towards the driver's head area;

[0167] (3) The car steering wheel is included in the lens;

[0168] (4) Detect the outlines of the front seats and steering wheel according to the computer vision target detection algorithm (as shown in the bold box in the figure);

[0169] (5) The seat on the same side as the steering wheel is determined to be the driver's seat;

[0170] (6) The original image is cropped using the outline of the seat on the same side as the steering wheel as the boundary to complete the acquisition of the driver's seat image.

[0171] Please refer to Figure 6(C). After acquiring the driver's seat image, the background is removed from the driver's seat image based on contour estimation to obtain the seat image. Face detection is then performed after acquiring the seat image.

[0172] Please refer to Figure 6(D). After acquiring the face image, the rearview mirror system determines whether the target object is facing the central rearview mirror based on the pitch angle and left and right turning angle of the face image relative to the frontal view.

[0173] based on Figure 3 and Figure 4 The method for controlling the rearview mirror surface in the illustrated embodiment is described below, and an application scenario of this application embodiment is provided below:

[0174] Please refer to Figure 7(A). This application embodiment provides a method for calculating and adjusting the angle of a rearview mirror relative to the driver's viewing angle.

[0175] On the rearview mirror, determine the positive direction of the X-axis as the left direction along the upper edge of the rearview mirror, determine the positive direction of the Z-axis as the upward direction of the perpendicular line M in the rearview mirror, and determine the mirror surface perpendicular normal plane P and mirror normal line N of the rearview mirror surface.

[0176] Determine the line L connecting the midpoint of the line connecting the corresponding eye positions of the left and right eyeballs to the origin of the rearview mirror. Determine the projection L' of line L onto the mirror surface, and the projection L” of line L onto the perpendicular normal plane of the rearview mirror surface. The angle between L' and the perpendicular bisector Y of the rearview mirror surface is angle α, and the angle between L” and the normal N of the mirror surface is angle β. Refer to Figures 7(B) and 7(C). Adjust the rearview mirror so that angle α = 0 and angle β = 0, meaning the rearview mirror is directly facing the driver's face, allowing the driver to clearly see the content displayed in the rearview mirror.

[0177] The rearview mirror system in the embodiments of this application is described below:

[0178] Please see Figure 8 In this embodiment of the application, the rearview mirror system includes an acquisition unit 801, a generation unit 802, and an adjustment unit 803.

[0179] Acquisition unit 801 is used to acquire facial feature information of the target object;

[0180] The generation unit 802 is used to generate control commands for the central rearview mirror when the facial feature information meets the command triggering conditions.

[0181] Adjustment unit 803 is used to adjust the central rearview mirror in response to the control command, according to the relative position of the central rearview mirror and the target object.

[0182] Please see Figure 9 In another configuration of the rearview mirror system according to an embodiment of this application, the rearview mirror system includes an acquisition unit 901, a generation unit 902, an adjustment unit 903, and a determination unit 904.

[0183] The acquisition unit 901 is specifically used to acquire the effective facial pose of the target object when the target object is facing the central rearview mirror.

[0184] The determining unit 904 is specifically used to determine that the facial feature information meets the instruction triggering condition when the duration of the effective facial pose is greater than or equal to the duration threshold.

[0185] The acquisition unit 901 is specifically used to acquire the face image sequence of the target object;

[0186] The acquisition unit 901 is further configured to acquire the effective facial pose corresponding to the target object when it is detected that the facial pose of each face image in the face image sequence is facing the central rearview mirror.

[0187] The determination unit 904 is specifically used to determine that the face posture is facing the central rearview mirror when the left and right turning angles and pitch angles of the face posture are less than the angle threshold.

[0188] The acquisition unit 901 is specifically used to acquire the driver's seat image sequence of the target object, and the driver's seat image sequence and the face image sequence have a corresponding relationship;

[0189] The acquisition unit 901 is also used to acquire a face image sequence based on the driver's seat image sequence.

[0190] The acquisition unit 901 is specifically used to acquire a seat image sequence based on the driver's seat image sequence, wherein the seat image sequence and the face image sequence have a corresponding relationship.

[0191] The acquisition unit 901 is also used to acquire a face image sequence based on the seat image sequence.

[0192] Acquisition unit 901 is specifically used to acquire images of the front row of the driver's cab;

[0193] The acquisition unit 901 is also used to acquire the left seat image, the right seat image, and the steering wheel image based on the front row image of the driver's cab;

[0194] The determining unit 904 is also used to determine the driver's seat based on the left seat image, the right seat image, and the steering wheel image.

[0195] The acquisition unit 901 is also used to acquire a driver's seat image sequence based on the driver's seat.

[0196] The determining unit 904 is specifically used to determine the position point of the human eye based on the effective facial pose;

[0197] The determining unit 904 is also used to determine the target line based on the position point of the human eye and the target position point on the central rearview mirror;

[0198] The acquisition unit 901 is specifically used to acquire the first projection of the target line onto the mirror-perpendicular normal plane of the central rearview mirror.

[0199] The acquisition unit 901 is also used to acquire the second projection of the target line on the mirror surface of the central rearview mirror;

[0200] The determining unit 904 is also used to determine the first included angle between the first projection and the vertical line of the center rearview mirror.

[0201] The determining unit 904 is also used to determine the second included angle between the second projection and the mirror normal of the central rearview mirror;

[0202] The adjustment unit 903 is specifically used to adjust the first angle of the central rearview mirror to a first preset value;

[0203] The adjustment unit 903 is also used to adjust the second angle of the central rearview mirror to a second preset value.

[0204] Figure 10 This is a schematic diagram of a rearview mirror system provided in an embodiment of this application. The rearview mirror system 1000 may include one or more central processing units (CPUs) 1001 and a memory 1005, in which one or more applications or data are stored.

[0205] The memory 1005 can be volatile or persistent storage. The program stored in the memory 1005 can include one or more modules, each module including a series of instruction operations on the server. Furthermore, the central processing unit 1001 can be configured to communicate with the memory 1005 and execute the series of instruction operations stored in the memory 1005 on the server 1000.

[0206] Server 1000 may also include one or more power supplies 1002, one or more wired or wireless network interfaces 1003, one or more input / output interfaces 1004, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0207] The central processing unit 1001 can perform the aforementioned... Figures 3 to 7(C) The specific operations performed by the rearview mirror system in the illustrated embodiment will not be described in detail here.

[0208] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0209] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0210] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0211] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0212] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for controlling a rearview mirror, characterized in that, include: Obtain facial feature information of the target object; If the facial feature information meets the command triggering conditions, a control command for the central rearview mirror is generated. In response to the control command, the central rearview mirror is adjusted according to the relative position of the central rearview mirror and the target object; The acquisition of facial feature information of the target object includes: If the target object is facing the central rearview mirror, then the effective facial pose corresponding to the target object is obtained; Adjusting the central rearview mirror according to the relative position of the central rearview mirror and the target object includes: The position of the human eye is determined based on the effective facial posture. A target line is determined based on the human eye position point and the target position point on the central rearview mirror. The target line is a direct line between a single human eye position point and the target position point, or a line between two human eye position points and the target position point. Obtain the first projection of the target line onto the vertical normal plane of the central rearview mirror; Obtain the second projection of the target line onto the mirror surface of the central rearview mirror; Determine the first included angle between the first projection and the mirror normal of the central rearview mirror; Determine the second included angle between the second projection and the vertical line of the center rearview mirror; Adjust the first included angle of the central rearview mirror to a first preset value; Adjust the second angle of the central rearview mirror to a second preset value.

2. The rearview mirror control method according to claim 1, characterized in that, After obtaining the facial feature information of the target object, the method further includes: If the duration of the effective facial pose is greater than or equal to the duration threshold, then the facial feature information is determined to meet the instruction triggering condition.

3. The rearview mirror control method according to claim 2, characterized in that, If the target object is facing the central rearview mirror, then the effective facial pose corresponding to the target object is obtained, including: Obtain the facial image sequence of the target object; If it is detected that the facial pose of each face image in the face image sequence is facing the central rearview mirror, then the effective facial pose corresponding to the target object is obtained.

4. The rearview mirror control method according to claim 3, characterized in that, After obtaining the facial image sequence of the target object, the method further includes: If the left and right turning angles and pitch angles of the facial posture are less than the angle threshold, then the facial posture is determined to be facing the central rearview mirror.

5. The rearview mirror control method according to claim 3, characterized in that, The step of obtaining the facial image sequence of the target object includes: Obtain a driver's seat image sequence of the target object, wherein the driver's seat image sequence and the face image sequence have a corresponding relationship; The face image sequence is obtained from the driver's seat image sequence.

6. The method for controlling a rearview mirror according to claim 5, characterized in that, The step of obtaining the face image sequence based on the driver's seat image sequence includes: A seat image sequence is obtained based on the driver's seat image sequence, and the seat image sequence and the face image sequence have a corresponding relationship; The face image sequence is obtained from the seat image sequence.

7. The rearview mirror control method according to claim 5, characterized in that, The step of obtaining the driver's seat image sequence of the target object includes: Acquire front view of the driver's cab; The left seat image, right seat image, and steering wheel image are obtained based on the front row image of the driver's cab; The driver's seat is determined based on the left seat image, the right seat image, and the steering wheel image; The driver's seat image sequence is obtained based on the driver's seat.

8. A rearview mirror system, characterized in that, include: The acquisition unit is used to acquire facial feature information of the target object; The generation unit is used to generate control commands for the central rearview mirror when the facial feature information meets the command triggering conditions. An adjustment unit is configured to adjust the central rearview mirror in response to the control command, based on the relative position of the central rearview mirror and the target object. The acquisition unit is specifically used to acquire the effective facial posture corresponding to the target object when the target object is facing the central rearview mirror; The determining unit is specifically used to determine the position point of the human eye based on the effective facial posture; The determining unit is further configured to determine a target line based on the human eye position point and the target position point on the central rearview mirror, wherein the target line is a direct line between a single human eye position point and the target position point, or a line between two human eye position points and the target position point. The acquisition unit is specifically used to acquire the first projection of the target line onto the vertical normal plane of the central rearview mirror. The acquisition unit is also used to acquire the second projection of the target line onto the mirror surface of the central rearview mirror; The determining unit is further configured to determine the first included angle between the first projection and the mirror normal of the central rearview mirror; The determining unit is also used to determine the second included angle between the second projection and the vertical line of the center rearview mirror. The adjustment unit is specifically used to adjust the first included angle of the central rearview mirror to a first preset value; The adjustment unit is also used to adjust the second included angle of the central rearview mirror to a second preset value.

9. The rearview mirror system according to claim 8, characterized in that, include: The determining unit is specifically used to determine that the facial feature information satisfies the instruction triggering condition when the duration of the effective facial pose is greater than or equal to the duration threshold.

10. The rearview mirror system according to claim 9, characterized in that, include: The acquisition unit is specifically used to acquire the face image sequence of the target object; The acquisition unit is further configured to acquire the effective facial pose corresponding to the target object when it is detected that the facial pose of each facial image in the facial image sequence is facing the central rearview mirror.

11. The rearview mirror system according to claim 10, characterized in that, include: The determining unit is specifically used to determine that the face posture is facing the central rearview mirror when the left and right turning angles and pitch angles of the face posture are less than the angle threshold.

12. The rearview mirror system according to claim 10, characterized in that, include: The acquisition unit is specifically used to acquire the driver's seat image sequence of the target object, wherein the driver's seat image sequence and the face image sequence have a corresponding relationship; The acquisition unit is further configured to acquire the face image sequence based on the driver's seat image sequence.

13. The rearview mirror system according to claim 12, characterized in that, include: The acquisition unit is specifically used to acquire a seat image sequence based on the driver's seat image sequence, wherein the seat image sequence and the face image sequence have a corresponding relationship. The acquisition unit is further configured to acquire the face image sequence based on the seat image sequence.

14. The rearview mirror system according to claim 12, characterized in that, include: The acquisition unit is specifically used to acquire images of the front row of the driver's cab; The acquisition unit is also used to acquire the left seat image, the right seat image, and the steering wheel image based on the front row image of the driver's cab; The determining unit is further configured to determine the driver's seat based on the left seat image, the right seat image, and the steering wheel image; The acquisition unit is further configured to acquire the driver's seat image sequence based on the driver's seat.

15. A rearview mirror system, characterized in that, The rearview mirror system includes a processor, memory, input / output devices, and a bus; The processor, memory, and input / output devices are connected to the bus; The processor is used to execute the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a program that, when executed by the computer, performs the method as described in any one of claims 1 to 7.

17. A vehicle, characterized in that, The vehicle includes the rearview mirror system as described in any one of claims 8 to 14.

Images