Vehicle rearview mirror adjusting method and device, vehicle and storage medium

Abstract

The application relates to a vehicle rearview mirror adjusting method and device, a vehicle and a storage medium, wherein the method comprises the following steps: performing head recognition of a driver based on multiple image data of the driver, and detecting a gaze fixation area of the driver according to a recognition result; recognizing a gesture change trend of the driver, matching a corresponding gesture adjusting instruction based on the gesture change trend, determining a target adjusting parameter of the vehicle rearview mirror according to the gesture adjusting instruction, determining a theoretical adjusting parameter of the vehicle rearview mirror according to a surrounding traffic environment parameter of the vehicle, and combining the target adjusting parameter, the theoretical adjusting parameter and a preset logical constraint to obtain an actual adjusting parameter of the vehicle rearview mirror, so as to adjust the vehicle rearview mirror by using the actual adjusting parameter. Therefore, the technical problem that in the prior art, a driver manually adjusts a button with low efficiency, the operation burden of the driver is large, the influence on the attention of the driver is high, there is a certain safety hazard, the intelligent level is low, and the voice control mode is not conducive to the driver with language barriers is solved.

Description

Technical Field

[0001] The present application relates to the technical field of rearview mirror control, and in particular to a vehicle rearview mirror adjustment method, device, vehicle, and storage medium. Background Art

[0002] In related technologies, the rearview mirror adjustment method requires the driver to manually push a button, which not only has low adjustment efficiency and increases the driver's operating burden, but may also distract their attention and reduce driving safety. The rearview mirror adjustment method through voice control commands is difficult to accurately identify the driver's actual adjustment intentions, and is not conducive to some drivers with language barriers, and needs to be improved. Summary of the Invention

[0003] The present application provides a vehicle rearview mirror adjustment method, device, vehicle and storage medium to solve the technical problems in the related art that the driver's manual adjustment by pulling buttons is inefficient, the driver's operating burden is heavy, the impact on the driver's attention is high, there are certain safety hazards, the intelligence level is low, and the voice control method is not conducive to some drivers with language barriers.

[0004] The first aspect of the present application provides a method for adjusting a rearview mirror of a vehicle, comprising the following steps: collecting multi-frame image data of a driver; performing head recognition of the driver based on the multi-frame image data to obtain a recognition result, and detecting the driver's gaze area based on the recognition result; using the multi-frame image data to identify the driver's gesture change trend, matching corresponding gesture adjustment instructions based on the gesture change trend, determining target adjustment parameters of the vehicle's rearview mirror according to the gesture adjustment instructions, determining theoretical adjustment parameters of the vehicle's rearview mirror according to the vehicle's surrounding traffic environment parameters, and obtaining actual adjustment parameters of the vehicle's rearview mirror in combination with the target adjustment parameters, the theoretical adjustment parameters and preset logical constraints, so as to adjust the vehicle's rearview mirror using the actual adjustment parameters.

[0005] Optionally, in one embodiment of the present application, before using the multi-frame image data to identify the driver's gesture change trend, it also includes: determining whether the vehicle rearview mirror exists in the gaze area; if the vehicle rearview mirror exists, controlling the vehicle to enter the rearview mirror adjustment mode to identify the gesture change trend, otherwise, prohibiting the vehicle from entering the rearview mirror adjustment mode.

[0006] Optionally, in one embodiment of the present application, the driver's head is recognized based on the multi-frame image data to obtain a recognition result, and the driver's gaze area is detected based on the recognition result, including: identifying the driver's facial area from the multi-frame image data; identifying multiple key points within the facial area to determine the positions of the driver's eyes and eyebrows based on the multiple key points; estimating the driver's head posture based on the positions of the driver's eyes and eyebrows, and determining the driver's initial gaze direction based on the head posture; performing pupil recognition and iris edge tracking recognition of the driver based on the positions of the driver's eyes and eyebrows to obtain the driver's eye tracking data, and determining the driver's actual gaze direction based on the head posture, the eye tracking data and the initial gaze direction to obtain the gaze area based on the actual gaze direction.

[0007] Optionally, in one embodiment of the present application, obtaining the gaze area based on the actual gaze direction includes: constructing an in-vehicle coordinate system of the vehicle; obtaining eye coordinates in the in-vehicle coordinate system based on the head posture and the positions of the eyes and eyebrows; and converting the actual gaze direction into a gaze area in the in-vehicle coordinate system based on the eye coordinates.

[0008] Optionally, in one embodiment of the present application, the use of the multiple frames of image data to identify the driver's gesture change trend includes: extracting the driver's hand features from each frame of image data, and sorting the hand features according to the time sequence of each frame of image data to obtain a sorting result; matching the corresponding hand features in a preset gesture database in sequence according to the time sequence to obtain a matching result; if the matching result is a successful match, taking the hand features corresponding to the current frame of image data as the gesture change starting frame, and determining the gesture change trend based on the gesture change starting frame and the hand features; otherwise, matching the hand features corresponding to the next frame of image data.

[0009] Optionally, in one embodiment of the present application, after using the multi-frame image data to identify the driver's gesture change trend, it also includes: identifying the driver's actual control intention based on the gesture change trend; if the actual control intention is the control intention of the vehicle rearview mirror, matching the corresponding adjustment instructions based on the gesture change trend, otherwise, controlling the vehicle to execute other control instructions other than the vehicle rearview mirror control based on the actual control intention.

[0010] Optionally, in one embodiment of the present application, the preset logical constraints include: the danger level posed by the vehicle's surrounding traffic environment to the vehicle, wherein the danger level is obtained by the surrounding traffic environment parameters and the vehicle's current driving parameters; the driver's emotional characteristics, wherein the driver's emotional characteristics are obtained by the multi-frame image data and the driver's voice data.

[0011] The second aspect of the present application provides a rearview mirror adjustment device for a vehicle, comprising: an acquisition module for acquiring multi-frame image data of a driver; a first recognition module for performing head recognition of the driver based on the multi-frame image data to obtain a recognition result, and detecting the driver's gaze area based on the recognition result; an adjustment module for using the multi-frame image data to identify the driver's gesture change trend, matching corresponding gesture adjustment instructions based on the gesture change trend, determining target adjustment parameters of the vehicle rearview mirror according to the gesture adjustment instructions, determining theoretical adjustment parameters of the vehicle rearview mirror according to the vehicle's surrounding traffic environment parameters, and obtaining actual adjustment parameters of the vehicle rearview mirror in combination with the target adjustment parameters, the theoretical adjustment parameters and preset logical constraints, so as to adjust the vehicle rearview mirror using the actual adjustment parameters.

[0012] Optionally, in one embodiment of the present application, it also includes: a judgment module for judging whether the vehicle rearview mirror exists in the gaze area; a first control module for controlling the vehicle to enter the rearview mirror adjustment mode and identify the gesture change trend when the vehicle rearview mirror exists, otherwise, prohibiting the vehicle from entering the rearview mirror adjustment mode.

[0013] Optionally, in one embodiment of the present application, the recognition module includes: a first recognition unit for identifying the driver's facial area from the multiple frames of image data; a second recognition unit for identifying multiple key points within the facial area to determine the positions of the driver's eyes and eyebrows based on the multiple key points; an estimation unit for estimating the driver's head posture based on the positions of the driver's eyes and eyebrows, and determining the driver's initial gaze direction based on the head posture; a third recognition unit for performing pupil recognition and iris edge tracking recognition of the driver based on the positions of the driver's eyes and eyebrows to obtain the driver's eye tracking data, and determine the driver's actual gaze direction in combination with the head posture, the eye tracking data and the initial gaze direction to obtain the gaze area based on the actual gaze direction.

[0014] Optionally, in one embodiment of the present application, the third recognition unit includes: a construction subunit for constructing the in-vehicle coordinate system of the vehicle; a calculation subunit for obtaining the eye coordinates in the in-vehicle coordinate system based on the head posture and the positions of the eyes and eyebrows; and a conversion subunit for converting the actual line of sight direction into a gaze area in the in-vehicle coordinate system based on the eye coordinates.

[0015] Optionally, in one embodiment of the present application, the adjustment module includes: an extraction unit, used to extract the driver's hand features from each frame of image data, and sort the hand features in the time sequence of each frame of image data to obtain a sorting result; a first matching unit, used to match the corresponding hand features in a preset gesture database in sequence according to the time sequence to obtain a matching result; a second matching unit, used to use the hand features corresponding to the current frame of image data as the gesture change starting frame when the matching result is a successful match, and determine the gesture change trend based on the gesture change starting frame and the hand features, otherwise, match the hand features corresponding to the next frame of image data.

[0016] Optionally, in one embodiment of the present application, it also includes: a second recognition module, used to identify the actual control intention of the driver based on the gesture change trend; a second control module, used to match the corresponding adjustment instructions based on the gesture change trend when the actual control intention is the control intention of the vehicle rearview mirror, otherwise, control the vehicle to execute other control instructions other than the vehicle rearview mirror control based on the actual control intention.

[0017] Optionally, in one embodiment of the present application, the preset logical constraints include: the danger level posed by the vehicle's surrounding traffic environment to the vehicle, wherein the danger level is obtained by the surrounding traffic environment parameters and the vehicle's current driving parameters; the driver's emotional characteristics, wherein the driver's emotional characteristics are obtained by the multi-frame image data and the driver's voice data.

[0018] A third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the rearview mirror adjustment method for the vehicle as described in the above embodiment.

[0019] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the vehicle rearview mirror adjustment method as described in the above embodiment.

[0020] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned vehicle rearview mirror adjustment method.

[0021] The embodiments of the present application can capture multiple frames of driver image data to perform driver head recognition and detect the driver's gaze area based on the recognition results. Simultaneously, the driver's gesture change trends are identified, and corresponding gesture adjustment instructions are matched based on the gesture change trends. Target adjustment parameters for the vehicle's rearview mirror are determined based on the gesture adjustment instructions. The theoretical adjustment parameters for the vehicle's rearview mirror are determined based on the vehicle's surrounding traffic environment parameters. The actual adjustment parameters for the vehicle's rearview mirror are then determined by combining the target adjustment parameters, the theoretical adjustment parameters, and preset logical constraints. The actual adjustment parameters are then used to adjust the vehicle's rearview mirror, achieving automatic adjustment of the vehicle's rearview mirror without the need for manual driver operation. This improves adjustment efficiency and, by incorporating factors from the surrounding traffic environment, avoids accidental touches, further improving the safety of vehicle rearview mirror adjustment. This solves the technical problems of related technologies, such as low efficiency of manual button adjustment by the driver, heavy operational burden on the driver, high impact on the driver's attention, certain safety risks, low intelligence level, and voice control methods that are unfavorable for drivers with language barriers.

[0022] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 This is a flow chart of a vehicle rearview mirror adjustment method provided according to an embodiment of the present application;

[0025] Figure 2 is a flow chart of a rearview mirror adjustment method according to one embodiment of the present application;

[0026] Figure 3 This is a schematic structural diagram of a vehicle rearview mirror adjustment device provided according to an embodiment of the present application;

[0027] Figure 4 A schematic structural diagram of a vehicle provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0029] The following describes a vehicle rearview mirror adjustment method, device, vehicle, and storage medium according to embodiments of the present application with reference to the accompanying drawings. In response to the technical issues mentioned in the background art above, manual adjustment by the driver using buttons is inefficient, burdens the driver's operation, significantly impacts the driver's attention, poses certain safety risks, has a low level of intelligence, and is not suitable for drivers with language barriers. The present application provides a vehicle rearview mirror adjustment method. In this method, multiple frames of driver image data can be collected to perform driver head recognition. Based on the recognition results, the driver's gaze area is detected. Simultaneously, the driver's gesture change trends are identified. Based on the gesture change trends, corresponding gesture adjustment instructions are matched. Target adjustment parameters for the vehicle rearview mirror are determined based on the gesture adjustment instructions. The theoretical adjustment parameters for the vehicle rearview mirror are determined based on the vehicle's surrounding traffic environment parameters. The actual adjustment parameters for the vehicle rearview mirror are obtained by combining the target adjustment parameters, the theoretical adjustment parameters, and preset logical constraints. The actual adjustment parameters for the vehicle rearview mirror are then adjusted using the actual adjustment parameters, achieving automatic adjustment of the vehicle rearview mirror without the need for manual driver operation. This improves adjustment efficiency and, by incorporating surrounding traffic environment factors, avoids accidental touches, further improving the safety of vehicle rearview mirror adjustment. This solves the technical problems in related technologies, such as low efficiency of manual button adjustment by the driver, heavy operating burden on the driver, high impact on the driver's attention, certain safety hazards, low intelligence level, and voice control method that is not conducive to some drivers with language barriers.

[0030] Specifically, Figure 1 A schematic flow chart of a vehicle rearview mirror adjustment method provided in an embodiment of the present application.

[0031] like Figure 1 As shown, the vehicle rearview mirror adjustment method includes the following steps:

[0032] In step S101 , multiple frames of image data of the driver are collected.

[0033] During actual implementation, the embodiments of the present application can utilize the vehicle's internal camera, or reuse the camera of the vehicle's driver status monitoring system, to obtain multi-frame image data of the driver within a certain period of time, so as to determine the driver's movement trend based on the acquisition time of the multi-frame image data.

[0034] In step S102 , the driver's head is recognized based on the multi-frame image data to obtain a recognition result, and the driver's gaze area is detected according to the recognition result.

[0035] Furthermore, the embodiment of the present application can capture the driver's facial and eye information in real time based on multiple frames of image data, and determine the driver's gaze area through algorithm analysis, etc., to determine whether the driver is looking at the left, right or center.

[0036] Optionally, in one embodiment of the present application, the driver's head is recognized based on multiple frames of image data to obtain a recognition result, and the driver's gaze area is detected based on the recognition result, including: identifying the driver's facial area from multiple frames of image data; identifying multiple key points in the facial area to determine the positions of the driver's eyes and eyebrows based on the multiple key points; estimating the driver's head posture based on the positions of the driver's eyes and eyebrows, and determining the driver's initial gaze direction based on the head posture; performing pupil recognition and iris edge tracking recognition on the driver based on the positions of the driver's eyes and eyebrows to obtain the driver's eye tracking data, and determining the driver's actual gaze direction in combination with the head posture, eye tracking data and initial gaze direction to obtain the gaze area based on the actual gaze direction.

[0037] For example, when performing head recognition, the embodiment of the present application may include the following steps:

[0038] Step S1: Face detection and positioning. In the embodiment of the present application, computer vision technologies such as Haar features, HOG+SVM, and deep learning models can be used to pre-process the image and identify and locate the driver's facial area.

[0039] Step S2: Key point detection: The embodiment of the present application can detect key points within the facial region, such as the positions of the eyes and eyebrows, to understand the head posture and eye direction.

[0040] Step S3: Head posture estimation. The embodiment of the present application can use key point information to estimate the head posture angle (pitch, yaw and roll) through a 3D model or machine learning method, thereby inferring the driver's general gaze direction, that is, the initial gaze direction.

[0041] Step S4: Eye tracking. The embodiment of the present application can be further refined to the eye area, using pupil detection and iris edge tracking technology, combined with head posture information, to accurately calculate the direction of sight, that is, the actual gaze direction.

[0042] Optionally, in one embodiment of the present application, obtaining a gaze area based on the actual gaze direction includes: constructing an in-vehicle coordinate system of the vehicle; obtaining eye coordinates in the in-vehicle coordinate system based on the head posture and the positions of the eyes and eyebrows; and converting the actual gaze direction into a gaze area in the in-vehicle coordinate system based on the eye coordinates.

[0043] Furthermore, the embodiments of the present application can combine the three-dimensional coordinate system inside the vehicle to convert the actual gaze direction into a specific gaze area inside or outside the vehicle.

[0044] During actual implementation, the embodiment of the present application can establish a model of the in-vehicle environment and construct a coordinate system with a certain point as the origin, so as to convert the line of sight direction into the actual gaze area.

[0045] In step S103, the driver's gesture change trend is identified using multi-frame image data, and corresponding gesture adjustment instructions are matched based on the gesture change trend. The target adjustment parameters of the vehicle rearview mirror are determined according to the gesture adjustment instructions, and the theoretical adjustment parameters of the vehicle rearview mirror are determined according to the vehicle's surrounding traffic environment parameters. The actual adjustment parameters of the vehicle rearview mirror are obtained by combining the target adjustment parameters, the theoretical adjustment parameters and the preset logical constraints, so as to adjust the vehicle rearview mirror using the actual adjustment parameters.

[0046] As a possible implementation method, the embodiment of the present application can use multi-frame image data to identify the driver's gesture change trend, thereby obtaining the driver's gesture control action, and then use the driver's gesture adjustment instructions to adjust the vehicle's rearview mirror, and eliminate the risk of accidental touch or adjustment according to the traffic environment. Intelligent and automatic rearview mirror adjustment can be achieved without the driver's manual operation or voice control.

[0047] The embodiment of the present application can determine the target adjustment parameters based on the gesture adjustment instruction and the driver's gaze area, so that the rearview mirror can display the reflection image required by the driver.

[0048] Among them, the embodiment of the present application can determine the driver's target vehicle rearview mirror based on the gaze area. For example, when the driver looks at the left vehicle rearview mirror and makes a gesture, it means that the driver wants to adjust the left vehicle rearview mirror.

[0049] For example, when the driver turns his head to the right by more than a certain value and his gaze area is to the right rear of the driver's seat, the driver's gaze area is the right side or right rear side of the vehicle. This means that the driver wants to obtain the situation on the right side or right rear side of the vehicle. The embodiment of the present application can adjust all the rearview mirrors of the vehicle to ensure that the rearview mirrors can reflect the image on the right side or right rear side of the vehicle.

[0050] Optionally, in one embodiment of the present application, the preset logical constraints include: the danger level posed to the vehicle by the vehicle's surrounding traffic environment, wherein the danger level is obtained by the surrounding traffic environment parameters and the vehicle's current driving parameters; the driver's emotional characteristics, wherein the driver's emotional characteristics are obtained by multiple frames of image data and the driver's voice data.

[0051] For example, the embodiment of the present application can combine the target adjustment parameters, theoretical adjustment parameters and preset logical constraints to obtain the actual adjustment parameters of the vehicle rearview mirror.

[0052] The target adjustment parameter is the adjustment parameter of the rearview mirror determined according to the driver's gesture.

[0053] The theoretical adjustment parameters are the adjustment parameters of the rearview mirror that are determined based on the current driving environment of the vehicle and surrounding traffic flow data so that the driver can ensure the maximum observation field of view by observing the rearview mirror.

[0054] The preset logical constraints may include the danger level posed to the vehicle by the surrounding traffic environment. For example, if there are pedestrians or animals close behind the vehicle, the danger level is high; if there are other vehicles traveling in the same direction but at a uniform speed behind the vehicle, the danger level is medium. Therefore, when the danger level is high, the embodiment of the present application can increase the weight of the logical constraint, so that the driver can ensure that the pedestrians or animals behind the vehicle are observed by adjusting the vehicle's rearview mirror.

[0055] Among them, the danger level can be obtained by using the vehicle's radar, camera and other perception parameters to obtain the surrounding traffic environment parameters, and combined with the vehicle's current driving parameters.

[0056] The preset logical constraints may also include the driver's emotional characteristics. For example, when the driver's emotions are positive and active, the embodiment of the present application may increase the weight of the target adjustment parameters. When the driver's emotions are negative and passive, the embodiment of the present application may increase the weight of the theoretical adjustment parameters to avoid the driver's erroneous operations under negative emotions.

[0057] Among them, emotional characteristics can be matched through image data analysis of the driver, and combined with the tone and timbre of the voice to determine the driver's actual emotions.

[0058] Optionally, in one embodiment of the present application, before using multi-frame image data to identify the driver's gesture change trend, it also includes: determining whether there is a vehicle rearview mirror in the gaze area; if there is a vehicle rearview mirror, controlling the vehicle to enter the rearview mirror adjustment mode and identifying the gesture change trend, otherwise, prohibiting the vehicle from entering the rearview mirror adjustment mode.

[0059] In some embodiments, the embodiments of the present application first determine whether there is a vehicle rearview mirror in the viewing area. When there is no vehicle rearview mirror, it means that the driver does not want to control the vehicle rearview mirror, and the vehicle is prohibited from entering the rearview mirror adjustment mode.

[0060] When a vehicle rearview mirror is present, it can be determined that the driver's intention is to control the vehicle rearview mirror, and gesture recognition is performed to control the vehicle rearview mirror.

[0061] In addition, the gaze area can also be expressed as the driver's area of ​​interest, that is, the area the driver wants to see. At this time, the embodiment of the present application can control the vehicle to enter the rearview mirror adjustment mode according to the area of ​​interest and gesture instructions, so that the vehicle's rearview mirror can reflect the driver's area of ​​interest after adjustment.

[0062] Optionally, in one embodiment of the present application, after using multi-frame image data to identify the driver's gesture change trend, it also includes: identifying the driver's actual control intention based on the gesture change trend; if the actual control intention is the control intention of the vehicle rearview mirror, matching the corresponding adjustment instructions based on the gesture change trend, otherwise, controlling the vehicle to execute other control instructions other than the vehicle rearview mirror control based on the actual control intention.

[0063] In other embodiments, when the actual control intention corresponding to the gesture change trend is to control the vehicle rearview mirror, the embodiments of the present application can match the corresponding adjustment instructions according to the gesture change trend.

[0064] When the actual control intention corresponding to the gesture change trend is not to control the vehicle rearview mirror, it means that the driver does not want to control the vehicle rearview mirror. The driver's gesture action may be used to control other functions of the vehicle. At this time, other functions of the vehicle can be controlled.

[0065] Optionally, in one embodiment of the present application, multiple frames of image data are used to identify the driver's gesture change trend, including: extracting the driver's hand features from each frame of image data, and sorting the hand features in chronological order of each frame of image data to obtain a sorting result; matching the corresponding hand features in a preset gesture database in chronological order to obtain a matching result; if the matching result is a successful match, the hand features corresponding to the current frame of image data are used as the gesture change starting frame, and based on the gesture change starting frame and the hand features, the gesture change trend is determined; otherwise, the hand features corresponding to the next frame of image data are matched.

[0066] It is understandable that when the driver performs gesture control, the first frame image is not necessarily the starting point of the driver's gesture action. In this case, performing gesture recognition is very likely to produce erroneous recognition results.

[0067] Therefore, the embodiment of the present application can extract the driver's hand features in each frame of image data and compare them in the gesture database in sequence. Those that fail to match are marked as meaningless actions. The image frame corresponding to the first matching hand feature is the starting point of the gesture, and then the subsequent image frames are compared in sequence.

[0068] In addition, if the hand features of the next frame of image data after the image frame corresponding to the gesture starting point cannot be matched with the data in the gesture database, it can also be determined that the gesture starting point is wrong, and the starting point can be reconfirmed.

[0069] Combine Figure 2 As shown, the working principle of the rearview mirror adjustment method of the vehicle in the embodiment of the present application is described in detail using an embodiment.

[0070] First, embodiments of the present application may comprise a driver status monitoring system, a gesture recognition module, a rearview mirror drive unit, and a powered rearview mirror. The driver status monitoring system uses a camera to capture the driver's facial and eye information, while the gesture recognition module uses the camera to capture changes in the driver's gestures. The rearview mirror drive unit receives commands from the gesture recognition module and drives the powered rearview mirror for adjustment.

[0071] The camera of the driver status monitoring system can be installed in the cab and use image processing technology to capture the driver's facial and eye information in real time. In the embodiment of the present application, an algorithm can be used to analyze the driver's gaze direction to determine whether the gaze area is on the left, right, or center.

[0072] The gesture recognition module can capture the driver's gesture changes through the camera and can accurately identify the driver's gesture actions such as sliding up and down, waving left and right, etc.

[0073] The mirror drive unit receives commands from the gesture recognition module and controls the electric mirrors for vertical and horizontal adjustment. High-precision motor control ensures accurate and rapid adjustment.

[0074] like Figure 2 As shown, the embodiment of the present application may include the following steps:

[0075] Step S1: Determine the gaze area.

[0076] The driver status monitoring system captures the driver's gaze in real time and determines the area of ​​their gaze. Based on this area of ​​gaze, the system determines whether the driver intends to control the left or right side of the rearview mirror, or whether there is no need to adjust the rearview mirror.

[0077] Step S2: Gesture recognition.

[0078] After determining the driver's intention, the gesture recognition module begins to capture the driver's gesture changes. Based on the recognized gestures, such as swiping up and down, waving left and right, it determines the driver's specific adjustment needs for the rearview mirror.

[0079] Step S3: Rearview mirror adjustment.

[0080] The mirror drive unit adjusts the electric mirrors based on commands from the gesture recognition module. During the adjustment process, the mirrors are monitored in real time to ensure they are accurately adjusted to the driver's desired angle.

[0081] In addition, the embodiment of the present application can also eliminate the possibility of driver's erroneous operation by combining factors such as the external traffic environment and the driver's emotional characteristics, so as to improve the safety of rearview mirror adjustment.

[0082] According to the vehicle rearview mirror adjustment method proposed in the embodiments of the present application, multiple frames of driver image data can be collected to perform driver head recognition and detect the driver's gaze area based on the recognition results. Simultaneously, the driver's gesture change trends are identified, and corresponding gesture adjustment instructions are matched based on the gesture change trends. Target adjustment parameters for the vehicle rearview mirror are determined based on the gesture adjustment instructions. The theoretical adjustment parameters for the vehicle rearview mirror are determined based on the vehicle's surrounding traffic environment parameters. The actual adjustment parameters for the vehicle rearview mirror are then determined by combining the target adjustment parameters, the theoretical adjustment parameters, and preset logical constraints. The actual adjustment parameters for the vehicle rearview mirror are then adjusted using the actual adjustment parameters, achieving automatic adjustment of the vehicle rearview mirror without the need for manual driver operation, resulting in higher adjustment efficiency. Furthermore, by incorporating surrounding traffic environment factors, accidental touches are avoided, further improving the safety of vehicle rearview mirror adjustment. This solves the technical problems of related technologies, such as low efficiency of manual button adjustment by the driver, heavy driver operational burden, high driver attention impact, certain safety risks, low intelligence level, and voice control methods that are unfavorable for drivers with language barriers.

[0083] Next, a rearview mirror adjustment device for a vehicle according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0084] Figure 3 It is a block diagram of a rearview mirror adjustment device for a vehicle according to an embodiment of the present application.

[0085] like Figure 3 As shown, the rearview mirror adjustment device 10 of the vehicle includes: a collection module 100 , a first recognition module 200 and an adjustment module 300 .

[0086] Specifically, the acquisition module 100 is used to acquire multiple frames of image data of the driver.

[0087] The first recognition module 200 is configured to perform head recognition of the driver based on multiple frames of image data, obtain a recognition result, and detect the driver's gaze area according to the recognition result.

[0088] The adjustment module 300 is used to use multi-frame image data to identify the driver's gesture change trend, match the corresponding gesture adjustment instruction based on the gesture change trend, determine the target adjustment parameters of the vehicle rearview mirror according to the gesture adjustment instruction, determine the theoretical adjustment parameters of the vehicle rearview mirror according to the vehicle's surrounding traffic environment parameters, and combine the target adjustment parameters, theoretical adjustment parameters and preset logical constraints to obtain the actual adjustment parameters of the vehicle rearview mirror, so as to adjust the vehicle rearview mirror using the actual adjustment parameters.

[0089] Optionally, in one embodiment of the present application, the vehicle rearview mirror adjustment device 10 further includes: a judgment module and a first control module.

[0090] The judgment module is used to judge whether there is a vehicle rearview mirror in the gaze area.

[0091] The first control module is used to control the vehicle to enter the rearview mirror adjustment mode when there is a vehicle rearview mirror and identify the gesture change trend; otherwise, the vehicle is prohibited from entering the rearview mirror adjustment mode.

[0092] Optionally, in one embodiment of the present application, the recognition module includes: a first recognition unit, a second recognition unit, an estimation unit and a third recognition unit.

[0093] The first recognition unit is used to recognize the driver's facial area from multiple frames of image data.

[0094] The second recognition unit is configured to recognize a plurality of key points in the facial region, so as to determine positions of the driver's eyes and eyebrows based on the plurality of key points.

[0095] The estimation unit is used to estimate the driver's head posture based on the positions of the driver's eyes and eyebrows, and determine the driver's initial gaze direction according to the head posture.

[0096] The third recognition unit is used to perform pupil recognition and iris edge tracking recognition of the driver based on the positions of the driver's eyes and eyebrows, obtain the driver's eye tracking data, and determine the driver's actual gaze direction in combination with the head posture, eye tracking data and initial gaze direction, so as to obtain a gaze area based on the actual gaze direction.

[0097] Optionally, in one embodiment of the present application, the third recognition unit includes: a construction subunit, a calculation subunit and a conversion subunit.

[0098] Among them, the construction subunit is used to construct the vehicle's in-vehicle coordinate system.

[0099] The calculation subunit is used to obtain the eye coordinates in the vehicle coordinate system based on the head posture and the positions of the eyes and eyebrows.

[0100] The conversion subunit is used to convert the actual sight direction into the gaze area in the vehicle coordinate system based on the eye coordinates.

[0101] Optionally, in one embodiment of the present application, the adjustment module 300 includes: an extraction unit, a first matching unit, and a second matching unit.

[0102] The extraction unit is used to extract the driver's hand features from each frame of image data, and sort the hand features according to the time sequence of each frame of image data to obtain a sorting result.

[0103] The first matching unit is used to match the corresponding hand features in the preset gesture database in chronological order to obtain a matching result.

[0104] The second matching unit is used to use the hand features corresponding to the current frame image data as the starting frame of the gesture change when the matching result is a successful match, and determine the gesture change trend based on the gesture change starting frame and the hand features; otherwise, match the hand features corresponding to the next frame image data.

[0105] Optionally, in one embodiment of the present application, the vehicle rearview mirror adjustment device 10 further includes: a second identification module and a second control module.

[0106] The second recognition module is used to recognize the driver's actual control intention based on the gesture change trend.

[0107] The second control module is used to match the corresponding adjustment instructions based on the gesture change trend when the actual control intention is the control intention of the vehicle rearview mirror; otherwise, based on the actual control intention, control the vehicle to execute other control instructions except the vehicle rearview mirror control.

[0108] Optionally, in one embodiment of the present application, the preset logical constraints include: the danger level posed to the vehicle by the surrounding traffic environment, where the danger level is derived from surrounding traffic environment parameters and the vehicle's current driving parameters; and the driver's emotional characteristics, where the driver's emotional characteristics are derived from multiple frames of image data and the driver's voice data. It should be noted that the aforementioned explanation of the embodiment of the vehicle rearview mirror adjustment method also applies to the vehicle rearview mirror adjustment device of this embodiment and will not be repeated here.

[0109] The vehicle rearview mirror adjustment device proposed in an embodiment of the present application can collect multiple frames of driver image data to perform driver head recognition and detect the driver's gaze area based on the recognition results. Simultaneously, it can identify the driver's gesture change trends, match corresponding gesture adjustment instructions based on the gesture change trends, determine target adjustment parameters for the vehicle rearview mirror based on the gesture adjustment instructions, determine theoretical adjustment parameters for the vehicle rearview mirror based on the vehicle's surrounding traffic environment parameters, and combine the target adjustment parameters, theoretical adjustment parameters, and preset logical constraints to obtain actual adjustment parameters for the vehicle rearview mirror. The actual adjustment parameters are then used to adjust the vehicle rearview mirror, achieving automatic adjustment of the vehicle rearview mirror without the need for manual driver operation, resulting in higher adjustment efficiency. Furthermore, by incorporating factors from the surrounding traffic environment, accidental touches can be avoided, further improving the safety of vehicle rearview mirror adjustment. This solves the technical problems of related art, such as low efficiency of manual button adjustment by the driver, heavy driver operation burden, high driver attention loss, certain safety risks, low intelligence level, and voice control methods that are not conducive to drivers with language barriers.

[0110] Figure 4 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle may include:

[0111] Memory 401 , processor 402 , and computer programs stored in the memory 401 and executable on the processor 402 .

[0112] When the processor 402 executes the program, the vehicle rearview mirror adjustment method provided in the above embodiment is implemented.

[0113] Furthermore, the vehicle further comprises:

[0114] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0115] The memory 401 is used to store computer programs that can be run on the processor 402 .

[0116] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0117] If the memory 401, the processor 402, and the communication interface 403 are implemented independently, the communication interface 403, the memory 401, and the processor 402 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0118] Optionally, in a specific implementation, if the memory 401 , the processor 402 and the communication interface 403 are integrated on a chip, the memory 401 , the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0119] The processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0120] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned vehicle rearview mirror adjustment method when executed by a processor.

[0121] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the vehicle rearview mirror adjustment method provided by an embodiment of the present invention.

[0122] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0123] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0124] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0125] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0126] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0127] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0128] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0129] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for adjusting a rearview mirror of a vehicle, characterized in that: The following steps are involved: Collecting multiple frames of image data of the driver; performing head recognition of the driver based on the multiple frames of image data to obtain a recognition result, and detecting a gaze area of ​​the driver according to the recognition result; identifying a trend of a change in the driver's gesture using the multiple frames of image data, matching a corresponding gesture adjustment instruction based on the gesture change trend, determining a target adjustment parameter of a vehicle rearview mirror according to the gesture adjustment instruction, determining a theoretical adjustment parameter of the vehicle rearview mirror according to a traffic environment parameter surrounding the vehicle, and obtaining an actual adjustment parameter of the vehicle rearview mirror by combining the target adjustment parameter, the theoretical adjustment parameter, and preset logical constraints, so as to adjust the vehicle rearview mirror using the actual adjustment parameter; Among them, the preset logical constraints include: the danger level posed by the vehicle's surrounding traffic environment to the vehicle, wherein the danger level is obtained by the surrounding traffic environment parameters and the vehicle's current driving parameters; the driver's emotional characteristics, wherein the driver's emotional characteristics are obtained by the multi-frame image data and the driver's voice data.

2. The method according to claim 1, characterized in that Before using the multiple frames of image data to identify the driver's gesture change trend, the method further includes: Determining whether the vehicle rearview mirror exists in the gaze area; If the vehicle rearview mirror exists, the vehicle is controlled to enter a rearview mirror adjustment mode and the gesture change trend is identified; otherwise, the vehicle is prohibited from entering the rearview mirror adjustment mode.

3. The method according to claim 1, characterized in that The step of performing head recognition of the driver based on the multiple frames of image data to obtain a recognition result, and detecting the driver's gaze area according to the recognition result, includes: identifying a driver's facial region from the plurality of frames of image data; identifying a plurality of key points within the facial region to determine positions of the driver's eyes and eyebrows based on the plurality of key points; estimating the driver's head posture based on the positions of the driver's eyes and eyebrows, and determining the driver's initial gaze direction according to the head posture; Based on the positions of the driver's eyes and eyebrows, the driver's pupil recognition and iris edge tracking recognition are performed to obtain the driver's eye tracking data, and the driver's actual gaze direction is determined in combination with the head posture, the eye tracking data and the initial gaze direction to obtain the gaze area based on the actual gaze direction.

4. The method according to claim 3, characterized in that The obtaining the gaze area based on the actual gaze direction includes: Constructing an in-vehicle coordinate system of the vehicle; Obtaining eye coordinates in the in-vehicle coordinate system based on the head posture and the positions of the eyes and eyebrows; Based on the eye coordinates, the actual sight direction is converted into a gaze area in the in-vehicle coordinate system.

5. The method according to claim 1, wherein The identifying a change trend of the driver's gesture using the multiple frames of image data includes: Extracting hand features of the driver from each frame of image data, and sorting the hand features according to the time sequence of each frame of image data to obtain a sorting result; Matching the corresponding hand features in the preset gesture database in sequence according to the time sequence to obtain a matching result; If the matching result is a successful match, the hand features corresponding to the current frame image data are used as the gesture change starting frame, and the gesture change trend is determined based on the gesture change starting frame and the hand features. Otherwise, the hand features corresponding to the next frame image data are matched.

6. The method according to claim 5, characterized in that After identifying the driver's gesture change trend using the multiple frames of image data, the method further includes: Based on the gesture change trend, identifying the driver's actual control intention; If the actual control intention is the control intention of the vehicle rearview mirror, the corresponding adjustment instruction is matched based on the gesture change trend; otherwise, the vehicle is controlled to execute other control instructions except the vehicle rearview mirror control based on the actual control intention.

7. A rearview mirror adjustment device for a vehicle, characterized in that: include: An acquisition module, used for acquiring multi-frame image data of the driver; a recognition module, configured to recognize the driver's head based on the multiple frames of image data, obtain a recognition result, and detect the driver's gaze area based on the recognition result; an adjustment module, configured to identify a trend of changes in the driver's gestures using the multiple frames of image data, match corresponding gesture adjustment instructions based on the trend of changes in the gestures, determine target adjustment parameters of the vehicle rearview mirror according to the gesture adjustment instructions, determine theoretical adjustment parameters of the vehicle rearview mirror according to parameters of the vehicle's surrounding traffic environment, and obtain actual adjustment parameters of the vehicle rearview mirror by combining the target adjustment parameters, the theoretical adjustment parameters, and preset logical constraints, so as to adjust the vehicle rearview mirror using the actual adjustment parameters; Among them, the preset logical constraints include: the danger level posed by the vehicle's surrounding traffic environment to the vehicle, wherein the danger level is obtained by the surrounding traffic environment parameters and the vehicle's current driving parameters; the driver's emotional characteristics, wherein the driver's emotional characteristics are obtained by the multi-frame image data and the driver's voice data.

8. A vehicle, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle rearview mirror adjustment method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the vehicle rearview mirror adjustment method according to any one of claims 1 to 6.

Images