Streaming media rearview mirror control method and controller

Abstract

The embodiment of the invention relates to the technical field of vehicle control, and discloses a streaming media rearview mirror control method and a controller, and the method comprises the steps that the controller can obtain pupil position information and head pose information of a user, and the sight line information of the user can be calculated. The sight information at least comprises the sight range of the user. The controller can generate a control instruction for the streaming media rearview mirror according to the position relation between the sight range and the streaming media rearview mirror. The controller can control the operation state of the streaming media rearview mirror through the control instruction. By means of the technical scheme, the response speed of the streaming media rearview mirror in the using process can be increased, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of vehicle control technology, and in particular to a control method and controller for a streaming media rearview mirror. Background Art

[0002] At present, streaming media rearview mirrors are increasingly being used in new energy vehicles. When the user is driving, the streaming media rearview mirror can start and display video information when it detects the user's gaze. However, this method has the problem of slow response to the operating status of the streaming media rearview mirror. Summary of the invention

[0003] In view of the above problems, an embodiment of the present invention provides a control method and a controller for a streaming media rearview mirror, which are used to solve the problem of untimely response to the operating status of the streaming media rearview mirror in the prior art.

[0004] According to one aspect of an embodiment of the present invention, a method for controlling a streaming media rearview mirror is provided, the method comprising:

[0005] Obtain the user's pupil position information and head posture information;

[0006] Determining the user's sight line information according to the pupil position information and the head posture information; the sight line information includes the user's sight range;

[0007] According to the visual range, the operating state of the streaming media rearview mirror is controlled

[0008] According to another aspect of an embodiment of the present invention, a control device for a streaming media rearview mirror is provided, comprising:

[0009] An acquisition module is used to obtain the user's pupil position information and head posture information;

[0010] A control module is used to determine the user's line of sight information based on the pupil position information and the head posture information; the line of sight information includes the user's line of sight range; and control the operating state of the streaming media rearview mirror based on the line of sight range.

[0011] According to another aspect of an embodiment of the present invention, there is provided a controller, comprising:

[0012] A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0013] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the above method.

[0014] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a controller to execute the above method.

[0015] The embodiment of the present invention calculates the user's line of sight information by acquiring the user's pupil position information and head posture information, and the line of sight information at least includes the user's line of sight range; generates a control instruction for the streaming media rearview mirror according to the positional relationship between the line of sight range and the streaming media rearview mirror; and controls the operating state of the streaming media rearview mirror through the control instruction, thereby improving the response speed of the streaming media rearview mirror during use and improving the user experience.

[0016] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0018] Figure 1 A first structural schematic diagram of a streaming media rearview mirror provided by the present invention is shown;

[0019] Figure 2 A flow chart showing a first embodiment of a method for controlling a streaming media rearview mirror provided by the present invention;

[0020] Figure 3 A flow chart showing a second embodiment of the method for controlling a streaming media rearview mirror provided by the present invention;

[0021] Figure 4 A flow chart showing a third embodiment of the method for controlling a streaming media rearview mirror provided by the present invention;

[0022] Figure 5 A second structural schematic diagram of the streaming media rearview mirror provided by the present invention is shown;

[0023] Figure 6 A schematic structural diagram of an embodiment of a control device for a streaming media rearview mirror provided by the present invention is shown;

[0024] Figure 7 A schematic structural diagram of an embodiment of a controller provided by the present invention is shown. DETAILED DESCRIPTION

[0025] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0026] With the continuous development and popularization of new energy vehicles and artificial intelligence, the intelligent control of vehicles is becoming more and more abundant, and the user experience of vehicles is constantly being optimized and improved. The streaming media rearview mirror is an intelligent and advanced version of the vehicle's rearview mirror. Figure 1 As shown, the streaming media rearview mirror may include a detection module, a control module and an execution module. Among them, the detection module can be used to obtain user information in the vehicle. The control module can realize the generation of control instruction pairs for the streaming media rearview mirror of the vehicle according to the user information obtained by the detection module. The execution module is recorded as the main body of the streaming media rearview mirror, which realizes the execution of the control instructions in the control module. For example, the detection module of the vehicle can detect user information, and if it is found that the user is looking at the streaming media rearview mirror, the detected user information is passed to the control module. The control module controls the streaming media rearview mirror to realize backlight turning on, video display and other operations according to the user information provided by the detection module. Optionally, the control module may specifically include a microcontroller unit (MCU).

[0027] At present, the activation process of the streaming media rearview mirror may specifically include the steps of powering on the main body of the streaming media rearview mirror, powering on the camera of the streaming media rearview mirror, transmitting the video data of the camera to the main body, and displaying the video data on the main body. If the controller executes the above-mentioned process of turning on the streaming media rearview mirror after the detection module detects that the user is looking at the streaming media rearview mirror, then for the user, the user will first see the information reflected by the rearview mirror through the physical reflection function of the mirror surface, and then can see the video information displayed by the streaming media rearview mirror. The data conversion process may consume more attention of the user, thereby bringing safety hazards. In addition, for many users, checking the rearview mirror may be a quick glance. Obviously, the method of starting after the gaze point is detected may appear when the streaming media rearview mirror displays the video and the user has completed the rearview mirror viewing.

[0028] In view of the above problems, the present application proposes a control method for a streaming media rearview mirror. The method can detect the user's line of sight and complete the power-on function of the streaming media rearview mirror in advance, so that the streaming media rearview mirror can respond to the video more quickly when the user is looking at it, so that the user can see the video at the first time, improving the user experience. Optionally, the control method for the streaming media rearview mirror can be implemented and executed by a controller. Optionally, the controller can be Figure 1 The control module shown. Optionally, the controller can be a controller in a vehicle.

[0029] Figure 2 FIG. 1 is a flow chart showing a first embodiment of a method for controlling a streaming media rearview mirror provided by the present invention, wherein the method is executed by a controller. Figure 2 As shown, the method comprises the following steps:

[0030] Step 110: Obtain the user's pupil position information and head posture information.

[0031] The controller can be Figure 1 The detection module shown realizes the acquisition of pupil position information and head posture information.

[0032] Optionally, the detection module may include a camera. Optionally, the camera may monitor the user's facial image in real time. Optionally, the camera may also detect the user's head image in real time. Optionally, the camera may be an infrared camera and / or a visual camera. The controller may obtain the facial image and / or head image through an infrared camera. Alternatively, the controller may obtain the facial image and / or head image through a visual camera. The controller may obtain the facial image and / or head image from a camera. A recognition algorithm may be preset in the controller. The controller may obtain pupil position information and head posture information through the recognition algorithm. Alternatively, the detection module may further include a processor. A recognition algorithm is preset in the processor. The processor may obtain pupil position information and head posture information through the recognition algorithm. The controller may obtain the pupil position information and head posture information from the processor.

[0033] Optionally, the user may be the driver. Alternatively, the user may also be other persons in the vehicle such as the co-pilot, the rear seat passenger, etc.

[0034] Step 120: Determine the user's sight line information according to the pupil position information and the head posture information. The sight line information includes the user's sight range.

[0035] Among them, the controller can determine the user's line of sight information after determining the pupil position based on the eye tracking technology. The controller can also optimize the line of sight information based on the head posture information to obtain more accurate line of sight information. The line of sight information includes at least a line of sight range. The line of sight range is the full range that the user's peripheral vision can see when the user looks in one direction. Optionally, the line of sight information can also include the user's gaze point information. Optionally, the user's gaze point information can specifically include the gaze point position, and / or, gaze point movement information. The gaze point position is the point that the user is currently looking at. Optionally, the gaze point can be a point on the target that the user's line of sight specifically reaches in three-dimensional space. Or, the gaze point can also be a point on the target plane of the present application. The controller can determine the content of the user's gaze based on the user's gaze point on the target plane. The gaze point movement information can be used to indicate the relationship between the two gaze points before and after. Optionally, the movement information of the gaze point can at least include moving into the area corresponding to the streaming media rearview mirror and moving out of the area corresponding to the streaming media rearview mirror. Optionally, the gaze point movement information may also include movement within the area corresponding to the streaming media rearview mirror, movement outside the area corresponding to the streaming media rearview mirror, etc.

[0036] Optionally, the area corresponding to the streaming media rearview mirror may be an area slightly larger than the main body of the streaming media rearview mirror. Alternatively, the area corresponding to the streaming media rearview mirror may be the main body of the streaming media rearview mirror.

[0037] In one example, the specific process of the controller determining the user's sight line information according to the pupil position information and the head posture information may include the following steps:

[0038] Step 121: Determine the user orientation based on the head posture information.

[0039] After obtaining the head posture information, the controller can determine the user's orientation according to the head posture information. Optionally, the direction parallel to the vehicle's driving direction and parallel to the ground can be used as the initial orientation. Optionally, the direction parallel to the vehicle's driving direction and parallel to the ground is the direction directly in front of the driver when driving. Based on the initial orientation, the user orientation can include turning left or right to a first angle, and turning up or down to a second angle.

[0040] Step 122: Determine the gaze point on the target plane according to the pupil position information and the user orientation. The target plane is a plane perpendicular to the vehicle's travel direction and the ground and passing through the streaming media rearview mirror.

[0041] Among them, the controller can determine the user's line of sight direction based on the user's orientation and pupil position information. The controller can determine the target plane based on the vehicle's travel direction, the ground and the streaming media rearview mirror. The target plane can be a plane that passes through the streaming media rearview mirror and is perpendicular to the vehicle's travel direction and the ground. The controller can determine the intersection of the line of sight and the target plane based on the line of sight, and determine the intersection as the gaze point. Optionally, the controller for calculating the gaze point based on the target plane does not need to determine the target object that intersects with the line of sight in three-dimensional space, which greatly reduces the need for spatial modeling and improves the calculation efficiency of the gaze point.

[0042] Step 123: Determine the sight range on the target plane according to the gaze point and the preset sight angle.

[0043] The controller may be preset with a field of view angle. The controller may determine a field of view range on the target plane according to the gaze point and the field of view angle. Optionally, the field of view angle may include four angles: up, down, left, and right. Optionally, the field of view range may be a quadrilateral.

[0044] Step 130: Control the operating state of the streaming media rearview mirror according to the visual range.

[0045] After determining the visual range, the controller can generate a control instruction for the streaming media rearview mirror according to the positional relationship between the visual range and the streaming media rearview mirror. The controller can control the operating state of the streaming media rearview mirror through the control instruction. Optionally, the operating state may include powering on the streaming media rearview mirror, powering off the streaming media rearview mirror, displaying a video, turning off a video, etc.

[0046] In one example, when the streaming media rearview mirror is in a powered-on state, the controller can control the operating state of the streaming media rearview mirror based on the line of sight and gaze point information.

[0047] Among them, when the streaming media rearview mirror is in the power-on state, it means that the streaming media rearview mirror has entered the user's field of vision. At this time, the controller can determine the operating state of the media rearview mirror based on the field of vision and according to the gaze point information. At this time, the adjustment of the operating state implemented by the controller according to the gaze point information may include two states: the streaming media rearview mirror displays the video and the streaming media rearview mirror turns off the video. For example, when the user's gaze point is outside the area corresponding to the streaming media rearview mirror, the controller can control the streaming media rearview mirror to turn off the video. For another example, when the user's gaze point is within the area corresponding to the streaming media rearview mirror, the controller can control the streaming media rearview mirror to display the video. Based on the power-on state, the controller can quickly realize the display and closing of the video, reduce the user perception time, and improve the user experience.

[0048] In another example, when the streaming media rearview mirror is in a powered-on state, the controller can control the operating state of the streaming media rearview mirror based on the line of sight and gaze point movement information.

[0049] The gaze point movement information may include movement within the area corresponding to the streaming media rearview mirror, movement outside the area corresponding to the streaming media rearview mirror, movement into the area corresponding to the streaming media rearview mirror, and movement out of the area corresponding to the streaming media rearview mirror. For different gaze point movement information, the controller can control different operating states of the streaming media rearview mirror.

[0050] In another example, the controller can also determine whether the streaming media rearview mirror falls within the visual range, thereby realizing the control of the operating state of the streaming media rearview mirror. Specifically, if the controller determines that the streaming media rearview mirror falls within the visual range at the third moment, the controller can power on the streaming media rearview mirror. And, at a fourth moment later than the third moment, if the controller determines that the streaming media rearview mirror does not fall within the visual range for a second preset time period, the streaming media rearview mirror is powered off.

[0051] Optionally, the second preset time length can be 2 minutes, 5 minutes, 7 minutes, etc. For example, after the user stops looking at the streaming media rearview mirror, the user status monitoring infrared camera promptly feeds back to the control system based on the real-time detection information. After two minutes, the control system turns off the streaming media rearview mirror backlight, and after 5 minutes, the streaming media rearview mirror assembly system enters the sleep standby mode again.

[0052] For example, after the user enters the car, when the user moves his sight towards the streaming media rearview mirror, when the streaming media rearview mirror enters the user's field of vision, the controller can adjust the state of the streaming media rearview mirror and power on the streaming media rearview mirror. When the user continues to look at the streaming media rearview mirror, the controller can make the streaming media rearview mirror display the video when it is determined that the gaze point reaches the area range corresponding to the streaming media rearview mirror. When the user retracts his sight, the controller can make the streaming media rearview mirror turn off the video when it is determined that the duration of the gaze point leaving the streaming media rearview mirror reaches a first preset duration. When the user continues to retract his sight, the controller can make the streaming media rearview mirror power off when it is determined that the duration of the streaming media rearview mirror leaving the field of vision reaches a second preset duration. Among them, if the user's gaze point enters the area range corresponding to the streaming media rearview mirror again when the gaze point leaves the streaming media rearview mirror, the controller restarts the timing when the gaze point leaves the streaming media rearview mirror.

[0053] In the present application, the controller can obtain the user's pupil position information and head posture information to calculate the user's sight information. The sight information includes at least the user's sight range. The controller can generate a control instruction for the streaming media rearview mirror according to the positional relationship between the sight range and the streaming media rearview mirror. The controller can control the operating state of the streaming media rearview mirror through the control instruction. In the present application, the response speed of the streaming media rearview mirror during use is improved by controlling the operating state of the streaming media rearview mirror according to the sight range, thereby improving the user experience. Optionally, the present application can also improve the energy saving effect of the streaming media rearview mirror by turning off the video after the first preset time and powering off after the second preset time, while ensuring the use effect of the streaming media rearview mirror. The power consumption of the streaming media rearview mirror is generally 10-20W when it is working normally. The operation of turning off the video and powering off makes the streaming media rearview mirror only have this power consumption for a very short time when the human eye looks at the rearview mirror, which can save the power consumption of the streaming media rearview mirror when the vehicle is driving. According to research, after taking into account the different habits of a large number of users in looking at the rearview mirror, the settings of the first preset time length and the second preset time length can generally save more than 90% of power consumption.

[0054] Figure 3 The flowchart of the second embodiment of the control method of the streaming media rearview mirror provided by the present invention is shown, and the method is executed by the controller. Figure 2 Based on the embodiment shown, Figure 3 As shown, when the streaming media rearview mirror is in a powered-on state, the specific process of the controller controlling the operating state of the streaming media rearview mirror according to the line of sight range and the gaze point information may include the following steps:

[0055] Step 210: If it is determined that the streaming media rearview mirror falls within the field of vision and the gaze point position falls within the area corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to display the video.

[0056] The controller can determine whether the streaming media rearview mirror falls within the visual range. If the streaming media rearview mirror falls within the visual range, it means that the streaming media rearview mirror is in a powered-on state, and the controller can continue to determine whether the gaze point falls within the area corresponding to the streaming media rearview mirror. If the gaze point falls within the area corresponding to the streaming media rearview mirror, the controller can control the streaming media rearview mirror to display the video.

[0057] And, if the gaze point does not fall within the area range corresponding to the streaming media rearview mirror, the controller needs to further determine whether the gaze point movement information is moving out of the area range corresponding to the streaming media rearview mirror. If the gaze point movement information indicates that it is moving out of the area range corresponding to the streaming media rearview mirror, the controller can record the duration of the gaze point moving out of the area range corresponding to the streaming media rearview mirror, and when the duration lasts for a first preset duration, the controller can turn off the video. Otherwise, if the gaze point movement information indicates that it is moving out of the area range corresponding to the streaming media rearview mirror and the duration does not reach the first preset duration, the controller will continue to display the video. If the gaze point movement information indicates that the gaze point is only moving outside the area range corresponding to the streaming media rearview mirror, the control should keep the video in the off state.

[0058] In one example, the process of the controller controlling the streaming media rearview mirror to display a video may include:

[0059] Step 211: At the first moment, if it is determined that the streaming media rearview mirror falls within the field of vision, and the gaze point position falls within the area corresponding to the streaming media rearview mirror for the first time, the streaming media rearview mirror is controlled to start displaying the video.

[0060] The first moment may be any moment when the streaming media rearview mirror is powered on. If the controller determines at the first moment that the streaming media rearview mirror falls within the visual range, and the gaze point position falls within the area corresponding to the streaming media rearview mirror for the first time, it means that the user has moved the visual line from other places to the streaming media rearview mirror during the process, and finally reached the streaming media rearview mirror. At this time, the controller can display the video, so that the user can view the situation of the rear vehicle through the streaming media rearview mirror.

[0061] Step 212: At a second moment later than the first moment, if it is determined that the gaze point movement information indicates that the gaze point is moving, and the gaze point position remains within the area corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to keep displaying the video.

[0062] The second moment is a time later than the first moment. That is, the second moment is the moment after the user's gaze point reaches the streaming media rearview mirror. If the gaze point movement information at the second moment indicates that the gaze point is moving, and the gaze point position remains within the area corresponding to the streaming media rearview mirror, it means that the user's gaze point moves within the area corresponding to the streaming media rearview mirror. At this time, the controller controls the streaming media rearview mirror to keep displaying the video.

[0063] Optionally, if the gaze point movement information at the second moment indicates that the gaze point is moving, and the time that the gaze point position leaves the area corresponding to the streaming media rearview mirror does not exceed a first preset time, the controller controls the streaming media rearview mirror to keep displaying the video.

[0064] Step 220: If the streaming media rearview mirror displays a video, and it is determined that the streaming media rearview mirror falls within the field of vision, and the gaze point position does not fall within the area corresponding to the streaming media rearview mirror within a first preset time period, then the streaming media rearview mirror is controlled to turn off the video.

[0065] Among them, the controller can determine whether the streaming media rearview mirror falls within the field of vision. If the streaming media rearview mirror falls within the field of vision, it means that the streaming media rearview mirror is in a powered-on state. At this time, the controller can continue to determine whether the streaming media rearview mirror displays the video. If the streaming media rearview mirror displays the video, it means that when the gaze point position does not fall within the area range corresponding to the streaming media rearview mirror, the user's gaze point moves out of the area range corresponding to the streaming media rearview mirror. At this time, the controller can determine whether the duration of the gaze point moving out of the area range corresponding to the streaming media rearview mirror reaches a first preset duration. If the duration of the gaze point moving out of the area range corresponding to the streaming media rearview mirror reaches the first preset duration, the streaming media rearview mirror is controlled to turn off the video.

[0066] Optionally, if the controller can determine that the duration that the gaze point moves out of the area corresponding to the streaming media rearview mirror does not reach a first preset duration, the controller can control the streaming media rearview mirror to continue displaying the video.

[0067] In the present application, after the streaming media rearview mirror is powered on, the controller can control the streaming media rearview mirror to display the video if it is determined that the streaming media rearview mirror falls within the visual range and the gaze point position falls within the area range corresponding to the streaming media rearview mirror. The controller can also control the streaming media rearview mirror to turn off the video if the streaming media rearview mirror displays the video and it is determined that the streaming media rearview mirror falls within the visual range and the gaze point position does not fall within the area range corresponding to the streaming media rearview mirror for a first preset time period after the streaming media rearview mirror is powered on. The present application determines the operating state of the streaming media rearview mirror by the relationship between the gaze point and the area range corresponding to the streaming media rearview mirror when the streaming media rearview mirror is powered on, thereby improving the control flexibility of the streaming media rearview mirror during use, improving the response speed of the streaming media rearview mirror, and improving the user experience.

[0068] Figure 4 The flowchart of the third embodiment of the control method of the streaming media rearview mirror provided by the present invention is shown, and the method is executed by the controller. Figure 2 and Figure 3 Based on the embodiment shown, Figure 4As shown, the gaze point movement information may include moving into and out of the area corresponding to the streaming media rearview mirror. Then, the method for the controller to control the streaming media rearview mirror according to the gaze point movement information may include the following steps:

[0069] Step 310: Determine gaze point movement information according to the positional relationship between gaze points at two consecutive moments.

[0070] The controller may obtain the gaze point position at the last moment and the gaze point position at the current moment, and may determine the gaze point movement information of the gaze point during the movement process by comparing the position relationship of the gaze point at the two moments.

[0071] In one example, if the gaze point at the previous moment is within the area corresponding to the streaming media rearview mirror, and the gaze point at the current moment is not within the area corresponding to the streaming media rearview mirror, then the gaze point movement information is determined to be moving out of the area corresponding to the streaming media rearview mirror.

[0072] In another example, if the gaze point at the previous moment is not within the area corresponding to the streaming media rearview mirror, and the gaze point at the current moment is within the area corresponding to the streaming media rearview mirror, then the gaze point movement information is determined to be within the area corresponding to the streaming media rearview mirror.

[0073] Step 320: When the streaming media rearview mirror is in the power-on state, if it is determined that the streaming media rearview mirror falls within the field of vision and the gaze point movement information is within the area corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to display the video.

[0074] When the streaming media rearview mirror is powered on, the streaming media rearview mirror is usually within the visual range. If it is determined that the streaming media rearview mirror is usually within the visual range when the streaming media rearview mirror is powered on, the controller can further determine whether the gaze point movement information is within the area range corresponding to the streaming media rearview mirror. If the gaze point movement information is within the area range corresponding to the streaming media rearview mirror, the controller controls the streaming media rearview mirror to display the video.

[0075] Otherwise, if the streaming media rearview mirror is in the powered-on state and it is determined that the streaming media rearview mirror is usually not within the visual range, the decision of the controller is abnormal. The controller can further improve the accuracy of the streaming media rearview mirror in the judgment process, improve user experience, and improve the execution accuracy of the controller through the dual judgment that the streaming media rearview mirror is in the powered-on state and the streaming media rearview mirror is usually not within the visual range.

[0076] Step 330: When the streaming media rearview mirror is powered on and the streaming media rearview mirror displays a video, if it is determined that the streaming media rearview mirror falls within the visual range and the gaze point movement information is out of the area corresponding to the streaming media rearview mirror, then start recording the duration of the gaze point position being out of the area corresponding to the streaming media rearview mirror. If the duration of the out-of-range movement reaches a first preset duration, the streaming media rearview mirror is controlled to turn off the video.

[0077] Among them, when the streaming media rearview mirror is in the power-on state, the controller can also determine whether the streaming media rearview mirror displays a video. If the streaming media rearview mirror displays a video, it means that the user's gaze point is within the area corresponding to the streaming media rearview mirror at the current moment. If at this time, the gaze point movement information is moved out of the area corresponding to the streaming media rearview mirror, the controller can start to record the duration of the gaze point moving out of the area corresponding to the streaming media rearview mirror. If the duration exceeds the first preset duration, it means that the user does not check the streaming media rearview mirror in a short time, and the controller can control the streaming media rearview mirror to turn off the video.

[0078] Optionally, the present application uses the streaming media rearview mirror to display video and viewpoint movement information for dual judgment of moving out of the area corresponding to the streaming media rearview mirror, thereby further improving the judgment accuracy of the controller.

[0079] In the present application, the controller can obtain the gaze point position at the last moment and the gaze point position at the current moment. The controller can determine the gaze point movement information of the gaze point during the movement process by comparing the positional relationship of the gaze points at the two moments. When the streaming media rearview mirror is in the power-on state, if the controller determines that the streaming media rearview mirror falls within the visual range and the gaze point movement information is moved into the area range corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to display the video. When the streaming media rearview mirror is in the power-on state and the streaming media rearview mirror displays the video, if the controller determines that the streaming media rearview mirror falls within the visual range and the gaze point movement information is moved out of the area range corresponding to the streaming media rearview mirror, the gaze point position is started to be recorded. The duration of the movement of the gaze point outside the area range corresponding to the streaming media rearview mirror. If the duration of the movement reaches the first preset duration, the controller can control the streaming media rearview mirror to turn off the video. In the present application, by using the gaze point movement information, the adjustment of the operating state of the streaming media rearview mirror is realized, the control flexibility of the streaming media rearview mirror during use is improved, the response speed of the streaming media rearview mirror is improved, and the user experience is improved.

[0080] On the basis of the above embodiment, the controller can also record the time that the user's gaze point stays on the streaming media rearview mirror when the gaze point stays on the streaming media rearview mirror. If the dwell time is greater than the duration threshold, the controller can send a first warning message. The first warning message can be sent by changing the color of the streaming media rearview mirror. Optionally, the first warning message is used to indicate that the user is looking at the streaming media rearview mirror for a long time. When the user's dwell time at the streaming media rearview mirror is greater than the duration threshold, the user may not pay attention to the road conditions due to the user's long-term gaze at the streaming media rearview mirror, resulting in safety hazards during vehicle driving. Therefore, the controller can implement the warning by sending the first warning message. Among them, the duration threshold can be determined according to the current speed of the vehicle. For example, when the speed is zero, the background color reminder will not change no matter how long the user stares. When the speed is 0 to 10 kilometers per hour, the duration threshold is 5S. When the speed is 10 to 30 kilometers per hour, the duration threshold is 3S. When the speed is 30 to 60 kilometers per hour, the duration threshold is 2S. When the vehicle speed is greater than 60 km / h, the duration threshold is 1S.

[0081] Based on the above embodiment, when the gaze point is within the area corresponding to the streaming media rearview mirror, the controller can also enlarge or highlight the image of the focus area according to the user's gaze point.

[0082] On the basis of the above-mentioned embodiment, when the streaming media rearview mirror displays a video, the streaming media rearview mirror can also automatically adjust the backlight brightness by sensing the intensity of the ambient light through a photosensitive integrated circuit (Sensor IC).

[0083] On the basis of the above-mentioned embodiment, in the driving mode of extreme power saving, the streaming media rearview mirror will not support the wake-up function, turn on the backlight, etc., and only the physical reflection function of the mirror will remain, so as to achieve the ultimate energy saving.

[0084] Based on the above embodiment, when the user status monitoring infrared camera assembly detects that the user is driving fatigued, that is, the person blinks frequently or does not open his eyes for more than one second, the controller will sound an alarm. At the same time, the controller will not execute the power-on and video display actions of the streaming media rearview mirror.

[0085] Based on the above embodiment, during driving, if the controller determines that the rear vehicle is too close to the vehicle based on the video, the controller can also control the streaming media rearview mirror to display a second warning message. The second warning message can be realized by automatically lighting up the screen and changing the color while giving a sound reminder. The second warning message is used to remind the user that the rear vehicle is too close to the vehicle.

[0086] Based on the above embodiment, the circuit of the streaming media rearview mirror can be as follows: Figure 5As shown. The circuit may also include a power supply. The power supply realizes DC power supply for the detection module, the control module and the execution module. The power supply can realize 14.5V DC power supply. Among them, the power supply circuit is used to output various voltages. The detection module may include an infrared camera assembly for monitoring the user status. The detection module can detect the user's status in real time, mainly to detect whether the user is looking at the streaming media rearview mirror, and transmit the information to the control module accurately and timely. The controller module may include an MCU, which is mainly responsible for powering the entire system. The MCU outputs control commands based on the information obtained. The streaming media rearview mirror assembly works according to the command of the control module, including screen wake-up, sleep screen, picture recording, zooming, etc. The execution module can be a streaming media rearview mirror assembly. When the streaming media rearview mirror is powered off, the power consumption of the circuit is extremely low, with only a bottom current of no more than 0.1mA. Meet the ultimate energy-saving needs.

[0087] Figure 6 FIG. 2 is a schematic diagram showing a structure of an embodiment of a control device for a streaming media rearview mirror provided by the present invention. Figure 6 As shown, the device 600 includes: an acquisition module 610 and a control module 620 .

[0088] An acquisition module is used to obtain the user's pupil position information and head posture information;

[0089] The control module 610 is used to determine the user's sight line information according to the pupil position information and the head posture information; the sight line information includes the user's sight range;

[0090] The operating state of the streaming media rearview mirror is controlled according to the visual range.

[0091] Optionally, the sight line information further includes: user's gaze point information; wherein the gaze point information includes gaze point position and / or gaze point movement information;

[0092] The control module 620 is used to:

[0093] When the streaming media rearview mirror is in a powered-on state, the operating state of the streaming media rearview mirror is controlled according to the visual range and the gaze point information.

[0094] Optionally, the control module 620 is configured to:

[0095] When the streaming media rearview mirror is in a powered-on state, if it is determined that the streaming media rearview mirror falls within the visual range and the gaze point position falls within the area range corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to display the video.

[0096] Optionally, the control module 620 is configured to:

[0097] At the first moment, if it is determined that the streaming media rearview mirror falls within the visual range, and the gaze point position falls within the area corresponding to the streaming media rearview mirror for the first time, the streaming media rearview mirror is controlled to start displaying the video;

[0098] At a second moment later than the first moment, if it is determined that the gaze point movement information indicates that the gaze point is moving, and the gaze point position remains within the area corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to keep displaying the video.

[0099] Optionally, the control module 620 is configured to:

[0100] When the streaming media rearview mirror is in a powered-on state and the streaming media rearview mirror displays a video, if it is determined that the streaming media rearview mirror falls within the field of vision and the gaze point position does not fall within the area corresponding to the streaming media rearview mirror for a first preset time period, the streaming media rearview mirror is controlled to turn off the video.

[0101] Optionally, when the gaze point movement information indicates that the gaze point moves into the area corresponding to the streaming media rearview mirror, the control module 620 is configured to:

[0102] When the streaming media rearview mirror is in a powered-on state, if it is determined that the streaming media rearview mirror falls within the visual range and the gaze point movement information is moved within the area corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to display the video.

[0103] Optionally, when the gaze point movement information indicates that the gaze point moves out of the area corresponding to the streaming media rearview mirror, the control module 620 is configured to:

[0104] When the streaming media rearview mirror is powered on and the streaming media rearview mirror displays a video, if it is determined that the streaming media rearview mirror falls within the visual range and the gaze point movement information is moved out of the area corresponding to the streaming media rearview mirror, then start recording the duration of the gaze point position moving out of the area corresponding to the streaming media rearview mirror;

[0105] If the duration of the removal reaches a first preset duration, the streaming media rearview mirror is controlled to turn off the video.

[0106] Optionally, the control module 620 is configured to:

[0107] At the third moment, if it is determined that the streaming media rearview mirror falls within the visual range, the streaming media rearview mirror is powered on;

[0108] At a fourth moment later than the third moment, if it is determined that the streaming media rearview mirror is not within the visual range for a second preset time period, the streaming media rearview mirror is powered off.

[0109] Optionally, the control module 620 is configured to:

[0110] Determining the user orientation according to the head posture information;

[0111] Determine a gaze point on a target plane according to the pupil position information and the user orientation; the target plane is a plane perpendicular to the vehicle's travel direction and the ground and passing through a streaming media rearview mirror;

[0112] The sight range is determined on the target plane according to the gaze point and the preset sight angle.

[0113] Optionally, the control module 620 is configured to:

[0114] The gaze point movement information is determined according to the positional relationship of the gaze point at two consecutive moments.

[0115] Optionally, the control module 620 is configured to:

[0116] If the gaze point at the previous moment is within the area corresponding to the streaming media rearview mirror, and the gaze point at the current moment is not within the area corresponding to the streaming media rearview mirror, then determine that the gaze point movement information is moved out of the area corresponding to the streaming media rearview mirror;

[0117] If the gaze point at the previous moment was not within the area corresponding to the streaming media rearview mirror, and the gaze point at the current moment is within the area corresponding to the streaming media rearview mirror, then the gaze point movement information is determined to be moved into the area corresponding to the streaming media rearview mirror.

[0118] Optionally, the control module 620 is configured to:

[0119] Determining, based on the gaze point position, the dwelling time of the user's gaze point on the streaming media rearview mirror;

[0120] If the stay time is greater than the duration threshold, a first warning message is sent; the first warning message indicates that the user is staring at the streaming media rearview mirror for a long time;

[0121] The duration threshold is determined according to the current speed of the vehicle.

[0122] The control device 600 of the streaming media rearview mirror provided in the embodiment of the present application can execute the above method embodiment. Its specific implementation principle and technical effects can be found in the above method embodiment, and this embodiment will not be repeated here.

[0123] Figure 7 The schematic diagram of the structure of the embodiment of the controller of the present invention is shown, and the specific embodiment of the present invention does not limit the specific implementation of the controller.

[0124] like Figure 7 As shown, the controller may include: a processor (processor) 702 , a communications interface (Communications Interface) 704 , a memory (memory) 706 , and a communication bus 708 .

[0125] The processor 702, the communication interface 704, and the memory 706 communicate with each other via a communication bus 708. The communication interface 704 is used to communicate with other devices such as a client or other server network elements. The processor 702 is used to execute a program 710, which can specifically execute the relevant steps in the above-mentioned control method for a streaming media rearview mirror.

[0126] Specifically, the program 710 may include program code including computer executable instructions.

[0127] The processor 702 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the controller may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0128] The memory 706 is used to store the program 710. The memory 706 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0129] The electronic device provided in this embodiment can be used to execute the above-mentioned streaming media rearview mirror control method, and its implementation method and technical effect are similar, which will not be repeated in this embodiment.

[0130] The present application also provides a computer-readable storage medium, wherein the storage medium stores at least one executable instruction. When the executable instruction is executed on a controller, the controller executes the methods provided in the various embodiments described above.

[0131] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system or other device. In addition, the embodiments of the present invention are not directed to any particular programming language.

[0132] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. Similarly, in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. Wherein, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself is a separate embodiment of the present invention.

[0133] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0134] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0135] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A control method for a streaming media rearview mirror, characterized in that: The method comprises: Obtain the user's pupil position information and head posture information; Determining the user's sight line information according to the pupil position information and the head posture information; the sight line information includes the user's sight range; The operating state of the streaming media rearview mirror is controlled according to the visual range.

2. The method according to claim 1, characterized in that The sight line information also includes: user's gaze point information; wherein the gaze point information includes gaze point position and / or gaze point movement information; According to the visual range, controlling the operating state of the streaming media rearview mirror includes: When the streaming media rearview mirror is in a powered-on state, the operating state of the streaming media rearview mirror is controlled according to the visual range and the gaze point information.

3. The method according to claim 2, characterized in that When the streaming media rearview mirror is in a powered-on state, controlling the operating state of the streaming media rearview mirror according to the visual range and the gaze point information includes: When the streaming media rearview mirror is in a powered-on state, if it is determined that the streaming media rearview mirror falls within the visual range and the gaze point position falls within the area range corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to display the video.

4. The method according to claim 3, characterized in that If it is determined that the streaming media rearview mirror falls within the visual range, and the gaze point position falls within the area corresponding to the streaming media rearview mirror, then controlling the streaming media rearview mirror to display the video includes: At the first moment, if it is determined that the streaming media rearview mirror falls within the visual range, and the gaze point position falls within the area corresponding to the streaming media rearview mirror for the first time, the streaming media rearview mirror is controlled to start displaying the video; At a second moment later than the first moment, if it is determined that the gaze point movement information indicates that the gaze point is moving, and the gaze point position remains within the area corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to keep displaying the video.

5. The method according to claim 3, characterized in that: When the streaming media rearview mirror is in a powered-on state, controlling the operating state of the streaming media rearview mirror according to the visual range and the gaze point information includes: When the streaming media rearview mirror is in a powered-on state and the streaming media rearview mirror displays a video, if it is determined that the streaming media rearview mirror falls within the field of vision and the gaze point position does not fall within the area corresponding to the streaming media rearview mirror for a first preset time period, the streaming media rearview mirror is controlled to turn off the video.

6. The method according to claim 2, characterized in that When the gaze point movement information indicates that the gaze point moves into the area corresponding to the streaming media rearview mirror, when the streaming media rearview mirror is in a powered-on state, the operating state of the streaming media rearview mirror is controlled according to the visual range and the gaze point information, including: When the streaming media rearview mirror is in a powered-on state, if it is determined that the streaming media rearview mirror falls within the visual range and the gaze point movement information is moved within the area corresponding to the streaming media rearview mirror, the streaming media rearview mirror is controlled to display the video.

7. The method according to claim 1, characterized in that According to the visual range, controlling the operating state of the streaming media rearview mirror includes: At the third moment, if it is determined that the streaming media rearview mirror falls within the visual range, the streaming media rearview mirror is powered on; At a fourth moment later than the third moment, if it is determined that the streaming media rearview mirror is not within the visual range for a second preset time period, the streaming media rearview mirror is powered off.

8. The method according to any one of claims 1 to 7, characterized in that Determine the user's sight line information according to the pupil position information and the head posture information; the sight line information includes the user's sight range, including: Determining the user orientation according to the head posture information; Determine a gaze point on a target plane according to the pupil position information and the user orientation; the target plane is a plane perpendicular to the vehicle's travel direction and the ground and passing through a streaming media rearview mirror; The sight range is determined on the target plane according to the gaze point and the preset sight angle.

9. The method according to any one of claims 1 to 7, characterized in that: The line of sight information further includes: user's gaze point information; wherein the gaze point information includes the gaze point position; the method includes: Determining, based on the gaze point position, the dwelling time of the user's gaze point on the streaming media rearview mirror; If the stay time is greater than the duration threshold, a first warning message is sent; the first warning message indicates that the user is staring at the streaming media rearview mirror for a long time; The duration threshold is determined according to the vehicle speed at the current moment.

10. A controller, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the method according to any one of claims 1-9.

Images