Control method, control device, streaming media rearview mirror and vehicle

Abstract

The invention provides a control method, a control device, a streaming media rearview mirror and a vehicle, and the method comprises the steps: obtaining first data which comprises second data collected by a sensor of the vehicle and / or third data from external equipment; and controlling a streaming media rearview mirror of the vehicle to prompt the driver according to the first data. The utilization rate of the streaming media rearview mirror can be improved, and the driving safety of a user can be improved.

Description

Technical Field

[0001] The present application relates to the field of smart vehicles, and more specifically, to a control method, a control device, a streaming media rearview mirror and a vehicle. Background Art

[0002] Currently, vehicles can only transmit images captured by the vehicle's camera to the streaming media rearview mirror for display. The driver can only obtain information on both sides of the vehicle from the streaming media rearview mirrors located on both sides of the vehicle doors when controlling the vehicle to turn, or only view the information behind the vehicle through the streaming media rearview mirror located in front of the windshield when it is necessary to view the situation behind the vehicle. In this way, the streaming media rearview mirror can only provide the function of viewing the images captured by the camera, and the utilization rate of the streaming media rearview mirror is low. Summary of the invention

[0003] The present application provides a control method, a control device, a streaming media rearview mirror and a vehicle, which are helpful to improve the utilization rate of the streaming media rearview mirror and also help to improve the driving safety of users.

[0004] In a first aspect, the present application provides a control method, which includes: acquiring first data, the first data including second data collected by a sensor of the vehicle and / or third data from an external device; and based on the first data, controlling a streaming media rearview mirror of the vehicle to prompt the driver.

[0005] In combination with the first aspect, in some possible implementations of the first aspect, the first data includes the second data, and the sensor includes a sensor outside the vehicle cabin, wherein the streaming media rearview mirror of the vehicle is controlled to prompt the driver based on the first data, including: when it is determined that there is a safety risk in the vehicle based on data collected by the sensor outside the cabin, the streaming media rearview mirror is controlled to warn the driver.

[0006] In combination with the first aspect, in some possible implementations of the first aspect, the first data includes second data and third data, the sensor includes a positioning sensor, and the external device includes a device for providing map data, wherein the streaming media rearview mirror of the vehicle is controlled to prompt the driver based on the first data, including: based on the data collected by the positioning sensor and the map data from the external device, the streaming media rearview mirror is controlled to prompt the driver with navigation information.

[0007] In combination with the first aspect, in some possible implementations of the first aspect, the first data includes the second data, and the sensor includes a camera in the cockpit, wherein, based on the first data, the streaming media rearview mirror of the vehicle is controlled to prompt the driver, including: when it is determined that the driver is in a fatigue state based on the image captured by the camera in the cockpit, the streaming media rearview mirror is controlled to issue a fatigue warning to the driver; or when it is determined that the driver is in a distracted state based on the image captured by the camera in the cockpit, the streaming media rearview mirror is controlled to issue a distraction warning to the driver.

[0008] In combination with the first aspect, in some possible implementations of the first aspect, before controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data, the method also includes: obtaining fourth data collected by the sensor; obtaining the driver's information according to the fourth data, the driver's information including one or more of the driver's biometrics, driving habits or driving proficiency; wherein, controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data includes: controlling the streaming media rearview mirror to prompt the driver according to the first data and the driver's information.

[0009] In a second aspect, the present application provides a control device, which includes: an acquisition unit for acquiring first data, the first data including second data collected by a sensor of the vehicle and / or third data from an external device; a control unit for controlling a streaming media rearview mirror of the vehicle to prompt the driver based on the first data.

[0010] In combination with the second aspect, in some possible implementations of the second aspect, the first data includes the second data, and the sensor includes a sensor outside the cabin, wherein the control unit is specifically used to: when it is determined that there is a safety risk in the vehicle based on the data collected by the sensor outside the cabin, control the streaming media rearview mirror to warn the driver.

[0011] In combination with the second aspect, in some possible implementations of the second aspect, the first data includes second data and third data, the sensor includes a positioning sensor, and the external device includes a device for providing map data, wherein the control unit is specifically used to: control the streaming media rearview mirror to prompt navigation information to the driver based on data collected by the positioning sensor and map data from the external device.

[0012] In combination with the second aspect, in some possible implementations of the second aspect, the first data includes the second data, and the sensor is a camera in the vehicle cabin, wherein the control unit is specifically used to: when it is determined that the driver is in a fatigue state based on the image captured by the camera in the cabin, control the streaming media rearview mirror to issue a fatigue warning to the driver; or, when it is determined that the driver is in a distracted state based on the image captured by the camera in the cabin, control the streaming media rearview mirror to issue a distraction warning to the driver.

[0013] In combination with the second aspect, in some possible implementations of the second aspect, the acquisition unit is also used to obtain fourth data collected by the sensor before the control unit controls the streaming media rearview mirror of the vehicle to prompt the driver; based on the fourth data, obtain the driver's information, the driver's information includes one or more of the driver's biometrics, driving habits or driving proficiency; wherein the control unit is specifically used to: control the streaming media rearview mirror to prompt the driver based on the first data and the driver's information.

[0014] In a third aspect, the present application provides a control device, comprising: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the device performs any of the methods described in the first aspect.

[0015] In a fourth aspect, the present application provides a streaming media rearview mirror, which includes one or more prompting devices and the device described in the second or third aspect above, and the one or more prompting devices include a display screen.

[0016] In combination with the fourth aspect, in some possible implementations of the fourth aspect, one or more prompting devices include a voice interaction device.

[0017] In combination with the fourth aspect, in some possible implementations of the fourth aspect, the streaming media rearview mirror includes a data interface, which is used to receive data from sensors and / or external devices.

[0018] In a fifth aspect, the present application provides a vehicle, which includes the device described in the second aspect or the third aspect, or includes the streaming media rearview mirror described in the fourth aspect.

[0019] In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is run on a computer, the computer executes any one of the methods in the first aspect.

[0020] In a seventh aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is run on a computer, the computer executes any one of the methods in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic block diagram of a vehicle provided in an embodiment of the present application.

[0022] Figure 2It is a schematic block diagram of the streaming media rearview mirror provided in an embodiment of the present application.

[0023] Figure 3 It is a schematic flow chart of the control method provided in the embodiment of the present application.

[0024] Figure 4 It is a schematic block diagram of a control device provided in an embodiment of the application.

[0025] Figure 5 It is another schematic block diagram of the streaming media rearview mirror provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. "At least one" means one or more. For example, "at least one of A and B" is similar to "A and / or B", describing the association relationship of associated objects, indicating that three relationships can exist, for example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0027] In the embodiments of the present application, prefixes such as "first" and "second" are used only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present application does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitation should be constituted due to the use of such prefixes. In addition, in the description of the present embodiment, unless otherwise specified, the meaning of "multiple" is two or more.

[0028] Figure 1 A schematic block diagram of a vehicle 100 provided in an embodiment of the present application is shown. The vehicle 100 may include a processor 110, a memory 120, a sensor 130, and a streaming media rearview mirror 140.

[0029] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, or a neural-network processing unit (NPU), etc. Different processing units may be independent components or integrated into one or more processors.

[0030] The memory 120 may include an internal memory. The internal memory may be used to store one or more computer programs, which include instructions. The processor 110 may enable the vehicle 100 to execute the control method provided in some embodiments of the present application by running the above instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. Among them, the program storage area may store an operating system; the program storage area may also store one or more applications (such as navigation applications), etc. The data storage area may store data created during the use of the electronic device 100. In addition, the internal memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disks (or disk storage components), a flash memory component, a universal flash storage (UFS), etc. In some embodiments, the processor 110 may enable the vehicle 100 to execute the control method provided in the embodiments of the present application, run various applications, and perform data processing by running instructions stored in the internal memory and / or instructions stored in a memory provided in the processor 110.

[0031] The sensor 130 includes a camera outside the cabin, a millimeter wave radar, a laser radar, an ultrasonic radar, and a positioning sensor, etc., or the sensor 130 may also include a camera inside the cabin, a biosensor, etc.

[0032] The streaming media rearview mirror 140 may be a streaming media rearview mirror for displaying images of the left area or the right area of ​​the vehicle. For example, the streaming media rearview mirror for displaying the left area of ​​the vehicle may be located near the handle of the left door in the cabin, and the streaming media rearview mirror for displaying the right area of ​​the vehicle may be located near the handle of the right door in the cabin.

[0033] The streaming media rearview mirror 140 may also be a streaming media rearview mirror for displaying an image behind the vehicle. Exemplarily, the streaming media rearview mirror for displaying an image behind the vehicle may be located in front of the windshield of the vehicle.

[0034] Optionally, there is a data interface in the streaming media rearview mirror 140, which can be used to receive data from the sensor 130 and / or an external device.

[0035] In the embodiment of the present application, the processor 110 can obtain data collected by the sensor 130 and / or data from an external device, and control the streaming media rearview mirror 140 to prompt the driver. Alternatively, the streaming media rearview mirror 140 can receive data from the sensor and / or external device through the above data interface, thereby controlling the streaming media rearview mirror 140 to prompt the driver.

[0036] Figure 2 1 is a schematic block diagram of a streaming media rearview mirror 140 provided in an embodiment of the present application. The streaming media rearview mirror 140 includes a processor 141, a memory 142, and a display screen 143. The processor 141 and the memory 142 can refer to the description of the processor 110 and the memory 120, respectively, and will not be repeated here.

[0037] The processor 141 can receive data from the sensor 130 and / or external devices through the data interface, and process the data. The data processing result can be used to control the display screen 143 and other prompting devices to prompt the driver.

[0038] The display screen 143 can be used to display images, videos, etc. The display screen 143 can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), MiniLed, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0039] Optionally, the display screen 143 includes a touch sensor for collecting a user's touch command on the display screen 143. The processor 143 may control the display screen 143 to display a related image based on the touch command.

[0040] In the embodiment of the present application, the streaming media rearview mirror 140 can obtain data collected by the sensor 130 and / or data from external devices, and the processor 141 can control the display screen 143 or other prompting devices in the streaming media rearview mirror to prompt the driver based on the data. This helps to improve the utilization rate of the streaming media rearview mirror and also helps to improve the driving safety of users.

[0041] Optionally, the streaming media rearview mirror 140 includes a voice interaction device 144 .

[0042] The streaming media rearview mirror provided in the embodiment of the present application may include one or more prompting devices, which may include a display screen 143, a voice interaction device 144, and a vibration feedback device. The display screen 143 may be used to provide an intuitive operation interface, and the driver may adjust the display content, reminder mode, etc. of the display screen 143 by touch operation on the display screen 143. The voice interaction device 144 may support the driver to interact with the streaming media rearview mirror 140 through voice commands, so as to facilitate convenient operation during driving. The vibration feedback device may provide the driver with tactile reminders, such as warnings, navigation instructions, etc., through different vibration modes.

[0043] The streaming media rearview mirror 140 also includes a main structure, which can be made of a light and strong material to ensure the durability and stability of the streaming media rearview mirror 140. The design of the main structure can take into account the influence of wind resistance and vibration to reduce the impact on the performance of the streaming media rearview mirror 140 during driving. At the same time, the installation method of the streaming media rearview mirror 140 can also be optimized to ensure that the installation is simple, stable and reliable.

[0044] The above one or more prompting devices may also be referred to as human-computer interaction devices.

[0045] Figure 3 A schematic flow chart of a control method 300 provided in an embodiment of the present application is shown. The method 300 may be executed by the vehicle 100; or, may be executed by the processor 110; or, may be executed by the streaming media rearview mirror 140; or, may be executed by the processor 141. The method 300 includes:

[0046] S310, acquiring first data, where the first data includes second data collected by a sensor of the vehicle and / or third data from an external device.

[0047] Optionally, the sensor may include a sensor outside the cabin, for example, one or more of a camera, a lidar, a millimeter-wave radar or an ultrasonic radar outside the cabin.

[0048] Exemplarily, the sensor includes a camera outside the cabin. The streaming media rearview mirror 140 can obtain images captured by the camera outside the cabin in real time. For example, the image can include one or more traffic elements (such as lane lines, traffic signs, obstacles, and surrounding vehicles, etc.). The processed image can be used to identify traffic elements, analyze road conditions, and detect potential risks.

[0049] Exemplarily, the sensor includes a radar (lidar or millimeter wave radar) outside the cockpit, and the radar can be further divided into long-range radar and short-range radar. Among them, the long-range radar can be used to detect long-distance obstacles in front of the vehicle, including other vehicles, pedestrians, etc. The long-range radar measures the distance and speed of the obstacle by sending electromagnetic waves and receiving the reflected signal, providing the driver with real-time collision warning and obstacle avoidance assistance. The short-range radar can be used to monitor close obstacles around the vehicle, such as obstacle detection when parking or blind spot monitoring when driving at low speed. The short-range radar can have high sensitivity and high resolution, and can accurately measure the position and size of obstacles to ensure that the driver can avoid collisions when parking or changing lanes.

[0050] Exemplarily, the camera outside the cabin and the streaming media rearview mirror 140 may be two independent components; or, the streaming media rearview mirror 140 may also be integrated with the camera outside the cabin.

[0051] Optionally, the sensor may include a sensor in the cabin, such as a camera in the cabin, an eye tracking sensor, an electrocardiogram (ECG) sensor, a biosensor, etc.

[0052] Exemplarily, the camera in the cockpit can be a driver monitoring system (DMS) camera. The data collected by the camera in the cockpit can be used to identify the driver in order to provide personalized driving assistance information. The data collected by the eye tracking sensor can be used to track the driver's gaze in real time and analyze his attention to the streaming media rearview mirror. By setting one or more ECG sensors in the seats in the cockpit, the user's heartbeat information can be read through clothing, and the electromagnetic changes caused by line fluctuations can be monitored, so that the driver's abnormal heartbeat changes can be detected. The data collected by the biosensor can be used to detect the driver's physiological state, such as heart rate, blood pressure or skin temperature, etc., and this information can be used to judge the driver's health status.

[0053] For example, the processor 141 is used as the execution subject of method 300. The processor 141 may have a built-in image processor for performing enhancement, denoising, feature extraction, and other processing on the image captured by the camera (camera inside the cabin or camera outside the cabin) to improve image quality and recognition accuracy. At the same time, deep learning algorithms may be used for image recognition and target tracking to achieve automatic detection and classification of traffic elements.

[0054] Optionally, the sensor may also include one or more of a speed sensor, an acceleration sensor or a steering sensor of the vehicle. The data collected by these sensors may be used to determine the driving habits (or driving style) of the driver.

[0055] Optionally, the sensor may comprise a positioning sensor.

[0056] Exemplarily, the positioning sensor may be a global positioning system (GPS) sensor or a Beidou positioning sensor. The data collected by the positioning sensor may be used to determine the position of the vehicle.

[0057] Optionally, the external device may include a vehicle navigation system or a map server. The vehicle navigation system or the map server may provide map data.

[0058] Exemplarily, the map data may be high-precision map data or standard map data.

[0059] The streaming media rearview mirror 140 can obtain high-precision map data by connecting to the vehicle navigation system or an external map server. The high-precision map data contains traffic elements such as road networks, traffic signs, and speed limit information, providing the driver with accurate navigation and route planning.

[0060] Optionally, the external device includes a wearable device of the driver, such as a smart watch or a smart bracelet of the driver.

[0061] For example, the streaming media rearview mirror can obtain data from the driver's wearable device, which can be used to determine the driver's heart rate, blood pressure, heartbeat and other physiological information.

[0062] Optionally, the external device includes other vehicles.

[0063] For example, the vehicle 100 can communicate with other vehicles via vehicle-to-vehicle (V2V). The streaming media rearview mirror 140 can communicate with other vehicles wirelessly, thereby obtaining information such as the driving status, location, and intention of other vehicles. Through V2V communication, the streaming media rearview mirror 140 can understand the dynamics of other vehicles in real time, predict potential traffic conflicts, and provide timely warnings and avoidance suggestions to the driver.

[0064] Optionally, the external device may include infrastructure on the road, such as traffic lights, road sensors, etc. Vehicles and infrastructure may communicate with each other via vehicle-to-infrastructure (V2I) communication technology.

[0065] S320: Control the streaming media rearview mirror of the vehicle to prompt the driver according to the first data.

[0066] By comprehensively utilizing multi-source information such as sensors and / or external devices, the streaming media rearview mirror 140 can achieve all-round perception and dynamic analysis of the vehicle's surrounding environment. After processing and integrating this information, the streaming media rearview mirror 140 can be controlled to provide the driver with real-time and accurate driving assistance information, effectively improving driving safety and convenience.

[0067] In the embodiment of the present application, by processing the data collected by the sensor and / or the external device, the streaming media rearview mirror 140 can be controlled to intelligently remind the driver, providing the driver with more comprehensive and accurate driving assistance. These intelligent reminder functions not only improve driving safety, but also enhance driving comfort and convenience.

[0068] Optionally, the first data includes third data, and the external device includes traffic lights and / or road sensors, wherein the vehicle's streaming media rearview mirror is controlled to prompt the driver based on the first data, including: controlling the streaming media rearview mirror to prompt the driver with information about the driving route based on data from the traffic lights and / or road sensors.

[0069] In the embodiment of the present application, the streaming media rearview mirror 140 can obtain real-time traffic signal status and road condition information by communicating with infrastructure such as traffic lights and road sensors. This information can be used to plan the driving route for the driver in advance, reduce the driver's waiting time, and also help improve driving efficiency.

[0070] Optionally, the first data includes the second data, and the sensor includes a sensor outside the cabin, wherein based on the first data, the streaming media rearview mirror of the vehicle is controlled to prompt the driver, including: when it is determined that there is a safety risk in the vehicle based on the data collected by the sensor outside the cabin, the streaming media rearview mirror is controlled to warn the driver.

[0071] Optionally, when it is determined that the vehicle poses a safety risk based on data collected by sensors outside the cockpit, the streaming media rearview mirror is controlled to give an early warning to the driver, including: when it is determined that the vehicle poses a safety risk based on data collected by sensors outside the cockpit, the streaming media rearview mirror is controlled to give the driver one or more of vehicle approach reminders, blind spot detection reminders, road abnormality reminders, or illegal driving reminders.

[0072] Taking vehicle approach reminder as an example, when the data collected by the sensor outside the cockpit determines that other vehicles are rapidly approaching or entering the field of view of the streaming media rearview mirror 140, and there is a potential risk of collision between other vehicles and the vehicle 100, the streaming media rearview mirror 140 can be controlled to mark the approaching vehicles through highlighting or a specific color frame, and can also be used in conjunction with a sound alarm (such as a beep) or vibration feedback to remind the driver's attention.

[0073] Taking the blind spot monitoring reminder as an example, when the data collected by the sensor outside the cockpit determines that other vehicles have entered the driver's blind spot area (or the area that the driver cannot directly observe through the streaming media rearview mirror 140), the blind spot monitoring area of ​​the streaming media rearview mirror 140 can be controlled to light up a warning light or display an icon, and further remind the driver to confirm before changing lanes through sound or vibration.

[0074] Taking road abnormality reminder as an example, when the data collected by the sensors outside the cockpit determines that there are obstacles, potholes or other abnormal conditions on the road ahead, the streaming media rearview mirror 140 can be controlled to display icons or marks at the corresponding positions, and at the same time combine with sound reminders to inform the driver to slow down or avoid.

[0075] Taking illegal driving reminders as an example, when data collected by sensors outside the cockpit determines that the vehicle has deviated from the lane, changed lanes without a turn signal, or violated traffic lights and other violations, the streaming media rearview mirror 140 can be controlled to display the violation information and continuously remind the driver through sound and vibration until the driver corrects the behavior.

[0076] Take the highway lane change scenario as an example. The driver is driving on the highway and is about to merge from the right lane into the left lane. At this time, there is a fast-approaching truck behind the left lane. When the fast-approaching truck is detected through the data collected by the radar outside the cockpit, and the image collected by the camera is used to perform target recognition and distance measurement to determine that there is a potential lane change conflict, the streaming media rearview mirror 140 can be controlled to immediately alert the driver through a sound alarm and highlight the left lane area.

[0077] In the embodiment of the present application, after receiving the reminder, the driver can adjust the vehicle speed in time or wait for a safer time to change lanes, thereby avoiding a collision with the vehicle behind.

[0078] When the vehicle is in special weather conditions (for example, the vehicle is in low visibility scenes such as fog or rain), the driver's line of sight is limited, and it is difficult to accurately judge the road conditions and obstacles ahead. The data collected by the radar and camera can detect and identify obstacles ahead under low visibility conditions. At the same time, the map data provided by the map server and the positioning information collected by the positioning sensor can predict the information of the road ahead (for example, curvature and slope). Based on the information of the obstacles and roads ahead, the streaming media rearview mirror 140 can be controlled to provide early warning information to the driver in advance. For example, the streaming media rearview mirror 140 can highlight potential obstacle areas or issue continuous sound alarms to remind the driver to slow down or avoid them.

[0079] In the embodiment of the present application, when the vehicle is in low visibility conditions, intelligent reminders can be given by controlling the streaming media rearview mirror 140, which can help the driver perceive potential risks in advance, reduce traffic accidents caused by poor visibility, and improve driving safety.

[0080] Optionally, the first data includes second data and third data, the sensor includes a positioning sensor, and the external device includes a device for providing map data, wherein the vehicle's streaming media rearview mirror is controlled to prompt the driver based on the first data, including: based on the data collected by the positioning sensor and the map data from the external device, the streaming media rearview mirror is controlled to prompt the driver with navigation information.

[0081] In an embodiment of the present application, by matching the data collected by the positioning sensor with the map data, the streaming media rearview mirror can be controlled to provide one or more of accurate navigation guidance, real-time traffic information, and predictive driving suggestions to help the driver choose the best route and avoid congested areas.

[0082] Optionally, the first data includes second data and third data, the sensor includes a positioning sensor, and the external device includes road infrastructure, wherein the vehicle's streaming media rearview mirror is controlled to prompt the driver based on the first data, including: controlling the streaming media rearview mirror to prompt the driver to slow down or stop based on the data collected by the positioning sensor and the status information of the traffic lights from the road infrastructure.

[0083] For example, a driver is driving on a city road and approaches an intersection. Because the vehicle ahead blocks the driver's sight, the driver cannot directly see the state of the traffic light.

[0084] The traffic light status information is received through the Internet of Vehicles technology, and the distance between the current position of the vehicle and the traffic light is determined in combination with the map data. When it is determined that the driver may miss the change of the traffic light status due to obstructed vision, the streaming media rearview mirror 140 can be controlled to display the status and countdown information of the traffic light in advance, and remind the driver to slow down or stop through sound or vibration.

[0085] In the embodiment of the present application, the driver can understand the status of the traffic lights in advance, which helps to avoid running red lights or emergency braking due to obstructed vision, thereby helping to improve the safety and smoothness of driving.

[0086] Optionally, the first data includes the second data, and the sensor includes a camera in the vehicle cabin, wherein, based on the first data, the vehicle's streaming media rearview mirror is controlled to prompt the driver, including: when it is determined that the driver is in a fatigue state based on the image captured by the camera in the cabin, the streaming media rearview mirror is controlled to issue a fatigue warning to the driver; or when it is determined that the driver is in a distracted state based on the image captured by the camera in the cabin, the streaming media rearview mirror is controlled to issue a distraction warning to the driver.

[0087] For example, when the data collected by the camera in the cockpit detects that the driver is fatigued, distracted or in other abnormal driving conditions, the streaming media rearview mirror 140 can be controlled to sound an alarm and display a prompt message through the display screen 143, which is used to prompt the driver to rest or concentrate.

[0088] Optionally, the method 300 further includes: when the driver is in a fatigued or distracted state, sending an instruction to the automatic driving controller, the instruction being used to instruct the vehicle to park by the roadside. In this way, in extreme cases, the streaming media rearview mirror 140 can be linked with the vehicle's automatic driving system to take safety measures such as controlling the vehicle to park by the roadside and emergency braking.

[0089] Optionally, determining whether the driver is in a fatigued or distracted state based on images captured by a camera in the cockpit includes: inputting the images captured by the camera in the cockpit into a prediction model to obtain a prediction result, and the preset result is used to indicate whether the driver is in a fatigued or distracted state.

[0090] Optionally, before controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data, method 300 also includes: obtaining fourth data collected by the sensor; obtaining driver information according to the fourth data, the driver information including one or more of the driver's biometrics, driving habits or driving proficiency; wherein, controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data includes: controlling the streaming media rearview mirror to prompt the driver according to the first data and the driver's information.

[0091] Exemplarily, the driver's biometrics include the driver's age or gender, etc. For example, the driver's biometrics indicates that the driver is a young driver or an elderly driver. For another example, the driver's biometrics indicates that the driver is a male driver or a female driver.

[0092] Exemplarily, the driver's driving habits can be used to indicate whether the driver is an aggressive driver or a conservative driver. For example, if the number of overtaking times during a period of time (e.g., a week) when the driver is driving the vehicle on the road is greater than or equal to a preset number of times as determined by the data collected by the sensor within a period of time (e.g., a week), the driver can be determined to be an aggressive driver; otherwise, the driver can be determined to be a conservative driver.

[0093] For another example, when it is determined through data collected by the pedal sensor over a period of time (for example, a week) that the number of times the pedal opening change rate is greater than or equal to a preset change rate within a preset time period is greater than or equal to a preset number of times, the driver can be determined to be an aggressive driver; otherwise, the driver can be determined to be a conservative driver.

[0094] The above pedal sensor may be an accelerator pedal sensor or a brake pedal sensor.

[0095] Exemplarily, the driving proficiency of the driver is used to indicate whether the driver is a highly skilled driver or a less skilled driver. For example, when the number of illegal driving of the driver is determined to be greater than or equal to a preset number through data collected by the sensor within a period of time (e.g., a week), the driver can be determined to be a less skilled driver; otherwise, the driver can be determined to be a more skilled driver.

[0096] Exemplarily, the fourth data collected by the above sensors can be used to establish a driver information database. For example, Table 1 shows the corresponding relationship between the driver's face information and the above driver information.

[0097] Table 1

[0098] Driver’s facial information Driver's information Driver A’s facial information Less skilled drivers Driver B’s facial information Elderly drivers … …

[0099] Optionally, as the driver's experience accumulates and changes, the information database will be automatically updated to ensure the accuracy of the reminder service. For example, in one month, the data collected by the sensor can determine that the number of illegal driving of driver A is greater than or equal to the preset number, and driver A can be determined as a driver with low proficiency. In the next month, the data collected by the sensor can determine that the number of illegal driving of driver A is less than the preset number, and driver A can be determined as a driver with high proficiency.

[0100] The corresponding relationship shown in Table 1 above can be integrated into the driver information database. The drivers are classified according to the above driver information, so as to provide appropriate reminder services for drivers of different categories.

[0101] For example, for less skilled drivers, the streaming media rearview mirror 140 can be controlled to provide more detailed reminder services, such as lane departure warning, obstacle detection, front and rear vehicle distance monitoring, etc., while adding multiple reminder methods such as sound and vibration to help novice drivers better master driving skills and improve driving safety.

[0102] For elderly drivers, taking into account their possible vision and hearing impairment problems, the display screen 143 of the streaming media rearview mirror 140 can be controlled to provide a display interface with larger fonts and higher brightness, and a voice prompt function can be added to ensure that the elderly drivers can clearly receive reminder information.

[0103] For more skilled drivers, according to their driving habits and preferences, the streaming media rearview mirror 140 can be controlled to provide a simpler and more efficient reminder service, such as reminding only through sound or slight vibration to avoid excessive interference.

[0104] In the embodiment of the present application, based on the information of different drivers, the streaming media rearview mirror can be controlled to provide personalized driving suggestions at the appropriate time, which helps to improve the flexibility of intelligent reminders and thus helps to improve the user's driving experience.

[0105] Figure 4A schematic block diagram of a control device 400 provided in an embodiment of the present application is shown. The device 400 includes: an acquisition unit 410, used to acquire first data, the first data including second data collected by a sensor of the vehicle and / or third data from an external device; a control unit 420, used to control a streaming media rearview mirror to prompt the driver according to the first data.

[0106] Optionally, the first data includes the second data, and the sensor includes a sensor outside the cabin, wherein the control unit 420 is specifically used to: when it is determined that there is a safety risk to the vehicle based on the data collected by the sensor outside the cabin, control the streaming media rearview mirror to warn the driver.

[0107] Optionally, the first data includes second data and third data, the sensor includes a positioning sensor, and the external device includes a device for providing map data, wherein the control unit 420 is specifically used to control the streaming media rearview mirror to prompt navigation information to the driver based on data collected by the positioning sensor and map data from the external device.

[0108] Optionally, the first data includes the second data, and the sensor includes a camera in the cockpit, wherein the control unit 420 is specifically used to: when it is determined that the driver is in a fatigue state based on the image captured by the camera in the cockpit, control the streaming media rearview mirror to issue a fatigue warning to the driver; or, when it is determined that the driver is in a distracted state based on the image captured by the camera in the cockpit, control the streaming media rearview mirror to issue a distraction warning to the driver.

[0109] Optionally, the acquisition unit 410 is also used to obtain fourth data collected by the sensor before the control unit controls the streaming media rearview mirror to prompt the driver; based on the fourth data, obtain the driver's information, the driver's information includes one or more of the driver's biometrics, driving habits or driving proficiency; wherein the control unit 420 is specifically used to: control the streaming media rearview mirror to prompt the driver based on the first data and the driver's information.

[0110] An embodiment of the present application further provides a control device, which includes: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the device performs the above method.

[0111] An embodiment of the present application also provides a streaming media rearview mirror, which includes one or more prompting devices and the above-mentioned control device, and the one or more prompting devices include a display screen.

[0112] Optionally, the streaming media rearview mirror may include a data interface, which can be used to communicate with sensors and / or external devices.

[0113] Optionally, one or more prompting devices include voice interaction devices.

[0114] For example, Figure 5 Another schematic block diagram of a streaming media rearview mirror provided by an embodiment of the present application is shown, and the streaming media rearview mirror includes an information processing and storage module and a human-computer interaction module, wherein the information processing and storage module is used to obtain data from one or more of the camera, radar, vehicle to everything (V2X) device or map server and perform fusion processing on the data. Based on the fusion processing result, the information processing and storage module can control the human-computer interaction module to prompt the driver, and the prompt content includes but is not limited to one or more of illegal driving reminders, personalized suggestions, driving status detection results, blind spot detection results, road abnormality reminders, and vehicle approaching reminders.

[0115] An embodiment of the present application provides a vehicle, which includes the above-mentioned control device, or includes the above-mentioned streaming media rearview mirror.

[0116] The embodiment of the present application further provides a computer program product, which includes: a computer program code, and when the computer program code is run on a computer, the computer executes the method in the above embodiment.

[0117] The embodiment of the present application further provides a computer-readable medium, wherein the computer-readable medium stores a program code. When the computer program code runs on a computer, the computer executes the method in the above embodiment.

[0118] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0119] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0120] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0123] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A control method, characterized in that: include: Acquire first data, where the first data includes second data collected by a sensor of the vehicle and / or third data from an external device; According to the first data, the streaming media rearview mirror of the vehicle is controlled to prompt the driver.

2. The method according to claim 1, characterized in that The first data includes the second data, the sensor includes a sensor outside the cabin, Wherein, controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data includes: When it is determined that the vehicle has a safety risk based on the data collected by the sensor outside the cabin, the streaming media rearview mirror is controlled to warn the driver.

3. The method according to claim 1 or 2, characterized in that: The first data includes the second data and the third data, the sensor includes a positioning sensor, and the external device includes a device for providing map data. Wherein, controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data includes: According to the data collected by the positioning sensor and the map data from the external device, the streaming media rearview mirror is controlled to prompt navigation information to the driver.

4. The method according to claim 1 or 2, characterized in that: The first data includes the second data, the sensor includes a camera in the cockpit, Wherein, controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data includes: When it is determined that the driver is in a fatigue state according to the image captured by the camera in the cockpit, controlling the streaming media rearview mirror to issue a fatigue warning to the driver; or, When it is determined that the driver is in a distracted state according to the image captured by the camera in the cockpit, the streaming media rearview mirror is controlled to issue a distraction warning to the driver.

5. The method according to claim 1 or 2, characterized in that: Before controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data, the method further includes: Acquiring fourth data collected by the sensor; acquiring the driver's information according to the fourth data, wherein the driver's information includes one or more of the driver's biological characteristics, driving habits or driving proficiency; Wherein, controlling the streaming media rearview mirror of the vehicle to prompt the driver according to the first data includes: According to the first data and the driver's information, the streaming media rearview mirror is controlled to prompt the driver.

6. A control device, characterized in that: include: An acquisition unit, configured to acquire first data, wherein the first data includes second data collected by a sensor of the vehicle and / or third data from an external device; A control unit is used to control the streaming media rearview mirror of the vehicle to prompt the driver according to the first data.

7. The device according to claim 6, characterized in that The first data includes the second data, the sensor includes a sensor outside the cabin, Wherein, the control unit is specifically used for: When it is determined that the vehicle has a safety risk based on the data collected by the sensor outside the cabin, the streaming media rearview mirror is controlled to warn the driver.

8. The device according to claim 6 or 7, characterized in that The first data includes the second data and the third data, the sensor includes a positioning sensor, and the external device includes a device for providing map data. Wherein, the control unit is specifically used for: According to the data collected by the positioning sensor and the map data from the external device, the streaming media rearview mirror is controlled to prompt navigation information to the driver.

9. The device according to claim 6 or 7, characterized in that The first data includes the second data, the sensor includes a camera in the cockpit, Wherein, the control unit is specifically used for: When it is determined that the driver is in a fatigue state according to the image captured by the camera in the cockpit, controlling the streaming media rearview mirror to issue a fatigue warning to the driver; or, When it is determined that the driver is in a distracted state according to the image captured by the camera in the cockpit, the streaming media rearview mirror is controlled to issue a distraction warning to the driver.

10. The device according to claim 6 or 7, characterized in that The acquisition unit is further configured to acquire fourth data collected by the sensor before the control unit controls the streaming media rearview mirror of the vehicle to prompt the driver; acquiring the driver's information according to the fourth data, wherein the driver's information includes one or more of the driver's biological characteristics, driving habits or driving proficiency; Wherein, the control unit is specifically used for: According to the first data and the driver's information, the streaming media rearview mirror is controlled to prompt the driver.

11. A control device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory so that the vehicle performs the method according to any one of claims 1 to 5.

12. A streaming media rearview mirror, characterized in that: The streaming media rearview mirror comprises a device as described in any one of claims 6 to 11 and one or more prompting devices, wherein the one or more prompting devices comprise a display screen.

13. A vehicle, characterized in that: The vehicle includes the device as described in any one of claims 6 to 11, or the vehicle includes the streaming media rearview mirror as described in claim 12.

14. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 5 is implemented.

15. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is run on a computer, the method according to any one of claims 1 to 5 is implemented.

Images