A device control method, device and system in a vehicle-mounted system

Abstract

This application provides a device control method, device, and system for an in-vehicle system, relating to the field of terminal technology. It can display road condition information and other data in real time on a device that the driver is focused on during vehicle operation, thereby preventing danger and improving vehicle safety. The in-vehicle system includes a first terminal and a control device. The first terminal detects whether the driver is looking at its display interface. If the driver is detected looking at the display interface, the first terminal sends a gaze message to the control device. Then, in response to the gaze message, the control device sends a real-time road condition image acquired by a second terminal to the first terminal. Upon receiving the road condition image, the first terminal can display or project the image.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a device control method, device and system in a vehicle system. Background Technology

[0002] With the development of intelligent vehicles and vehicle-to-everything (V2X) technology, vehicles are increasingly equipped with intelligent devices. For example, intelligent rearview mirrors, intelligent dashboards, intelligent center consoles, and intelligent dashcams can be installed. Furthermore, drivers can bring their mobile devices such as smartphones and tablets into the vehicle.

[0003] These devices in vehicles not only provide drivers with certain conveniences while driving, but also pose certain safety hazards. For example, if drivers spend a long time looking at their mobile phones or the content on the smart dashboard while driving, they may easily overlook the current road conditions, which could easily lead to traffic accidents. Summary of the Invention

[0004] This application provides a device control method, device, and system for an in-vehicle system, which can display road conditions and other information in a timely manner on the device that the driver is paying attention to during vehicle operation, thereby avoiding danger and improving vehicle driving safety.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, this application provides a device control method for an in-vehicle system. The in-vehicle system may include a first terminal and a control device. The first terminal is equipped with a display screen or has a projection display function. The method includes: the first terminal detecting whether the driver is looking at the display interface of the first terminal, which may be an interface on the display screen or a projected interface; if the driver is detected looking at the display interface of the first terminal, the first terminal may send a gaze message to the control device, which is used to instruct the driver to look at the display interface of the first terminal; then, in response to the gaze message, the control device may send a road condition image obtained in real time by a second terminal using a first camera to the first terminal, so that the first terminal displays or projects the road condition image.

[0007] In this way, when the driver's gaze shifts to any terminal inside the vehicle (i.e., the first terminal), the control equipment in the vehicle system can control the first terminal that the driver is looking at to display the corresponding road condition image, so that the driver can effectively observe the road conditions, thereby avoiding danger and improving the safety of vehicle driving.

[0008] In one possible implementation, a second camera may be positioned around the first terminal. The first terminal detects whether the driver is looking at its display interface by: the first terminal using the second camera to capture an image; when the captured image meets preset conditions, the first terminal determines that the driver is looking at the display interface; otherwise, the first terminal determines that the driver is not looking at the display interface. The preset conditions include that the captured image includes facial features or eye features. In other words, when the first terminal uses the second camera to capture an image containing facial features or eye features, it can be confirmed that the driver is looking at the display interface.

[0009] In one possible implementation, the aforementioned preset condition further includes: the user corresponding to the aforementioned facial or eye features is a driver. That is, after the first terminal uses the second camera to capture an image containing facial or eye features, it can further determine whether the facial or eye features belong to the driver. If the user corresponding to the aforementioned facial or eye features is the driver, it can be confirmed that the driver is looking at the display interface of the first terminal.

[0010] In one possible implementation, before the first terminal detects whether the driver is looking at its display interface, the method further includes: the first terminal determining that the vehicle it is in is in motion. For example, when the vehicle speed exceeds a speed threshold, the first terminal can determine that the vehicle it is in is in motion. In this case, after detecting that the driver is looking at its display interface, the first terminal can display real-time traffic information as described above, enhancing the user experience.

[0011] In one possible implementation, if it is detected that the driver is looking at the display interface of the first terminal, the first terminal sends a gaze message to the control device, including: if it is detected that the driver is looking at the display interface of the first terminal, the first terminal can further determine the duration of the driver's gaze at the display interface of the first terminal; if the duration of the driver's gaze at the display interface of the first terminal is greater than a first threshold, the safety hazard caused by ignoring the road conditions will increase significantly, then the first terminal can send a gaze message to the control device so that the first terminal can display real-time road condition images according to the above method, thereby improving the user experience.

[0012] In one possible implementation, before the control device sends the real-time road condition image acquired by the second terminal using the first camera to the first terminal, the method further includes: the control device determining, based on the vehicle's operating status, that a road condition image in a first direction needs to be displayed; wherein, the control device sending the real-time road condition image acquired by the second terminal using the first camera to the first terminal includes: if the first camera can record the road condition image in the first direction, then the control device sends the real-time road condition image in the first direction acquired by the second terminal to the first terminal. In this way, when the user is looking at the first terminal, the control device can send a road condition image in a specific direction to the first terminal for display.

[0013] In one possible implementation, the control device sends a real-time road condition image acquired by the second terminal using the first camera to the first terminal. This includes: the control device acquiring the real-time road condition image acquired by the first camera from the second terminal (e.g., a dashcam). This road condition image can be the original image captured by the first camera, or it can be an image processed by the second terminal (e.g., image stitching, exposure adjustment, etc.). Then, the control device can send the road condition image to the first terminal for display. Alternatively, the first terminal can also perform image processing on the received road condition image.

[0014] Alternatively, the control device can send a first notification message to the second terminal, causing the second terminal to send the real-time traffic information to the first terminal based on the first notification message. Or, the control device can send a second notification message to the first terminal, causing the first terminal to obtain the real-time traffic information from the second terminal based on the second notification message.

[0015] In one possible implementation, before displaying traffic information on the first terminal, the system further includes: the first terminal displaying the interface of a first application; wherein, displaying traffic information on the first terminal includes: the first terminal displaying a first window and a second window, the first window being used to continue displaying the interface of the first application, and the second window being used to display traffic information. That is, when the user is looking at the first terminal, the first terminal can simultaneously display the interface of the running first application and the latest traffic information via a split-screen method. In this way, when the driver is looking at the first terminal while driving, the first terminal can not only display the application the driver was originally focusing on, but also display real-time traffic information, thereby reducing the safety risk caused by the driver neglecting traffic conditions due to using devices in the in-vehicle system, and without affecting the driver's normal use of devices in the in-vehicle system.

[0016] In one possible implementation, a monitoring app is installed on the first terminal. After the first terminal sends a look message to the control device, the process further includes: the control device sending a start message to the first terminal, instructing the first terminal to run the monitoring app; in response to the start message, the first terminal runs the monitoring app in a second window, so that the monitoring app displays traffic information in the second window. Alternatively, the monitoring app can also run in full-screen mode on the first terminal, in which case the first terminal can display the received traffic information in full-screen mode.

[0017] In one possible implementation, after the monitoring app runs in the second window on the first terminal, the method further includes: the monitoring app sending a traffic condition request to the control device, which requests to obtain traffic condition images; wherein, the control device sends the traffic condition images obtained in real time by the second terminal using the first camera to the first terminal, including: in response to the traffic condition request, the control device sending the traffic condition images obtained in real time by the second terminal using the first camera to the first terminal. In other words, the control device can use the traffic condition request sent by the monitoring app as a trigger condition to trigger the sending of the real-time traffic condition images to the first terminal.

[0018] In one possible implementation, after the first terminal sends a gaze message to the control device, the method further includes: the control device receiving a first alarm message from a third terminal (the third terminal can be any terminal in the vehicle system other than the first and second terminals), the first alarm message indicating that the third terminal has detected an abnormal situation while driving, such as a fuel level warning or tire pressure warning light; in response to the first alarm message, the control device can send a first notification message corresponding to the first alarm message to the first terminal; subsequently, the first terminal can display the first notification message. Since the first terminal is the device currently being gazed at by the driver, displaying the aforementioned first notification message by the gaze device can promptly warn the driver of the currently detected abnormal situation, preventing the driver from missing any abnormal situation occurring in the vehicle.

[0019] Alternatively, when the first terminal detects that the duration of the driver's gaze at its display screen exceeds a preset value, the first terminal may also display a corresponding notification message (such as a second notification message) so that the driver can see the notification message in a timely manner, thereby reducing the safety risks caused by excessive use of the first device.

[0020] In one possible implementation, after the first terminal sends a gaze message to the control device, the method further includes: if no driver's gaze is detected on the display interface of the first terminal within a preset time, the first terminal may send a gaze-off message to the control device, which instructs the driver to look away from the display interface of the first terminal; in response to the gaze-off message, the control device may stop sending the real-time traffic information obtained by the second terminal to the first terminal. Correspondingly, the first terminal may also stop displaying the traffic information.

[0021] In one possible implementation, the first terminal is an in-vehicle terminal (e.g., an electronic rearview mirror, dashboard lights) or a mobile terminal (e.g., a mobile phone, tablet computer, etc.).

[0022] Secondly, this application provides a device control method for an in-vehicle system, the in-vehicle system including a first terminal and a control device; the first terminal is equipped with a display screen or has a projection display function, wherein the device control method is executed by the first terminal, the method including: the first terminal detecting whether the driver is looking at the display interface of the first terminal; if the driver is detected looking at the display interface of the first terminal, the first terminal sends a gaze message to the control device, the gaze message being used to instruct the driver to look at the display interface of the first terminal; the first terminal obtaining a road condition image obtained in real time by a second terminal using a first camera from the control device; the first terminal displaying or projecting the road condition image.

[0023] In one possible implementation, a second camera is disposed around the first terminal; wherein, the first terminal detects whether the driver is looking at the display interface of the first terminal by: the first terminal using the second camera to capture images; when the captured images meet preset conditions, the first terminal determines that the driver is looking at the display interface of the first terminal; otherwise, the first terminal determines that the driver is not looking at the display interface of the first terminal; wherein, the preset conditions include that the captured images include facial features or eye features.

[0024] In one possible implementation, the aforementioned preset conditions also include that the user corresponding to the facial features or eye features is a driver.

[0025] In one possible implementation, before the first terminal detects whether the driver is looking at the display interface of the first terminal, the method further includes: the first terminal determining that the vehicle in which the first terminal is located is in motion.

[0026] In one possible implementation, if it is detected that the driver is looking at the display interface of the first terminal, the first terminal sends a gaze message to the control device, including: if it is detected that the driver is looking at the display interface of the first terminal, the first terminal determines the duration of the driver's gaze at the display interface of the first terminal; if the duration of the driver's gaze at the display interface of the first terminal is greater than a first threshold, the first terminal sends a gaze message to the control device.

[0027] In one possible implementation, before the first terminal obtains the real-time traffic footage acquired by the second terminal using the first camera from the control device, the method further includes: the first terminal displaying the display interface of the first application; wherein, the first terminal displaying the traffic footage includes: the first terminal displaying a first window and a second window, the first window being used to continue displaying the display interface of the first application, and the second window being used to display the traffic footage.

[0028] In one possible implementation, after the first terminal sends a gaze message to the control device, the method further includes: the first terminal receiving a start message sent by the control device, the start message instructing the first terminal to run the monitoring APP; in response to the start message, the first terminal running the monitoring APP in a second window so that the monitoring APP displays traffic conditions in the second window.

[0029] In one possible implementation, after the first terminal sends a gaze message to the control device, the method further includes: if no driver's gaze is detected on the display interface of the first terminal within a preset time, the first terminal sends a gaze-off message to the control device, the gaze-off message being used to instruct the driver to look away from the display interface of the first terminal.

[0030] Thirdly, this application provides a device control method for an in-vehicle system, the in-vehicle system including a first terminal and a control device; the first terminal is equipped with a display screen or has a projection display function, wherein the above-mentioned device control method can be executed by the control device, the road condition image method includes: the control device determining that the driver is looking at the display interface of the first terminal; the control device acquiring a road condition image obtained in real time by a second terminal using a first camera; and the control device sending the road condition image to the first terminal.

[0031] In one possible implementation, the control device determines whether the driver is looking at the display interface of the first terminal by: if the control device receives a first gaze message sent by the first terminal, then the control device determines that the driver is looking at the display interface of the first terminal, the first gaze message indicating that the driver is looking at the display interface of the first terminal; or, if the control device receives a second gaze message sent by a second camera, then the control device determines that the driver is looking at the display interface of the first terminal, the second gaze message indicating that the driver is looking at the second camera, the second camera being located near the first terminal. That is, the first terminal can use its own camera to detect whether the user is looking at the display interface of the first terminal, or a device with a camera located near the first terminal can also detect whether the user is looking at the display interface of the first terminal.

[0032] In one possible implementation, after the control device determines that the driver is looking at the display interface of the first terminal, the method further includes: the control device sending a start message to the first terminal, which instructs the first terminal to run the monitoring APP; wherein, the control device acquiring the real-time road condition image obtained by the second terminal using the first camera includes: if a road condition request is received from the monitoring APP, the control device acquires the real-time road condition image obtained by the second terminal using the first camera.

[0033] In one possible implementation, after the control device determines that the driver is looking at the display interface of the first terminal, the method further includes: the control device receiving an alarm message sent by the third terminal, which is used to indicate that the third terminal has detected an abnormal situation while driving; and the control device sending a notification message corresponding to the alarm message to the first terminal.

[0034] In one possible implementation, after the control device determines that the driver is looking at the display interface of the first terminal, the control device further includes: receiving a look-away message sent by the first terminal, the look-away message being used to instruct the driver to look away from the display interface of the first terminal; in response to the look-away message, the control device stops sending traffic images to the first terminal.

[0035] Fourthly, this application provides a terminal, including: a memory, a communication module, a display screen, and one or more processors; the memory, communication module, display screen, and processors are coupled together. The memory stores computer program code, which includes computer instructions; when the terminal is running, the processor executes one or more computer instructions stored in the memory to cause the terminal to perform the device control method described in any of the preceding claims.

[0036] Fifthly, this application provides a control device, including: a memory, a communication module, and one or more processors; the memory, the communication module, and the processors are coupled together. The memory stores computer program code, including computer instructions; when the control device is running, the processor executes one or more computer instructions stored in the memory to cause the control device to perform the device control method as described in any of the preceding claims.

[0037] Sixthly, this application provides an in-vehicle system including the aforementioned terminal and control device, wherein the terminal and control device can execute the device control method described in any one of the first aspects above when they interact.

[0038] In a seventh aspect, this application provides a computer storage medium including computer instructions that, when executed on a terminal (or control device), cause the device control method described in any of the above claims to be performed on the terminal (or control device).

[0039] Eighthly, this application provides a computer program product that, when run on a terminal (or control device), causes the device control method described in any of the above claims to be executed on the terminal (or control device).

[0040] Understandably, the terminals, control devices, vehicle systems, computer storage media, and computer program products provided above are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0041] Figure 1 A schematic diagram of the architecture of an in-vehicle system provided in this application embodiment. Figure 1 ;

[0042] Figure 2 This is a schematic diagram of the structure of an electronic rearview mirror provided in an embodiment of this application;

[0043] Figure 3 This application provides an illustration of an application scenario for a device control method in a vehicle system. Figure 1 ;

[0044] Figure 4 A schematic diagram of the architecture of an in-vehicle system provided in this application embodiment. Figure 2 ;

[0045] Figure 5 This is a schematic diagram of the structure of a mobile phone provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of the structure of a mobile phone operating system provided in an embodiment of this application;

[0047] Figure 7 A schematic diagram of the structure of a control device provided in this application embodiment. Figure 1 ;

[0048] Figure 8 An interactive schematic diagram of a device control method in a vehicle system provided in an embodiment of this application;

[0049] Figure 9 This application provides an illustration of an application scenario for a device control method in a vehicle system. Figure 2 ;

[0050] Figure 10 This application provides an illustration of an application scenario for a device control method in a vehicle system. Figure 3 ;

[0051] Figure 11 This application provides an illustration of an application scenario for a device control method in a vehicle system. Figure 4 ;

[0052] Figure 12 This application provides an illustration of an application scenario for a device control method in a vehicle system. Figure 5 ;

[0053] Figure 13 This application provides an illustration of an application scenario for a device control method in a vehicle system. Figure 6 ;

[0054] Figure 14 This application provides an illustration of an application scenario for a device control method in a vehicle system. Figure 7 ;

[0055] Figure 15 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application;

[0056] Figure 16 A schematic diagram of the structure of a control device provided in this application embodiment. Figure 2 . Detailed Implementation

[0057] The implementation of this embodiment will now be described in detail with reference to the accompanying drawings.

[0058] refer to Figure 1 The device control method provided in this application embodiment can be applied to Figure 1 The vehicle-mounted system 100 shown.

[0059] The in-vehicle system 100 is generally installed inside a vehicle. For example, the in-vehicle system 100 may include an instrument panel 101a, a center console 101b, an electronic rearview mirror 101c, or a dashcam. Figure 1 (Not shown in the image) Vehicle-mounted terminals 101, etc. These vehicle-mounted terminals 101 can communicate with each other after connecting to the vehicle's bus. The vehicle-mounted terminals 101 are generally fixedly installed on the vehicle body, and these devices are generally not removed from the vehicle by users (such as drivers or passengers). The vehicle-mounted terminal 101 can also be called vehicle-mounted equipment or vehicle-mounted system, etc.

[0060] Still Figure 1 As shown, the in-vehicle system 100 may also include mobile terminals 102 such as mobile phones 102a, tablet computers 102b, or wearable devices 102c. Mobile terminals 102 are generally not permanently installed in the vehicle, but are carried into the vehicle by the user (e.g., driver or passenger) when using the vehicle.

[0061] In the vehicle system 100, both the vehicle terminal 101 and the mobile terminal 102 can be interconnected via a communication network. For example, this communication network can be a wired network or a wireless network. For instance, the communication network can be a local area network (LAN) or a wide area network (WAN), such as the Internet. The aforementioned communication network can be implemented using any known network communication protocol, which can be various wired or wireless communication protocols, such as Ethernet, Universal Serial Bus (USB), FireWire, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), Bluetooth, Wireless Fidelity (Wi-Fi), NFC, Voice over Internet Protocol (VoIP), communication protocols supporting network slicing architecture, or any other suitable communication protocol.

[0062] In this embodiment, the vehicle terminal 101 or mobile terminal 102 in the vehicle system 100 can be equipped with a display screen. For example, the display screen can be an LCD display screen or an OLED display screen. The vehicle terminal 101 or mobile terminal 102 can display relevant content to the user through its own display screen. For example, the instrument panel 101a can display information such as fuel level, vehicle speed, mileage, water temperature, engine tachometer, and various fault indicator lights through its display screen. The center console 101b can display relevant content such as maps or audio-visual entertainment through its display screen. The electronic rearview mirror 101c can display the road conditions on the left and right sides of the vehicle through its display screen.

[0063] Furthermore, in this embodiment, the vehicle terminal 101 or mobile terminal 102 can also monitor whether the driver's gaze is falling on its own display screen. Taking the electronic rearview mirror 101c as an example... Figure 2 As shown, the electronic rearview mirror 101c may include a first camera 201, a second camera 202, and a display screen 203. The first camera 201 is located on the outside of the vehicle and can be used to capture road conditions outside the vehicle, and then display the captured road conditions on the display screen 203. The second camera 202 and the display screen 203 can be located inside or outside the vehicle. For example, the second camera 202 can be located around the display screen 203. When the driver looks at the display screen 203 of the electronic rearview mirror 101c, the second camera 202 can capture the driver's eye image, thereby determining that the driver is looking at the electronic rearview mirror 101c, that is, the driver's line of sight is on the display screen 203 of the electronic rearview mirror 101c.

[0064] In some embodiments, one or more devices in the vehicle system 100 may be configured as control devices (also referred to as servers, central control devices, or master devices, etc.). Figure 3As shown, taking the center console 101b as an example of a control device, when other devices in the vehicle system 100 (such as the electronic rearview mirror 101c) detect that the driver is looking at it, they can send message 1 to the center console 101b. Message 1 is used to report the information that the driver is looking at the electronic rearview mirror 101c. Then, the center console 101b can respond to message 1 to decide whether to display the current road conditions to the driver. For example, the center console 101b can mark the electronic rearview mirror 101c as the device currently being looked at by the driver. Furthermore, if the vehicle is currently in drive, the center console 101b can obtain the road condition image in front of the vehicle from the driving recorder 301 and then send the road condition image in front of the vehicle to the electronic rearview mirror 101c in real time. The electronic rearview mirror 101c can then display the road condition image in front of the vehicle on its display screen. For example, the electronic rearview mirror 101c can simultaneously display the road conditions on the side of the vehicle and the road conditions in front of the vehicle on the display screen through split screen or floating window, so that the driver can pay attention to the road conditions in front of the vehicle at the same time when using the electronic rearview mirror 101c.

[0065] In other words, when the system detects that the driver is looking at any device in the vehicle system 100, in order to prevent the driver from ignoring the current road conditions, the device can report the driver's gaze message to the control device in the vehicle system 100, trigger the control device to determine the specific road conditions that need to be shown to the driver at this time, and then obtain the corresponding road condition image from the dashcam and other related devices, and control the device that the driver is looking at to display the road condition image.

[0066] Thus, during driving, when the driver's gaze shifts to the in-vehicle terminal 101 or mobile terminal 102 inside the vehicle, the control device in the in-vehicle system 100 can control the device the driver is looking at to display the corresponding road condition image, so that the driver can effectively observe the road conditions, thereby avoiding danger and improving the safety of vehicle driving.

[0067] It should be noted that when the control device in the vehicle system 100 detects that the driver is looking at the vehicle's own screen, the corresponding road condition image can also be displayed on the control device in the manner described above. Taking the center console 101b as an example of the control device, when the center console 101b detects that the driver is looking at its own display screen, the center console 101b can also obtain the road condition image in front of the vehicle from the driving recorder 301 and display the road condition image in front of the vehicle in real time on the display screen of the center console 101b, so that the driver can still effectively observe the current road conditions while looking at the center console.

[0068] Or, such as Figure 4As shown, the control device in the vehicle system 100 can also be a server 401 on the network side. In this case, when the vehicle terminal 101 or mobile terminal 102 in the vehicle system 100 detects that the driver is looking at its display screen, it can send a corresponding gaze message to the server 401. Then, the server 401 can respond to the gaze message, instructing a dashcam or other related devices to send the real-time road condition image to the device the driver is looking at, so that the device instructing the driver to look at can display the road condition image, allowing the driver to effectively observe the current road conditions.

[0069] The specific details of how the various devices in the vehicle system 100 interact to implement the above-mentioned device control method will be elaborated in subsequent embodiments, and therefore will not be repeated here.

[0070] For example, taking mobile phone 102a as an example, the mobile terminal 102 in the above-mentioned vehicle system 100 is described. Figure 5 A schematic diagram of the structure of a mobile phone 102a is shown.

[0071] like Figure 5 As shown, mobile phone 102a may include processor 410, external memory interface 420, internal memory 421, universal serial bus (USB) interface 430, antenna 1, antenna 2, mobile communication module 450, wireless communication module 460, audio module 470, speaker 470A, receiver 470B, microphone 470C, headphone jack 470D, sensor module 480, camera 493, display screen 494, etc.

[0072] It is understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the mobile phone 102a. In other embodiments of this application, the mobile phone 102a may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0073] Processor 410 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0074] The processor 410 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. This memory can store instructions or data that the processor 410 has just used or that are used repeatedly. If the processor 410 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0075] In some embodiments, the processor 410 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0076] The wireless communication function of mobile phone 102a can be realized through antenna 1, antenna 2, mobile communication module 450, wireless communication module 460, modem processor and baseband processor.

[0077] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in mobile phone 102a can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0078] The mobile communication module 450 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the mobile phone 102a. The mobile communication module 450 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 450 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 450 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 450 may be housed in the processor 410. In some embodiments, at least some functional modules of the mobile communication module 450 and at least some modules of the processor 410 may be housed in the same device.

[0079] The wireless communication module 460 can provide solutions for wireless communication applications on the mobile phone 102a, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 460 can be one or more devices integrating at least one communication processing module. The wireless communication module 460 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 410. The wireless communication module 460 can also receive signals to be transmitted from processor 410, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0080] In some embodiments, antenna 1 of mobile phone 102a is coupled to mobile communication module 450, and antenna 2 is coupled to wireless communication module 460, enabling mobile phone 102a to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0081] The mobile phone 102a implements its display function through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 410 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0082] Display screen 494 is used to display images, videos, etc. Display screen 494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, mobile phone 102a may include one or N displays 494, where N is a positive integer greater than 1.

[0083] The 102a phone can achieve shooting functions through the ISP, camera 493, video codec, GPU, display 494, and application processor.

[0084] The ISP (Image Signal Processor) is used to process data fed back from the camera 493. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 493.

[0085] Camera 493 is used to capture still images or videos. In some embodiments, mobile phone 102a may include one or N cameras, where N is a positive integer greater than 1. Camera 493 may be a front-facing camera or a rear-facing camera.

[0086] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when the mobile phone 102a is selecting a frequency, the DSP performs Fourier transforms on the frequency energy.

[0087] Video codecs are used to compress or decompress digital video. The 102a mobile phone can support one or more video codecs. Thus, the 102a mobile phone can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0088] The external storage interface 420 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone 102a. The external storage card communicates with the processor 410 through the external storage interface 420 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0089] The internal memory 421 can be used to store computer executable program code, which includes instructions. The processor 410 executes various functional applications and data processing of the mobile phone 102a by running the instructions stored in the internal memory 421. The internal memory 421 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the mobile phone 102a (such as audio data, phonebook, etc.). Furthermore, the internal memory 421 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0090] The 102a mobile phone can perform audio functions, such as music playback and recording, through the audio module 470, speaker 470A, receiver 470B, microphone 470C, headphone jack 470D, and application processor.

[0091] The audio module 470 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. The audio module 470 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 470 may be located in the processor 410, or some functional modules of the audio module 470 may be located in the processor 410.

[0092] The speaker 470A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The mobile phone 102a can listen to music or make hands-free calls through the speaker 470A.

[0093] The receiver 470B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the mobile phone 102a answers a call or voice message, the receiver 470B can be brought close to the user's ear to listen to the voice.

[0094] Microphone 470C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 470C, inputting the sound signal into microphone 470C. Mobile phone 102a can be equipped with at least one microphone 470C. In some embodiments, mobile phone 102a can be equipped with two microphones 470C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, mobile phone 102a can also be equipped with three, four, or more microphones 470C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0095] The 470D headphone jack is used to connect wired headphones. The 470D headphone jack can be a USB 430 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0096] The sensor module 480 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0097] Of course, the mobile phone 102a may also include a charging management module, a power management module, a battery, buttons, an indicator, and one or more SIM card interfaces, etc., and this application embodiment does not impose any restrictions on this.

[0098] Taking mobile phone 102a as an example of the mobile terminal 102 in the aforementioned vehicle system 100, the software system of mobile phone 102a can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of mobile phone 102a.

[0099] Figure 6 This is a software structure block diagram of mobile phone 102a according to an embodiment of this application.

[0100] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime, the system libraries, and the kernel layer.

[0101] like Figure 6 As shown, the application layer can include a series of applications.

[0102] The aforementioned applications may include calling, contacts, camera, gallery, calendar, maps, navigation, Bluetooth, music, video, SMS, and other applications (APPs).

[0103] In this embodiment, a pre-installed monitoring app can be installed in the application layer. The monitoring app can be a system-level application; when the phone 102a is powered on, the monitoring app runs as a process in the application layer. Users generally cannot manually kill the running monitoring app.

[0104] like Figure 6 As shown, the application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.

[0105] Still Figure 6 As shown, the application framework layer can be configured with modules such as the connect observer 601, the eyefocus observer 602, and the first message client 603.

[0106] The first message center 603 can act as a client to interact with other devices in the vehicle system 100. For example, the first message center 603 may include a first sending unit and a first receiving unit. The first sending unit is used to send data to other devices (e.g., control devices) in the vehicle system 100. For example, the first sending unit can encapsulate the data to be sent into corresponding data packets according to preset protocol requirements and send the data packets to the control device. Correspondingly, the first receiving unit can be used to receive data sent by other devices (e.g., control devices) in the vehicle system 100. For example, the first receiving unit can receive data packets sent by the control device and decapsulate the data packets according to preset protocol requirements to obtain the specific data carried in the data packets.

[0107] The connection monitoring module 601 is used to monitor whether the mobile phone 102a has joined the vehicle system 100. For example, when the connection monitoring module 601 detects that the mobile phone 102a has accessed a preset in-vehicle Wi-Fi network, the connection monitoring module 601 can determine that the mobile phone 102a has joined the vehicle system 100. Then, the connection monitoring module 601 can notify the gaze monitoring module 602 to start operating to monitor whether the driver is looking at the mobile phone 102a. Furthermore, after determining that the mobile phone 102a has joined the vehicle system 100, the connection monitoring module 601 can interact with the control device in the vehicle system 100 through the first message center 603, so that the control device knows that the mobile phone 102a is the device that the user is currently looking at in the vehicle system 100.

[0108] For example, after the gaze monitoring module 602 starts running, it can monitor whether the driver is looking at the mobile phone 102a by acquiring images or other means. If it detects that the driver is looking at the mobile phone 102a, it means that the driver's current attention is on the APP (such as a video APP) running on the mobile phone 102a and not on the real-time road conditions. Then the gaze monitoring module 602 can interact with the control device in the vehicle system 100 through the first message center 603 to request the control device to obtain the real-time road condition image.

[0109] Subsequently, the control device can send real-time traffic images to the monitoring app on the mobile phone 102a, which will then display the images on the phone's screen. For example, the monitoring app can utilize the split-screen function of the WindowManager in the application framework layer, displaying the real-time traffic images in one window while simultaneously displaying the video app originally running on the phone 102a in another. This allows the real-time traffic images to be displayed on the device the driver is looking at, preventing the driver from being unable to observe the current traffic conditions while using the devices in the in-vehicle system 100, thus improving driving safety.

[0110] In other embodiments, those skilled in the art may also integrate the relevant functions implemented by one or more modules such as the connection monitoring module 601, the line-of-sight monitoring module 602, and the first message center 603 into the monitoring APP or other APP at the application layer. This application embodiment does not impose any restrictions on this.

[0111] For example, the application framework layer may also include an activity manager, a window manager, a content provider, a resource manager, an input method manager, etc.

[0112] The Activity Manager manages the lifecycle of each application. Applications typically run as activities in the operating system. The Activity Manager schedules the application's activity processes and manages the lifecycle of each application. The Window Manager manages windowed applications. It can retrieve screen size, determine if a status bar is present, lock the screen, and capture screenshots. The Content Provider stores and retrieves data, making it accessible to applications. This data can include videos, images, audio, made and received phone calls, browsing history and bookmarks, phonebook entries, etc. The Resource Manager provides applications with various resources, such as localized strings, icons, images, layout files, and video files.

[0113] like Figure 6 As shown, the Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0114] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0115] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0116] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0117] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics.

[0118] like Figure 6 As shown, the kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, etc., but this application embodiment does not impose any limitations on this.

[0119] See Figure 7 This is a structural diagram of a control device in the vehicle-mounted system 100.

[0120] For example, the control device may include a second message server 701, a device manager 702, and a notification manager 703.

[0121] The second message center 701 can act as a server to interact with other devices in the vehicle system 100. For example, the second message center 701 may include a second sending unit and a second receiving unit. The second sending unit is used to send data to other devices (e.g., mobile phone 102a) in the vehicle system 100. Correspondingly, the second receiving unit is used to receive data sent from other devices (e.g., mobile phone 102a) in the vehicle system 100.

[0122] The device management module 702 can be used to manage the access and deregistration processes of devices in the vehicle system 100. For example, when mobile phone 102a accesses a preset in-vehicle Wi-Fi network, mobile phone 102a can send an access message to the second message center 701 of the control device. The second message center 701 can send the access message of mobile phone 102a to the device management module 702. The device management module 702 can respond to the access message and record mobile phone 102a as a device in the device list. The devices in the device list are the devices currently in the vehicle system 100. Subsequently, the device management module 702 can periodically receive heartbeat data packets sent by mobile phone 102a through the second message center 701. If no heartbeat data packet is received from mobile phone 102a after a preset time, it means that mobile phone 102a has left the aforementioned in-vehicle Wi-Fi network. At this time, the device management module 702 can delete mobile phone 102a from the aforementioned device list, and mobile phone 102a will no longer be included in the vehicle system 100.

[0123] In some embodiments, the device management module 702 can also receive a gaze message from a device in the vehicle system 100 that the driver has registered the device for viewing, via the second message center 701. For example, when mobile phone 102a detects that the driver is looking at it, mobile phone 102a can send a gaze message to the device management module 702 controlling the device. At this time, the device management module 702 can respond to the gaze message and mark mobile phone 102a as the device the driver is looking at in the device list. Furthermore, the device management module 702 can also call the preset interface A when interacting with the dashcam to obtain the real-time road condition image captured by the dashcam. Then, the device management module 702 can send the road condition image to mobile phone 102a for display via the second message center 701, thereby displaying the real-time road condition image on the device the driver is looking at, preventing the driver from being unable to observe the current road conditions while using the device in the vehicle system 100, and improving driving safety.

[0124] In addition, the notification management module 703 can receive alarm messages reported by various devices in the vehicle system 100. For example, when the dashboard detects that the vehicle's fuel level is lower than a preset value, the dashboard can send a fuel level alarm message to the notification management module 703 of the control device. At this time, the notification management module 703 can interact with the device management module 702 to obtain the device that the driver is currently looking at (e.g., mobile phone 102a). Then, the notification management module 703 can send the fuel level alarm message to the mobile phone 102a through the second message center 701, and the mobile phone 102a can display the fuel level alarm message. In this way, the alarm messages that appear in the vehicle in real time in the vehicle system 100 can be displayed on the device that the driver is looking at, so that the driver can promptly perceive any abnormalities that occur in the vehicle system 100.

[0125] It is understood that the software system of the aforementioned control device may adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc., and the embodiments of this application do not impose any restrictions on this.

[0126] In some embodiments, the control device can be any in-vehicle terminal 101 or mobile terminal 102 in the vehicle system 100. For example, the control device can be the center console 101b in the vehicle system 100. Therefore, the center console 101b can function as both an in-vehicle terminal 101 and a vehicle terminal 102. Figure 6 The functions shown can also be implemented as a control device. Figure 7 The relevant functions are shown. At this time, the center console 101b includes both... Figure 6 The related modules shown also include Figure 7 The relevant modules are shown.

[0127] The control method of the device in the above-mentioned vehicle system 100 will be described in detail below with reference to the accompanying drawings and examples.

[0128] For example, after the driver starts the vehicle, the vehicle can provide a preset in-vehicle Wi-Fi network. At this time, in-vehicle terminals such as the dashboard, dashcam, center console, and electronic rearview mirror can access the in-vehicle Wi-Fi network. Furthermore, the driver can connect their mobile terminal, such as a smartphone, to the in-vehicle Wi-Fi network. In this case, the in-vehicle Wi-Fi network and the various devices connected to it form an in-vehicle system 100.

[0129] Alternatively, the driver can use the hotspot function of a mobile device such as a smartphone to provide a pre-configured in-vehicle Wi-Fi network as an access point (AP). The driver can then connect in-vehicle devices such as the dashboard, dashcam, center console, and rearview mirror to the in-vehicle Wi-Fi network created by the smartphone. Similarly, the driver can also connect wearable devices and other mobile devices to the aforementioned in-vehicle Wi-Fi network. In this case, the in-vehicle Wi-Fi network and the various devices connected to it form an in-vehicle system 100.

[0130] In addition to networking various vehicle terminals and mobile terminals to form the vehicle system 100 via Wi-Fi as described above, the vehicle system 100 can also be networked via Bluetooth, ZigBee, or Ultra Wide Band (UWB). This application embodiment does not impose any restrictions on this.

[0131] Taking the center console in a vehicle as an example of the control device in the vehicle system 100, after the vehicle is started, the center console can provide a preset in-vehicle Wi-Fi network. Taking the driver's mobile phone accessing the aforementioned in-vehicle Wi-Fi network as an example, such as... Figure 8 As shown, after the mobile phone connects to the current in-vehicle Wi-Fi network, it can send a connection message to the center console (i.e., the control device), such as MSG_CONNECT.

[0132] For example, in combination Figure 6 The schematic diagram of the mobile phone shows that the connection monitoring module 601 can monitor in real time whether the mobile phone is connected to a preset in-vehicle Wi-Fi network. When the connection monitoring module 601 detects that the mobile phone has connected to the preset in-vehicle Wi-Fi network, it can send a connection message to the central control console through the first message center 603. This connection message indicates that the mobile phone has joined the current in-vehicle Wi-Fi network. For example, the connection message may include device identifiers such as the mobile phone's MAC address or IP address.

[0133] Combination Figure 7The schematic diagram of the control device shown illustrates that after the mobile phone sends the aforementioned connection message to the central control station, the central control station can receive the connection message from the mobile phone through the second message center 701. Furthermore, as... Figure 8 As shown, the central control unit can respond to the connection message and register the mobile phone as a device in the current vehicle system 100. For example, the second message center 701 of the central control unit can send the connection message to its device management module 702. Then, the device management module 702 can add the device identifier of the mobile phone in the connection message to a preset device list, where all devices currently connected to the vehicle system 100 are listed. In this way, each mobile terminal or vehicle terminal, after accessing the vehicle Wi-Fi network, can notify the central control unit using the above method, enabling the central control unit to be aware of all devices connected to the current vehicle system 100.

[0134] Still Figure 8 As shown, after the mobile phone sends the connection message to the central control unit, it can also periodically send heartbeat data packets (MSG_HEARTBEAT) to the central control unit according to the heartbeat mechanism. This allows the central control unit to determine whether the mobile phone has left the current in-vehicle Wi-Fi network, i.e., the current in-vehicle system 100, based on the heartbeat data packets sent by the mobile phone. For example, the mobile phone's first message center 603 can periodically send heartbeat data packets to the central control unit. If the central control unit's second message center 701 receives the heartbeat data packet sent by the mobile phone within a preset time, it means that the mobile phone is still in the current in-vehicle Wi-Fi network, i.e., the mobile phone is online. If the central control unit's second message center 701 does not receive the heartbeat data packet sent by the mobile phone within the preset time, it means that the mobile phone has likely left the current in-vehicle Wi-Fi network, i.e., the mobile phone is offline. In this way, the central control unit can remove the mobile phone's device identifier from the aforementioned device list. Thus, the central control unit will no longer manage the mobile phone as a device in the in-vehicle system 100.

[0135] Of course, the central control unit can also use other methods to detect whether the devices in the above device list have left the vehicle Wi-Fi network, and this application embodiment does not impose any restrictions on this. For example, the central control unit can periodically send detection messages to the devices in the device list. If a response message is received from the corresponding device within a preset time, it means that the device has not left the current vehicle Wi-Fi network; otherwise, it means that the device has left the current vehicle Wi-Fi network.

[0136] Still Figure 8As shown, after the mobile phone sends the aforementioned connection message to the central control unit, it can also begin monitoring whether the driver is looking at the phone's display screen. For example, the phone's gaze monitoring module 602 can call the phone's front-facing camera to start capturing images and identify whether the images contain facial or eye features. When the image captured by the phone contains facial or eye features, the phone's gaze monitoring module 602 can confirm that the driver is looking at the phone's display screen.

[0137] Alternatively, when the image captured by the mobile phone contains facial or eye features, the phone's gaze monitoring module 602 can further determine whether the captured facial or eye features belong to the driver. For example, the phone can pre-store the driver's facial features. When the similarity between the facial features captured by the phone and the pre-stored driver's facial features is greater than a threshold, the phone's gaze monitoring module 602 can confirm that the driver is looking at the phone's screen. As another example, since the driver's seat in the vehicle is generally fixed, when the phone captures facial or eye features, the phone's gaze monitoring module 602 can analyze the environment in which the facial or eye features are located to confirm that the captured facial or eye features belong to the driver.

[0138] Besides mobile phones, other devices in the vehicle system 100 can also monitor whether the driver is looking at their display screen using the above method after connecting to the vehicle's Wi-Fi network. This application embodiment does not impose any restrictions on this. For example, after the electronic rearview mirror is connected to the above-mentioned vehicle Wi-Fi network, the electronic rearview mirror can use a camera to capture images from the driver's seat direction. When the captured image includes facial features or eye features, the electronic rearview mirror can confirm that the driver is looking at the electronic rearview mirror's display screen.

[0139] Of course, the devices in the vehicle system 100 can also monitor whether the driver is looking at their display screen through other methods, and this application embodiment does not impose any limitations on this. For example, fingerprint recognition functions can be set up on each device in the vehicle system 100, and the driver's fingerprint can be pre-stored in each device. In this way, the devices in the vehicle system 100 can identify whether the fingerprint entered by the user is the driver's fingerprint by acquiring the fingerprint entered by the user. If it is the driver's fingerprint, it can be confirmed that the driver is looking at their own display screen; otherwise, it can be confirmed that the driver is not looking at their own display screen at this time.

[0140] For example, a camera can be placed near a device (such as an electronic rearview mirror) in the vehicle system 100. This camera can capture images in real time and send them to the center console (i.e., the control device). When the center console detects that the image sent by the camera contains the driver's facial or eye features, it can confirm that the driver is looking at the electronic rearview mirror. Alternatively, the camera can also identify in real time whether the captured image contains the driver's facial or eye features; if it does, the camera can notify the center console that the driver is looking at the electronic rearview mirror.

[0141] In other embodiments, before monitoring whether the driver is looking at the display screen, the mobile phone or other devices in the vehicle system 100 can also detect whether the vehicle is in motion. For example, after the mobile phone connects to the aforementioned in-vehicle Wi-Fi network, the central control unit can obtain the current vehicle speed. If the current vehicle speed is greater than a speed threshold, it indicates that the vehicle is in motion, and the central control unit can instruct the mobile phone and other devices in the vehicle system 100 to start monitoring whether the driver is looking at the display screen. In this way, when the vehicle is stationary or at a low speed, the driver can normally use the mobile terminal and in-vehicle terminal devices in the vehicle system 100. However, when the vehicle speed is greater than the speed threshold, the driver may ignore the current road conditions when using or operating the devices in the vehicle system 100. At this time, the device control method provided in this application embodiment can be activated to start monitoring whether the driver is looking at the display screen of a certain device in the vehicle system 100.

[0142] In other embodiments, some electronic devices in the vehicle system 100 may also have projection display functionality. For example, device 1 may project the display interface onto a preset area of ​​the vehicle's windshield. In this case, the devices in the vehicle system 100 may also monitor whether the driver is looking at the display interface displayed in the preset area, as described above. For example, a camera may be installed in the preset area, and when the camera detects an image containing the driver's facial features or eye features, it can be determined that the driver is looking at the display interface displayed in the preset area.

[0143] Still Figure 8As shown, when the mobile phone detects that the driver is looking at the phone's screen (or the screen projected by the phone), the phone can send a gaze message (MSG_EYE_FOCUS) to the central control unit. This gaze message indicates that the driver is looking at the phone's screen. For example, the phone's gaze monitoring module 602 can send a preset gaze message to the central control unit through the first message center 603. After receiving the gaze message from the phone, the central control unit can mark the phone as the device the driver is currently looking at. For example, after receiving the gaze message from the phone, the central control unit's second message center 701 can send the gaze message to the central control unit's device management module 702. Then, the device management module 702 can respond to the gaze message and mark the phone as the device being looked at in the device list. In this way, the central control unit, as the control device in the vehicle system 100, can know that the device the driver is currently looking at is the mobile phone. For example, the phone may be running a video app, and the driver may be watching or operating the video app while driving.

[0144] In other embodiments, when the mobile phone detects that the driver is looking at its display screen, it can also monitor the duration of the driver's gaze. For example, if the driver's eye features are detected for three consecutive seconds, the mobile phone can send the aforementioned gaze message to the central control unit. Otherwise, the mobile phone may not send the gaze message to the central control unit. In other words, when the driver continuously focuses on the display screen of a device in the vehicle system 100, the safety hazard caused by neglecting road conditions increases significantly. Therefore, a gaze message can be sent to the control device in the vehicle system 100 so that the control device knows the specific device the driver is currently looking at.

[0145] Still Figure 8 As shown, after the central control unit in the vehicle system 100 marks the mobile phone as a viewing device based on the viewing message sent by the mobile phone, it can obtain real-time traffic images from devices such as dashcams, and instruct the mobile phone to use a preset monitoring APP to display the real-time traffic images. For example, the device management module 702 of the central control unit can send a start message for the monitoring APP to the mobile phone through the second message center 701, such as MSG_START_MONITORAPP. After receiving the start message for the monitoring APP, if the mobile phone is not running the monitoring APP at this time, the mobile phone can open the monitoring APP; if the mobile phone is running the monitoring APP in the background, the mobile phone can switch the monitoring APP to run in the foreground.

[0146] For example, such as Figure 9As shown in (a), before receiving the startup message from the monitoring app, the mobile phone can run the video app in the foreground. At this time, the mobile phone displays the video app's display interface 901. That is, the interface the driver is looking at is display interface 901. After receiving the startup message from the monitoring app, the mobile phone can activate the split-screen function to create two windows: one window to continue running the video app, and the other window to run the monitoring app. For example, the mobile phone can call the split-screen function in the WindowManager to create two windows. At this time, the mobile phone can send a traffic condition request (e.g., MSG_SHOW_TRAFFIC) to the central control unit, requesting the central control unit to obtain real-time traffic condition footage. Then, the central control unit can respond to the traffic condition request, obtain real-time traffic condition footage from the dashcam, and send it to the mobile phone's monitoring app as a video stream. At this time, as... Figure 9 As shown in (b), the mobile phone can continue to display the video APP's display interface 901 in the first window 902 after splitting the screen, and run the monitoring APP in the second window 903 after splitting the screen, displaying the received real-time traffic footage 904 through the monitoring APP.

[0147] It should be noted that the aforementioned road condition image 904 can be the original image captured in real time by the camera on the dashcam, or it can be an image processed by the dashcam (or a device such as a center console or mobile phone) after image processing of the original image captured. For example, the dashcam can stitch together original images captured by multiple cameras at different locations. Another example is that the dashcam can highlight other vehicles that are close to the vehicle in the captured original image, etc. This application embodiment does not impose any limitations on this.

[0148] Alternatively, after receiving the startup message from the monitoring app, the mobile phone can create a floating window on the display interface 901 of the currently displayed video app. The mobile phone can then run the monitoring app within the floating window, displaying real-time traffic information. In this way, when the driver is looking at the phone while driving, the phone can display not only the applications the driver was originally focused on, but also real-time traffic information, thereby reducing the safety risk of the driver neglecting traffic conditions due to using the devices in the vehicle system 100, without affecting the driver's normal use of the devices in the vehicle system 100. Of course, after receiving the startup message from the monitoring app, the mobile phone can also run the monitoring app in full-screen mode, displaying real-time traffic information on the entire mobile phone screen; this embodiment does not impose any limitations on this.

[0149] In some embodiments, after receiving a traffic condition request from a mobile phone or other device, the center console, acting as a control device in the vehicle system 100, can determine whether to obtain a specific traffic condition image for a particular direction based on parameters such as the current vehicle's operating status. For example, when the center console receives a traffic condition request from a mobile phone or other device, if the vehicle is in drive, the center console can determine that it needs to send a traffic condition image of the area in front of the vehicle to the mobile phone. Conversely, if the vehicle is in reverse, the center console can determine that it needs to send a traffic condition image of the area behind and / or to the sides of the vehicle to the mobile phone. In this way, the control device can send the traffic condition image that the driver needs most at this moment to the device that the driver is currently looking at.

[0150] Of course, in addition to obtaining real-time road condition images from the dashcam, the control device can also obtain real-time road condition images from other devices. For example, the control device can obtain road condition images behind the vehicle from a camera at the rear of the vehicle. Or, for example, the control device can obtain road condition images on both sides of the vehicle from the electronic rearview mirrors on both sides of the vehicle; this application embodiment does not impose any limitations on this.

[0151] It can be seen that when a driver is driving, if their gaze shifts to the in-vehicle terminal or mobile terminal, the control equipment in the in-vehicle system can control the device the driver is looking at to display the corresponding road condition image, so that the driver can effectively observe the road conditions, thereby avoiding danger and improving the safety of driving.

[0152] The above embodiments illustrate how a central control unit (i.e., a control device) acquires traffic footage from a dashcam and sends it to a mobile phone. It is understood that a mobile phone can also directly acquire real-time traffic footage from a dashcam. For example, after receiving a traffic request from a mobile phone, the central control unit can include the mobile phone's device identifier in a control message and send it to the dashcam. This allows the dashcam to communicate with the mobile phone based on the device identifier in the control message, thereby sending real-time traffic footage to the mobile phone. Alternatively, after receiving a traffic request from a mobile phone, the central control unit can send the dashcam's identifier to the mobile phone, allowing the mobile phone to communicate with the dashcam based on the identifier and acquire real-time traffic footage from it. Or, the mobile phone can also acquire real-time traffic footage from a network-side server; this embodiment does not impose any limitations on this.

[0153] In some embodiments, after the mobile phone detects that the driver is looking at the phone's display and sends a gaze message to the central control unit, it can continue to detect whether the driver is looking at the phone's display. For example, the phone's gaze monitoring module 602 can continuously capture images through the front-facing camera. After capturing the driver's facial or eye features for a period of time, if no facial or eye features of the driver are captured for a continuous period of time (e.g., 2 seconds), the phone's gaze monitoring module 602 can confirm that the driver's gaze has left the phone's display. At this time, it still... Figure 8 As shown, the mobile phone can send an eye-out message (e.g., MSG_EYE_UNFOCUS) to the central control unit. For example, the mobile phone's eye-tracking module 602 can send an eye-out message to the central control unit through the first message center 603.

[0154] Still Figure 8 As shown, after receiving a gaze-away message from the mobile phone, the central control unit can remove the mobile phone from the gaze-attention device list. For example, after receiving the gaze-away message from the mobile phone, the second message center 701 of the central control unit can send the gaze-away message to the device management module 702 of the central control unit. Then, the device management module 702 can respond to the gaze-away message and remove the mobile phone from the gaze-attention device list. In this way, the central control unit, as the control device in the vehicle system 100, can update the gaze-attention devices in the device list in a timely manner.

[0155] Additionally, even after the central control panel removes the label for the mobile phone as a viewing device, it still works as before. Figure 8 As shown, the central control unit can instruct the monitoring app on the mobile phone to stop displaying real-time traffic footage. For example, the device management module 702 of the central control unit can send a stop message to the monitoring app, such as MSG_CLOSE_MONITORAPP, to the mobile phone via the second message center 701. Simultaneously, the central control unit can also stop acquiring real-time traffic footage from the dashcam and stop sending the traffic footage to the mobile phone. After receiving the stop message from the monitoring app, the first message center 603 of the mobile phone can send the stop message to the monitoring app. Then, the monitoring app can respond to the stop message, exit split-screen mode, and stop displaying real-time traffic footage. At this time, the video app running in another window on the mobile phone can resume full-screen mode and continue running.

[0156] The above embodiment uses a mobile phone as an example of the driver's gaze device. It can be understood that other devices in the vehicle system 100 can also be connected to the vehicle system 100 in the same way as described above, and when the driver is detected looking at the display screen, they can interact with the control device to display the real-time road condition on the display screen, so that the driver can still effectively observe the road condition while looking at the device in the vehicle system 100.

[0157] For example, when changing lanes, a driver's gaze is primarily focused on the left and right sides of the vehicle or on the electronic rearview mirror inside the car to confirm whether there are vehicles in the target lane. However, while checking the electronic rearview mirror, the driver may easily overlook the road conditions in front of the vehicle. If a vehicle in front suddenly slows down, it can easily lead to a dangerous situation.

[0158] In this scenario, after connecting to the vehicle's Wi-Fi network, the electronic rearview mirror can function as a device within the vehicle system 100, monitoring whether the driver is looking at the mirror's display screen. Taking the left-side electronic rearview mirror as an example... Figure 10 As shown, a driver may look at the display screen of the electronic rearview mirror while changing lanes. At this time, the electronic rearview mirror display screen shows the road conditions on the left side of the vehicle (1001). Figure 8 The interaction between the mobile phone and the center console is similar. If the electronic rearview mirror detects that the driver is looking at its display, it can send a gaze message to the center console. Upon receiving this message, the center console can mark the electronic rearview mirror as the device the driver is currently looking at. At this point, the center console can obtain real-time traffic footage from devices such as a dashcam and send it to the electronic rearview mirror. Simultaneously, the center console can instruct the electronic rearview mirror to display the real-time traffic footage using a monitoring app.

[0159] For example, such as Figure 11 As shown, after the electronic rearview mirror activates the monitoring app according to the instructions on the central control panel, it can use the split-screen function to continue displaying the road condition image 1001 on the left side of the vehicle in the first window 1101 after the split screen, and display the received road condition image 1103 in the second window 1102 after the split screen. In this way, when the driver looks at the electronic rearview mirror while driving, the electronic rearview mirror can not only display the road condition image on the side of the vehicle in real time, but also the road condition image in front of the vehicle in real time, so that the driver can effectively observe the road conditions in all directions of the vehicle, thereby avoiding danger and improving driving safety.

[0160] The interaction details between the electronic rearview mirror, as a viewing device in the vehicle system 100, and the center console can be found in the interaction details between the mobile phone and the center console in the above embodiment, so they will not be repeated here.

[0161] In addition to mobile phones and electronic rearview mirrors, other devices in the vehicle system 100 can detect whether the driver is looking at their own display screen using the above method, and interact with the control device when the driver is looking at their own display screen, so that the control device can display real-time road conditions on the display screen that the driver is looking at, thereby avoiding danger and improving vehicle driving safety.

[0162] In other embodiments, the center console is still used as an example of the control device in the above-mentioned vehicle system 100. In addition to interacting with other devices in the vehicle system 100 as a control device, the center console can also be used as a vehicle terminal to detect whether the driver is looking at its own display screen according to the above method.

[0163] For example, a driver might look at the content displayed on the center console while driving. At this time, the driver's attention is easily diverted from the current road conditions, which can easily lead to danger.

[0164] In this scenario, after the center console connects to the vehicle's Wi-Fi network, it can function as a device within the vehicle system 100, monitoring whether the driver is looking at the center console's display screen. For example... Figure 12 As shown, the driver may look at the center console display screen while driving. At this time, the center console display screen shows interface 1201 of the broadcast function. Similar to the above embodiment, if the center console's gaze monitoring module 602 detects that the driver is looking at the center console display screen, the center console's gaze monitoring module 602 can send a gaze message to its own device management module 702. Then, the center console's device management module 702 can mark the center console as the device the driver is currently looking at. At this time, on the one hand, the center console can obtain real-time road condition images of the road ahead of the vehicle from devices such as a dashcam; on the other hand, the center console can launch a monitoring APP through the split-screen function and display the obtained road condition images through the monitoring APP.

[0165] For example, such as Figure 13 As shown, after the center console activates the split-screen function, the broadcast function interface 1201 can continue to be displayed in the first window 1301 after the split screen, and the received road condition image 1303 ahead of the vehicle can be displayed in the second window 1302 after the split screen. In this way, the center console can not only serve as a control device in the vehicle system 100, but also display the current road condition image in real time when the driver is detected looking at the center console, so that the driver can effectively observe the current road conditions, thereby avoiding danger and improving the safety of vehicle driving.

[0166] In other embodiments, after devices such as mobile phones, electronic rearview mirrors, and center consoles detect that the driver is looking at their displays, they can further detect the duration of the driver's gaze at the displays. If the duration of the driver's gaze at their displays exceeds a preset value (e.g., 30 seconds), it indicates that the driver's prolonged gaze at the devices in the vehicle system 100 poses a significant safety risk.

[0167] For example, if a mobile phone detects that the driver is looking at its display screen for more than 30 seconds, it can send an alarm message to the control device (e.g., the center console) in the vehicle system 100. Upon receiving the alarm message, the center console can determine that the device the driver is currently looking at is the mobile phone. Then, the center console can send a notification message corresponding to the alarm message to the mobile phone, which will then display the notification message, thus warning the driver not to look at the phone for extended periods and to pay attention to the current road conditions. Because the center console can accurately send the notification message corresponding to the received alarm message to the device the driver is currently looking at, the driver can see the notification message promptly, reducing the safety risks caused by excessive use of the devices in the vehicle system 100.

[0168] For example, the mobile phone can send an alarm message to the notification management module 703 of the central control unit via the first message center 603. Then, the notification management module 703 of the central control unit can interact with the device management module 702 to obtain information from the device list recorded by the device management module 702 that the device the driver is currently looking at is the mobile phone. Furthermore, the notification management module 703 of the central control unit can send the corresponding notification message to the monitoring APP on the mobile phone via the second message center 701. Figure 14 As shown, when the phone receives a notification message from the central control unit, it is running a monitoring app via split-screen functionality. At this time, the monitoring app can display the notification message 1401 indicating excessive gaze time via banner, animation, or floating window. Alternatively, the central control unit can send the corresponding notification message to the phone's notification manager, which is used to handle notifications. The notification manager can then display the message using existing notification display mechanisms. Of course, the phone can also alert the driver to the excessive gaze time notification message via vibration or voice.

[0169] Alternatively, when the central control unit receives an alarm message from the mobile phone indicating that the driver's gaze time has been too long, the central control unit can also control the mobile phone to forcibly turn off the screen or forcibly close the applications running on the mobile phone, so as to avoid the driver from gazing at the devices in the vehicle system 100 for a long time, which may cause greater safety risks.

[0170] Alternatively, when the phone detects that the driver is looking at its display screen for more than 30 seconds, since the phone has already detected that the driver is looking at its own screen, the phone can directly display the aforementioned warning message about excessive gaze time, without having to send the aforementioned warning message to the center console.

[0171] It should be noted that, in addition to reporting alarm messages to the center console when the driver looks at the display screen for a duration exceeding a preset value, the devices in the vehicle system 100 can also report alarm messages to the center console when an abnormal situation occurs in the vehicle or an abnormal situation is detected.

[0172] For example, when the vehicle detects that the fuel level is below a certain threshold, it can report a corresponding warning message to the center console. Similarly, when the vehicle detects abnormal tire pressure, it can report a corresponding warning message to the center console. Likewise, when the vehicle detects that it is too close to the vehicle in front or behind, it can report a corresponding warning message to the center console. Finally, when the vehicle detects an engine malfunction, it can report a corresponding warning message to the center console.

[0173] After receiving alarm messages from different devices in the vehicle system 100, the central control console can query the device the driver is currently looking at using the method described above. Then, the central control console can send a notification message corresponding to the alarm message to the currently looking device, which will then display the notification message, thereby promptly alerting the driver to the detected anomaly and preventing the driver from missing any abnormalities in the vehicle.

[0174] Of course, when an abnormal situation occurs in the vehicle or an abnormal situation is detected, the various devices in the vehicle system 100 can, in addition to reporting alarm messages to the central control panel, also alert the driver to the abnormal situation in the existing alarm methods. For example, the instrument panel can display a low fuel indicator light, and the vehicle's horn can play a warning sound when the distance to the vehicle in front or behind is too close. This application embodiment does not impose any limitations on this.

[0175] It should be noted that the above embodiments use the center console as an example of the control device in the vehicle system 100. It is understood that the above control device can also be an electronic device with or without a display screen, such as a mobile phone or a car audio system. This application embodiment does not impose any restrictions on this.

[0176] like Figure 15As shown in the illustration, this application discloses a terminal, which can be either the vehicle terminal 101 (e.g., an electronic rearview mirror) in the aforementioned vehicle system 100, or the mobile terminal 102 (e.g., a mobile phone) in the aforementioned vehicle system 100. Specifically, the terminal may include: a display screen 1501; one or more processors 1502; a memory 1503; a communication module 1506; one or more application programs (not shown); and one or more computer programs 1204. These components can be connected via one or more communication buses 1505. Of course, the terminal may also include devices such as a camera and a touch sensor. The one or more computer programs 1504 are stored in the aforementioned memory 1503 and configured to be executed by the one or more processors 1502. The one or more computer programs 1504 include instructions that can be used to execute the relevant steps performed by the vehicle terminal 101 or the mobile terminal 102 in the above embodiments.

[0177] like Figure 16 As shown in the illustration, this application discloses a control device, which can be the center console or similar component in the aforementioned vehicle system 100. Specifically, the control device may include: a communication module 1601; one or more processors 1602; a memory 1603; one or more application programs (not shown); and one or more computer programs 1604. These components can be connected via one or more communication buses 1605. The one or more computer programs 1604 are stored in the memory 1603 and configured to be executed by the one or more processors 1602. The one or more computer programs 1604 include instructions that can be used to perform the relevant steps executed by the control device in the above embodiment.

[0178] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0179] In the embodiments of this application, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0181] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A device control method in a vehicle-mounted system, characterized in that, The vehicle-mounted system includes a first terminal and a control device; the first terminal is equipped with a display screen or has a projection display function, wherein... The first terminal is a vehicle-mounted terminal, which includes a first camera and a second camera. The first camera is disposed on the outside of the vehicle body and is used to collect real-time images of the first road conditions in the first direction. The second camera is disposed around the display interface of the first terminal and is used to collect images. The first terminal detects whether the driver is looking at the display interface of the first terminal; If the driver is detected looking at the display interface of the first terminal, the first terminal determines the duration of the driver looking at the display interface of the first terminal. If the driver gazes at the display interface of the first terminal for a duration greater than a first threshold, the first terminal sends a gaze message to the control device. In response to the gaze message, the control device sends a real-time road condition image of a second direction, acquired by the second terminal using a third camera, to the first terminal; the second direction is different from the first direction. When the first terminal receives the traffic image from the second direction, the first terminal simultaneously displays or projects the traffic image in a split-screen or overlay manner; wherein, the traffic image includes the first traffic image captured in real time by the first camera and the traffic image from the second direction received by the control device.

2. The method according to claim 1, characterized in that, The first terminal detects whether the driver is looking at the display interface of the first terminal, including: The first terminal uses the second camera to capture images; When the acquired image meets the preset conditions, the first terminal determines that the driver is looking at the display interface of the first terminal; otherwise, the first terminal determines that the driver is not looking at the display interface of the first terminal; wherein, the preset conditions include that the acquired image includes facial features or eye features.

3. The method according to claim 2, characterized in that, The preset conditions also include that the user corresponding to the facial features or eye features is a driver.

4. The method according to any one of claims 1-3, characterized in that, Before the first terminal detects whether the driver is looking at the display interface of the first terminal, the method further includes: The first terminal determines that the vehicle in which the first terminal is located is in motion.

5. The method according to any one of claims 1-3, characterized in that, Before the control device sends the real-time road condition image in the second direction obtained by the second terminal using the third camera to the first terminal, the method further includes: The control device determines whether to display the road condition image in the second direction based on the vehicle's operating status; the second direction is any direction other than the operating status. The control device transmits the real-time road condition image in the second direction, acquired by the second terminal using the third camera, to the first terminal, including: If the third camera can record the road conditions in the second direction, the control device will send the road conditions in the second direction, which are acquired in real time by the second terminal, to the first terminal.

6. The method according to any one of claims 1-3, characterized in that, The control device sends the real-time road condition image in the second direction, acquired by the second terminal using the third camera, to the first terminal, including: The control device obtains real-time road condition images from the second terminal using the third camera; The control device sends the traffic information to the first terminal.

7. The method according to any one of claims 1-3, characterized in that, The method further includes: The first terminal displays the interface of the first application; The first terminal displays a first window and a second window. The first window is used to continue displaying the display interface of the first application, and the second window is used to display the traffic situation image.

8. The method according to claim 7, characterized in that, The first terminal is equipped with a monitoring app. After the first terminal sends a gaze message to the control device, the app further includes: The control device sends a start message to the first terminal, the start message being used to instruct the first terminal to run the monitoring APP; In response to the startup message, the first terminal runs the monitoring APP in the second window, so that the monitoring APP displays the traffic situation image in the second window.

9. The method according to claim 8, characterized in that, After the first terminal runs the monitoring APP in the second window, the method further includes: The monitoring APP sends a traffic condition request to the control device, and the traffic condition request is used to request to obtain traffic condition images; The control device transmits real-time road condition images acquired by the second terminal using the first camera to the first terminal, including: In response to the traffic condition request, the control device sends the traffic condition image, which is captured in real time by the second terminal using the first camera, to the first terminal.

10. The method according to any one of claims 1-3, characterized in that, After the first terminal sends a gaze message to the control device, the process further includes: The control device receives a first alarm message sent by a third terminal, the first alarm message being used to indicate that the third terminal has detected an abnormal situation while driving; In response to the first alarm message, the control device sends a first notification message corresponding to the first alarm message to the first terminal; The first terminal displays the first notification message.

11. The method according to any one of claims 1-3, characterized in that, After the first terminal sends a gaze message to the control device, the process further includes: If the driver is not detected looking at the display interface of the first terminal within a preset time, the first terminal sends a gaze-off message to the control device. The gaze-off message is used to instruct the driver to look away from the display interface of the first terminal. In response to the "eyes out" message, the control device stops sending the real-time traffic images obtained by the second terminal to the first terminal.

12. A device control method in a vehicle-mounted system, characterized in that, The vehicle-mounted system includes a first terminal and a control device; the first terminal is equipped with a display screen or has a projection display function, wherein the device control method is executed by the first terminal, and the method includes: The first terminal is a vehicle-mounted terminal, which includes a first camera and a second camera. The first camera is disposed on the outside of the vehicle body and is used to collect real-time images of the first road conditions in the first direction. The second camera is disposed around the display interface of the first terminal and is used to collect images. The first terminal detects whether the driver is looking at the display interface of the first terminal; If the driver is detected looking at the display interface of the first terminal, the first terminal determines the duration of the driver looking at the display interface of the first terminal. If the driver gazes at the display interface of the first terminal for a duration greater than a first threshold, the first terminal sends a gaze message to the control device. The first terminal obtains a real-time road condition image of a second direction from the control device, which is captured by the second terminal using a third camera; the second direction is different from the first direction. The first terminal simultaneously displays or projects the road condition images in a split-screen or overlay manner; wherein, the road condition images include the first road condition images captured in real time by the first camera and the road condition images in the second direction received by the control device.

13. The method according to claim 12, characterized in that, The first terminal detects whether the driver is looking at the display interface of the first terminal, including: The first terminal uses the second camera to capture images; When the acquired image meets the preset conditions, the first terminal determines that the driver is looking at the display interface of the first terminal; otherwise, the first terminal determines that the driver is not looking at the display interface of the first terminal; wherein, the preset conditions include that the acquired image includes facial features or eye features.

14. The method according to claim 13, characterized in that, The preset conditions also include that the user corresponding to the facial features or eye features is a driver.

15. The method according to any one of claims 12-14, characterized in that, Before the first terminal detects whether the driver is looking at the display interface of the first terminal, the method further includes: The first terminal determines that the vehicle in which the first terminal is located is in motion.

16. The method according to any one of claims 12-14, characterized in that, Before the first terminal obtains the road condition image in the second direction obtained in real time by the second terminal using the third camera from the control device, the method further includes: The first terminal displays the interface of the first application; The display of the traffic condition image on the first terminal includes: The first terminal displays a first window and a second window. The first window is used to continue displaying the display interface of the first application, and the second window is used to display the traffic situation image.

17. The method according to claim 16, characterized in that, After the first terminal sends a gaze message to the control device, the process further includes: The first terminal receives a start message sent by the control device, the start message being used to instruct the first terminal to run a monitoring APP; In response to the startup message, the first terminal runs the monitoring APP in the second window, so that the monitoring APP displays the traffic situation image in the second window.

18. The method according to any one of claims 12-14, characterized in that, After the first terminal sends a gaze message to the control device, the process further includes: If the driver is not detected looking at the display interface of the first terminal within a preset time, the first terminal sends a gaze-off message to the control device. The gaze-off message is used to instruct the driver to look away from the display interface of the first terminal.

19. A device control method in a vehicle-mounted system, characterized in that, The vehicle-mounted system includes a first terminal and a control device; the first terminal is equipped with a display screen or has a projection display function, wherein the device control method is executed by the control device, and the method includes: The first terminal is a vehicle-mounted terminal, which includes a first camera and a second camera. The first camera is disposed on the outside of the vehicle body and is used to collect real-time images of the first road conditions in the first direction. The second camera is disposed around the display interface of the first terminal and is used to collect images. The control device determines that the driver is looking at the display interface of the first terminal; The control device acquires real-time road condition images in a second direction from a third camera used by the second terminal; the second direction is different from the first direction. The control device sends the traffic condition image in the second direction to the first terminal.

20. The method according to claim 19, characterized in that, The control device determines that the driver is looking at the display interface of the first terminal, including: If the control device receives a first gaze message sent by the first terminal, the control device determines that the driver is gazing at the display interface of the first terminal, and the first gaze message is used to indicate that the driver is gazing at the display interface of the first terminal; or, If the control device receives a second gaze message from the second camera, the control device determines that the driver is looking at the display interface of the first terminal, and the second gaze message is used to instruct the driver to look at the second camera.

21. The method according to claim 19, characterized in that, After the control device determines that the driver is looking at the display interface of the first terminal, it further includes: The control device sends a start message to the first terminal, the start message being used to instruct the first terminal to run the monitoring APP; The control device acquires real-time road condition images captured by the second terminal using the first camera, including: If a traffic condition request is received from the monitoring APP, the control device obtains the traffic condition image captured in real time by the second terminal using the first camera.

22. The method according to any one of claims 19-21, characterized in that, After the control device determines that the driver is looking at the display interface of the first terminal, it further includes: The control device receives an alarm message sent by a third terminal, the alarm message being used to indicate that the third terminal has detected an abnormal situation while driving; The control device sends a notification message corresponding to the alarm message to the first terminal.

23. The method according to any one of claims 19-21, characterized in that, After the control device determines that the driver is looking at the display interface of the first terminal, it further includes: The control device receives a gaze-off message sent by the first terminal, the gaze-off message being used to instruct the driver to look away from the display interface of the first terminal; In response to the "eyes out" message, the control device stops sending the traffic images to the first terminal.

24. A terminal, characterized in that, include: One or more processors; Memory; Communication module; The memory stores one or more computer programs, the one or more computer programs including instructions that, when executed by the terminal, cause the terminal to perform the device control method performed by the first terminal as claimed in any one of claims 12-18.

25. A control device, characterized in that, include: One or more processors; Memory; Communication module; The memory stores one or more computer programs, the one or more computer programs including instructions that, when executed by the control device, cause the control device to perform the device control method as described in any one of claims 19-23.

26. A vehicle-mounted system, characterized in that, The system includes the terminal as described in claim 24 and the control device as described in claim 25.

27. A computer-readable storage medium storing instructions, characterized in that, When the instruction is executed on a terminal, the terminal performs the device control method as described in any one of claims 12-18; or, when the instruction is executed on a control device, the control device performs the device control method as described in any one of claims 19-23.

Images