Monitoring method, in-vehicle infotainment system, electronic equipment and computer readable medium

Abstract

The invention provides a monitoring method, a vehicle machine system, electronic equipment and a computer readable medium, and belongs to the technical field of intelligent cabins. The monitoring method is applied to a vehicle machine system in an intelligent cabin, and the monitoring method comprises the following steps: obtaining a camera monitoring data stream sent by a mobile terminal which is located in the intelligent cabin and is independent of the intelligent cabin; analyzing and processing the camera monitoring data stream to obtain an analysis and processing result; and sending the analysis processing result to an intranet unit controller and / or a vehicle machine interface application in the intelligent cabin. According to the technical scheme, under the conditions that camera hardware and serializer and deserializer chips are reduced and a CPU, a GPU and a memory of the system are not occupied, image taking and image analysis processing of the camera can be completed, the driving and riding monitoring function is achieved, and therefore the effects of reducing the hardware cost and improving the stability of the vehicle machine system can be achieved.

Description

Technical Field

[0001] This invention relates to the field of smart cockpit technology, and in particular to a monitoring method, vehicle infotainment system, electronic device and computer-readable medium. Background Technology

[0002] Intelligent cockpits are becoming increasingly popular among car users. They typically come pre-installed with intelligent vehicle infotainment systems, which offer a wealth of practical functions such as navigation and travel assistance, vehicle status monitoring and control, intelligent interaction, and networking and communication.

[0003] With the development of automotive intelligence, some in-vehicle systems are now capable of monitoring occupants within the intelligent cockpit, such as Driver Monitoring System (DMS) and Occupant Monitoring System (OMS). Specifically, these systems monitor the driver or passengers within the intelligent cockpit, analyze the monitoring data using pre-defined algorithms, and finally implement appropriate response controls based on the analysis results.

[0004] However, in practical applications, it has been found that current in-vehicle infotainment systems, while capable of monitoring people in the smart cockpit, require high hardware costs and also have a significant negative impact on the stability of the system's operation. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art, and proposes a monitoring method, vehicle system, electronic device and computer-readable medium.

[0006] In a first aspect, this disclosure provides a monitoring method applied to an in-vehicle infotainment system within a smart cockpit, the monitoring method comprising:

[0007] Acquire camera monitoring data streams sent by mobile terminals located within and independent of the smart cockpit;

[0008] The camera monitoring data stream is analyzed and processed to obtain the analysis and processing results;

[0009] The analysis and processing results are sent to the intranet unit controller and / or vehicle interface application in the smart cockpit.

[0010] In some embodiments, prior to the step of acquiring camera monitoring data streams sent by a mobile terminal located within the smart cockpit and independent of the smart cockpit, the method further includes:

[0011] Establish a P2P network with the mobile terminal;

[0012] The steps for acquiring camera monitoring data streams sent by mobile terminals located within and independent of the smart cockpit include:

[0013] The camera monitoring data stream sent by the mobile terminal is obtained through a P2P network.

[0014] In some embodiments, the step of establishing a P2P network with the mobile terminal includes:

[0015] Establish a Bluetooth connection with the mobile terminal;

[0016] The device connects to the mobile terminal via Bluetooth for Wi-Fi authentication and establishes a Wi-Fi P2P network.

[0017] In some embodiments, after the step of establishing a WIFI P2P network and before the step of acquiring camera monitoring data streams sent by mobile terminals located within the smart cockpit and independent of the smart cockpit, the method further includes:

[0018] Obtain the network configuration information of the mobile terminal in the P2P network;

[0019] Based on the network configuration information and the preset whitelist information, identify whether the mobile terminal is a whitelist object;

[0020] When the mobile terminal is identified as a whitelisted object, the step of obtaining the camera monitoring data stream sent by the mobile terminal, which is independent of the smart cockpit, is executed.

[0021] In some embodiments, the network configuration information includes: the port number and / or IP address of the mobile terminal;

[0022] The whitelist information includes: port number whitelist and / or IP address whitelist.

[0023] In some embodiments, the P2P network is configured to be isolated from the Internet.

[0024] In some embodiments, the P2P network is configured to allow simultaneous access to multiple mobile terminals.

[0025] Secondly, embodiments of this disclosure also provide a vehicle infotainment system configured to implement any of the monitoring methods provided in the first aspect, the vehicle infotainment system comprising:

[0026] A data stream receiving module is used to acquire camera monitoring data streams sent by mobile terminals located in the smart cockpit and independent of the smart cockpit.

[0027] The data stream processing module is used to analyze and process the camera monitoring data stream, obtain the analysis and processing results, and send the analysis and processing results to the intranet unit controller and / or vehicle interface application in the smart cockpit.

[0028] Thirdly, embodiments of this disclosure also provide an electronic device, characterized in that it includes:

[0029] One or more processors;

[0030] Memory, used to store one or more programs;

[0031] When the one or more programs are executed by the one or more processors, the one or more processors implement the monitoring method as provided in the first aspect.

[0032] Thirdly, embodiments of this disclosure also provide a computer-readable medium storing a computer program, characterized in that, when executed by a processor, the computer program implements the steps of the monitoring method provided in the first aspect.

[0033] In this disclosure, the vehicle is not equipped with driver and passenger monitoring camera hardware and serializer / deserializer chips at the factory. Instead, it utilizes the driver's or passenger's mobile terminal (e.g., mobile phone, tablet, laptop, etc.) as a camera to extract camera monitoring data streams (essentially camera image output information streams) of occupants inside the vehicle. This data is then transmitted to a data stream processing module for processing, achieving real-time monitoring of occupants inside the vehicle. Compared to related technologies, the technical solution of this disclosure can complete camera image acquisition and image analysis processing, and realize driver and passenger monitoring functions, while reducing camera hardware and serializer / deserializer chips and without consuming the system's CPU, GPU, and memory. This achieves the effects of reducing hardware costs and improving the stability of the vehicle's infotainment system.

[0034] At the same time, the position of the mobile terminal in this disclosure can be adjusted at any time inside the cockpit according to actual needs, so as to realize the monitoring of the entire cockpit area using only one mobile terminal. Attached Figure Description

[0035] Figure 1 This is a structural block diagram of a vehicle-mounted infotainment system with personnel monitoring capabilities, as per relevant technologies.

[0036] Figure 2 This is a schematic diagram showing the distribution of multiple cameras used for personnel monitoring within a smart cockpit in related technologies.

[0037] Figure 3 A structural block diagram of a vehicle infotainment system provided in an embodiment of this disclosure;

[0038] Figure 4 A flowchart of a monitoring method provided in this embodiment of the disclosure;

[0039] Figure 5 A flowchart of another monitoring method provided in this disclosure embodiment;

[0040] Figure 6 A flowchart illustrating another monitoring method provided in this disclosure embodiment;

[0041] Figure 7 This is a schematic diagram of the P2P network of the vehicle-mounted system in this disclosure when multiple mobile terminals are simultaneously connected.

[0042] Figure 8 This is a structural block diagram of another vehicle infotainment system provided in an embodiment of the present disclosure;

[0043] Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0045] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0046] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0047] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0048] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0049] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0050] Figure 1 This is a structural block diagram of a vehicle-mounted infotainment system with personnel monitoring capabilities, which is related to relevant technologies. Figure 2 This is a schematic diagram illustrating the distribution of multiple cameras used for personnel monitoring within a smart cockpit in related technologies. (Example:) Figure 1 Ganghe Figure 2 As shown, the vehicle system includes a sensing end and a planning end.

[0051] The sensing end specifically includes at least one camera fixed inside the smart cockpit. These in-vehicle cameras capture the behavior and state of the driver or passengers, generating corresponding camera monitoring data streams. These cameras can monitor the driver's or passenger's actions and expressions in real time, such as eye opening and closing, and head position. To ensure data transmission efficiency, serializer / deserializer chips are often used to transmit the camera monitoring data streams to the planning end.

[0052] The planning end specifically includes a data stream processing module, which receives camera monitoring data streams sent by the cameras and analyzes and processes the camera monitoring data streams based on preset data analysis and processing algorithms to identify the current status of the driver or passengers.

[0053] Generally, a smart cockpit includes an execution unit (such as an intranet unit controller and / or vehicle interface application) that works in conjunction with the planning unit to take appropriate measures based on the analysis and processing results of the data stream processing module. For example, if the data stream processing module detects driver fatigue, it may issue an alert through the vehicle interface application to remind the driver, or automatically adjust certain vehicle settings through the intranet unit controller to improve the driving environment.

[0054] join Figure 2 As shown, Figure 2 The document describes a scenario where a smart cockpit is equipped with one DMS camera and two OMS cameras; the DMS camera monitors the driver's seat, while the two OMS cameras monitor the front passenger seat and the rear seats, respectively.

[0055] The data stream processing module has a built-in data analysis and processing algorithm, which has two main functions: 1) DMS function (such as fatigue driving monitoring, distracted driving monitoring, abnormal driver status monitoring, identity recognition and personalized services, etc.), which analyzes and processes the camera monitoring data stream from the DMS camera and outputs the analysis and processing results to ensure driving safety and prevent accidents caused by driver error or abnormal status; 2) OMS function (such as passenger safety monitoring, ride comfort adjustment, behavioral safety warning, and item leaving reminder), which analyzes and processes the camera monitoring data stream from the OMS camera and outputs the analysis and processing results to ensure passenger safety and comfort and improve the ride experience.

[0056] For ease of description, the data analysis and processing algorithm within the data stream processing module that supports the DMS function is referred to as the vehicle-mounted DMS algorithm, and the data analysis and processing algorithm within the data stream processing module that supports the OMS function is referred to as the vehicle-mounted OMS algorithm. The specific algorithm logic of the vehicle-mounted DMS algorithm and the vehicle-mounted OMS algorithm can be pre-designed and adjusted according to actual needs; for example, the vehicle-mounted DMS algorithm and the vehicle-mounted OMS algorithm used in this disclosure can be existing algorithms in the prior art, or algorithms not yet mentioned in the prior art but capable of achieving data analysis and processing functions. The technical solution of this disclosure does not limit the specific algorithm logic of the two algorithms.

[0057] In related technologies, it is necessary to pre-install camera hardware and add deserializer chips in the vehicle, which results in high hardware costs. At the same time, the driving and image output process of the pre-installed camera hardware in the vehicle puts a large demand on the memory, CPU, GPU and other components of the vehicle system, which has a negative impact on the overall stability of the vehicle system.

[0058] In order to effectively improve or even completely solve at least one of the technical problems existing in the related technologies, this disclosure provides a new technical solution.

[0059] Figure 3 This is a structural block diagram of a vehicle infotainment system provided in an embodiment of the present disclosure. Figure 4 This is a flowchart illustrating a monitoring method provided in an embodiment of this disclosure. Figure 3 and Figure 4 As shown, this monitoring method is applied to the vehicle infotainment system, and the monitoring method includes:

[0060] Step S1: Obtain the camera monitoring data stream sent by the mobile terminal located in the smart cockpit and independent of the smart cockpit.

[0061] Step S2: Analyze and process the camera monitoring data stream to obtain the analysis and processing results.

[0062] Step S3: Send the analysis and processing results to the intranet unit controller and / or vehicle interface application in the smart cockpit.

[0063] In this embodiment of the disclosure, steps S2 and S3 are both implemented by the data stream processing module, and the relevant descriptions can be found in the preceding content.

[0064] In practical applications, the data stream processing module can send the analysis and processing results to the vehicle's infotainment system for display or notification. Of course, for some special scenarios, the data stream processing module can also send the analysis and processing results to the Vehicle Intranet Unit (VIU) controller, so that the VIU controller can automatically adjust some of the vehicle's settings.

[0065] In this disclosure, the vehicle is not equipped with camera hardware and serializer / deserializer chip for driver and passenger monitoring when it leaves the factory. Instead, the driver's or passenger's mobile terminal (e.g., mobile phone, tablet, laptop, etc.) is used as a camera to extract the camera monitoring data stream of the people in the vehicle (essentially the camera image output information stream), and the data stream is passed to the data stream processing module for processing, so as to achieve the purpose of real-time monitoring of the people in the vehicle.

[0066] Compared to related technologies, the technical solution disclosed herein can complete the image acquisition and image analysis processing of the camera and realize the driving and riding monitoring function under the conditions of reducing camera hardware, adding a deserializer chip, and not occupying the system's CPU, GPU and memory, thereby achieving the effect of reducing hardware costs and improving the stability of the vehicle system.

[0067] At the same time, the position of the mobile terminal in this disclosure can be adjusted at any time inside the cockpit according to actual needs, so as to realize the monitoring of the entire cockpit area using only one mobile terminal.

[0068] Figure 5 A flowchart illustrating another monitoring method provided in this disclosure embodiment. Figure 5 As shown, the monitoring method not only includes steps S1 to S3 in the previous embodiments, but also includes step Sa before step S1.

[0069] Step Sa: Establish a P2P network with the mobile terminal.

[0070] Optionally, step Sa specifically includes:

[0071] Step Sa1: Connect to the mobile terminal via Bluetooth.

[0072] Specifically, the vehicle infotainment system establishes a Bluetooth connection with the mobile terminal through the RFCOMM communication protocol with built-in Bluetooth.

[0073] Step Sa2: Connect with the mobile terminal via Bluetooth to perform WIFI authentication and establish a WIFI P2P network.

[0074] Specifically, the vehicle system and the mobile terminal exchange WIFI authentication information and perform WIFI authentication. Based on the WIFI information provided by the vehicle system, the mobile terminal actively connects to the vehicle's WIFI Direct hotspot. The vehicle system assigns an IP address and establishes a TCP connection with the mobile terminal to complete the establishment of the WIFI P2P network.

[0075] In some embodiments, the vehicle-mounted system in the P2P network acts as the Group Owner, and the mobile terminal acts as the Group Client.

[0076] At this point, step S1 specifically includes: obtaining the camera monitoring data stream sent by the mobile terminal through the P2P network. The mobile terminal uses a proprietary protocol to transmit the camera monitoring data stream over the P2P network via a proprietary application layer protocol (a pre-configured proprietary protocol).

[0077] It should be noted that the process of establishing a P2P network between the mobile terminal and the vehicle system in this disclosure can use a seamless connection scheme, that is, Bluetooth connection authorization is only required when the mobile terminal and the vehicle system connect for the first time.

[0078] In some embodiments, the P2P network is configured to be isolated from the Internet. In this disclosure, by using a proprietary protocol at the application layer and ensuring the P2P network is not connected to the Internet, data is ensured to reside only on the local network, preventing user privacy from being leaked and maximizing the protection of user privacy and security.

[0079] Figure 6 This is a flowchart illustrating another monitoring method provided in an embodiment of the present disclosure. Figure 6 As shown, the monitoring method not only includes steps Sa to S3 in the previous embodiments, but also includes steps Sb and Sc between steps Sa and S1.

[0080] Step Sb: Obtain the network configuration information of the mobile terminal in the P2P network.

[0081] The network configuration information includes the core parameters for mobile terminals to access the P2P network and achieve communication.

[0082] Step Sc: Identify whether the mobile terminal is a whitelisted object based on network configuration information and preset whitelist information.

[0083] When a mobile terminal is identified as a whitelisted object, the step of obtaining camera monitoring data streams sent by the mobile terminal, independent of the smart cockpit, is executed. When a mobile terminal is identified as a whitelisted object, it is determined that the mobile terminal cannot access the P2P network.

[0084] Network configuration information includes: the mobile terminal's port number and / or IP address; whitelist information includes: port number whitelist and / or IP address whitelist.

[0085] In this disclosure, by setting a whitelist mechanism, it can be ensured that the P2P network accessed by the vehicle system is a trusted mobile terminal, thus preventing the P2P network from being subjected to malicious network attacks.

[0086] Figure 7 This is a schematic diagram illustrating the P2P network of the vehicle-to-everything (V2X) system in this disclosure when multiple mobile terminals are simultaneously connected. For example... Figure 7 As shown, in some embodiments, the P2P network is configured to allow multiple mobile terminals to access simultaneously. When the vehicle-to-mobile terminal establishes a P2P network, it is a WLAN hotspot established on the vehicle system side, which allows multiple mobile terminals to access simultaneously, enabling the deployment of DMS / OMS at different locations within the smart cockpit. In practical use, specific monitoring strategies can be flexibly configured as needed.

[0087] Based on the same inventive concept, this disclosure provides an in-vehicle infotainment system. See also Figure 3 As shown, the vehicle system is configured to perform the monitoring method provided in the previous embodiment. The vehicle system includes a data stream receiving module and a data stream processing module.

[0088] The data stream receiving module is used to acquire camera monitoring data streams sent by mobile terminals located inside and outside the smart cockpit.

[0089] The data stream processing module is used to analyze and process the camera monitoring data stream, obtain the analysis and processing results, and send the analysis and processing results to the intranet unit controller and / or vehicle interface application in the smart cockpit.

[0090] In some embodiments, the data stream receiving module is further configured to establish a P2P network with the mobile terminal. Specifically, the data stream receiving module first establishes a Bluetooth connection with the mobile terminal, then performs Wi-Fi authentication with the mobile terminal via Bluetooth, and establishes a Wi-Fi P2P network.

[0091] Figure 8 This is a structural block diagram of another vehicle infotainment system provided in an embodiment of this disclosure. Figure 8As shown, in some embodiments, the vehicle-mounted infotainment system further includes a whitelist module, wherein the whitelist module is used to obtain network configuration information of the mobile terminal in the P2P network; and to identify whether the mobile terminal is a whitelist object based on the network configuration information and preset whitelist information.

[0092] When the whitelist module identifies the mobile terminal as a whitelisted object, the data stream receiving module begins to receive the camera monitoring data stream sent by the mobile terminal.

[0093] For a detailed description of each of the above functional modules, please refer to the corresponding content in the previous embodiments, which will not be repeated here.

[0094] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 9 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the monitoring methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0095] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0096] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0097] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0098] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps of any of the monitoring methods described in the above embodiments. The computer-readable storage medium may be volatile or non-volatile.

[0099] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described monitoring method.

[0100] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0101] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0102] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0103] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0104] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0105] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0106] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0107] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0109] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A monitoring method, characterized in that, The monitoring method, applied to in-vehicle infotainment systems within smart cockpits, includes: Acquire camera monitoring data streams sent by mobile terminals located within and independent of the smart cockpit; The camera monitoring data stream is analyzed and processed to obtain the analysis and processing results; The analysis and processing results are sent to the intranet unit controller and / or vehicle interface application in the smart cockpit.

2. The method according to claim 1, characterized in that, Prior to the step of acquiring camera monitoring data streams sent by mobile terminals located within and independent of the smart cockpit, the method further includes: Establish a P2P network with the mobile terminal; The steps for acquiring camera monitoring data streams sent by mobile terminals located within and independent of the smart cockpit include: The camera monitoring data stream sent by the mobile terminal is obtained through a P2P network.

3. The method according to claim 2, characterized in that, The steps for establishing a P2P network with the mobile terminal include: Establish a Bluetooth connection with the mobile terminal; The device connects to the mobile terminal via Bluetooth for Wi-Fi authentication and establishes a Wi-Fi P2P network.

4. The method according to claim 2, characterized in that, After establishing the WIFI P2P network and before acquiring the camera monitoring data stream sent by a mobile terminal located within the smart cockpit and independent of the smart cockpit, the method further includes: Obtain the network configuration information of the mobile terminal in the P2P network; Based on the network configuration information and the preset whitelist information, identify whether the mobile terminal is a whitelist object; When the mobile terminal is identified as a whitelisted object, the step of obtaining the camera monitoring data stream sent by the mobile terminal, which is independent of the smart cockpit, is executed.

5. The method according to claim 4, characterized in that, The network configuration information includes: the port number and / or IP address of the mobile terminal; The whitelist information includes: port number whitelist and / or IP address whitelist.

6. The method according to claim 2, characterized in that, The P2P network is configured to be isolated from the Internet.

7. The method according to any one of claims 2 to 6, characterized in that, The P2P network is configured to allow multiple mobile terminals to access simultaneously.

8. A vehicle infotainment system, characterized in that, The vehicle system is configured to implement the monitoring method as described in any one of claims 1 to 7, and includes: A data stream receiving module is used to acquire camera monitoring data streams sent by mobile terminals located in the smart cockpit and independent of the smart cockpit. The data stream processing module is used to analyze and process the camera monitoring data stream, obtain the analysis and processing results, and send the analysis and processing results to the intranet unit controller and / or vehicle interface application in the smart cockpit.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images