Integrated TBOX intelligent vehicle-mounted terminal system and control method thereof

Abstract

The invention discloses an integrated TBOX intelligent vehicle-mounted terminal system and a control method. The system comprises a main control unit, a video acquisition module, a communication module and a storage unit, the main control unit is respectively connected to the video acquisition unit, the communication unit and the storage unit; the main control unit is used for realizing data processing and interaction control strategies corresponding to a DVR function, a DMS function and a TBOX function, and the video acquisition module is used for providing required images for the main control module to realize the DVR function; the communication module is used for providing a communication signal for the main control module to realize the TBOX function; and the storage unit is used for providing data storage when the main control unit realizes the DVR function, the DMS function and the TBOX function. Through deep integration of DVR, DMS and TBOX functions and optimization of hardware architecture and software algorithms, resource integration and function collaboration are realized, the safety, the intelligent level and the user experience of the vehicle are improved, and the hardware cost and the installation and maintenance complexity are reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent connected vehicle electronic equipment, and in particular to an integrated terminal system and its control method that integrates a driving recorder (DVR), a driver monitoring system (DMS), and a telematics unit (TBOX). Background Technology

[0002] With the rapid development of automotive intelligence and connectivity, the problem of functional fragmentation in in-vehicle electronic devices is becoming increasingly prominent. Traditional DVRs only have video recording capabilities and cannot effectively interact with other vehicle systems; DMSs rely on independent cameras and computing units, resulting in wasted hardware resources; and TBOXs, as independent communication modules, lack coordination with other devices in data transmission and processing. This traditional solution suffers from high hardware redundancy, poor data coordination, and high installation and maintenance costs. Furthermore, the independent processors configured for DVRs, DMSs, and TBOXs prevent the sharing of computing power, leading to insufficient NPU utilization across modules. This reliance on CAN bus transmission for cross-device data interaction also impacts CAN network load and controller response speed.

[0003] Therefore, developing a low-cost, high-reliability integrated solution for DVR, DMS, and TBOX has become an urgent need in the industry. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an integrated TBOX intelligent vehicle terminal system and its control method. By deeply integrating the functions of DVR, DMS and TBOX, optimizing the hardware architecture and software algorithm, resource integration and functional synergy are achieved, thereby improving the safety, intelligence level and user experience of the vehicle, and reducing hardware costs and installation and maintenance complexity.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] An integrated TBOX intelligent vehicle terminal system includes a main control unit, a video acquisition module, a communication module, and a storage unit. The main control unit is connected to the video acquisition unit, the communication unit, and the storage unit. The main control unit is used to implement data processing and interactive control strategies corresponding to DVR, DMS, and TBOX functions. The video acquisition module provides the necessary images for the main control module to implement the DVR function. The communication module provides communication signals for the main control module to implement the TBOX function. The storage unit provides data storage for the main control unit to implement the DVR, DMS, and TBOX functions.

[0007] The main control unit adopts a heterogeneous computing chipset, which integrates a CPU, an NPU, and a baseband chip. The CPU is responsible for the overall system scheduling and data processing. The NPU runs deep learning algorithms to realize image recognition for DMS function and intelligent video analysis for DVR function. The baseband chip provides 4G / 5G communication network.

[0008] The video acquisition module includes a driving camera and a DMS camera. The driving camera is used to acquire images required for DVR functions and images required for driver assistance.

[0009] The communication module includes a 4G / 5G communication module, a Bluetooth module, and a Wi-Fi module. The 4G / 5G communication module supports high-speed data transmission to enable remote control of the TBOX and data upload / download functions between the vehicle and the cloud server. The Bluetooth module is used for short-range communication with mobile devices, enabling users to configure devices and transmit data. The Wi-Fi module enables rapid connection between the vehicle and home networks or hotspots, facilitating large file data synchronization and OTA software upgrades.

[0010] The storage unit uses a high-capacity eMMC storage chip and performs a cyclic overwrite mechanism to store data. Under normal vehicle driving conditions, the main control unit controls the storage unit to record videos in cyclic order. When an abnormal event is detected, the relevant video clips are encrypted and locked to prevent data loss.

[0011] The system also includes a sensor module, which is used to collect vehicle status data in real time, and its output is connected to the main control unit. The driving status information is used to assist the DVR function in recording more accurate vehicle driving data, assist the DMS function in judging the rationality of the driver's driving behavior, and also to upload vehicle status information for the TBOX function.

[0012] A control method for an integrated TBOX intelligent vehicle terminal system, the method comprising: a main control unit connected to a video acquisition unit, a communication unit, and a storage unit; the main control unit being used to implement data processing and interactive control strategies corresponding to DVR, DMS, and TBOX functions; the video acquisition module being used to provide the necessary images for the main control module to implement the DVR function; the communication module being used to provide communication signals for the main control module to implement the TBOX function; and the storage unit being used to provide data storage when the main control unit implements the DVR, DMS, and TBOX functions.

[0013] The main control unit performs time-series analysis on the collected driver status data and vehicle driving data through a multimodal data fusion algorithm to accurately judge driving scenarios and potential risks, and outputs corresponding warning signals based on the judgment results. Specifically, a spatiotemporal alignment algorithm is used to align the video data collected by the DVR, the driver status data obtained by the DMS, and the vehicle status data received by the TBOX in the time and space dimensions. A deep learning algorithm is used to extract features from the video images, and a Long Short-Term Memory (LSTM) network is combined to perform time-series analysis on the driver status data and vehicle driving data to obtain the judgment results.

[0014] The data processing and interactive control strategies corresponding to the DVR function include: the main control unit uses a loop recording storage method to store the recorded image data; when an abnormal event is detected by the sensor module, the emergency recording function is triggered, at which time the video is recorded at a higher bitrate and the video clips within the time range of the abnormal event are encrypted and stored.

[0015] The data processing and interactive control strategies corresponding to the DMS function include: deep learning-based face recognition and pose estimation algorithms, which establish a high-precision facial expression recognition model by training massive amounts of face image data, and detect various dangerous driving behaviors of the driver in real time through the model. When a dangerous behavior is detected, an alarm is immediately issued to remind the driver, and or the vehicle equipment is linked to eliminate or reduce the driver's dangerous behavior or dangerous state according to the driver's state.

[0016] The advantages of this invention are: high hardware integration: the core functional modules of DVR, DMS, and TBOX are integrated into one device, and a single DIN standard size design is adopted. Compared with the traditional independent device solution, the hardware cost is reduced, the installation space is significantly reduced, the length of in-vehicle wiring is reduced, the overall vehicle cost and failure rate are effectively reduced, and the installation efficiency and vehicle interior space utilization are improved at the same time.

[0017] Deep Data Fusion: Through multimodal data fusion algorithms, deep integration of video data, driver status data, and vehicle status data is achieved. In accident assessment scenarios, in addition to relying on video footage recorded by the DVR, the system also combines collision signals detected by the accelerometer, driver status monitored by the DMS, and driving data fed back by the vehicle's ECU to more accurately determine the severity and cause of the accident and take timely emergency measures, such as automatically sending accident information and location coordinates to emergency contacts.

[0018] Functional Collaboration Optimization: High-level collaboration is achieved between various functional modules. For example, when the DMS detects driver fatigue, in addition to issuing an alarm, it also sends a request to the cloud server via the TBOX to obtain information on nearby rest areas and displays a prompt on the in-vehicle navigation system. Simultaneously, the DVR automatically marks and records the driver's driving process while fatigued, providing data support for subsequent safety analysis and driving behavior improvement. Furthermore, in remote control scenarios, after receiving commands, the TBOX can collaborate with the DMS for authentication, ensuring operational security.

[0019] Intelligent Interactive Experience: Based on integrated data and algorithms, it provides users with a more intelligent and personalized interactive experience. By learning the driver's habits and preferences, it automatically adjusts the settings of in-vehicle equipment such as audio and air conditioning; users can remotely monitor and control the vehicle in all aspects through a mobile application, including real-time viewing of DVR video, receiving DMS alarm information, and remotely controlling vehicle equipment, truly enabling them to keep track of the vehicle's status anytime, anywhere.

[0020] Protocol compatibility and security: Supports adaptive switching between JT / T808-2019 and GB / T32960 dual protocol stacks to ensure reliable data communication in different application scenarios and communication environments. SSL / TLS encryption technology is used for end-to-end data encryption to ensure secure data transmission between the vehicle and the cloud server, preventing data leakage and malicious attacks. Attached Figure Description

[0021] The following is a brief explanation of the contents of each of the accompanying drawings and the markings in the drawings:

[0022] Figure 1 This is a system architecture diagram of the present invention;

[0023] Figure 2 This is a flowchart of the system interaction of the present invention. Detailed Implementation

[0024] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and the description of the preferred embodiments.

[0025] This embodiment provides an integrated TBOX intelligent vehicle terminal system, including a main control unit, a video acquisition module, a communication module, and a storage unit;

[0026] The main control unit is connected to the video acquisition unit, the communication unit, and the storage unit. The main control unit is used to implement the data processing and interactive control strategies corresponding to the DVR, DMS, and TBOX functions. The video acquisition module is used to provide the images required for the main control module to implement the DVR function. The communication module is used to provide communication signals for the main control module to implement the TBOX function. The storage unit is used to provide data storage when the main control unit implements the DVR, DMS, and TBOX functions.

[0027] The main control unit adopts a heterogeneous computing chipset, which integrates a CPU, an NPU, and a baseband chip. The CPU is responsible for the overall system scheduling and data processing. The NPU runs deep learning algorithms to realize image recognition for DMS function and intelligent video analysis for DVR function. The baseband chip provides 4G / 5G communication network.

[0028] The video acquisition module includes a driving camera and a DMS camera. The driving camera is used to acquire the images required for the DVR function and the image data required for driver assistance, so as to facilitate the implementation of the video data required for the DVR function and the DMS function.

[0029] The communication module includes a 4G / 5G communication module, a Bluetooth module, and a Wi-Fi module. The 4G / 5G communication module supports high-speed data transmission, enabling remote control of the TBOX and data upload / download functions between the vehicle and the cloud server. The Bluetooth module is used for short-range communication with mobile devices, allowing users to configure devices and transfer data. The Wi-Fi module enables rapid connection between the vehicle and home networks or hotspots, facilitating large file data synchronization and OTA software upgrades.

[0030] The storage unit uses a high-capacity eMMC storage chip and implements a cyclic overwrite mechanism to store data. Under normal vehicle operation, the main control unit controls the storage unit to record video in cyclic order. When an abnormal event is detected, the relevant video clips are encrypted and locked to prevent data loss.

[0031] The sensor module is used to collect vehicle status data in real time, and its output is connected to the main control unit. Driving status information assists the DVR function in recording more accurate vehicle driving data, assists the DMS function in judging the rationality of driver behavior, and is also used by the TBOX function to upload vehicle status information. The sensor module includes inertial sensors such as accelerometers and gyroscopes to detect vehicle speed, acceleration, steering angle, and other driving status information in real time.

[0032] A control method for an integrated TBOX intelligent vehicle terminal system includes: a main control unit connected to a video acquisition unit, a communication unit, and a storage unit; the main control unit implements data processing and interactive control strategies corresponding to DVR, DMS, and TBOX functions; the video acquisition module provides the necessary images for the main control unit to implement the DVR function; the communication module provides communication signals for the main control unit to implement the TBOX function; and the storage unit provides data storage for the main control unit to implement the DVR, DMS, and TBOX functions. The integrated main control unit implements the DVR, DMS, and TBOX functions. The core components of the main control unit include a CPU, an NPU, and a baseband chip. Various algorithms and logics are run in the main control unit to implement the corresponding DVR, DMS, and TBOX function strategies. Because the main control unit integrates an NPU, it provides powerful computing power for algorithm processing, thereby meeting the processing and control requirements of the functional strategies. The functions of the main control unit include:

[0033] The main control unit has a built-in multimodal data fusion algorithm module. The main control unit performs time-series analysis on the collected driver status data and vehicle driving data through the multimodal data fusion algorithm to achieve accurate judgment of driving scenarios and potential risks, and outputs corresponding warning signals based on the judgment results. Among them, the spatiotemporal alignment algorithm is used to align the video data collected by DVR, the driver status data obtained by DMS, and the vehicle status data received by TBOX in the time and space dimensions. Deep learning algorithm is used to extract features from video images, and combined with Long Short-Term Memory (LSTM) network, time-series analysis of driver status data and vehicle driving data is performed to obtain the judgment results.

[0034] The data processing and interactive control strategies implemented by the main control unit for the DVR function include: The main control unit stores recorded image data using a loop recording storage method. When an abnormal event is detected by the sensor module, an emergency recording function is triggered. At this time, video is recorded at a higher bitrate, and video segments within the time range of the abnormal event are encrypted and stored. Image data acquired by the image acquisition module is sent to the main control unit. The main control unit performs image signal processing and video encoding on the video data sent by the image acquisition module (camera) to obtain video stream data. The main control unit encodes the video data according to a pre-set base bitrate to obtain video stream data and stores it in the storage unit using a loop storage method. When an abnormal event is detected, such as sudden braking or a collision, an abnormal bitrate is used to encode the video data to obtain video stream data. The abnormal bitrate is a pre-set encoding parameter higher than the base bitrate. The purpose of this is to reduce the storage space occupied by the video in non-abnormal situations, reduce the impact of repeated data read / write on the lifespan of the storage unit, and at the same time, increase the bitrate and improve the clarity during abnormal situations to further meet the clarity requirements. In this embodiment, the efficient H.265 encoding standard can be adopted. While ensuring 4K high-definition image quality, it reduces data storage and transmission bandwidth requirements by approximately 50% compared to H.264 encoding. The H.265 encoding rate is increased only in abnormal situations such as collisions to improve the bitrate in emergency situations. In a preferred embodiment, the video stream data obtained by increasing the bitrate is encrypted and stored. Encryption is prohibited from modification within a set time range, while allowing segmented storage and uploading to the cloud platform. This ensures secure and reliable data storage and remote retrieval, and prohibiting modification guarantees data reliability. After the set time range is exceeded, new video stream data is allowed to overwrite the existing data. Within the set time range, the main control unit uploads the encrypted portion of the video stream data to the cloud platform via the communication module, thus achieving bidirectional storage and preventing data loss. If the encrypted data storage is not completed within the set time range, the encrypted video stream data remains prohibited until the delay time threshold is reached before modification and overwriting by other data are permitted. This ensures data reliability, reduces data loss due to modification or overwriting of encrypted data, and improves the reliability of data storage in abnormal situations.

[0035] The data processing and interactive control strategies corresponding to the DMS function in the main control unit include: a deep learning-based face recognition and pose estimation algorithm. By training on massive amounts of facial image data, a high-precision facial expression recognition model is established. This model detects various dangerous driving behaviors of the driver in real time. When a dangerous behavior is detected, an alarm is immediately issued to remind the driver, and / or, based on the driver's state, the onboard equipment is linked to eliminate or mitigate the driver's dangerous behavior or state. Dangerous driving behaviors include fatigued driving, distracted driving, talking on the phone, smoking, etc. After detecting dangerous driving behaviors, the onboard equipment is linked to eliminate fatigue or issue a warning to avoid distracting the driver; for example, when driver fatigue is detected, the air conditioning temperature is automatically increased and refreshing music is played to improve the driving environment. When the DMS function detects that the driver is fatigued or drowsy, appropriate stimulation is needed, such as lowering the air conditioning temperature to wake the driver or playing uplifting music to stimulate the driver. During vehicle operation, the main control unit collects real-time carbon dioxide concentration data inside the vehicle via onboard sensor modules. When it detects a continuous increase in carbon dioxide concentration within a set time, it intermittently opens the sunroof slightly to allow air exchange between the inside and outside of the vehicle, thereby ensuring that the carbon dioxide concentration decreases to a preset range. Alternatively, it activates the air conditioning and switches its recirculation mode to external circulation until the carbon dioxide concentration inside the vehicle drops to the set range. If rain is detected outside the vehicle, the sunroof is prohibited from being opened, and the air conditioning is activated. Otherwise, the opening size of the sunroof and the power of the air conditioning blower are controlled according to the rate of decrease in carbon dioxide concentration, thereby reducing the carbon dioxide concentration to a reasonable set range and avoiding fatigue and drowsiness caused by carbon dioxide. Because when driving for a long time, if the interior of the vehicle is relatively sealed, the carbon dioxide concentration inside will gradually increase, which can lead to drowsiness and affect driving safety. Therefore, when the DMS function is executed, it monitors the carbon dioxide concentration and controls the sunroof ventilation or the air conditioning external circulation mode to reduce the carbon dioxide concentration when the rate of increase exceeds the set speed or the concentration exceeds the set range.In a preferred embodiment of this application, when the detected carbon dioxide concentration exceeds a set threshold, the presence of carbon dioxide may cause driver drowsiness. Therefore, to avoid driver fatigue and drowsiness caused by excessive carbon dioxide concentration, a multi-angle reminder system is used to stimulate the driver. This prevents drowsiness caused by the inability of the air conditioner or sunroof to quickly reduce the carbon dioxide concentration when it exceeds the standard. This solution stimulates the driver through multi-sensory synergy or environmental means, including massaging or pulsating the driver through the seat's built-in massage device, emitting an enhancing fragrance through the in-vehicle fragrance system, adjusting the headlights to a cool color scheme and dynamic flashing mode through the in-vehicle ambient lighting system, and playing uplifting music, etc., to remind the driver in multiple ways, achieving multi-faceted reminders through touch, hearing, smell, and sight, and avoiding the occurrence of situations caused by excessive carbon dioxide concentration.

[0036] The data processing and interactive control strategies corresponding to the TBOX function in the main control unit include: designing a secure and reliable communication protocol, and using SSL / TLS encryption technology to encrypt data between the vehicle and the cloud server to prevent data theft and tampering. It achieves accurate parsing and execution of remote control commands. When a user sends remote control commands (such as unlocking, locking, or starting the engine) via a mobile application, the TBOX can quickly parse the command and send it to the corresponding vehicle control unit via the vehicle's CAN bus, enabling remote operation. Simultaneously, it periodically uploads vehicle status information such as location, speed, fuel consumption, and fault codes to the cloud server, supporting adaptive switching between JT / T808-2019 and GB / T32960 dual protocol stacks to ensure data communication reliability in different scenarios. This allows users to view vehicle status in real time via a mobile application and provides data support for remote vehicle diagnostics and maintenance.

[0037] In this embodiment, when the main control module executes the data processing and control strategies for the DMS, DVR, and TBOX functions, the real-time data processing and interaction of these three functions may lead to insufficient computing power or lag. To avoid this, considering the relative importance of the three functions, timing control adjustments are made to the implementation of the DMS, DVR, and TBOX functions. Specifically, this includes: real-time monitoring of the computing power requirements of the main control unit's operating functions; when the main control unit's NPU algorithm can meet the needs of all functions, all functions are operated in parallel; otherwise, the DMS function is run periodically within a set time frame. Between two DMS function cycles, the DVR, TBOX, and other functions are activated. After the set time frame is completed, the computing power requirement is assessed, and timing control adjustments are made again based on the computing power requirement. This adjustment method satisfies the control of the computing power requirements of the functions. When the computing power is sufficient, control is achieved through parallel operation. When the computing power requirement is insufficient, the DMS function, which involves security, is operated as the core function, and computing power is used for the DVR, TBOX, and other functions during its operation intervals.

[0038] like Figure 1 , 2 As shown, the TBOX intelligent vehicle terminal solution integrating DVR and DMS functions provided in this embodiment specifically includes:

[0039] (I) Hardware Architecture Design

[0040] Main control unit: Employs a heterogeneous computing chipset, integrating a CPU, NPU (Neural Processing Unit), and baseband chip. The CPU handles overall system scheduling and routine data processing; the NPU is specifically designed for deep learning algorithms, providing powerful computing support for image recognition in DMS and intelligent video analysis in DVR; the baseband chip ensures stable operation of 4G / 5G communication. Taking an ARM-based multi-core processor as an example, the CPU clock speed reaches 2.5GHz, and the NPU computing power reaches 8 TOPS, meeting the requirements for multi-task parallel processing and ensuring smooth operation of functions such as DVR video encoding / decoding, DMS real-time monitoring, and TBOX data communication.

[0041] Video capture module: Equipped with multiple high-performance cameras, the front-facing camera features 4K resolution and 60fps frame rate, used not only for DVR driving image recording but also to support ADAS (Advanced Driver Assistance Systems) functions, enabling lane departure warning, forward collision warning, and more. An in-cabin infrared camera with 850nm infrared illumination technology provides night vision capabilities, clearly capturing driver facial expressions and eye movements in low-light conditions, providing accurate data for DMS (Driver Monitoring System) functions.

[0042] Communication Module: Integrates a 4G / 5G communication module, Bluetooth module, and Wi-Fi module. The 4G / 5G communication module supports high-speed data transmission, enabling remote control of the TBOX and data upload / download functions between the vehicle and the cloud server. The Bluetooth module is used for short-range communication with mobile phones and other devices, facilitating user device settings and data transfer. The Wi-Fi module enables quick connection between the vehicle and home networks or hotspots, supporting large file data synchronization and OTA software upgrades. When abnormal events such as collisions, rapid acceleration, or sudden braking are detected, relevant video clips are automatically uploaded to the platform.

[0043] Storage Unit: Employs a high-capacity eMMC storage chip with a unique cyclic overwrite mechanism. Under normal driving conditions, videos are recorded cyclically in chronological order; when abnormal events such as collisions, rapid acceleration, or sudden braking are detected, the relevant video clips are automatically encrypted and locked to prevent them from being overwritten by subsequent videos, ensuring that critical evidence is not lost.

[0044] Sensor module: Includes inertial sensors such as accelerometers and gyroscopes to detect vehicle speed, acceleration, steering angle, and other driving status information in real time. This information assists the DVR in recording more accurate vehicle driving data, provides a reference for the DMS to judge the rationality of the driver's driving behavior, and is also an important component of the vehicle status information uploaded by the TBOX. In addition, it directly connects to the vehicle ECU via the CAN bus to obtain real-time operating data of the vehicle, such as motor speed and power consumption, further enriching the data sources.

[0045] (II) Software Algorithm Design

[0046] Multimodal data fusion algorithm: A multimodal data fusion model is constructed, employing a spatiotemporal alignment algorithm to ensure consistency in time and space between video data acquired by the DVR, driver status data obtained by the DMS, and vehicle status data received by the TBOX. Deep learning algorithms, such as convolutional neural networks (CNNs), are used to extract features from video images, combined with long short-term memory networks (LSTMs) for temporal analysis of driver status and vehicle driving data, enabling accurate judgment of driving scenarios and potential risks. For example, when the vehicle is traveling at high speed and the driver is fatigued, the system can provide early warning of potential hazards.

[0047] DVR Function Algorithm: The video encoding adopts the efficient H.265 encoding standard, which reduces data storage and transmission bandwidth requirements by approximately 50% compared to H.264 encoding while ensuring 4K high-definition image quality. An intelligent video recording strategy is designed: in addition to regular loop recording, when the sensor detects abnormal events such as collisions, rapid acceleration, or sudden braking, an emergency recording function is automatically activated to record video at a higher bitrate and encrypt and store the relevant segments. Simultaneously, it supports segmented video storage and uploading to platforms, facilitating users to quickly find and play back video content from specific time periods.

[0048] DMS (Driver Monitoring System) algorithm: Based on deep learning-based face recognition and pose estimation algorithms, it establishes a high-precision facial expression recognition model by training on massive amounts of facial image data. It can detect various dangerous driving behaviors in real time, such as driver fatigue, distraction, phone calls, and smoking. When abnormal behavior is detected, it immediately issues sound and light warnings to alert the driver. Furthermore, it can also coordinate with the vehicle's air conditioning, audio system, and other equipment based on the driver's condition. For example, when driver fatigue is detected, it automatically raises the air conditioning temperature and plays refreshing music to improve the driving environment.

[0049] TBOX Functional Algorithm: A secure and reliable communication protocol is designed, employing SSL / TLS encryption technology to encrypt data between the vehicle and the cloud server, preventing data theft and tampering. It enables precise parsing and execution of remote control commands. When a user sends remote control commands (such as unlocking, locking, or starting the engine) via a mobile application, TBOX can quickly parse the command and send it to the corresponding vehicle control unit via the vehicle's CAN bus, enabling remote operation. Simultaneously, it periodically uploads vehicle status information such as location, speed, fuel consumption, and fault codes to the cloud server. It supports adaptive switching between JT / T808-2019 and GB / T32960 dual protocol stacks, ensuring reliable data communication in different scenarios. This allows users to view vehicle status in real-time via a mobile application and provides data support for remote vehicle diagnostics and maintenance.

[0050] (III) Hardware Setup and Deployment

[0051] Main control unit installation: Select a multi-core heterogeneous computing chipset based on ARM architecture (such as a CPU with a main frequency of 2.5GHz and an NPU with a computing power of 8TOPS), and solder it to the core position of the motherboard to ensure stable signal transmission between the CPU, NPU and baseband chip, and connect a heat dissipation device to prevent the chip from overheating.

[0052] Video capture module deployment: A 4K resolution, 60fps front-facing camera is installed above the vehicle's windshield and connected to the motherboard video interface; an 850nm infrared-illuminated in-cabin camera is installed at the top of the cockpit or in a suitable position on the dashboard to ensure complete capture of the driver's facial information.

[0053] Communication module integration: The 4G / 5G communication module, Bluetooth 5.0 module and Wi-Fi 6 module are soldered or connected to the motherboard through interfaces. The antenna is external or reasonably arranged inside the vehicle to ensure the quality of signal reception and transmission.

[0054] Sensor module integration: Inertial sensors such as accelerometers and gyroscopes are soldered or connected to the motherboard via interfaces to ensure the acquisition of real-time operating data.

[0055] Storage unit configuration: Solder high-capacity eMMC storage chips to the motherboard and configure firmware with cyclic overwrite and abnormal data protection mechanisms to ensure safe and reliable data storage.

[0056] (iv) Software system deployment and configuration

[0057] System initialization: After the device is started, the CPU of the main control unit loads the operating system, initializes hardware resources such as the NPU and baseband chip, and establishes communication connections with each module.

[0058] Multimodal data fusion:

[0059] A spatiotemporal alignment algorithm is used to synchronize the timestamps and calibrate the spatial coordinates of DVR video data, driver status data collected by DMS, and vehicle status data received by TBOX.

[0060] The CNN algorithm is used to extract features from video images (such as the vehicle's surrounding environment and the driver's facial features), and the LSTM algorithm is combined to analyze the temporal changes in the driver's state and vehicle driving data to construct a driving scene model.

[0061] DVR Functionality Implementation:

[0062] The H.265 encoding standard is used to encode video data in real time, reducing storage and transmission pressure.

[0063] The sensors monitor the vehicle's status in real time. When events such as collisions, rapid acceleration, or sudden braking are detected, the emergency recording function is triggered to record video at a high bit rate and store it in encrypted form. Under normal conditions, the video is recorded in a loop in chronological order.

[0064] DMS Functionality Implementation:

[0065] Based on deep learning models (such as well-trained facial expression recognition models), the system uses in-cabin cameras to detect driver fatigue, distraction, phone calls, smoking, and other behaviors in real time.

[0066] When abnormal behavior is detected, an alarm is triggered via buzzer, dashboard lights, etc., and the air conditioner is turned up to raise the temperature and the audio system plays refreshing music. At the same time, the abnormal information is uploaded to the cloud via TBOX.

[0067] TBOX Functionality: Employs SSL / TLS encryption technology to establish a secure communication channel between the vehicle and the cloud, supporting adaptive switching between JT / T808-2019 and GB / T32960 protocols.

[0068] It receives remote commands from the mobile application (such as unlocking or starting the engine), parses them, and sends them to the vehicle control unit via the CAN bus for execution. It also feeds back the execution results to the cloud and the mobile application. It periodically uploads information such as vehicle location, speed, and fault codes to the cloud.

[0069] Software system interaction flow: The video captured by the front 4K camera and the infrared cabin camera is transmitted to the heterogeneous processor via LVDS; the vehicle signal is transmitted to the heterogeneous processor via the CAN bus.

[0070] The video processing unit adopts the efficient H.265 encoding standard to ensure high-definition 4K video quality. Simultaneously, an intelligent video recording strategy is designed. In addition to regular loop recording, when the sensor detects abnormal events such as collisions, rapid acceleration, or sudden braking, an emergency recording function is automatically activated to record video at a higher bitrate. The relevant segments are encrypted and stored in the eMMC, and then uploaded to the cloud platform for encrypted storage via the 4G / 5G communication module.

[0071] Based on the DMS analysis engine's deep learning-based face recognition and pose estimation algorithms, a high-precision facial expression recognition model is established by training on massive amounts of facial image data. It can detect various dangerous driving behaviors in real time, such as driver fatigue, distraction, phone calls, and smoking. When abnormal behavior is detected, it immediately issues warnings to the driver via sound and lights.

[0072] A secure and reliable communication protocol stack is designed, employing SSL / TLS encryption technology to encrypt data between the vehicle and the cloud server, preventing data theft and tampering. High-speed data transmission via a 4G / 5G communication module enables remote control of the TBOX and data upload / download functions between the vehicle and the cloud server.

[0073] Obviously, the specific implementation of this invention is not limited to the above-described methods. Any non-substantial improvements made using the inventive concept and technical solution of this invention are within the protection scope of this invention.

Claims

1. An integrated TBOX intelligent vehicle terminal system, characterized in that: The system includes a main control unit, a video acquisition module, a communication module, and a storage unit. The main control unit is connected to the video acquisition unit, the communication unit, and the storage unit. The main control unit is used to implement the data processing and interactive control strategies corresponding to the DVR, DMS, and TBOX functions. The video acquisition module is used to provide the necessary images for the main control module to implement the DVR function. The communication module is used to provide communication signals for the main control module to implement the TBOX function. The storage unit is used to provide data storage when the main control unit implements the DVR, DMS, and TBOX functions.

2. The integrated TBOX intelligent vehicle terminal system as described in claim 1, characterized in that: The main control unit adopts a heterogeneous computing chipset, which integrates a CPU, an NPU, and a baseband chip. The CPU is responsible for the overall system scheduling and data processing. The NPU runs deep learning algorithms to realize image recognition for DMS function and intelligent video analysis for DVR function. The baseband chip provides 4G / 5G communication network.

3. The integrated TBOX intelligent vehicle terminal system as described in claim 1, characterized in that: The video acquisition module includes a driving camera and a DMS camera. The driving camera is used to acquire images required for DVR functions and images required for driver assistance.

4. The integrated TBOX intelligent vehicle terminal system as described in claim 1, characterized in that: The communication module includes a 4G / 5G communication module, a Bluetooth module, and a Wi-Fi module. The 4G / 5G communication module supports high-speed data transmission to enable remote control of the TBOX and data upload / download functions between the vehicle and the cloud server. The Bluetooth module is used for short-range communication with mobile devices, enabling users to configure devices and transmit data. The Wi-Fi module enables rapid connection between the vehicle and home networks or hotspots, facilitating large file data synchronization and OTA software upgrades.

5. The integrated TBOX intelligent vehicle terminal system as described in claim 1, characterized in that: The storage unit uses a high-capacity eMMC storage chip and executes a cyclic overwrite mechanism to store data. Under normal vehicle driving conditions, the main control unit controls the storage unit to record video in cyclic order according to time. When an abnormal event is detected, the relevant video clips are encrypted and stored, and locked to prevent data loss.

6. An integrated TBOX intelligent vehicle terminal system as described in any one of claims 1-5, characterized in that: The system also includes a sensor module, which is used to collect vehicle status data in real time, and its output is connected to the main control unit. The driving status information is used to assist the DVR function in recording more accurate vehicle driving data, assist the DMS function in judging the rationality of the driver's driving behavior, and also to upload vehicle status information for the TBOX function.

7. A control method for an integrated TBOX intelligent vehicle terminal system as described in any one of claims 1-6, characterized in that: The method includes: a main control unit connected to a video acquisition unit, a communication unit, and a storage unit; the main control unit is used to implement data processing and interactive control strategies corresponding to DVR, DMS, and TBOX functions; the video acquisition module is used to provide the required images for the main control module to implement DVR functions; the communication module is used to provide communication signals for the main control module to implement TBOX functions; and the storage unit is used to provide data storage when the main control unit implements DVR, DMS, and TBOX functions.

8. The control method of an integrated TBOX intelligent vehicle terminal system as described in claim 7, characterized in that: The main control unit performs time-series analysis on the collected driver status data and vehicle driving data through a multimodal data fusion algorithm to accurately judge driving scenarios and potential risks, and outputs corresponding warning signals based on the judgment results; The process employs a spatiotemporal alignment algorithm to align the video data collected by the DVR, the driver status data acquired by the DMS, and the vehicle status data received by the TBOX in both time and space dimensions. Deep learning algorithms are used to extract features from the video images, and a Long Short-Term Memory (LSTM) network is combined to perform temporal analysis on the driver status data and vehicle driving data to obtain the judgment result.

9. A control method for an integrated TBOX intelligent vehicle terminal system as described in claim 7 or 8, characterized in that: The data processing and interactive control strategies corresponding to the DVR function include: the main control unit uses a loop recording storage method to store the recorded image data; when an abnormal event is detected by the sensor module, the emergency recording function is triggered, at which time the video is recorded at a higher bitrate and the video clips within the time range of the abnormal event are encrypted and stored.

10. A control method for an integrated TBOX intelligent vehicle terminal system as described in claim 7 or 8, characterized in that: The data processing and interactive control strategies corresponding to the DMS function include: deep learning-based face recognition and pose estimation algorithms, which establish a high-precision facial expression recognition model by training massive amounts of face image data, and detect various dangerous driving behaviors of the driver in real time through the model. When a dangerous behavior is detected, an alarm is immediately issued to remind the driver, and or the vehicle equipment is linked to eliminate or reduce the driver's dangerous behavior or dangerous state according to the driver's state.

Images