Ai electronic rearview mirror

Abstract

The present invention discloses an AI Dashcam including an edge computing device, wherein the edge computing device includes an NPU and an AI model coupled to the NPU. The AI model and the NPU provide the edge computing for intelligence driving assistance, wherein the AI model includes an artificial intelligence driving assistance model for improving driving safety. The artificial intelligence driving assistance includes ADAS, DMS, OMS or the combination thereof.

Description

[Technical Field]

[0001] This invention relates to a rearview mirror, and more particularly to an electronic rearview mirror with an edge computing device. [Previous Technology]

[0002] Early car rearview mirrors were simple mirrors used to allow drivers to see what was behind the vehicle. These mirrors had limited fields of view and were susceptible to glare under certain conditions. With advancements in automotive technology, anti-glare technology was introduced into rearview mirrors to address glare issues during nighttime driving, reducing glare from rear headlights and improving driving safety. In recent years, electronic rearview mirrors have gradually replaced traditional rearview mirrors, utilizing imaging technology to provide a wider field of view and more auxiliary functions, such as blind spot monitoring and distance warning.

[0003] The working principle of an electronic rearview mirror in a vehicle is based on the combination of a camera and a display. The typical workflow involves a camera capturing images; for example, a high-resolution camera installed on the outer side of the rear of the vehicle captures real-time images of the rear, providing a wider field of view than a traditional mirror. The images captured by the camera are processed by an image processing system, which includes image enhancement and optimization to ensure clear visibility under different lighting conditions. For example, night vision functions and anti-glare technology can improve image quality. The processed image is then displayed instantly on the display. The display is usually located in the position of a traditional rearview mirror for easy viewing by the driver. To reduce the harm of glare to the driver, previous technologies have used one or more layers of anti-glare film coating on the display screen to reduce the impact of glare on the human eye.

[0004] Some electronic rearview mirrors are also equipped with auxiliary functions such as distance warning, blind spot monitoring, and lane departure warning. These functions are achieved through image processing technology and the collaboration of vehicle sensors to improve driving safety. The development of electronic rearview mirrors not only improves the clarity of vision but also helps to improve vehicle maintenance and efficiency. Modern electronic rearview mirrors have become very diverse. In addition to recording basic vehicle data, some types can also record various data such as vehicle position, acceleration, and braking force, and can provide more comprehensive data.

[0005] Prior art, such as TWM660161U, announced on September 1, 2024, discloses a rearview mirror display assembly comprising a main control module, a display screen, a liquid crystal mirror, a first photosensitive sensor, and a second photosensitive sensor. The first photosensitive sensor is used to detect the brightness of the environment in front of the vehicle and generates a first detection signal accordingly. The second photosensitive sensor is configured to detect the brightness of the environment behind the vehicle and generates a second detection signal accordingly, which is transmitted to the main control module. Based on the comparison of the first and second detection signals, the main control module controls the transmittance and reflectance of the liquid crystal mirror, so that the liquid crystal mirror can have different transmittance and reflectance according to different conditions, which is beneficial to driving safety.

[0006] Patent TWM653976U, published on April 11, 2024, discloses a vehicle electronic rearview mirror warning system with an input interface for users to set display data. A detection unit is used to detect whether an object is approaching and generates a sensing signal. When a threat is detected, the electronic rearview mirror displays a warning message. The detection unit includes at least one vibration sensor, which senses the vibration state of the vehicle and generates a sensing signal.

[0007] Patent TWM574119U discloses a vision-enhancing electronic rearview mirror, comprising a light modulation module, a control module for controlling the operation of the vision-enhancing electronic rearview mirror, a camera module for detecting image signals and transmitting the image signals to the control module, and a display module coupled to the control module for displaying the image signals. Patent No. TWM310832U discloses an electronic rearview mirror that can reduce electromagnetic interference, comprising a first shielding layer, which can effectively shield electromagnetic interference generated when the electronic device is powered on, thereby reducing electromagnetic interference. The prior art rearview mirror TWM579728U includes an electro-optic dielectric layer disposed between a first light-transmitting component and a second light-transmitting component, a light-transmitting electrode disposed between the first light-transmitting component and the electro-optic dielectric layer, and the electro-optic dielectric layer disposed between the first light-transmitting component and a reflective layer; a power supply transmits electrical energy to the electro-optic dielectric layer, thereby changing the transparency of the electro-optic dielectric layer.

[0008] With the popularization and development of the Internet, artificial intelligence technology is becoming increasingly mature. However, its current applications are mostly in customer service or chat. Customer service applications provide services through AI customer service, replacing traditional customer service personnel or robots. However, many previous rearview mirror technologies have not yet involved integration in this field. Based on the need to expand the application level, this invention proposes a new application area. [Summary of the Invention]

[0009] To achieve the above objectives, the present invention proposes an AI electronic rearview mirror with an edge computing device, integrating intelligent services to provide safety. In one embodiment of the present invention, the edge computing device includes an acceleration circuit, which includes a neural processing unit (NPU). An AI model is stored in memory and coupled to the neural processing unit. The AI ​​model and the neural processing unit provide intelligent driving assistance. The AI ​​model includes an artificial intelligence driving assistance model to improve driving safety.

[0010] According to one aspect of the present invention, the above model includes an artificial intelligence driving assistance model to assist the driver in improving driving safety and driving experience. The above artificial intelligence driving model includes one or a combination of Advanced Driver Assistance Systems (ADAS), Driver Management Systems (DMS), and Occupation Management Systems (OMS).

[0011] According to one aspect of the present invention, the AI ​​electronic rearview mirror of the present invention captures dynamic images of the driver and passengers through an image sensor, analyzes the behavior of the driver and passengers through an artificial intelligence driving model, and issues warnings or actively intervenes in a timely manner. In other words, in one embodiment, the AI ​​dashcam of the present invention captures driving images through an image sensor and analyzes driving behavior through a driving management system; it also captures passenger images through an image sensor and analyzes passenger behavior through a passenger management system.

[0012] In one aspect of the present invention, by capturing images of the front, rear, and surrounding areas of a vehicle through sensors, and using artificial intelligence for data analysis, identification, and warning of possible errors, driving safety is improved. The advanced driver assistance system can perform one or any combination of the following tasks: monitoring road conditions, identifying obstacles, maintaining vehicle speed, maintaining lane, and maintaining distance.

[0013] According to one aspect of the present invention, the AI ​​electronic rearview mirror includes a G-sensor and a communication module coupled to a processor, wherein the G-sensor includes one or a combination of an accelerometer and a gyroscope. The communication module includes one or a combination of Wi-Fi, Bluetooth, 4G, 5G, and 6G; the AI ​​electronic rearview mirror includes a GPS module for positioning.

[0014] In another aspect of the present invention, AI electronic rearview mirrors based on edge computing can expand services and applications through AI. The AI ​​electronic rearview mirror of the present invention can be connected to one or a combination of user devices and in-vehicle systems. The AI ​​electronic rearview mirror built on a neural computing platform can expand driving assistance, electronic map applications, and optimize navigation and positioning accuracy, etc.

Implementation Method

[0015] This invention will be described in detail here with reference to specific embodiments and their viewpoints. Such descriptions are for illustrative purposes only and are not intended to limit the scope of the invention. Therefore, in addition to the specific and preferred embodiments described in the specification, the invention can also be widely implemented in other different embodiments. The following describes the implementation of the invention through specific embodiments. Those skilled in the art can easily understand the effectiveness and advantages of the invention from the content disclosed in this specification. Furthermore, the invention can also be used and implemented through other specific embodiments, and the various details set forth in this specification can be applied based on different needs, and various modifications or changes can be made without departing from the spirit of the invention.

[0016] This invention proposes an AI electronic rearview mirror 104 based on edge computing, which can expand AI services and AI applications. According to one embodiment, referring to FIG1, the AI ​​electronic rearview mirror 104 includes an edge computing device 1050 embedded in the AI ​​electronic rearview mirror 104. In one embodiment, the edge computing device 1050 includes an AI model 1046 electrically connected to an AI acceleration circuit 1052, wherein the AI ​​acceleration circuit 1052 includes at least one neural processing unit (NPU) 1054. The AI ​​model 1046 is stored in memory 1053 and coupled to the AI ​​acceleration circuit 1052 to facilitate the provision of intelligent services. The edge computing device 1050 adopts a distributed computing model, transferring data processing and storage from data centers or the cloud to edge devices closer to the data source, such as the AI ​​electronic rearview mirror 104 of this invention. Edge computing can reduce message latency, improve computing efficiency, and reduce bandwidth usage.

[0017] The AI ​​model 1046 has a driving assistance model that utilizes a trained artificial intelligence driving model to assist the driver in improving driving safety and driving experience. The artificial intelligence driving model can help the driver monitor road conditions, identify potential obstacles, and maintain vehicle speed and distance. The AI ​​model 1046 stores basic vehicle data, such as driving assistance model data, autonomous driving model data, and other data, and can connect with other devices, such as various sensors 1042, to help the driver obtain more comprehensive data analysis, improving driving safety and comfort. The edge computing device 1050 processes and analyzes data such as vehicle driving route, speed, and acceleration, and provides real-time feedback to help the driver better understand the vehicle's status. Sensors 1042 are equipped around the vehicle or built into the AI ​​electronic rearview mirror 104. High-resolution cameras can record images around the vehicle, providing a more comprehensive accident record. The AI ​​model 1046 uses artificial intelligence technology to analyze data through the AI ​​acceleration circuit 1052, identifying and warning the driver of potential errors, thereby improving driving safety.

[0018] The AI ​​model 1046 includes a trained artificial intelligence driving model to improve driving safety. The artificial intelligence driving model includes Advanced Driver Assistance Systems (ADAS), Driver Management Systems (DMS), and Occupation Management Systems (OMS). Advanced Driver Assistance Systems (ADAS) aim to improve driving safety and comfort. ADAS systems typically utilize various sensors 1042, such as image sensors, light, radar, and ultrasound, to monitor the vehicle's surroundings and provide real-time information and warnings to the driver, even automatically intervening when necessary. ADAS functions include lane departure warning systems, adaptive cruise control, blind spot detection systems, and parking assistance systems. DMS and OMS respectively manage and analyze the behavioral patterns of the driver and passengers, such as mental state and driving behavior. The AI ​​electronic rearview mirror 104 captures dynamic images of the driver and passengers inside the vehicle through its sensor 1042. The neural processing unit 1054 analyzes the behavior of the driver and passengers through the driver management system (DMS) and passenger management system (OMS) and issues warnings or intervenes proactively as appropriate.

[0019] The AI ​​acceleration circuit 1052 is designed to accelerate artificial intelligence computing tasks. The AI ​​acceleration circuit 1052 includes at least an NPU 1054. The AI ​​acceleration circuit 1052 improves the inference and training speed of the AI ​​model 1046 through parallel computing and specialized algorithms. To handle large amounts of data, the AI ​​acceleration circuit 1052 is typically equipped with high-bandwidth memory to ensure fast data transfer. To improve efficiency, the AI ​​acceleration circuit 1052 supports a dedicated instruction set to facilitate the optimization of AI computing tasks.

[0020] The NPU 1054 is a dedicated processing unit for artificial intelligence and machine learning, assisting in accelerating the inference of the AI ​​model 1046, improving efficiency and reducing energy consumption. The main functions of the NPU include: efficient data processing, capable of rapidly processing large amounts of data; low energy consumption, compared to traditional CPUs and GPUs, making it suitable for mobile devices and embedded systems; and optimization for specific AI and ML algorithms, enabling more efficient task execution. For object recognition and detection, the NPU 1054 can process large amounts of data from the vehicle sensors 1042 in real time, identifying and marking pedestrians, other vehicles, road signs, etc., helping the autonomous driving system make immediate decisions. The NPU 1054 can quickly calculate the optimal path and dynamically adjust according to real-time traffic conditions to ensure safe and efficient driving. In one embodiment, the NPU 1054 processes data from sensors 1042, etc., to form a comprehensive perception of the vehicle's surrounding environment. Furthermore, the AI ​​model 1046 and the NPU 1054 can analyze driving data to optimize driving strategies, improving driving efficiency and safety. In one embodiment, the NPU 1054 can process data in real time to ensure the correctness of the autonomous driving system.

[0021] The AI ​​electronic rearview mirror 104 of the present invention includes a processor 1041, and a sensor 1042 coupled to the processor 1041. The sensor 1042 includes an image sensor for capturing image information. For example, the sensor 1042 includes a panoramic image sensor with a range of 130 to 160 degrees. The processor 1041 can convert the captured images into digital signals, compress them, and store them. A memory 1040 coupled to the processor 1041 is used to store video files, typically using an SD card or built-in memory. A G-sensor 1043 coupled to the processor 1041 is used to sense vehicle acceleration and collisions, and automatically records when a collision occurs.

[0022] In another embodiment, the AI ​​electronic rearview mirror 104 may optionally be configured to include a communication module 1045, such as one or both of a short-range communication module and a long-range communication module. In other words, the communication module 1045 includes one or any combination of Wi-Fi, Bluetooth, 4G, 5G, and 6G. The communication module 1045 is coupled to the processor 1041, allowing the user to view the recorded video and settings in real time via an application on the user device 102 (e.g., a mobile phone) through Wi-Fi or Bluetooth. The long-range communication module 1045 (e.g., 4G, 5G, 6G) is coupled to the processor 1041, allowing the user to communicate with a remote location. The communication module 1045, whether short-range or long-range, can download or update data, and can also receive dynamic information, such as weather information and traffic information. While in the vehicle or in motion, the user can connect to a remote location through the communication module 1045 to receive the required information at any time. Input unit 1047 (e.g., touch panel, microphone) and output unit 1048 (e.g., display, speaker), individually coupled to processor 1041, allow user input or user device output. In one embodiment, communication module 1045 enables edge computing device 1050 to communicate with server 106. AI electronic rearview mirror 104 includes G-sensor 1043 and GPS module 1049 coupled to processor 1041. GPS module 1049 is used to record the vehicle's position and speed. G-sensor 1043 includes a gyroscope and accelerometer. The gyroscope measures the angular velocity of an object, helping the device detect changes in its orientation and attitude. The accelerometer measures the acceleration of an object, enabling the detection of device movement and tilt. These two types of sensors work together to provide more accurate and richer data for tracking device movement and orientation. A power supply (not shown) provides power to the dashcam, and in one embodiment, it may be powered by the vehicle.

[0023] Data and operating system data are stored in memory 1040. Memory 1053 and 1040 can be selected from the following or any combination thereof: including read-only memory, random access memory, non-volatile flash memory, etc. Communication module 1045 can handle signal reception, baseband processing, etc.; signals are sent to output unit 1048, such as a speaker or display. Through communication module 1045, voice or video signals or both can be transmitted or received, or the above information can be shared with other contacts or friends in social applications, enabling the present invention to synchronously and wirelessly share driving information.

[0024] Figure 2 shows the system architecture of the present invention. The AI ​​electronic rearview mirror 104 can be connected to the user device 102, and the user device 102 can also be connected to the server 106. The server 106 can be a cloud server built by others, or it can be a self-built or rented cloud server. The AI ​​electronic rearview mirror 104 has an embedded or built-in intelligent model, which can improve the calculation results. The AI ​​electronic rearview mirror 104 can be connected to the user device 102 through a communication network, and the AI ​​electronic rearview mirror 104 can also communicate with the server 106, or indirectly connect to the server 106 through the user device 102. The aforementioned server 106 is a server under a cloud architecture. In one embodiment, the server 106 can be a dedicated server or a combination of a traditional computing server and a dedicated AI accelerator to provide higher performance and flexibility. In one embodiment, the user device 102 can be selected from smartphones 102a, tablets 102b, etc. According to an embodiment of the present invention, the AI ​​electronic rearview mirror 104 can obtain the required information or update the built-in intelligent model through the server 106. According to one embodiment of the present invention, the communication network may include a LAN network, a WAN network, etc., but is not limited thereto. The communication network may be, for example, a Wi-Fi network, GSM, CDMA, TDMA, Bluetooth, or any other wireless communication protocol.

[0025] This invention utilizes the advantages of edge computing's distributed computing model to process and store data in devices closer to the data source. Specific advantages of this invention include: (1) Low latency: Since data processing is performed locally on the AI ​​dashcam 104, data transmission latency can be significantly reduced, which is highly effective for applications requiring immediate response. (2) Reduced bandwidth requirements: Processing large amounts of data on the AI ​​dashcam 104 eliminates the need to transmit large amounts of information to the cloud, thereby reducing bandwidth resources and costs. (3) Enhanced privacy and security: Data is processed and stored locally on the AI ​​dashcam 104, reducing the risk of data being intercepted or leaked during transmission. (4) Distributed processing capability: Distributing computing resources across various AI dashcams 104 improves system reliability and flexibility.

[0026] The methods or embodiments proposed above in this invention can be executed in computer systems, servers, or similar computing systems. These server / computer systems typically include at least one controller, and peripheral devices may include storage systems, such as memory and file storage, user output interface devices, user input interface devices, and network interfaces. The network interface provides a connection interface to an external network and is coupled to corresponding interface devices of other computing devices. User input devices may include microphones, keyboards, mice, trackballs, touchscreens, touchpads, or pointing devices, and may be integrated into displays and other types of input devices. Computing devices can be of various types, including workstations, servers, computing clusters, or other data processing systems or computing devices.

[0027] The above description is a preferred embodiment of the present invention. Those skilled in the art should understand that it is used to illustrate the present invention and not to limit the scope of the patent rights claimed by the present invention. The scope of patent protection shall be determined by the appended claims and their equivalent fields. Any modifications or refinements made by those skilled in the art without departing from the spirit or scope of this patent are equivalent changes or designs made under the spirit disclosed in the present invention and should be included in the scope of the following claims. [Simplified Explanation of the Diagram]

[0028] [Figure 1] shows the functional block diagram of the AI ​​electronic rearview mirror of the present invention.

[0029] [Figure 2] shows a schematic diagram of the system architecture of the present invention.

Claims

1. An AI electronic rearview mirror, comprising: processor; An edge computing device is electrically connected to the processor; wherein the edge computing device includes an AI acceleration circuit, the AI ​​acceleration circuit includes a neural processing unit and an AI model stored in memory, the AI ​​acceleration circuit includes dedicated instruction set optimization computing, the AI ​​acceleration circuit improves the inference and training speed of the AI ​​model through parallel computing and specialized algorithms; and wherein intelligent driving assistance is provided through the neural processing unit and the AI ​​model, wherein the AI ​​model is a trained intelligent model, and the AI ​​dashcam can update the trained intelligent model through a server.

2. As in Request 1, the AI ​​electronic rearview mirror, wherein the AI ​​acceleration circuit is equipped with high-bandwidth memory to ensure fast data transmission.

3. As in request item 1, the AI ​​electronic rearview mirror, wherein the AI ​​model includes an artificial intelligence driving model.

4. The AI ​​electronic rearview mirror as claimed in claim 3, wherein the artificial intelligence driving model includes one or a combination of an advanced driver assistance system, a driver management system, and a passenger management system.

5. As in request item 4, the AI ​​electronic rearview mirror captures driving images through sensors, and the driving management system analyzes driving behavior or status.

6. As in request item 4, the AI ​​electronic rearview mirror captures occupant images through sensors, and the passenger management system analyzes occupant behavior or status.

7. The AI ​​electronic rearview mirror as requested in item 4, wherein the advanced driver assistance system performs one or any combination of the following tasks: monitoring road conditions, identifying obstacles, maintaining vehicle speed, maintaining lane position, and maintaining following distance.

8. The AI ​​electronic rearview mirror as described in claim 1, which includes one or a combination of an accelerometer and a gyroscope.

9. The AI ​​electronic rearview mirror as requested in item 1, which includes one or a combination of Wi-Fi, Bluetooth, 4G, 5G, and 6G modules.

10. The AI ​​electronic rearview mirror as requested in item 1, which includes a GPS module.

Images