Controller system and vehicle for an intelligent vehicle

Abstract

The present application relates to the field of intelligent vehicle technology in the field of artificial intelligence technology, and in particular to a controller system. The controller system includes an intelligent driving area control unit, a human-machine interaction area control unit, and a sensor interface unit connected to the intelligent driving area control unit and the human-machine interaction area control unit. The sensor interface unit is connected to a sensor and transmits data of the sensor to the intelligent driving area control unit and the human-machine interaction area control unit. The controller system further includes a third area control unit and a network switching unit. The network switching unit is connected to the intelligent driving area control unit, the human-machine interaction area control unit, and the third area control unit. The third area control unit is connected to any one of a chassis system control unit, a power system control unit, or a body system control unit. The present application can simplify external cable connection, reduce the occupation of data stream forwarding resources, and reduce the power consumption of the vehicle.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicle technologies, and in particular, to a controller system, a vehicle control system, a data processing method, a control method, and a system startup method.

Background Art

[0002] Intelligent driving is a key technology in realizing intelligent vehicles and intelligent transportation, and is an inevitable future vehicle development trend. According to the prediction of the Institute of Electrical and Electronics Engineers, 75% of new vehicles will provide intelligent driving functions by 2040. From a technical perspective, intelligent driving brings revolutionary challenges and opportunities to the automotive industry, and can improve driving safety, avoid traffic jams, enhance energy efficiency, and improve urban operation efficiency. From the perspective of the integrated industrial development of the Internet of Things, cloud computing, and artificial intelligence (AI), intelligent driving will become an important driving force for the future development of many industries and will promote the rapid development of intelligent manufacturing technologies and next-generation information technologies.

[0003] The intelligent driving system of a vehicle uses sensors such as cameras, radars, and lidars to obtain information about the vehicle and the information around the vehicle, analyzes and processes the obtained information to realize functions such as obstacle detection, target recognition, vehicle positioning, route planning, and driver monitoring / attention arousal, and improves the driving safety, automation, and comfort of the vehicle.

[0004] As the demand for intelligence in vehicle electronic functions increases, the electrical and / or electronic (E / E) architecture of vehicles is gradually evolving from a distributed architecture to a centralized architecture. To obtain rich information, the number and types of sensors arranged in the vehicle are also increasing. These sensors are connected to the vehicle to provide data, and the data needs to be considered in the design.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Summary of the Invention

Problems to be Solved by the Invention

[0006] This application provides a controller system, a vehicle control system, a data processing method, a control method, and a system startup method to simplify external cable connection, reduce the occupation of data stream transfer resources, and reduce the power consumption of the vehicle.

Means for Solving the Problems

[0007] To achieve the above object, the first aspect of this application provides a controller system. [Item 1] A controller system for an intelligent vehicle, comprising: Intelligent driving area control unit, Human-machine interaction area control unit, A sensor interface unit configured to be connected to an intelligent driving area control unit and a human-machine interaction area control unit through separate interfaces on the sensor interface unit, wherein an external interface of the sensor interface unit is connected to a sensor and configured to transmit data of the sensor to the intelligent driving area control unit and the human-machine interaction area control unit, the sensor interface unit; including signals transmitted by each of the separate interfaces include a synchronization signal, a control signal, and / or a video data stream signal; the synchronization signal is used as a trigger signal for the sensor or a frame rate control signal for the sensor for video capture, and the control signal is used for setting the sensor or reading data settings; the sensor interface unit is configured to select either the intelligent driving area control unit or the human-machine interaction area control unit connected to the sensor interface unit to transmit the synchronization signal and / or the control signal to the sensor through the external interface based on at least one of the priorities, safety levels, operating modes of the intelligent driving area control unit and the human-machine interaction area control unit, and the operating states of the intelligent driving area control unit and the human-machine interaction area control unit; when the priority and the safety level are high, the intelligent driving area control unit is selected first; the operating mode of the controller system is in the center remote mode, only the human-machine interaction area control unit is selected; in the surround view mode, it is selected according to the priority and the safety level, the controller system. [Item 2] The controller system according to claim 1, wherein the synchronization signal transmitted by the intelligent driving area control unit is synchronized in time with the synchronization signal transmitted by the human-machine interaction area control unit. [Item 3] A first display interface unit connected to the intelligent driving area control unit, wherein the first display interface unit is connected to an instrument display unit, or A second display interface unit connected to the human-machine interaction area control unit, wherein the second display interface unit is connected to a display screen, The controller system according to claim 1 or 2, further comprising at least one of the above. [Item 4] A network switching unit connected to the intelligent driving area control unit and the human-machine interaction area control unit The controller system according to any one of claims 1 to 3, further comprising the above. [Item 5] Further comprising a third area control unit connected to the network switching unit, The third area control unit is connected to any one of the following control units, namely, a chassis system control unit, a power system control unit, or a vehicle body system control unit, and the third area control unit Collects information from the connected control unit, and Is configured to transmit the collected information to the intelligent driving area control unit using the network switching unit. The controller system according to claim 4. [Item 6] The network switching unit further Is configured to be connected to at least one of a lidar sensor, a millimeter wave radar sensor, an event data recorder, a vehicle Internet communication box, and an in-vehicle recorder. The controller system according to claim 4 or 5. [Item 7] The intelligent driving area control unit is configured to perform an assisted driving or autonomous driving function, a part of the vehicle control function, or a part of the vehicle body control function. The third area control unit is configured to implement a chassis system control function, a power system control function, other functions of the vehicle control, or other functions of the vehicle body control. The human-machine interaction area control unit is configured to implement an entertainment area application function or a human-machine user interface function. The controller system according to claim 5. [Item 8] The sensor interface unit is further configured to receive data of the sensor. The human-machine interaction area control unit is further configured to perform human proximity detection or intrusion detection based on the data of the sensor, and transmit the data of the sensor to an in-vehicle recorder using the network switching unit. The controller system according to any one of claims 4 to 7. [Item 9] The human-machine interaction area control unit further generates alert data when the human proximity or intrusion is detected, and is configured to transmit the alert data to an audio device for playback, or transmit the alert data to a display screen for display. The controller system according to claim 8. [Item 10] The sensor interface unit is further configured to receive data of the sensor. The human-machine interaction area control unit is further configured to perform image processing based on the data of the sensor to generate surround view image data, and transmit the surround view image data to a display screen for display. The controller system according to any one of claims 1 to 7. [Item 11] The speed at which the intelligent driving area control unit performs initialization is lower than the speed at which the human-machine interaction area control unit performs initialization. The human-machine interaction area control unit is further configured to receive a synchronization signal or a control signal of the human-machine interaction area control unit through the sensor interface unit to enable the surround view mode, and transmit the synchronization signal or the control signal to the sensor. The intelligent driving area control unit is configured to receive a synchronization signal or a control signal of the intelligent driving area control unit through the sensor interface unit and transmit the synchronization signal or the control signal to the sensor in order to undertake control of the sensor interface unit or the sensor. The controller system according to any one of claims 1 to 7. [Article 12] A vehicle including the controller system according to any one of claims 1 to 11 and at least one sensor, wherein the at least one sensor is configured to collect data and transmit the collected data to the controller system.

[0008] Thus, in the system with this structure, sensor data can be replicated to the intelligent driving area control unit and the human-machine interaction area control unit module, and there is no need to be transferred by the intelligent driving area control unit in the center mode or surround view mode. Therefore, the surround view serialization / deserialization chip and cable are not required, the external cable connection is simplified, the occupation of data stream transfer resources is reduced, and the power consumption of the vehicle is further reduced.

[0009] In a possible implementation of the first aspect, the sensor interface unit is respectively connected to the intelligent driving area control unit and the human-machine interaction area control unit through two sets of interfaces, and the signals transmitted by each set of interfaces include a synchronization signal, a control signal, and a video data stream signal.

[0010] The synchronization signal is used as the trigger signal of the sensor or the frame rate control signal of the sensor for video capture, and the control signal is used for the configuration or data reading of the sensor.

[0011] In this way, since the two sets of signals are separately connected, the sensor data can be transmitted to the two sets of interfaces in a replication manner, the synchronization signal and the control signal can be received separately, and the sensor can be configured and controlled. This connection form is simplified. In some implementations, the sensor may be a camera.

[0012] In a possible implementation of the first aspect, the sensor interface unit selects either the intelligent driving area control unit or the human-machine interaction area control unit connected to the sensor interface unit to transmit the synchronization signal and / or the control signal.

[0013] According to the above description, through the sensor interface unit, for example, through the camera interface unit, one of the two sets of signals, the synchronization signal and the control signal, can be selected and transmitted to the sensor, for example, to the camera. Therefore, the sensor receives either the synchronization signal or the control signal.

[0014] In a possible implementation of the first aspect, the selection is made based on at least one of the following decision materials, namely, the priorities and safety levels of the intelligent driving area control unit and the human-machine interaction area control unit, the operating mode of the controller system, and the operating state of the controller system.

[0015] According to the above description, for example, the functional safety level required by the intelligent driving area control unit to control the sensor is ASIL B (that is, the safety level is B), and the control requirement of the human-machine interaction area control unit for the sensor is QM (QM represents the quality management level and there is no requirement for the safety level). Therefore, priorities can be set here, or selections can be made based on their safety levels. By setting the above priorities, after the two sets of signals, the synchronization signal and the control signal, are received separately, in order to meet the safety requirements, the controller can be taken over by the intelligent driving area control unit with a higher priority. The operation model and the operating state can also be flexibly set based on various requirements.

[0016] In a possible implementation of the first aspect, the synchronization signal transmitted between the camera interface unit and the intelligent driving area control unit is time-synchronized with the synchronization signal transmitted between the camera interface unit and the human-machine interaction area control unit.

[0017] In this way, since the time synchronization of the two sets of synchronization signals is set, when the control right of the sensor interface unit is switched, in order to avoid the jitter of the captured image, it is possible to smoothly switch the trigger timing of the image capture exposure of the sensor and the frame rate.

[0018] In a possible implementation of the first aspect, the controller system a first display interface unit connected to the intelligent driving area control unit, wherein the first display interface unit is connected to the instrument display unit, or a second display interface unit connected to the human-machine interaction area control unit, wherein the second display interface unit is connected to the display screen, and further includes at least one of the second display interface units.

[0019] In this way, since the first display interface unit is used, the intelligent driving area control unit can transmit some vehicle information such as driving information (vehicle speed, rotational speed, and total mileage) and vehicle state information (water temperature, fuel level, electricity level, temperature, etc.) and other contents that need to be displayed using the instrument to the instrument for display. Since the second display interface unit is used, the human-machine interaction area control unit can transmit, for display, any content that needs to be displayed using a display screen inside the vehicle, such as a central control display screen and a rear display screen, including an image outside the vehicle, a navigation image, and a human-machine interaction user interface, to the display screen.

[0020] In a possible implementation of the first aspect, the controller system further includes a network switching unit connected to the intelligent driving area control unit and the human-machine interaction area control unit.

[0021] In this way, since the gateway switching function is integrated into the controller system to implement internal data exchange, an external in-vehicle Ethernet gateway and connection cables are not required, the external cable connection is simplified, and the occupation of data stream transfer resources is reduced.

[0022] In a possible implementation of the first aspect, the controller system further includes a third area control unit connected to a network switching unit.

[0023] The third area control unit is connected to any one of a chassis system control unit, a power system control unit, or a vehicle body system control unit.

[0024] In this way, the third area control unit can implement the control of the chassis area, the power area, and the vehicle body area. Since a plurality of areas are combined, the external cable connection is simplified, and the occupation of data stream transfer resources is reduced.

[0025] In a possible implementation of the first aspect, the network switching unit further is configured to be connected to at least any one of a lidar sensor, a millimeter-wave radar sensor, an event data recorder, a vehicle Internet communication box, and an in-vehicle recorder.

[0026] In a possible implementation of the first aspect, the intelligent driving area control unit is configured to implement an assisted driving or autonomous driving function, a part of the vehicle control function, or a part of the vehicle body control function, the third area control unit is configured to implement a chassis system control function, a power system control function, other functions of the vehicle control, or other functions of the vehicle body control, and the human-machine interaction area control unit is configured to implement an entertainment area application function or a human-machine user interface function.

[0027] In this way, some applications of the vehicle control (VCU) and body control (BCM) function software are flexibly deployed based on the computing capabilities of the intelligent driving area control unit and the third area control unit, and the capabilities of the area control unit with higher computing capabilities can be fully utilized.

[0028] In a possible implementation of the first aspect, the intelligent driving area control unit further is configured to connect to at least any one of a millimeter-wave radar, an ultrasonic radar, and an integrated positioning unit including, for example, BeiDou satellites, GPS, GLONASS, and another positioning unit.

[0029] In a possible implementation of the first aspect, the human-machine interaction area control unit is further configured to connect to an audio device.

[0030] In a possible implementation of the first aspect, the intelligent driving area control unit or the human-machine interaction area control unit includes an image processing module, a graphics rendering module, a network / video interface module, an artificial intelligence (AI) computing module, and a control module.

[0031] The control module is configured to perform scheduling and general computing of other modules.

[0032] According to the above structure, for the intelligent driving area control unit, the function of directly outputting instrument display by the intelligent driving area control unit is implemented, the existing dedicated instrument SOC chip is replaced, and the entire data path is simplified.

[0033] The second aspect of this application provides a vehicle control system including any one implementation of a controller system.

[0034] Another aspect of the present application provides a vehicle. The vehicle includes any one of the implementation controller systems and at least one sensor connected to the vehicle controller system.

[0035] In a possible implementation, the sensor includes any one or more of the following sensors, namely, sensors for collecting image information such as cameras, infrared cameras, or three-color depth (RGB-D) cameras, millimeter-wave radars for collecting the distance, speed, and direction of an object, lidars for collecting point cloud information, millimeter-wave radars, ultrasonic radars, integrated positioning units, steering wheel pressure sensors, inertial sensors, and acceleration sensors. Optionally, the integrated positioning unit may include any one of a BeiDou positioning unit, a GPS positioning unit, and a GLONASS positioning unit.

[0036] In a possible implementation, the vehicle controller system may further be connected to at least any one of the following devices, namely, an event data recorder, a vehicle Internet communication box (TBOX), an in-vehicle recorder, a display screen, a power amplifier, a speaker, etc. Optionally, the display screen may include a liquid crystal display screen and / or a virtual display screen. The virtual display screen includes a virtual head-up display.

[0037] In a possible implementation, when at least any one of the aforementioned sensors or the aforementioned devices is connected to the vehicle control system of the present application, the sensor or the aforementioned device may be connected to a corresponding interface, such as the aforementioned sensor interface unit such as a camera interface unit, a CAN interface unit, a network switching unit, or a display interface unit, in order to communicate with the corresponding area control unit of the present application. Optionally, if it is possible to support the direct connection of the area control unit, the area control unit may be directly connected to the aforementioned sensor or device.

[0038] The third aspect of this application provides a data processing method. The method includes: receiving sensor data through a sensor interface unit; and transmitting the sensor data to an intelligent driving area control unit and a human-machine interaction area control unit.

[0039] In a possible implementation of the third aspect, the method further includes: selecting either the connection between the sensor interface unit and the intelligent driving area control unit or the connection between the sensor interface unit and the human-machine interaction area control unit to transmit one or more of the synchronization signal and the control signal to the sensor.

[0040] The synchronization signal is used as a trigger signal for the sensor or as a frame rate control signal for the sensor for video capture, and the control signal is used for the configuration or data reading of the sensor.

[0041] In a possible implementation of the third aspect, the selection is made based on at least one of the following criteria: the priorities and safety levels of the intelligent driving area control unit and the human-machine interaction area control unit, the operating mode of the controller system, and the operating state of the controller system.

[0042] In a possible implementation of the third aspect, the method further includes enabling the synchronization signal transmitted by the intelligent driving area control unit to be time-synchronized with the synchronization signal transmitted by the human-machine interaction area control unit.

[0043] In a possible implementation of the third aspect, the method includes: generating content to be displayed using an instrument display unit by using the intelligent driving area control unit; and Using the first display interface unit, further including the step of transmitting the generated content to be displayed on the instrument display unit to the instrument display unit for display.

[0044] In a possible implementation of the third aspect, the method Using the human-machine interaction area control unit, including the step of generating the content to be displayed on the display screen, Using the second display interface unit, further including the step of transmitting the generated content to be displayed on the display screen to the display screen for display.

[0045] The fourth aspect of this application provides a control method applied to any one of the implementation controller systems, and the method Including the step of receiving the sensor data through the sensor interface unit, Using the human-machine interaction area control unit, including the step of performing human proximity detection or intrusion detection based on the sensor data, When an exception is detected, including the step of using the network switching unit to transmit the sensor data to the in-vehicle recorder.

[0046] Therefore, when the sentry mode is implemented, there is almost no data stream and hardly any hardware resources are occupied, so the power consumption is very low.

[0047] In a possible implementation of the fourth aspect, when an exception is detected, the method Using the human-machine interaction area control unit, including the step of generating alert data, Including the step of transmitting the alert data to the audio device for playback, or using the second display interface unit to transmit the alert data to the display screen for display.

[0048] The fifth aspect of the present application provides a control method applicable to any one of the implementation controller systems, and the method includes: receiving sensor data through a sensor interface unit; performing image processing based on the sensor data using a human-machine interaction area control unit to generate surround view image data; transmitting the surround view image data to a display screen for display using a second display interface unit.

[0049] Therefore, when the surround view mode is implemented, there is almost no data stream and hardly any hardware resources are occupied, so the power consumption is very low.

[0050] The sixth aspect of the present application provides a system startup method applicable to any one of the implementation controller systems. The method includes: separately initializing by an intelligent driving area control unit and a human-machine interaction area control unit, where the speed at which the intelligent driving area control unit performs initialization is lower than the speed at which the human-machine interaction area control unit performs initialization; after the initialization performed by the human-machine interaction area control unit is completed, receiving, through a sensor interface unit, a synchronization signal or a control signal of the human-machine interaction area control unit to enable the surround view mode, and transmitting the synchronization signal or the control signal to the sensor; after the initialization performed by the intelligent driving area control unit is completed, receiving, through a sensor interface unit, a synchronization signal or a control signal of the intelligent driving area control unit to take over control of the sensor interface unit or the sensor, and transmitting the synchronization signal or the control signal to the sensor.

[0051] Thus, when starting the vehicle's power supply, first, the surround view mode is quickly enabled. Then, if the vehicle is normal after the power supply is turned on, a highly secure intelligent driving area control unit takes over the control of the sensor interface unit and the sensors. To ensure high safety, the switching based on the above-mentioned priority control and takeover is implemented.

[0052] These aspects of the present application and other aspects will become clearer and easier to understand in the following description of the (multiple) embodiments.

[0053] Hereinafter, with reference to the accompanying drawings, the features of the present application and the relationships between the features will be further described. All of the accompanying drawings are examples, and some features are not shown at actual ratios. In addition, in some of the accompanying drawings, general features that are not essential for the present application in the field of the present application may be omitted, or additional features that are not essential for the present application may be shown. The combinations of features shown in the accompanying drawings are not intended to limit the present application. In addition, in this specification, the content referred to by the same reference numerals is also the same. The specific accompanying drawings will be described below.

Brief Description of the Drawings

[0054]

Figure 1A

Figure 1B

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 7A

Figure 7B

Figure 8

Figure 9

Figure 10A

Figure 10B

Figure 11

Figure 12

Figure 13A

Figure 13B

Figure 13C

Mode for Carrying Out the Invention

[0055] In this specification and the claims, terms such as "first, second, third, etc." and similar terms like Module A, Module B, and Module C are only used to distinguish similar objects and do not represent a specific order of the objects. If a specific order or sequence is permitted, it may be exchanged. Thus, it should be understood that the embodiments of the present application described in this document may be implemented in an order other than the order illustrated or described in this document.

[0056] In the following description, related reference signs such as S110, S120 indicating steps do not necessarily indicate that the steps should be performed based on that order. If permitted, consecutive steps may be interchanged or performed simultaneously.

[0057] The term "comprising" used in this specification and the claims should not be construed as being limited to the content listed below and does not exclude other elements or steps. It should be construed as specifying the presence of the recited features, wholes, steps, or parts, but does not exclude the presence or addition of one or more other features, wholes, steps, or parts, and combinations thereof. Thus, the expression "a device comprising device A and device B" should not be limited to a device that includes only components A and B.

[0058] As used herein, the phrase "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present application. Thus, the phrases "in one embodiment" or "in an embodiment" that appear in this specification do not necessarily refer to the same embodiment, but may do so. Further, as will be apparent to those skilled in the art from this disclosure, in one or more embodiments, the particular features, structures, or characteristics may be combined in any suitable manner.

[0059] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. In case of conflict, the meaning as explained in this specification, or the meaning obtained according to the content recorded in this specification, shall be used. In addition, the terms used in this specification are only for the purpose of explaining the embodiments of this application and are not intended to limit this application. In order to accurately explain the technical content of this application and accurately understand this application, the following explanations, descriptions, or definitions of the terms used in this specification are provided before describing specific implementations.

[0060] (1) A domain control unit, abbreviated as DCU (Domain Control Unit) or DCM (Domain Control Module). The "domain" means dividing the electronic system of an automobile into several functional blocks based on functions. Each functional block may be called a domain, such as a vehicle domain, a body domain, an intelligent driving domain, and a human-machine interaction domain. The domain control unit can reduce the number of ECUs in the vehicle and relieve the complexity of the system. Each domain has a domain controller, which may also be called a domain control unit, and includes one or more processors responsible for processing functions and transfer functions within the domain. Components or modules within the domain are usually connected by low-speed communication, and the domain control unit is connected by high-speed communication. Note that the domain division method is not the only one in this document. For example, the vehicle domain may also be divided into a power domain and a chassis domain by the manufacturer. In another example, the vehicle domain is divided into a vehicle control domain, an intelligent driving domain, an intelligent cockpit domain, etc. The general settings of the domain control unit are as follows.

[0061] The intelligent driving area control unit may also be referred to as an advanced driver assistance systems / autonomous driving (ADAS / AD) area control unit or an ADAS / AD area controller, and may be configured to aggregate data from various sensors in order to perform detection and decision-making to implement assisted driving or autonomous driving. For example, it aggregates sensor data from cameras, ultrasonic radars, millimeter-wave radars, lidars, GPS / inertial measurement units (IMUs), and maps, and realizes assisted driving or autonomous driving through detection and decision-making. The intelligent driving area control unit has high requirements for computing power and data processing capabilities and needs to meet a high automotive safety integration level (ASIL) such as ASIL B, ASIL C, or ASIL D.

[0062] The human-machine interaction area control unit may also be referred to as an in-vehicle infotainment / human-machine interaction (IVI / HMI) area control unit, and is configured to provide functions such as in-vehicle information and in-vehicle entertainment, and can provide users with geographical information, multimedia entertainment, intelligent transportation services, etc., and can implement interactions related to entertainment information, content display, audio playback, etc. through the interaction between humans and machines. This includes collecting user information using sensors and displaying information to the user using a display or by sound, for example, collecting the driver's face information, fingerprint information, voice information, steering wheel pressure information, and pedal pressure information, etc. The human-machine interaction area control unit may also be referred to as a cockpit area control unit or a Cockpit Domain Controller (CDC).

[0063] The vehicle control unit (VCU) is an assembly controller for the power system of electric vehicles (hybrid and pure electric vehicles). The vehicle control unit controls vehicle chassis systems, such as the braking system, parking system, vehicle driving stability system, and steering system, and is used to control vehicle power systems, such as the power supply system, charging system, motor (in the case of electric vehicles), and engine system (in the case of fuel vehicles). This is sometimes called the Vehicle Domain Controller (VDC).

[0064] The body control module (BCM) is used to control body systems, such as the doors, windows, and seats of the vehicle body.

[0065] Note that the above division and naming of the domain control units are merely examples and are not limited thereto. With the evolution and development of technology, the names of the domain control units may also vary.

[0066] (2) Surround view display refers to capturing images around the vehicle using multiple vehicle cameras, stitching the images together, and displaying them using the vehicle display screen.

[0067] (3) Sentry Mode, a vehicle monitoring mode used when the vehicle is parked. Cameras are used to monitor the vehicle environment. When a person outside the vehicle approaches or intrudes, video data is recorded and an alarm is generated.

[0068] (4) The serializer / deserializer (Ser / Des) is a high-speed communication interface circuit. The serializer is configured to convert a low-speed parallel signal into a high-speed serial signal for transmission, and the deserializer is configured to convert a high-speed serial signal into a low-speed parallel signal for transmission. The serializer and the deserializer may be separately arranged or may be installed integrally. The serializer is installed on the transmission side, and the deserializer is installed on the reception side. The serializer / deserializer can achieve high-speed transmission of multi-bit video data and may be called a display SERDES interface when applied to video transmission using display components, or may be called a camera SERDES interface when applied to data transmission using a camera sensor.

[0069] (5) The Display Serial Interface (DSI) and the Camera Serial Interface (CSI), which are respectively called the display interface and the camera interface, are standard interfaces of the Mobile Industry Processor Interface (MIPI) and are respectively used to connect to a display and a camera.

[0070] Each component or function of a conventional vehicle requires one or more electronic control units (ECUs) for control. The current main solution for intelligent vehicles is an architecture that includes multiple independent domain controllers. For example, refer to FIG. 13A for the vehicle's electronic and electrical architecture. The electronic and electrical architecture is usually divided into a vehicle body control domain, a human-machine interaction domain, an intelligent driving domain, and a vehicle control domain. Each domain controller communicates with the vehicle Internet communication box (TBOX) through a controller area network (CAN) bus / Ethernet cable via a central gateway. Hereinafter, the data stream transfer process of the architecture will be described using an example.

[0071] Example 1: As shown in FIG. 13B, the data stream in the surround view / center mode is as follows.

[0072] Surround view display: First, the camera sends video data to the ADAS / AD domain controller. The internal image signal processing (ISP) module of the ADAS / AD domain controller processes the original data into data in the RGB / YUV (RGB and YUV are two color encoding methods) data format and transfers the data to the human-machine interaction domain controller. The human-machine interaction domain controller transfers the video data to the central control screen for display.

[0073] Center mode: First, the camera sends video data to the ADAS / AD area controller. The internal ISP of the ADAS / AD area controller processes the original data into data in RGB / YUV data format and transfers the data to the human-machine interaction area controller. The human-machine interaction area controller performs human proximity / intrusion detection. If an unauthorized intrusion is detected, the video data is transferred to the in-vehicle digital video recorder (DVR) for storage, and an alarm is generated through the central control screen or the audio amplifier.

[0074] Example 2: As shown in Figure 13C, the data stream of the instrument display data is as follows. The VCU collects information regarding chassis control and the power ECU and transfers the information necessary for display, such as the vehicle speed, to the instrument processor. The instrument processor performs image rendering using the image processing module (GPU) and then sends the image to the instrument screen for display. In the aforementioned architecture, multiple independent area controllers are used. Therefore, the vehicle's equipment occupies a large space and additional video / Ethernet cables are required. In addition, in the aforementioned example, since the transfer of the data stream is performed multiple times in this architecture, a large number of hardware resources are occupied and the power consumption increases.

[0075] It should be understood that the aforementioned architecture is only an exemplary solution and the vehicle architecture is also constantly evolving. In addition to some area control units, the vehicle may further include some ECUs that are independent of the area control units and are used for specific components or functions. These in-vehicle devices have computing power and processing power and may also be called in-vehicle computing devices or computing platforms.

[0076] Furthermore, as the number and types of sensors arranged in the vehicle increase, problems arise in the connection and signal transmission between the sensors in the vehicle and the controller or control unit.

[0077] This application provides another controller system and an application based on the controller system to reduce the occupation of data stream transfer resources, reduce the power consumption of vehicles in various in-vehicle application scenarios, improve the function startup speed of vehicles, and improve the user experience. First, the intelligent driving area controller, the human-machine interaction area controller, vehicle control, and vehicle body control are thoroughly integrated to simplify external cable connection. Second, a flexible connection method between sensors and in-vehicle computing devices is also provided to improve the sharing of sensor data.

[0078] The controller system provided in this application may be applied to vehicles such as intelligent vehicles or ships equipped with intelligent driving functions, or may also be applied to application scenarios such as robots.

[0079] Hereinafter, this application will be described in detail with reference to the accompanying drawings.

[0080] One embodiment of the present application provides a controller system. The controller system includes one or more in-vehicle control units and a sensor interface unit. The in-vehicle control unit may be a region control unit or an ECU. These in-vehicle control units may be deployed on the same hardware. For example, multiple SoCs (system on chip) are deployed on one hardware platform, and each SoC corresponds to one in-vehicle control unit. The in-vehicle control units may also be separately deployed on independent hardware. This is not limited to the present embodiment of the present application. For ease of explanation, hereinafter, an example where the in-vehicle control unit is an intelligent driving region control unit and an example where the in-vehicle control unit is a human-machine interaction region control unit will be used for the explanation. It should be understood that the embodiments of the present application are not limited thereto. In actual applications, the in-vehicle control unit may be another region control unit or an electronic control unit. As shown in FIG. 1A, an intelligent driving region control unit, a human-machine interaction region control unit, and a sensor interface unit are included. The sensor interface unit is connected to the intelligent driving region control unit and the human-machine interaction region control unit. The sensor interface unit further has an external interface for connecting to an external sensor. The sensor interface unit may be configured to transmit sensor data to the intelligent driving region control unit and the human-machine interaction region control unit. Note that the names of the region control units in the present application are only examples. The intelligent driving region control unit, the human-machine interaction region control unit, etc. are used as examples for explanation and are not limited thereto. With the evolution and development of technology, it should be understood that units or modules configured to implement corresponding functions may be named with other names.

[0081] The intelligent driving area control unit can perform detection and decision-making by using the received data of one or more sensors to implement assisted driving or autonomous driving. As the first area control unit of this application, the intelligent driving area control unit may include an electronic control unit (ECU), a micro control unit (MCU), a central processing unit (CPU), a graphics processing unit (GPU), or another control unit, and may be implemented by a single control unit or multiple control units. FIG. 6A shows one of the embodiments of the implementation described below.

[0082] One or more sensors are configured to collect external vehicle information and may include any one or more of the following sensors, namely, sensors for collecting image information such as a camera, an infrared camera, or a three-color depth (RGB-Depth, RGB-D) camera, a millimeter-wave radar for collecting the distance, speed, and direction of an object, a lidar for collecting point cloud information, or other sensors for collecting data. In this embodiment, an example of using the camera shown in FIG. 1B as a sensor is used in the following description. The corresponding sensor interface unit is a camera detection unit. The data collected may be image data outside the vehicle, for example, image data in front of the vehicle obtained during the driving process. The image data is used to detect vehicles, pedestrians, etc. in the image. The detection results are used for decision-making during assisted driving and autonomous driving.

[0083] The aforementioned human-machine interaction area control unit can perform face recognition, motion recognition, etc. using the received data of an external sensor, such as a camera, and provide the functions of entertainment services. As the second area control unit of this application, the human-machine interaction area control unit may include an ECU, an MCU, a CPU, a GPU, or another control unit, and may be implemented by a single control unit or a plurality of control units. For example, it may include the control unit of an in-vehicle infotainment system and the control unit of a human-machine user interface. FIG. 6B shows one of the embodiments to be described later.

[0084] When the human-machine interaction area control unit is applied to the sentry mode, the data of the external sensor, such as the data of the camera, can be a still face image outside the vehicle or a short-distance human body image. When the vehicle is in a stationary state, the image data obtained by the sensor is transmitted to the human-machine interaction area control unit for face detection or intrusion detection based on motion recognition, and the human-machine interaction area control unit can further control the transfer of the data of the relevant camera to the in-vehicle recorder for recording.

[0085] In some embodiments, when the sensor interface unit is a camera interface unit, the sensor interface unit may specifically be a camera serialization / deserialization interface for implementing high-speed serial communication with an external camera.

[0086] In some embodiments, the way the sensor interface unit is connected to the intelligent driving area control unit and the human-machine interaction area control unit includes that the sensor interface unit is respectively connected to the intelligent driving area control unit and the human-machine interaction area control unit through two sets of interfaces, and the signals transmitted between the sensor interface unit and the two area control units may include a synchronization signal, a control signal, and a video data stream signal. For ease of explanation, the interfaces may be referred to as a synchronization signal interface, a control signal interface, and a video data stream signal interface based on the types of signals transmitted. It will be understood that two or more of these interfaces may be integrated, and two or more of the synchronization signal, the control signal, and the video data stream signal may be combined for transmission. For example, the synchronization signal and the control signal are transmitted in one connection, and the video data stream signal is transmitted in one connection. In another example, the synchronization signal, the control signal, and the video data stream signal are transmitted in one connection. The interfaces can also be used as independent interfaces for transmitting the synchronization signal, the control signal, and the video data stream signal in three connections.

[0087] In some embodiments, the control signal may be transmitted using an I2C signal, and the video data stream signal may be transmitted using a CSI-2 signal.

[0088] In some embodiments, the video data stream signal may also be transmitted between the sensor interface unit and the two control units by using low voltage differential signaling (LVDS).

[0089] In some embodiments, corresponding to the two sets of interfaces, the sensor interface unit specifically is Receive the synchronization signals of the intelligent driving area control unit or the human-machine interaction area control unit using two sets of synchronization signal interfaces, send the synchronization signals to the sensor, and the synchronization signals are used as the exposure trigger signal or the frame rate control signal of the image captured by the sensor. Receive the control signals of the intelligent driving area control unit or the human-machine interaction area control unit using two sets of control signal interfaces (for example, I2C signal interfaces), send the control signals to the sensor, and the control signals are used for the configuration of the sensor (for example, a camera) or the read data, for example, for performing high dynamic range rendering (HDR) mode configuration or exposure parameter configuration, and Receive sensor data, the sensor interface unit duplicates the data, and then is configured to send the data to the intelligent driving area control unit and the human-machine interaction area control unit respectively using two sets of video data stream signal interfaces (such as CSI-2 signal interfaces and LVDS signal interfaces).

[0090] Each set of interfaces may include one or more physical transmission channels. A physical transmission channel in this document may be one or more signal lines or cables. For example, when CSI-2 signals are transmitted, each physical transmission channel may include a plurality of signal lines, for example, configured to transmit a pair of clock synchronization signals, a pair of data signals, two pairs of data signals, or four pairs of data signals. As another example, when LVDS signals are transmitted, each physical transmission channel may include a pair of cables used to provide LVDS signals. When each set of interfaces includes a plurality of physical transmission channels, video data can be received and transmitted separately using separate transmission channels to increase data transmission efficiency or to provide differentiated data transmission. For example, video data may be allocated to the corresponding transmission channel for transmission based on the load of various transmission channels. In another example, video data may also be allocated to the corresponding transmission channel for transmission based on various priorities, delay requirements, etc.

[0091] In a possible implementation, the sensor interface unit can transmit the video data stream signal separately to the intelligent driving area control unit and the human-machine interaction area control unit, but can transmit the synchronization signal or the control signal, or both the synchronization signal and the control signal, to only one of the area control units. In this way, the selection of the two control units is not required.

[0092] In some embodiments, the sensor interface unit selects either the intelligent driving area control unit or the human-machine interaction area control unit connected to the sensor interface unit for transmitting the synchronization signal or the control signal for configuration.

[0093] When the sensor interface unit is connected to multiple area control units, a specific area control unit can be selected for transmitting a synchronization signal or a control signal based on one or more decision-making factors such as the priority, safety level, system operation mode, and operating state of the area controller unit. In a possible implementation, an area controller unit with a higher priority can be selected for transmitting the synchronization signal and the control signal. For example, in this embodiment, the sensor interface unit is connected to an intelligent driving area control unit and a human-machine interaction area control unit, and the priority of the intelligent driving area control unit is higher than that of the human-machine interaction area control unit. Optionally, the method of selecting any one of the area control units for transmission includes that after receiving the synchronization signal or the control signal of the intelligent driving area control unit, the sensor interface unit selects to transmit the synchronization signal or the control signal of the intelligent driving area control unit.

[0094] In yet another possible implementation, an area control unit with a high safety level can be selected to receive or transmit the synchronization signal and the control signal. Continuing to use this embodiment as an example. The functional safety level required by the intelligent driving area control unit for controlling the sensor is ASIL B (i.e., the safety level is B), and the control requirement of the human-machine interaction area control unit for the sensor is QM (QM represents the quality management level and there is no requirement for the safety level). Therefore, the safety level requirement of the intelligent driving area control unit is higher than that of the human-machine interaction area control unit. The synchronization signal and the control signal of the intelligent driving area control unit are transmitted to an external sensor connected to the sensor interface unit. In other words, the sensor is taken over by the intelligent driving area control unit with a higher priority.

[0095] In yet another possible implementation, based on the operating state of the area control unit, for example, the activation and initialization of the human-machine interaction area control unit are faster than those of the intelligent driving area control unit. After the human-machine interaction area control unit is activated, the intelligent driving area control unit is not yet operating properly. In this case, after the initialization performed by the human-machine interaction area control unit is completed, the synchronization signal and control signal of the human-machine interaction area control unit can be sent to the sensor. After the initialization of the intelligent driving area control unit is completed, based on the priority, safety level, and other decision-making materials, it can also be determined whether the intelligent driving area control unit should take over the control of the sensor, and whether the synchronization signal and control signal of the intelligent driving area control unit should be sent to the sensor.

[0096] In yet another possible implementation, the system is in the standby mode. In this mode, the power of the human-machine interaction area control unit is turned on, but the power of the intelligent driving area control unit is not turned on. Therefore, the sensor interface unit receives only the synchronization signal and control signal of the human-machine interaction area control unit, and sends the synchronization signal and control signal to the external sensor.

[0097] In yet another possible implementation, the system is in the surround view mode, in which both the human-machine interaction area control unit and the intelligent driving area control unit are powered on. Therefore, based on the aforementioned priority or safety level, the sensor interface unit selects to send the synchronization signal and control signal of the intelligent driving area control unit to the external sensor.

[0098] In some embodiments, the synchronization signal transmitted between the sensor interface unit and the intelligent driving area control unit is time-synchronized with the synchronization signal transmitted between the sensor interface unit and the human-machine interaction area control unit.

[0099] In this way, since the time synchronization of the two sets of synchronization signals is set, when the control right of the sensor interface unit is switched, it is possible to smoothly switch the trigger timing of the image capture exposure of the sensor and the frame rate. Since the intelligent driving area control unit and the human-machine interaction area control unit can perform precise time synchronization based on Time-Sensitive Networking (TNS), the transmitted synchronization signals are synchronized with each other in time alignment.

[0100] In some embodiments, as shown in FIG. 2, the controller system includes a first display interface unit connected to the intelligent driving area control unit, where the first display interface unit is connected to the instrument display unit, or a second display interface unit connected to the human-machine interaction area control unit, where the second display interface unit is connected to the display screen. In addition, note that in the example shown in FIG. 2, a camera interface unit is used as an example of the sensor interface unit. Correspondingly, an example where a camera is used as a sensor is used for illustration. In addition, FIG. 2 further shows sensors such as millimeter-wave radar or ultrasonic radar, and the sensors can be connected to the intelligent driving area control unit through a CAN interface (for simplicity, the CAN interface unit included as the sensor interface is not depicted). In the example shown in FIG. 2, the video signal of the camera interface unit may be a CSI-2 signal provided by one or more physical transmission channels, or may be an LVDS signal provided by one or more physical transmission channels.

[0101] In this way, since the first display interface unit is used, the intelligent driving area control unit can transmit several vehicle information such as driving information (vehicle speed, rotational speed, and total mileage) and vehicle state information (water temperature, fuel quantity, electricity quantity, temperature, etc.) and other content that needs to be displayed using the instrument display screen to the instrument display screen for display.

[0102] Since the second display interface unit is used, the human-machine interaction area control unit can transmit some content that needs to be displayed using the display screens in the vehicle, such as the central control display screen and the rear display screen, including images outside the vehicle, navigation images, and human-machine interaction user interfaces, to the display screen for display.

[0103] In some embodiments, the display screen may be a liquid crystal display screen, or a virtual display screen, such as a virtual head-up display (Augmented Reality-Head Up Display, AR-HUD).

[0104] In some embodiments, as shown in FIG. 2 or FIG. 3, the controller system further includes a network switching unit connected to the intelligent driving area control unit and the human-machine interaction area control unit.

[0105] In this way, since the gateway switching function is integrated into the controller system to implement internal data exchange, an external in-vehicle Ethernet gateway and a connection cable are not required. The implementation of the network interface includes, but is not limited to, internal network interfaces such as Reduced Gigabit Media Independent Interface (RGMII), Reduced Media Independent Interface (RMII), Serial Gigabit Media Independent Interface (SGMII), 10 Gigabit Media Independent Interface (XGMII), and 10 G_base_R.

[0106] In some embodiments, as shown in FIG. 2 or FIG. 3, the controller system further includes a third area control unit connected to the network switching unit, and the third area control unit is connected to any one of the chassis system control unit, the power system control unit, or the vehicle body system control unit.

[0107] In some embodiments, the third area control unit may be implemented by an MCU. As shown in FIG. 2 or FIG. 4, the third area control unit can collect information such as the chassis system control unit, the power system control unit, or the vehicle body system control unit, or can control these units. The collected information can be transferred to the intelligent driving area control unit through the network switching unit. After performing internal processing (such as image processing, image rendering, etc.), the intelligent driving area control unit transmits the information to the instrument display unit for display through the first display interface unit.

[0108] The chassis system control unit manages an electric power steering system (EPS), an electronic stability program (ESP), an electrical park brake (EPB), an intelligent brake system (IBS), an electronic stability control (ESC), etc. The power system control unit manages an electric motor system (for electric vehicles), a battery management system (for electric vehicles), a power conversion system (DC-DC, i.e., direct current - direct current conversion) (for electric vehicles), an on-board charger (OBC) (for electric vehicles), an engine system (for fuel vehicles), etc. The vehicle body system control unit manages the doors, windows, seats, etc. of the vehicle and may further include other control units such as an airbag system and a thermal management system.

[0109] In some embodiments, as shown in FIG. 2, the network switching unit is further configured to connect to at least any one of a lidar sensor, a millimeter-wave radar sensor, an event data recorder, a vehicle internet communication box (TBOX), and an in-vehicle recorder.

[0110] The aforementioned devices may be connected through an Ethernet interface to implement the aforementioned collected data transferred to the controller system and may be provided for the corresponding area control unit or for communication with the area control unit.

[0111] In some embodiments, as shown in FIG. 2, the intelligent driving area control unit is further configured to connect to at least any one of a millimeter-wave radar, an ultrasonic radar, and an integrated positioning unit (including, for example, BeiDou, GPS, GLONASS, etc.).

[0112] The aforementioned device may be connected through a CAN interface, whereby the intelligent driving area control unit obtains sensor information.

[0113] Since the intelligent driving area control unit, the human-machine interaction area control unit, and the third area control unit are connected to separate devices, the necessary functional software corresponding to the three area control units can be flexibly deployed. As shown in FIG. 5, one deployment method may be as follows.

[0114] In addition to the intelligent driving area functional software, some functional software for vehicle control or some functional software for vehicle body control is integrally deployed in the intelligent driving area control unit. The underlying OS of the intelligent driving area control unit may be closed to ensure safety.

[0115] In addition to the chassis / power function software, some functional software for vehicle control or some functional software for vehicle body control is integrally deployed in the third area control unit. The underlying OS of the third area control unit may be closed to ensure safety.

[0116] The entertainment area application software and the human-machine user interface application software are deployed in the human-machine interaction area control unit. To facilitate the use of various entertainment application software, the underlying OS of the human-machine interaction area control unit may be open.

[0117] Some functional software for vehicle control or some applications for vehicle body control may be deployed based on the computing capabilities of the intelligent driving area control unit and the third area control unit. For example, when the computing capability of the third area control unit is lower than that of the intelligent driving area control unit, some functions of vehicle control may be deployed in the intelligent driving area control unit. When it is necessary to use these functions, the third area control unit exchanges relevant data through the network switching unit, and the execution of the functions is carried out by the intelligent driving area control unit. However, the direct information exchange with the chassis ECU and the power ECU related to vehicle control (such as control signals or collected sensor signals) is carried out by the third area control unit.

[0118] In some embodiments, as shown in FIG. 2, the human-machine interaction area control unit is further configured to connect to an audio device.

[0119] The audio device may include a power amplifier, a speaker, etc., and may be configured to play alert information, the voice of the interaction between humans and machines, etc.

[0120] In some embodiments, the intelligent driving area control unit or the human-machine interaction area control unit includes an image processing module, a graphics rendering module, a network / video interface module, an AI computing module, and a control module. The control module is configured to schedule other modules and perform general computing. A detailed description will be provided below.

[0121] As shown in FIG. 6A, in order to achieve an overall functional safety level of ASIL B or higher, at least the following modules are integrated into the intelligent driving area control unit. An ISP module mainly responsible for image processing, a GPU module responsible for graphic rendering, where the graphic rendering is used for instrument display, a network / video interface module configured to connect to an external interface device, an ARM CPU module configured to perform overall scheduling and multi-purpose computing, and an AI module configured to accelerate the computing in intelligent driving detection computing with strong computing power.

[0122] Since the GPU function is integrated into the intelligent driving area control unit, the function of directly outputting the instrument display by the intelligent driving area control unit is implemented, the existing dedicated instrument system on chip (SOC) is replaced, and the entire data path is simplified.

[0123] As shown in FIG. 6B, in order to achieve an overall functional safety level of ASIL B or higher, at least the following modules are integrated into the human-machine interaction area control unit. An ISP module mainly responsible for image processing, a GPU module responsible for graphics rendering, where the graphics rendering is used for the interaction between humans and machines and the display of in-vehicle infotainment (IVI) services, a network / video interface module configured to connect to an external interface device, an ARM CPU module responsible for overall scheduling and multi-purpose computing, and an artificial intelligence (AI) module configured to perform AI computing related to human-machine interaction and AI computing in the center mode with weak computing power.

[0124] In this book, as shown in FIG. 7A, it should be noted that the hardware form of the controller system of this application may be a circuit board and may be implemented by stacking a plurality of boards as shown in FIG. 7B. For example, the I / O interface of the MCU may be borne on one board or divided and borne on a plurality of boards, but is not limited thereto. To transmit power signals, low-speed data signals, video signals (such as CSI-2 described above), image control signals (such as DSI described above), and network signals (XGMII, SGMII, etc.), a plurality of boards may be connected to each other using a daughter board connector, a flexible cable, or the like.

[0125] The controller system of this application may further include a plurality of independent hardware devices and a sensor interface unit. One or more control units are deployed in each hardware device, and the hardware devices are connected to each other using cables.

[0126] The sensor interface unit is configured to obtain a sensor signal from a connected sensor and transmit the sensor signal to one or more control units within the controller system. Further, the sensor interface unit is configured to obtain a synchronization signal or a control signal from one or more control units within the controller system for configuring the connected sensor. For example, the connected sensor is a camera that captures image information, and correspondingly, the sensor signal includes a video signal.

[0127] The sensor interface unit may include one or more serialization / deserialization interfaces and may further include an LVDS signal interface.

[0128] The serialization / deserialization interface is configured to transmit a video signal, for example, between a camera and a control unit or between control units.

[0129] In one possible implementation, the serialization / deserialization interface is configured separately for each control unit that needs to transmit a video signal. If a video signal is transmitted between two control units, and those two control units are deployed on separate hardware devices, and those hardware devices are connected using a coaxial cable, then the first LVDS interface and the second LVDS interface need to be configured for the two control units respectively, such that the video signal is transmitted in the form of an LVDS signal between the control units deployed on separate hardware devices. Configuring the serialization / deserialization interface may involve deploying a serializer / deserializer that implements the serializer / deserializer interface function and the corresponding control unit on the same hardware. The first LVDS interface includes one or more pairs of cables, and the second LVDS interface includes one or more pairs of cables. The video signal may be transmitted separately on one or more physical transmission channels on the two interfaces.

[0130] An example is used to illustrate a controller system including a first control unit, a second control unit, and a sensor interface unit. The sensor interface unit includes a first serialization / deserialization interface and a second serialization / deserialization interface. The first control unit includes a SOC 1 configured to process sensor signals, and the second control unit includes a SOC 2 configured to process sensor signals. The first control unit obtains a video signal through the first serialization / deserialization interface, and the second control unit obtains a video signal through the second serialization / deserialization interface.

[0131] In one possible implementation, the video signal is replicated within the sensor interface unit, and two sets of signals are respectively transmitted to a first control unit and a second control unit through a first serialization / deserialization interface.

[0132] In yet another possible implementation, the video signal may be transmitted to the first control unit through the first serialization / deserialization interface. Then, the first serialization / deserialization interface transmits the video signal in the form of an LVDS signal to the second LVDS interface through the first LVDS interface, and the second LVDS interface transmits the video signal to the second serialization / deserialization interface. After converting the data into serial data, the second serialization / deserialization interface transmits the serial data to the second control unit. Alternatively, note that the video signal may be transmitted to the second control unit through the second serialization / deserialization interface, and then the second serialization / deserialization interface transmits the video signal in the form of an LVDS signal to the first LVDS interface through the second LVDS interface, and the first LVDS interface transmits the video signal to the first serialization / deserialization interface.

[0133] In the controller system provided in the foregoing embodiments, sensors can be shared by multiple control units to reduce cost and the space for deployment.

[0134] Furthermore, when the first control unit is started faster than the second control unit, the configuration of the sensor may be performed after the first control unit is started, thereby enabling the sensor to be started in a timely manner to obtain information around the vehicle and improving the user experience.

[0135] Furthermore, when the processing capacity of the second control unit is stronger than that of the first control unit, for example, when the image processing module is deployed in the second control unit, the video signal processed by the second control unit can be further transmitted to the first control unit using the sensor interface unit, so that the first control unit can obtain the processed video signal.

[0136] The first control unit may be a human-machine interaction area control unit or another ECU, and the second control unit may be an intelligent driving area control unit or another ECU.

[0137] Another embodiment of the present application provides a vehicle control system. This vehicle control system includes any one of the implemented controller systems. Another embodiment of the present application further provides a vehicle. This vehicle includes any one of the implemented controller systems, and this vehicle further includes at least any one of the following sensors, or at least any one of the following devices.

[0138] Sensor: The sensor includes any one or more of the following sensors, that is, sensors for collecting image information such as cameras, infrared cameras, three-color depth (RGB-Depth, RGB-D) cameras, millimeter-wave radars for collecting the distance, speed, and direction of an object, lidar for collecting point cloud information, millimeter-wave radars, ultrasonic radars, integrated positioning units (having positioning units such as BeiDou, GPS, GLONASS), steering wheel pressure sensors, inertial sensors, and acceleration sensors.

[0139] Device: Event data recorder, vehicle internet communication box (TBOX), in-vehicle recorder, display screen, power amplifier, and speaker. The display screen may be a liquid crystal display screen, or may be a virtual display screen, such as a virtual head-up display.

[0140] Please refer to FIG. 2. When at least one of the aforementioned sensors or devices is connected to the vehicle control system of the present application, the sensor or device communicates with the corresponding area control unit of the present application by connecting to the corresponding interface, for example, the aforementioned sensor interface unit (such as a camera interface unit, a CAN interface unit, etc.), or a network switching unit, or a display interface unit.

[0141] Another embodiment of the present application provides a control method, which is applied to the aforementioned controller system. Here, the sentry mode applied to the vehicle is used as an example for explanation. As shown in FIG. 8, the method includes the following steps.

[0142] S110: After initialization is performed when powering on in sentry mode, receive camera data through the camera interface unit.

[0143] To achieve low power consumption in this mode, only the camera, camera interface unit, human-machine interaction area control unit, and related display module related to the sentry mode are powered on, and other irrelevant modules do not need to be powered on. In addition, in this mode, the camera and the camera interface unit receive the synchronization signal and control signal of the human-machine interaction area control unit, that is, the human-machine interaction area control unit takes over the camera and the camera interface unit.

[0144] S120: Use the human-machine interaction area control unit to perform human proximity detection or intrusion detection based on the camera data. For example, after performing image processing using the internal ISP module of the human-machine interaction area control unit, the human-machine interaction area control unit uses the internal AI module to perform human proximity detection or intrusion detection.

[0145] S130: When an exception is detected, use the network switching unit to transmit the camera data obtained by the human-machine interaction area control unit to the in-vehicle recorder.

[0146] In some embodiments, when an exception is detected, the method further includes generating alert data using the human-machine interaction area control unit, and transmitting the alert data to an audio device for playback, or transmitting the alert data to a display screen for display using a second display interface unit.

[0147] Another embodiment of the present application provides a control method, which is applied to a controller system. Here, an example where a surround view mode is applied in a vehicle is used for explanation. As shown in FIG. 9, the method includes the following steps.

[0148] S210: Receive camera data through a camera interface unit.

[0149] S220: Use the human-machine interaction area control unit to perform image processing on the camera data to generate surround view image data. For example, the human-machine interaction area control unit performs image processing using the internal ISP module of the human-machine interaction area control unit, which includes converting the image format from RAW format to RGB format, performing image rendering using the internal GPU module, performing processes such as 3D surround view splicing based on the image data of multiple cameras, and overlaying the image on the vehicle model image to generate surround view image data.

[0150] S230: Use the second display interface unit to transmit the surround view image data to a display screen for display.

[0151] Another embodiment of the present application provides a system startup method, which is applied to the aforementioned controller system. In this embodiment, an example is used for description where the sensor interface unit is a camera interface unit and the sensor is a camera. As shown in the flowchart of FIG. 10A and the schematic diagram shown in FIG. 10B, the method includes the following steps.

[0152] S310: Initialization at power-on: including initialization separately performed by the intelligent driving area control unit and the human-machine interaction area control unit. When the intelligent driving area control unit performs initialization, due to the existence of the functional safety monitoring mechanism, the initialization speed of the intelligent driving area control unit is slower than the speed at which the human-machine interaction area control unit performs initialization. Therefore, the human-machine interaction area control unit completes initialization first and enters the operating state.

[0153] After the initialization performed by the human-machine interaction area control unit is completed, in order to enable the surround view mode, receive the synchronization signal or control signal of the human-machine interaction area control unit through the camera interface unit and send the synchronization signal or control signal to the camera. For this process, refer to the control method applied to the surround view mode, and the details will not be described again.

[0154] Note that if the initialization of the intelligent driving area control unit has not been completed yet, after the intelligent driving area control unit completes initialization, the configuration and startup of another camera and camera interface unit independent of the surround view mode may be performed by the intelligent driving area control unit.

[0155] After the initialization performed by the intelligent driving area control unit is completed, receive the synchronization signal or control signal of the intelligent driving area control unit through the camera interface unit, send the synchronization signal or control signal to the camera, and assume the control of the camera interface unit or the camera based on the above-mentioned priority reasons.

[0156] Since the above-mentioned startup method can be applied to the normal startup scenario of the vehicle, in the startup when the vehicle is powered on, first, the surround view mode is quickly enabled. Then, when the vehicle is normal after being powered on, the intelligent driving area control unit with a high safety level assumes the control of the camera interface unit and the camera. To ensure high safety, the switching based on the above-mentioned priority control and assumption is implemented.

[0157] Another embodiment of the present application provides a control method. For the specific implementation of the steps, or the technical problems that can be solved, or the corresponding effects, please refer to the above-mentioned embodiments of the area control system. Here, only a brief description is provided. As shown in the schematic diagrams shown in FIG. 11 or FIG. 10B, the present control method includes the following steps.

[0158] S410: Receive the sensor data through the sensor interface unit.

[0159] In some embodiments, the sensor interface unit and the sensor may be the camera interface unit and the camera for collecting video data outside the vehicle. For other optional embodiments, please refer to the above-mentioned embodiments of the area control system, and the details will not be described again.

[0160] S420: Send the sensor data to the intelligent driving area control unit and the human-machine interaction area control unit.

[0161] The intelligent driving area control unit can make decisions for intelligent driving based on received data, such as video data or other received data. The human-machine interaction area control unit can execute functions related to the interaction between humans and machines based on received data, such as video data or other received data, perform face detection (in the sentry mode), and display using a display screen (e.g., in the surround view mode).

[0162] In some embodiments, as shown in FIG. 12, the method further includes the following steps.

[0163] S510: Use two sets of interfaces of the sensor interface unit to separately receive the synchronization signal or control signal transmitted by the intelligent driving area control unit and the human-machine interaction area control unit.

[0164] S520: The sensor interface unit selects either the intelligent driving area control unit or the human-machine interaction area control unit to transmit the synchronization signal or control signal to the sensor. The synchronization signal is used as a trigger signal for the sensor or a frame rate control signal for the sensor for video capture, and the control signal is used for the configuration or data reading of the sensor.

[0165] In some embodiments, the step of selecting either the intelligent driving area control unit or the human-machine interaction area control unit includes the step that after receiving the synchronization signal or control signal of the intelligent driving area control unit, the sensor interface unit selects to transmit the synchronization signal or control signal of the intelligent driving area control unit.

[0166] In some embodiments, the method further includes enabling a synchronization signal transmitted by an intelligent driving area control unit to be time synchronized with a synchronization signal transmitted by a human-machine interaction area control unit.

[0167] In some embodiments, the method includes using an intelligent driving area control unit to generate content to be displayed using an instrument display unit; and using a first display interface unit to transmit the generated content to be displayed using the instrument display unit to the instrument display unit for display.

[0168] In some embodiments, the method includes using a human-machine interaction area control unit to generate content to be displayed using a display screen; and using a second display interface unit to transmit the generated content to be displayed using the display screen to the display screen for display.

[0169] Those skilled in the art can realize that, in combination with the examples described in the embodiments disclosed herein, the units and algorithm steps can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether the function is executed by hardware or software depends on the specific application of the technical solution and the design constraints. Those skilled in the art can implement the functions described in various ways for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0170] For the sake of simplicity, those skilled in the art will clearly understand that for the detailed working processes of the aforementioned systems, apparatuses, and units, reference should be made to the corresponding processes of the embodiments of the aforementioned methods. Details will not be described again here.

[0171] In some embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other ways. For example, the described embodiments of the apparatus are merely examples. For example, the division into units is only a logical functional division, and in actual implementation, it may be a different division. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the disclosed couplings or direct couplings or communication connections shown or discussed may be implemented using some interfaces. The indirect coupling or communication connection between apparatuses or units may be implemented in electronic form, mechanical form, or other forms.

[0172] The units described as separate parts may or may not be physically separate, and the parts shown as units may or may not be physical units. They may be arranged in one place or distributed among multiple network units. To achieve the objectives of the solutions of the embodiments, some or all of the units may be selected based on actual requirements.

[0173] In addition, the functional units of the embodiments of this application may be integrated into one processing unit. Each of the units may physically exist alone, or two or more units may be integrated into one unit.

[0174] Note that the above are only exemplary embodiments and technical principles of the present application. Those skilled in the art will understand that the present application is not limited to the specific embodiments described in this document, and those skilled in the art can make various obvious changes, readjustments, and substitutions without departing from the protection scope of the present application. Therefore, although the present application has been described in detail using the foregoing embodiments, the present application is not limited to the foregoing embodiments, and without departing from the concept of the present application, additional other equivalent embodiments can be included, and all of them fall within the protection scope of the present application.

Claims

Claim 1 A controller system for an intelligent vehicle, comprising: an intelligent driving area control unit; a human-machine interaction area control unit; a sensor interface unit configured to be connected to the intelligent driving area control unit and the human-machine interaction area control unit through separate interfaces on the sensor interface unit, wherein an external interface of the sensor interface unit is connected to a sensor and configured to transmit data of the sensor to the intelligent driving area control unit and the human-machine interaction area control unit; wherein signals transmitted by respective interfaces of the separate interfaces include a synchronization signal, a control signal, and / or a video data stream signal; the synchronization signal is used as a trigger signal of the sensor or a frame rate control signal of the sensor for video capture, and the control signal is used for setting the sensor or reading data settings; the sensor interface unit is configured to select either the intelligent driving area control unit or the human-machine interaction area control unit connected to the sensor interface unit to transmit the synchronization signal and / or the control signal to the sensor through the external interface based on at least one of the priority, safety level of the intelligent driving area control unit and the human-machine interaction area control unit, the operating mode of the controller system, and the operating states of the intelligent driving area control unit and the human-machine interaction area control unit; when the priority and the safety level are high, the intelligent driving area control unit is selected first; when the operating mode of the controller system is in the center mode, only the human-machine interaction area control unit is selected; in the surround view mode, it is selected according to the priority and the safety level. A controller system. Claim 2 The controller system according to claim 1, wherein the synchronization signal transmitted by the intelligent driving area control unit is time-synchronized with the synchronization signal transmitted by the human-machine interaction area control unit.

3. A first display interface unit connected to the intelligent driving area control unit, wherein the first display interface unit is connected to an instrument display unit, or A second display interface unit connected to the human-machine interaction area control unit, wherein the second display interface unit is connected to a display screen, The controller system according to claim 1 or 2, further comprising at least one of the above.

4. A network switching unit connected to the intelligent driving area control unit and the human-machine interaction area control unit The controller system according to any one of claims 1 to 3, further comprising the above.

5. The controller system according to claim 4, further comprising a third area control unit connected to the network switching unit, The third area control unit is connected to any one of the following control units, namely, a chassis system control unit, a power system control unit, or a vehicle body system control unit, and the third area control unit Collects information from the connected control unit, and Is configured to transmit the collected information to the intelligent driving area control unit using the network switching unit.

6. The network switching unit further Is configured to be connected to at least one of a lidar sensor, a millimeter wave radar sensor, an event data recorder, a vehicle Internet communication box, and an in-vehicle recorder. The controller system according to claim 4 or 5.

7. The intelligent driving area control unit is configured to implement an assisted driving or autonomous driving function, a part of the vehicle control function, or a part of the vehicle body control function. The third area control unit is configured to implement a chassis system control function, a power system control function, other functions of the vehicle control, or other functions of the vehicle body control. The human-machine interaction area control unit is configured to implement an entertainment area application function or a human-machine user interface function. The controller system according to claim 5.

8. The sensor interface unit is further configured to receive data from the sensor. The human-machine interaction area control unit is further configured to perform human proximity detection or intrusion detection based on the data of the sensor, and transmit the data of the sensor to an in-vehicle recorder using the network switching unit. The controller system according to any one of claims 4 to 7.

9. The human-machine interaction area control unit further generates alert data when the human proximity or intrusion is detected, and is configured to transmit the alert data to an audio device for playback or to a display screen for display. The controller system according to claim 8.

10. The sensor interface unit is further configured to receive data from the sensor. The human-machine interaction area control unit is further configured to perform image processing based on the data of the sensor to generate surround view image data, and transmit the surround view image data to a display screen for display. The controller system according to any one of claims 1 to 7.

11. The speed at which the intelligent driving area control unit initializes is lower than the speed at which the human-machine interaction area control unit initializes. The human-machine interaction area control unit is further configured to receive a synchronization signal or a control signal of the human-machine interaction area control unit through the sensor interface unit to enable the surround view mode, and transmit the synchronization signal or the control signal to the sensor. The intelligent driving area control unit is configured to receive a synchronization signal or a control signal of the intelligent driving area control unit through the sensor interface unit in order to take over the control of the sensor interface unit or the sensor, and to transmit the synchronization signal or the control signal to the sensor. The controller system according to any one of claims 1 to 7.

12. A vehicle including the controller system according to any one of claims 1 to 11 and at least one sensor, wherein the at least one sensor is configured to collect data and transmit the collected data to the controller system.

Images