Controller system and control method

Abstract

To provide a system, a data processing method, and a control method that reduce power consumption of a vehicle.SOLUTION: A controller system includes an intelligent operation region control unit, a human-machine interaction region control unit, and a sensor interface unit. Sensor data of the sensor interface unit is not transferred by the intelligent operation region control unit in a Sentry mode or a surround view mode. Accordingly, surround view serialization / deserialization chip and cable are not required, external cable connection is simplified, exclusive using of data stream transfer resources is reduced, and power consumption of a vehicle can be reduced.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] The present application relates to the field of intelligent vehicle technology, and in particular to a controller system, a vehicle control system, a data processing method, a control method, and a system activation method. [Background technology]

[0002] Intelligent driving is a key technology for realizing intelligent vehicles and intelligent transportation, and is an unavoidable trend in future vehicle development. According to the Institute of Electrical and Electronics Engineers (IEEE), 75% of new vehicles will offer intelligent driving functions by 2040. From a technological perspective, intelligent driving brings revolutionary challenges and opportunities to the automotive industry, improving driving safety, avoiding congestion, increasing energy efficiency, and enhancing urban operation efficiency. In terms of industrial development integrated with the Internet of Things, cloud computing, and artificial intelligence (AI), intelligent driving will become an important driving force for the development of many industries in the future and promote the rapid development of intelligent manufacturing technology and next-generation information technology.

[0003] A vehicle's intelligent driving system uses sensors such as cameras, radar, and laser radar to obtain information about the vehicle and its surroundings, analyzes and processes the information obtained, and performs functions such as obstacle detection, target recognition, vehicle positioning, route planning, and driver monitoring / attention warning, thereby improving the safety, automation, and comfort of vehicle driving.

[0004] As the demand for intelligence in vehicle electronic functions increases, the vehicle's electrical and / or electronic (E / E) architecture is gradually evolving from a distributed architecture to a centralized architecture. To obtain a wealth of information, the number and types of sensors placed in the vehicle are also increasing. These sensors are connected to the vehicle and provide data that must be taken into account in the design. Summary of the Invention [Means for solving the problem]

[0005] The present application provides a controller system, a vehicle control system, a data processing method, a control method, and a system startup method to simplify external cable connections, reduce the occupation of data stream transmission resources, and reduce the power consumption of a vehicle.

[0006] To achieve the above object, a first aspect of the present application provides a controller system, comprising: an intelligent driving range control unit; a human-machine interaction area control unit; and a sensor interface unit connected to the intelligent driving area control unit and the human-machine interaction area control unit, the sensor interface unit connected to the sensor and configured to transmit sensor data to the intelligent driving area control unit and the human-machine interaction area control unit.

[0007] In this way, in a system with this structure, sensor data can be duplicated in the intelligent driving area control unit and the human-machine interaction area control unit module, and does not need to be transferred by the intelligent driving area control unit in sentry mode or surround view mode. Therefore, surround view serialization / deserialization chips and cables are not required, simplifying external cable connections, reducing the occupation of data stream transfer resources, and further reducing the vehicle's power consumption.

[0008] In one possible implementation of the first aspect, the sensor interface unit is respectively connected to the intelligent driving area control unit and the human-machine interaction area control unit through two sets of interfaces, and the signals transmitted by each set of interfaces include synchronization signals, control signals, and video data stream signals.

[0009] The synchronization signal is used as a trigger signal for the sensor or as a frame rate control signal for the sensor for video capture, and the control signal is used to configure the sensor or read data.

[0010] In this way, two sets of signals are connected separately, allowing sensor data to be sent to two sets of interfaces in a duplicated manner and receiving synchronization and control signals separately to configure and control the sensor. This connection format is simplified. In some implementations, the sensor may be a camera.

[0011] In one possible implementation of the first aspect, the sensor interface unit selects one of the intelligent driving area control unit and the human-machine interaction area control unit connected to the sensor interface unit to send the synchronization signal and / or the control signal.

[0012] According to the above description, through a sensor interface unit, for example through a camera interface unit, either one of two sets of signals, synchronization signals and control signals, can be selected and sent to a sensor, for example to a camera, so that the sensor receives either the synchronization signals or the control signals.

[0013] In one possible implementation of the first aspect, the selection is based on at least one of the following criteria: the priority and safety level of the intelligent driving area control unit and the human-machine interaction area control unit, the operating mode of the controller system, and the operating state of the controller system.

[0014] According to the above description, an example is that the functional safety level required by the intelligent operating domain control unit to control the sensor is ASIL B (i.e., the safety level is B), and the control requirement of the human-machine interaction domain control unit for the sensor is QM (QM stands for quality management level, and there is no requirement for a safety level), so a priority can be set here, or a selection can be made based on the safety level. By setting the above priority, after the above two sets of signals, the synchronization signal and the control signal, are separately received, the controller can be taken over by the intelligent operating domain control unit with a higher priority to meet the safety requirement. The operating model and operating state can also be flexibly set based on various requirements.

[0015] In one possible implementation of the first aspect, the synchronization signal transmitted between the camera interface unit and the intelligent driving area control unit is time synchronized with the synchronization signal transmitted between the camera interface unit and the human-machine interaction area control unit.

[0016] In this way, time synchronization of the two sets of synchronization signals is established, so that when control of the sensor interface unit is switched, the trigger time and frame rate of the sensor's image capture exposure can be smoothly switched to avoid jitter in the captured image.

[0017] In one possible implementation of the first aspect, the controller system comprises: a first display interface unit connected to the intelligent driving region control unit, the first display interface unit being connected to an instrument display unit; or and further including at least one of a second display interface unit connected to the human-machine interaction area control unit, the second display interface unit being connected to a display screen.

[0018] In this way, the first display interface unit is used so that the intelligent driving area control unit can send some vehicle information, such as driving information (vehicle speed, rotational speed, and total mileage), vehicle status information (water temperature, fuel quantity, electricity quantity, temperature, etc.), and other content that needs to be displayed using the instrument, to the instrument for display. The second display interface unit is used so that the human-machine interaction area control unit can send some content that needs to be displayed using a display screen within the vehicle, such as a central control display screen or a rear display screen, including, for example, an image outside the vehicle, a navigation image, and a human-machine interaction user interface, to the display screen for display.

[0019] In one possible implementation of the first aspect, the controller system further includes a network switching unit connected to the intelligent driving area control unit and the human-machine interaction area control unit.

[0020] In this way, the gateway switching function is integrated into the controller system to implement internal data exchange, eliminating the need for an external automotive Ethernet gateway and connecting cables, simplifying external cable connections and reducing the occupation of data stream transmission resources.

[0021] In one possible implementation of the first aspect, the controller system further includes a third area control unit connected to the network switching unit.

[0022] The third area control unit is connected to any one of the chassis system control unit, the power system control unit, or the body system control unit.

[0023] In this way, the third area control unit can control the chassis area, the power area, and the body area, and since multiple areas are combined, the external cable connection is simplified and the occupation of data stream transmission resources is reduced.

[0024] In one possible implementation of the first aspect, the network switching unit further comprises: The sensor is configured to be connected to at least one of a laser radar sensor, a millimeter wave radar sensor, an event data recorder, a vehicle internet communication box, and an in-vehicle recorder.

[0025] In one possible implementation of the first aspect, the intelligent driving area control unit is configured to perform an assisted driving or autonomous driving function, a part of a vehicle control function, or a part of a vehicle body control function, the third area control unit is configured to perform a chassis system control function, a power system control function, other vehicle control functions, or other vehicle body control functions, and the human-machine interaction area control unit is configured to perform an entertainment area application function or a human-machine user interface function.

[0026] In this way, some applications of the vehicle control (VCU) and body control (BCM) function software can be flexibly deployed based on the computing power of the intelligent driving area control unit and the computing power of the third area control unit, and can fully utilize the capabilities of the area control unit with higher computing power.

[0027] In one possible implementation of the first aspect, the intelligent operating region control unit further comprises: The device is configured to connect to at least one of a millimeter wave radar, an ultrasonic radar, and an integrated positioning unit including, for example, BeiDou satellites, GPS, GLONASS, and another positioning unit.

[0028] In one possible implementation of the first aspect, the human-machine interaction area control unit is further configured to connect to an audio device.

[0029] In one possible implementation of the first aspect, the intelligent driving area control unit or the human-machine interaction area control unit comprises: It includes an image processing module, a graphics rendering module, a network / video interface module, an artificial intelligence (AI) calculation module, and a control module.

[0030] The control module is configured to perform scheduling and general calculations for the other modules.

[0031] According to the above structure, for the intelligent driving range control unit, the function of directly outputting the instrument display is implemented by the intelligent driving range control unit, replacing the existing dedicated instrument SOC chip, and simplifying the entire data path.

[0032] A second aspect of the present application provides a vehicle control system including the controller system of any one of the implementations.

[0033] Another aspect of the present application provides a vehicle including a controller system of any one of the implementations and at least one sensor connected to the vehicle controller system.

[0034] In one possible implementation, the sensors include any one or more of the following sensors: a sensor for collecting image information such as a camera, an infrared camera, or a three-color-depth (RGB-D) camera; a millimeter-wave radar for collecting distance, speed, and direction of an object; a laser radar, a millimeter-wave radar, an ultrasonic radar for collecting point cloud information; an integrated positioning unit; a steering wheel pressure sensor; an inertial sensor; and an acceleration sensor. Optionally, the integrated positioning unit includes any one of a BeiDou positioning unit, a GPS positioning unit, and a GLONASS positioning unit.

[0035] In one possible implementation, the vehicle controller system may further be connected to at least one of the following devices: an event data recorder, a vehicle internet communication box (TBOX), an on-board recorder, a display screen, a power amplifier, a speaker, etc. Optionally, the display screen may include a liquid crystal display screen and / or a virtual display screen. The virtual display screen may include a virtual head-up display.

[0036] In one possible implementation, when at least one of the aforementioned sensors or the aforementioned devices is connected to the vehicle control system of the present application, the sensor or the aforementioned device can be connected to a corresponding interface, for example, to the aforementioned sensor interface unit such as a camera interface unit, a CAN interface unit, a network switching unit, or a display interface unit, to communicate with the corresponding region control unit of the present application. Optionally, if the region control unit can support such a direct connection, the region control unit may be directly connected to the aforementioned sensor or device.

[0037] A third aspect of the present application provides a data processing method, the method comprising: receiving sensor data through a sensor interface unit; and transmitting the sensor data to the intelligent driving area control unit and the human-machine interaction area control unit.

[0038] In one possible implementation of the third aspect, the method comprises: The method further includes selecting one of a connection between the sensor interface unit and the intelligent driving area control unit and a connection between the sensor interface unit and the human-machine interaction area control unit to transmit one or more of the synchronization signal and the control signal to the sensor.

[0039] The synchronization signal is used as a trigger signal for the sensor or as a frame rate control signal for the sensor for video capture, and the control signal is used to configure the sensor or read data.

[0040] In one possible implementation of the third aspect, the selection is based on at least one of the following criteria: the priority and safety level of the intelligent driving area control unit and the human-machine interaction area control unit, the operating mode of the controller system, and the operating state of the controller system.

[0041] In one possible implementation of the third aspect, the method further includes a step of enabling a synchronization signal transmitted by the intelligent driving area control unit to be time synchronized with a synchronization signal transmitted by the human-machine interaction area control unit.

[0042] In one possible implementation of the third aspect, the method comprises: generating, using the intelligent driving region control unit, content to be displayed using the instrument display unit; and transmitting, using the first display interface unit, the generated content to the instrument display unit for display, the generated content being displayed using the instrument display unit.

[0043] In one possible implementation of the third aspect, the method comprises: generating content using a human-machine interaction area control unit to be displayed using the display screen; and transmitting, using the second display interface unit, the generated content to the display screen for display, the generated content being displayed using the display screen.

[0044] A fourth aspect of the present application provides a control method to be applied to any one of the controller systems of the implementations, the method comprising: receiving sensor data through a sensor interface unit; using a human-machine interaction area control unit to perform human proximity detection or intrusion detection based on the sensor data; and transmitting the sensor data to the vehicle recorder using the network switching unit if an exception is detected.

[0045] Therefore, when the sentry mode is implemented, there is almost no data stream and almost no hardware resources are occupied, so the power consumption is very low.

[0046] In one possible implementation of the fourth aspect, if an exception is detected, the method comprises: generating alert data using a human-machine interaction area control unit; The method further includes transmitting the alert data to an audio device for playback, or transmitting the alert data to a display screen for display using a second display interface unit.

[0047] A fifth aspect of the present application provides a control method to be applied to a controller system of any one of the implementations, the method comprising: receiving sensor data through a sensor interface unit; performing image processing based on the sensor data using a human-machine interaction area control unit to generate surround view image data; and transmitting, using the second display interface unit, the surround view image data to a display screen for display.

[0048] Therefore, when the surround view mode is implemented, there is little data stream and little hardware resource is occupied, so the power consumption is very low.

[0049] A sixth aspect of the present application provides a system startup method to be applied to any one of the controller systems of the implementation, the method comprising: Separately performing initialization by the intelligent driving area control unit and the human-machine interaction area control unit, wherein the speed at which the intelligent driving area control unit performs initialization is slower than the speed at which the human-machine interaction area control unit performs initialization; receiving a synchronization signal or a control signal from the human-machine interaction area control unit through a sensor interface unit to enable a surround view mode after the initialization performed by the human-machine interaction area control unit is completed; and transmitting the synchronization signal or the control signal to the sensor; After the initialization performed by the intelligent driving area control unit is completed, the step of receiving a synchronization signal or a control signal from the intelligent driving area control unit through the sensor interface unit to assume control over the sensor interface unit or the sensor includes a step of transmitting the synchronization signal or the control signal to the sensor.

[0050] In this way, when the vehicle is powered on, the surround view mode is enabled immediately, and then, if the vehicle is normal after powering on, the intelligent driving range control unit with a high safety level takes over the control of the sensor interface unit and sensors. To ensure high safety, the switching is performed based on the above-mentioned priority control and takeover.

[0051] These and other aspects of the present application will become clearer and easier to understand in the following description of the embodiment(s).

[0052] The features and relationships between features of the present application will be further described below with reference to the accompanying drawings. All of the accompanying drawings are examples, and some features are not shown to actual scale. In addition, some of the accompanying drawings may omit common features that are not essential to the field of the present application, or show additional features that are not essential to the application. The combinations of features shown in the accompanying drawings are not intended to limit the present application. In addition, in this specification, the same reference numerals refer to the same content. Specific accompanying drawings are described below. [Brief explanation of the drawings]

[0053] [Figure 1A] FIG. 1 is a schematic diagram of a controller system according to an embodiment of the present application. [Figure 1B] FIG. 2 is a schematic diagram of a controller system according to another embodiment of the present application. [Figure 2] FIG. 1 is a schematic diagram of a controller system according to certain implementations of the present application. [Figure 3] 1 is a schematic diagram of a controller system including a network switching unit according to an embodiment of the present application; [Figure 4] 2 is a schematic diagram of a data stream of data displayed by an instrument according to an embodiment of the present application; [Figure 5] FIG. 2 is a schematic diagram of functional software within a region controller according to an embodiment of the present application. [Figure 6A] 2 is a schematic diagram of the interior of an intelligent operating region control unit according to an embodiment of the present application; FIG. [Figure 6B] 1 is a schematic diagram of the interior of a human-machine interaction area control unit according to an embodiment of the present application; FIG. [Figure 7A] 1 is a schematic diagram of a first hardware form of a controller system according to an embodiment of the present application. [Figure 7B] FIG. 2 is a schematic diagram of a second hardware form of a controller system according to an embodiment of the present application. [Figure 8]1 is a schematic flowchart of a control method according to an embodiment of the present application. [Figure 9] 4 is a schematic flowchart of a control method according to another embodiment of the present application. [Figure 10A] 1 is a schematic flowchart of a system startup method according to an embodiment of the present application. [Figure 10B] FIG. 1 is a schematic diagram of a system startup method according to an embodiment of the present application. [Figure 11] 1 is a schematic diagram of a data processing method according to an embodiment of the present application; [Figure 12] FIG. 3 is a schematic diagram of a data processing method according to another embodiment of the present application; [Figure 13A] 1 is a schematic diagram of an architecture including independent region controllers in the current technology. [Figure 13B] FIG. 1 is a schematic diagram of a data stream in surround view / sentry mode in the current technology. [Figure 13C] 1 is a schematic diagram of a data stream of data displayed by an instrument in the current state of the art; DETAILED DESCRIPTION OF THE INVENTION

[0054] In this specification and claims, the terms "first, second, third, etc." and similar terms such as module A, module B, and module C are used merely to distinguish between similar objects and do not represent a particular order of the objects. It will be understood that the specific order or sequence may be interchanged, where permissible, and that the embodiments of the present application described herein may be implemented in orders other than those illustrated or described herein.

[0055] In the following description, the reference numerals S110, S120, etc. indicating steps do not necessarily indicate that the steps should be performed in that order, and consecutive steps may be interchanged or performed simultaneously, if permitted.

[0056] The term "comprising" as used in this specification and claims should not be construed as being limited to the contents listed below, nor should it exclude other elements or steps. It should be interpreted as specifying the presence of the mentioned feature, whole, step, or part, but not excluding the presence or addition of one or more other features, wholes, steps, or parts, and their collections. Thus, the phrase "a device including apparatus A and apparatus B" should not be limited to a device including only components A and B.

[0057] References herein to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, appearances of the terms "in one embodiment" or "in an embodiment" herein do not necessarily refer to the same embodiment, but may refer to the same embodiment. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. In the event of any conflict, the meaning described herein or obtained in accordance with the contents recorded herein shall prevail. In addition, the terms used herein are merely for the purpose of describing the embodiments of this application and are not intended to limit this application. In order to accurately describe the technical content of this application and to accurately understand this application, the following explanations, descriptions, or definitions of the terms used herein are provided before describing specific implementations.

[0059] (1) Domain control units, abbreviated as DCUs (Domain Control Units) or DCMs (Domain Control Modules). A "domain" refers to the division of a vehicle's electronic system into several functional blocks based on its function. Each functional block can be called a domain, such as the vehicle domain, body domain, intelligent driving domain, and human-machine interaction domain. Domain control units can reduce the number of ECUs in a vehicle and reduce system complexity. Each domain has a domain controller, also called a domain control unit, which contains one or more processors responsible for processing and forwarding functions within the domain. Components or modules within a domain are typically connected via low-speed communication, while domain control units are connected via high-speed communication. Note that the domain division method used in this document is not unique. For example, a vehicle domain may also be divided into a power domain and a chassis domain by a manufacturer. In another example, a vehicle domain may be divided into a vehicle control domain, an intelligent driving domain, an intelligent cockpit domain, etc. A typical configuration of a domain control unit is as follows:

[0060] An intelligent driving domain control unit, sometimes referred to as an advanced driver assistance systems / autonomous driving (ADAS / AD) domain control unit or ADAS / AD domain controller, may be configured to aggregate data from various sensors to perform sensing and decision-making to enable assisted or autonomous driving. For example, it aggregates sensor data from cameras, ultrasonic radar, millimeter-wave radar, laser radar, GPS / inertial measurement unit (IMU), and maps to perform sensing and decision-making to enable assisted or autonomous driving. The intelligent driving domain control unit has high requirements for computing power and data processing capabilities and must meet a high automotive safety integration level (ASIL), such as ASIL B, ASIL C, or ASIL D.

[0061] The human-machine interaction area control unit, sometimes called an in-vehicle infotainment / human-machine interaction (IVI / HMI) area control unit, is configured to provide functions such as in-vehicle information and in-vehicle entertainment, and can provide users with geographic information, multimedia entertainment, intelligent transportation services, etc., and can perform interactions related to entertainment information, content display, audio playback, etc. through human-machine interaction, including collecting user information using sensors and displaying information to the user using a display or by sound, for example, collecting the driver's face information, fingerprint information, voice information, steering wheel pressure information, pedal pressure information, etc. The human-machine interaction area control unit is sometimes called a cockpit area control unit or cockpit domain controller (CDC).

[0062] A vehicle control unit (VCU) is the assembly controller of the power system of an electric vehicle (hybrid and pure electric vehicle). The vehicle control unit is used to control the vehicle chassis systems, such as the braking system, parking system, body stability system, steering system, and to control the vehicle power system, such as the power supply system, charging system, motor (in the case of an electric vehicle), and engine system (in the case of a fuel-powered vehicle). It is sometimes called a Vehicle Domain Controller (VDC).

[0063] Body area control units (BCMs) are used to control body systems, for example, to control the doors, windows, and seats of the body.

[0064] It should be noted that the divisions and names of the area control units described above are merely examples and are not limiting. As technology evolves and develops, the names of the area control units may also be different.

[0065] (2) Surround view display refers to using multiple vehicle cameras to capture images of the vehicle's surroundings, stitching the images together, and displaying them using the vehicle's display screen.

[0066] (3) Sentry Mode, a vehicle surveillance mode used when the vehicle is parked. The camera is used to monitor the vehicle environment. When a person outside the vehicle approaches or enters, video data is recorded and an alarm is generated.

[0067] (4) A serializer / deserializer (Ser / Des) is a high-speed communication interface circuit. The serializer is configured to convert low-speed parallel signals into high-speed serial signals for transmission, and the deserializer is configured to convert high-speed serial signals into low-speed parallel signals for transmission. The serializer and deserializer may be located separately or may be integrated and fixed. The serializer is fixed on the transmitting side, and the deserializer is fixed on the receiving side. A serializer / deserializer can realize high-speed transmission of multi-bit video data, and when applied to video transmission using a display component, it is sometimes called a display SERDES interface, and when applied to data transmission using a camera sensor, it is sometimes called a camera SERDES interface.

[0068] (5) The Display Serial Interface (DSI) and Camera Serial Interface (CSI), also known as the display interface and camera interface, respectively, are standard interfaces of the Mobile Industry Processor Interface (MIPI) and are used to connect to displays and cameras, respectively.

[0069] Each component or function in a conventional vehicle requires one or more electronic control units (ECUs) for control. The current main solution for intelligent vehicles is an architecture that includes multiple independent area controllers. For example, see Figure 13A for a vehicle's electronic and electrical architecture. The electronic and electrical architecture is usually divided into a body control area, a human-machine interaction area, an intelligent driving area, and a vehicle control area. Each area controller communicates with a vehicle internet communication box (TBOX) through a central gateway via a controller area network (CAN) bus / Ethernet cable. The following uses an example to explain the data stream transfer process of the architecture.

[0070] Example 1: As shown in FIG. 13B, the data stream in surround view / sentry mode is as follows:

[0071] Surround view display: First, the camera sends video data to the ADAS / AD area controller, and then the ADAS / AD area controller's internal image signal processing (ISP) module processes the original data into RGB / YUV (RGB and YUV are two color encoding methods) data format data, and then forwards the data to the human-machine interaction area controller, which then forwards the video data to the central control screen for display.

[0072] Sentry mode: First, the camera sends video data to the ADAS / AD area controller, and the internal ISP of the ADAS / AD area controller processes the original data into RGB / YUV data format and transfers the data to the human-machine interaction area controller. The human-machine interaction area controller performs human proximity / intrusion detection. If unauthorized intrusion is detected, the video data is transferred to the on-board digital video recorder (DVR) for storage, and an alarm is generated through the central control screen or audio amplifier.

[0073] Example 2: As shown in FIG. 13C, the data stream of instrument display data is as follows: The VCU collects information about the chassis body control and power ECUs and transfers the information required for display, such as vehicle speed, to the instrument processor. The instrument processor uses an image processing module (GPU) to perform image rendering and then sends the image to the instrument screen for display. The above architecture uses multiple independent area controllers, which occupies a large amount of space in the vehicle equipment and requires additional video / Ethernet cables. In addition, in the above example, the architecture transfers data streams multiple times, occupying many hardware resources and increasing power consumption.

[0074] It will be appreciated that the above architectures are merely exemplary solutions, and vehicle architectures are constantly evolving. In addition to several area control units, a vehicle may further include several ECUs that are used for specific components or functions independent of the area control units. These on-board devices have computing and processing capabilities and may also be referred to as on-board computing devices, computing platforms, etc.

[0075] Furthermore, as the number and variety of sensors deployed in a vehicle increases, challenges arise in connecting and transmitting signals between the sensors and a controller or control unit within the vehicle.

[0076] This application provides a separate controller system and an application based on the controller system to reduce the occupation of data stream transmission resources, reduce vehicle power consumption in various vehicle application scenarios, improve vehicle function activation speed, and enhance user experience. First, it thoroughly integrates the intelligent driving area controller, human-machine interaction area controller, vehicle control, and vehicle body control, simplifying external cable connections. Second, it also provides a flexible connection method between sensors and on-board computing devices, improving the sharing of sensor data.

[0077] The controller system provided in this application may be applied to vehicles such as intelligent vehicles and ships with intelligent driving functions, or may be applied to application scenarios such as robots.

[0078] The present application will now be described in detail with reference to the accompanying drawings.

[0079] An embodiment of the present application provides a controller system. The controller system includes one or more on-board control units and a sensor interface unit. The on-board control units may be area control units or ECUs. These on-board control units may be implemented on the same hardware. For example, multiple SoCs (system on chips, SOCs) may be implemented on one hardware platform, with each SOC corresponding to one on-board control unit. The on-board control units may also be implemented separately on independent hardware. This is not limited to this embodiment of the present application. For ease of explanation, the following uses an example in which the on-board control unit is an intelligent driving area control unit and an example in which the on-board control unit is a human-machine interaction area control unit. It is understood that the embodiment of the present application is not limited thereto. In practical applications, the on-board control unit may be another area control unit or an electronic control unit. As shown in FIG. 1A , the controller system includes an intelligent driving area control unit, a human-machine interaction area control unit, and a sensor interface unit. The sensor interface unit is connected to the intelligent driving area control unit and the human-machine interaction area control unit. The sensor interface unit further has an external interface for connecting to an external sensor. The sensor interface unit may be configured to transmit sensor data to the intelligent driving area control unit and the human-machine interaction area control unit. It should be noted that the names of the area control units in this application are only examples. The intelligent driving area control unit, the human-machine interaction area control unit, etc. are used as examples for explanation and are not limited thereto. It should be understood that with the evolution and development of technology, units or modules configured to perform corresponding functions may be named by other names.

[0080] The intelligent driving area control unit can perform sensing and decision-making by using data received from one or more sensors to perform assisted or autonomous driving. As the first area control unit of the present application, the intelligent driving area control unit can include an electronic control unit (ECU), a microcontrol unit (MCU), a central processing unit (CPU), a graphics processing unit (GPU), or another control unit, and can be implemented by a single control unit or multiple control units. Figure 6A shows an embodiment of any one of the implementations described below.

[0081] The one or more sensors are configured to collect outside-vehicle information and may include any one or more of the following sensors: a sensor for collecting image information, such as a camera, an infrared camera, or a three-color depth (RGB-D) camera; a millimeter-wave radar for collecting distance, speed, and direction of an object; a laser radar for collecting point cloud information; or a sensor for collecting other data. In this embodiment, an example in which a camera shown in FIG. 1B is used as a sensor is used in the following description. The corresponding sensor interface unit is a camera detection unit. The collected data may be image data outside the vehicle, for example, image data of the area in front of the vehicle obtained during driving. The image data is used to detect vehicles, pedestrians, etc. in the image. The detection results are used for decision-making during assisted driving and autonomous driving.

[0082] The aforementioned human-machine interaction area control unit can perform face recognition, action recognition, etc. using data received from an external sensor, such as a camera, to provide entertainment service functions. As the second area control unit of the present application, the human-machine interaction area control unit can include an ECU, an MCU, a CPU, a GPU, or another control unit, and can be implemented by a single control unit or multiple control units, such as a control unit for an in-vehicle infotainment system and a control unit for a human-machine user interface. Figure 6B shows an embodiment of any one of the implementations described below.

[0083] When the human-machine interaction area control unit is applied to the sentry mode, the data of the external sensor, for example, the data of the camera, can be a still face image or a close-range human body image outside the vehicle. When the vehicle is in a stationary state, the image data obtained by the sensor is sent to the human-machine interaction area control unit to perform face detection or intrusion detection based on motion recognition, and the human-machine interaction area control unit can further control the transfer of the data of the relevant camera to the on-board recorder for recording.

[0084] In some embodiments, when the sensor interface unit is a camera interface unit, the sensor interface unit may specifically be a camera serialization / deserialization interface for performing high-speed serial communication with an external camera.

[0085] In some embodiments, the manner in which the sensor interface unit is connected to the intelligent driving area control unit and the human-machine interaction area control unit includes connecting the sensor interface unit to the intelligent driving area control unit and the human-machine interaction area control unit, respectively, through two sets of interfaces, and signals transmitted between the sensor interface unit and the two area control units may include synchronization signals, control signals, and video data stream signals. For ease of explanation, the interfaces may be referred to as a synchronization signal interface, a control signal interface, and a video data stream signal interface based on the types of signals transmitted. It should be understood that two or more of these interfaces may be integrated, and two or more of the synchronization signals, control signals, and video data stream signals may be combined for transmission. For example, the synchronization signals and control signals are transmitted through one connection, and the video data stream signals are transmitted through one connection. In another example, the synchronization signals, control signals, and video data stream signals are transmitted through one connection. The interfaces can also be used as independent interfaces for transmitting the synchronization signals, control signals, and video data stream signals through three connections.

[0086] In some embodiments, the control signals may be transmitted using I2C signals and the video data stream signals may be transmitted using CSI-2 signals.

[0087] In some embodiments, the video data stream signal may also be transmitted between the sensor interface unit and the two control units by using low voltage differential signaling (LVDS).

[0088] In some embodiments, corresponding to the two sets of interfaces, the sensor interface unit specifically includes: Use two sets of synchronization signal interfaces to receive synchronization signals from the intelligent driving area control unit or the human-machine interaction area control unit, and send the synchronization signals to the sensor, where the synchronization signals are used as exposure trigger signals or frame rate control signals for the images captured by the sensor; Using two sets of control signal interfaces (e.g., I2C signal interfaces) to receive control signals from the intelligent driving area control unit or the human-machine interaction area control unit, and sending the control signals to sensors, where the control signals are used to configure the sensors, e.g., cameras, or read data, for example, to perform high dynamic range rendering (HDR) mode configuration or exposure parameter configuration; and The sensor interface unit is configured to receive sensor data, duplicate the data, and then transmit the data to the intelligent driving area control unit and the human-machine interaction area control unit respectively using two sets of video data stream signal interfaces (such as a CSI-2 signal interface and an LVDS signal interface).

[0089] Each set of interfaces may include one or more physical transmission channels. A physical transmission channel herein may be one or more signal lines or cables. For example, when CSI-2 signals are transmitted, each physical transmission channel may include multiple signal lines, and may be configured to transmit, for example, one pair of clock synchronization signals, one pair of data signals, two pairs of data signals, or four pairs of data signals. As another example, when LVDS signals are transmitted, each physical transmission channel may include a pair of cables used to provide the LVDS signals. When each set of interfaces includes multiple physical transmission channels, separate transmission channels can be used to receive and transmit video data separately to improve data transmission efficiency or to provide differentiated data transmission. For example, video data may be distributed to corresponding transmission channels for transmission based on the load of the various transmission channels. In another example, video data may be distributed to corresponding transmission channels for transmission based on various priorities, delay requirements, etc.

[0090] In one possible implementation, the sensor interface unit can send video data stream signals to the intelligent driving area control unit and the human-machine interaction area control unit separately, but can send synchronization signals or control signals, or synchronization signals and control signals, to only one of the area control units. In this manner, there is no need to select between the two control units.

[0091] In some embodiments, the sensor interface unit selects either the intelligent driving area control unit or the human-machine interaction area control unit connected to the sensor interface unit to send synchronization or control signals for configuration.

[0092] When the sensor interface unit is connected to multiple zone control units, it can select a specific zone control unit for transmitting the synchronization signal or control signal based on one or more factors, such as the zone controller unit's priority, safety level, system operation mode, and operation status. In one possible implementation, a zone controller unit with a higher priority can be selected for transmitting the synchronization signal and control signal. For example, in this embodiment, the sensor interface unit is connected to an intelligent driving zone control unit and a human-machine interaction zone control unit, and the priority of the intelligent driving zone control unit is higher than the priority of the human-machine interaction zone control unit. Optionally, the manner of selecting one of the zone control units for transmission includes the sensor interface unit selecting the intelligent driving zone control unit to transmit the synchronization signal or control signal after receiving the synchronization signal or control signal of the intelligent driving zone control unit.

[0093] In yet another possible implementation, a region control unit with a higher safety level can be selected to receive or transmit synchronization and control signals. Continuing to use this embodiment as an example, the functional safety level required by the intelligent driving region control unit to control the sensor is ASIL B (i.e., the safety level is B), and the control requirement of the human-machine interaction region control unit for the sensor is QM (QM stands for quality management level, and there is no requirement for the safety level), so the safety level requirement of the intelligent driving region control unit is higher than the safety level requirement of the human-machine interaction region control unit. The synchronization and control signals of the intelligent driving region control unit are sent to the external sensor connected to the sensor interface unit; in other words, the sensor is taken over by the intelligent driving region control unit with a higher priority.

[0094] In yet another possible implementation, based on the operating state of the region control unit, for example, the startup and initialization of the human-machine interaction region control unit is faster than the startup and initialization of the intelligent driving region control unit, and the intelligent driving region control unit is not yet operating normally after the human-machine interaction region control unit is started up. In this case, after the initialization performed by the human-machine interaction region control unit is completed, the synchronization signal and control signal of the human-machine interaction region control unit can be sent to the sensor. After the initialization of the intelligent driving region control unit is completed, it can also be determined based on priority, safety level, and other considerations whether the intelligent driving region control unit should assume control of the sensor and whether the synchronization signal and control signal of the intelligent driving region control unit should be sent to the sensor.

[0095] In yet another possible implementation, the system is in sentry mode, in which the human-machine interaction area control unit is powered on but the intelligent driving area control unit is not powered on, and the sensor interface unit therefore only receives synchronization and control signals from the human-machine interaction area control unit and transmits synchronization and control signals to the external sensors.

[0096] In yet another possible implementation, the system is in a surround view mode, in which both the human-machine interaction area control unit and the intelligent driving area control unit are powered on. Therefore, based on the aforementioned priority or safety level, the sensor interface unit selects to send synchronization signals and control signals of the intelligent driving area control unit to the external sensors.

[0097] In some embodiments, the synchronization signal transmitted between the sensor interface unit and the intelligent driving area control unit is time synchronized with the synchronization signal transmitted between the sensor interface unit and the human-machine interaction area control unit.

[0098] In this way, time synchronization of the two sets of synchronization signals is set, so that when the control right of the sensor interface unit is switched, the trigger time point and frame rate of the sensor's image capture exposure can be smoothly switched.The intelligent driving area control unit and the human-machine interaction area control unit can implement precise time synchronization based on time-sensitive networking (TNS), so that the transmitted synchronization signals are synchronized with each other through time alignment.

[0099] In some embodiments, as shown in FIG. 2 , the controller system includes a first display interface unit connected to the intelligent driving area control unit, the first display interface unit being connected to an instrument display unit, or a second display interface unit connected to the human-machine interaction area control unit, the second display interface unit being connected to a display screen. Additionally, in the example shown in FIG. 2 , a camera interface unit is used as an example of a sensor interface unit. Accordingly, an example in which a camera is used as a sensor is used for the description. Additionally, FIG. 2 further illustrates a sensor, such as a millimeter-wave radar or an ultrasonic radar, which can be connected to the intelligent driving area control unit through a CAN interface (for simplicity, the CAN interface unit included as a sensor interface is not depicted). In the example shown in FIG. 2 , the video signal of the camera interface unit may be a CSI-2 signal provided by one or more physical transmission channels, or an LVDS signal provided by one or more physical transmission channels.

[0100] In this way, since the first display interface unit is used, the intelligent driving area control unit can send some vehicle information, such as driving information (vehicle speed, rotation speed, and total mileage), vehicle status information (water temperature, fuel quantity, electricity quantity, temperature, etc.), and other contents that need to be displayed using the instrument display screen, to the instrument display screen for display.

[0101] A second display interface unit is used so that the human-machine interaction area control unit can send any content that needs to be displayed using a display screen within the vehicle, such as a central control display screen or a rear display screen, including, for example, an image outside the vehicle, a navigation image, and a human-machine interaction user interface, to the display screen for display.

[0102] In some embodiments, the display screen may be a liquid crystal display screen, or may be a virtual display screen, such as a virtual Augmented Reality Head Up Display (AR-HUD).

[0103] In some embodiments, as shown in FIG. 2 or FIG. 3, the controller system further includes a network switching unit connected to the intelligent driving area control unit and the human-machine interaction area control unit.

[0104] In this way, the gateway switching function is integrated into the controller system to perform internal data exchange, eliminating the need for external automotive Ethernet gateways and connecting cables. Network interface implementations include, but are not limited to, internal network interfaces such as the Reduced Gigabit Media Independent Interface (RGMII), the Reduced Media Independent Interface (RMII), the Serial Gigabit Media Independent Interface (SGMII), the 10 Gigabit Media Independent Interface (XGMII), and the 10G Basic Transmission Protocol (10G_base_R).

[0105] In some embodiments, as shown in FIG. 2 or FIG. 3, the controller system further includes a third regional control unit connected to the network switching unit, and the third regional control unit is connected to any one of the chassis system control unit, the power system control unit, or the body system control unit.

[0106] In some embodiments, the third area control unit may be implemented by an MCU. As shown in FIG. 2 or 4, the third area control unit may collect information from or control units such as a chassis system control unit, a power system control unit, or a body system control unit. The collected information may be transferred to the intelligent driving area control unit through a network switching unit. After performing internal processing (such as image processing and image rendering), the intelligent driving area control unit sends the information to the instrument display unit for display through the first display interface unit.

[0107] The chassis system control unit manages the electric power steering system (EPS), electronic stability program (ESP), electric park brake (EPB), intelligent brake system (IBS), electronic stability control (ESC), etc. The power system control unit manages the electric motor system (for electric vehicles), battery management system (for electric vehicles), power conversion system (DC-DC, i.e., direct current-to-direct current conversion) (for electric vehicles), on-board charger (OBC) (for electric vehicles), engine system (for fuel-powered vehicles), etc. The body system control unit manages the vehicle's doors, windows, seats, etc., and may further include other control units such as an airbag system and a thermal management system.

[0108] In some embodiments, as shown in FIG. 2 , the network switching unit is further configured to connect to at least one of a laser radar sensor, a millimeter wave radar sensor, an event data recorder, a vehicle internet communication box (TBOX), and an on-board recorder.

[0109] The aforementioned devices may be connected through an Ethernet interface to effect the aforementioned collected data being transferred to a controller system, and may be provided for communication with or to a corresponding regional control unit.

[0110] In some embodiments, as shown in FIG. 2, the intelligent driving area control unit is further configured to connect to at least one of a millimeter wave radar, an ultrasonic radar, and an integrated positioning unit (e.g., including BeiDou, GPS, GLONASS, etc.).

[0111] The aforementioned devices may be connected through a CAN interface, so that the intelligent driving area control unit obtains sensor information.

[0112] The intelligent driving area control unit, the human-machine interaction area control unit, and the third area control unit are connected to separate devices, so that the three area control units can be flexibly deployed with corresponding required function software. As shown in Figure 5, one deployment method may be as follows:

[0113] In addition to the intelligent driving area function software, some function software for vehicle control or some function software for vehicle body control are integrated into the intelligent driving area control unit. The underlying OS of the intelligent driving area control unit may be closed to ensure safety.

[0114] In addition to the chassis / power function software, some function software for vehicle control or some function software for vehicle body control is integrated into the third area control unit. The underlying OS of the third area control unit may be closed to ensure safety.

[0115] The entertainment area application software and the human-machine user interface application software are deployed in the human-machine interaction area control unit. To facilitate the use of various entertainment application software, the underlying OS of the human-machine interaction area control unit may be opened.

[0116] Some functional software for vehicle control or some applications for vehicle body control may be deployed based on the computing capabilities of the intelligent driving area control unit and the third area control unit. For example, if the computing capability of the third area control unit is lower than that of the intelligent driving area control unit, some functions of vehicle control may be deployed in the intelligent driving area control unit. When these functions need to be used, the third area control unit exchanges relevant data through the network switching unit, and the execution of the functions is performed by the intelligent driving area control unit, but direct information exchange (e.g., control signals or collected sensor signals) with the chassis ECU and power ECU related to vehicle control is performed by the third area control unit.

[0117] In some embodiments, as shown in FIG. 2, the human-machine interaction area control unit is further configured to connect to an audio device.

[0118] The audio device may include a power amplifier, a speaker, etc., and may be configured to reproduce alert information, human-machine interaction sounds, etc.

[0119] In some embodiments, the intelligent driving area control unit or human-machine interaction area control unit includes an image processing module, a graphics rendering module, a network / video interface module, an AI calculation module, and a control module, where the control module is configured to perform scheduling and general calculations for the other modules. A detailed description is provided below.

[0120] As shown in Figure 6A, to achieve an overall functional safety level of ASIL B or above, at least the following modules are integrated into the intelligent driving domain control unit: an ISP module mainly responsible for image processing, a GPU module responsible for graphics rendering, which is used for instrument display, a network / video interface module configured to connect to external interface devices, an ARM CPU module configured to perform overall scheduling and multi-purpose calculations, and an AI module configured to accelerate calculations in intelligent driving detection calculations with its strong computing power.

[0121] With the GPU function integrated into the intelligent driving range control unit, the function of directly outputting the instrument display is performed by the intelligent driving range control unit, replacing the existing dedicated instrument system-on-chip (SOC) and simplifying the entire data path.

[0122] As shown in Figure 6B, to achieve an overall functional safety level of ASIL B or above, at least the following modules are integrated into the human-machine interaction area control unit: an ISP module primarily responsible for image processing; a GPU module responsible for graphics rendering, which is used to display human-machine interaction and entertainment services (IVI); a network / video interface module configured to connect to external interface devices; an ARM CPU module responsible for overall scheduling and general-purpose calculations; and an artificial intelligence (AI) module configured with weak computing power to perform AI calculations related to human-machine interaction and AI calculations in sentry mode.

[0123] It should be noted herein that the hardware form of the controller system of the present application may be a circuit board, as shown in FIG. 7A, or may be implemented by stacking multiple boards, as shown in FIG. 7B. For example, the I / O interface of the MCU may be carried on a single board, or may be split and carried on multiple boards, but is not limited to this. The multiple boards may be connected to each other using daughter board connectors, flexible cables, or the like, to transmit power signals, low-speed data signals, video signals (such as the aforementioned CSI-2), image control signals (such as the aforementioned DSI), and network signals (such as XGMII and SGMII).

[0124] The controller system of the present application may further include a plurality of independent hardware devices and a sensor interface unit, each of which is equipped with one or more control units, and the hardware devices are connected to each other using cables.

[0125] The sensor interface unit is configured to obtain sensor signals from the connected sensors and transmit the sensor signals to one or more control units in the controller system, and further configured to obtain synchronization or control signals for configuration of the connected sensors from the one or more control units in the controller system. For example, the connected sensors are cameras that capture image information, and accordingly, the sensor signals include video signals.

[0126] The sensor interface unit may include one or more serialization / deserialization interfaces and may further include an LVDS signal interface.

[0127] The serialization / deserialization interface is configured to transmit video signals, for example, between a camera and a control unit or between control units.

[0128] In one possible implementation, a serialization / deserialization interface is configured separately for each control unit that needs to transmit a video signal. If a video signal is transmitted between two control units that are located on separate hardware devices and connected using a coaxial cable, a first LVDS interface and a second LVDS interface need to be configured for each of the two control units so that the video signal is transmitted in the form of an LVDS signal between the control units located on the separate hardware devices. Configuring a serialization / deserialization interface may involve arranging a serializer / deserializer that performs the serializer / deserializer interface function and the corresponding control unit on the same hardware. The first LVDS interface includes one or more pairs of cables, and the second LVDS interface includes one or more pairs of cables. The video signal may be transmitted separately over one or more physical transmission channels on the two interfaces.

[0129] An example will be described in which the controller system includes a first control unit, a second control unit, and a sensor interface unit. The sensor interface unit includes a first serialization / deserialization interface and a second serialization / deserialization interface. The first control unit includes a SOC 1 configured to process a sensor signal, and the second control unit includes a SOC 2 configured to process the sensor signal. The first control unit obtains a video signal through the first serialization / deserialization interface, and the second control unit obtains a video signal through the second serialization / deserialization interface.

[0130] In one possible implementation, the video signal is duplicated within the sensor interface unit, and two sets of signals are sent to the first control unit and the second control unit, respectively, through the first serialization / deserialization interface.

[0131] In yet another possible implementation, the video signal may be transmitted to the first control unit through the first serialization / deserialization interface, which then transmits the video signal in the form of an LVDS signal to the second LVDS interface through the first LVDS interface, and the second LVDS interface transmits the video signal to the second serialization / deserialization interface. The second serialization / deserialization interface converts the data into serial data and then transmits the serial data to the second control unit. Alternatively, it should be noted that the video signal may be transmitted to the second control unit through the second serialization / deserialization interface, which then transmits the video signal in the form of an LVDS signal to the first LVDS interface through the second LVDS interface, and the first LVDS interface transmits the video signal to the first serialization / deserialization interface.

[0132] The controller system provided in the above embodiment allows sensors to be shared among multiple control units to reduce costs and deployment space.

[0133] Furthermore, if the first control unit is started up faster than the second control unit, the sensor configuration may be performed after the first control unit is started up, thereby enabling the sensor to be started up in a timely manner to obtain information about the vehicle's surroundings and improve the user experience.

[0134] Furthermore, if the processing capability of the second control unit is stronger than that of the first control unit, for example, if an image processing module is disposed in the second control unit, the video signal processed by the second control unit can be further transmitted to the first control unit using the sensor interface unit, so that the first control unit obtains the processed video signal.

[0135] The first control unit may be a human-machine interaction domain control unit or another ECU, and the second control unit may be an intelligent driving domain control unit or another ECU.

[0136] Another embodiment of the present application provides a vehicle control system. The vehicle control system includes the controller system of any one of the implementations. Another embodiment of the present application further provides a vehicle. The vehicle includes the controller system of any one of the implementations, and the vehicle further includes at least one of the following sensors or at least one of the following devices:

[0137] Sensor: The sensor includes any one or more of the following sensors: a sensor for collecting image information such as a camera, an infrared camera, or a three-color depth (RGB-Depth, RGB-D) camera; a millimeter-wave radar for collecting distance, speed, and direction of an object; a laser radar for collecting point cloud information, a millimeter-wave radar, an ultrasonic radar; an integrated positioning unit (having a positioning unit such as BeiDou, GPS, or GLONASS); a steering wheel pressure sensor; an inertial sensor; and an acceleration sensor.

[0138] Devices: an event data recorder, a vehicle internet communication box (TBOX), an in-vehicle recorder, a display screen, a power amplifier, and a speaker. The display screen may be a liquid crystal display screen or a virtual display screen, for example, a virtual head-up display.

[0139] See Figure 2. When at least one of the aforementioned sensors or devices is connected to the vehicle control system of the present application, the sensor or device can be connected to a corresponding interface, for example, to the aforementioned sensor interface unit (camera interface unit, CAN interface unit, etc.), or network switching unit, or display interface unit, to communicate with the corresponding area control unit of the present application.

[0140] Another embodiment of the present application provides a control method, which is applied to the above-mentioned controller system. Here, the sentry mode applied to a vehicle is used as an example for explanation. As shown in Figure 8, the method includes the following steps:

[0141] S110: After power-on initialization is performed in sentry mode, camera data is received through the camera interface unit.

[0142] To achieve low power consumption in this mode, only the camera, camera interface unit, human-machine interaction area control unit, and related display module related to the sentry mode are powered on, and other unrelated modules do not need to be powered on. In addition, in this mode, the camera and camera interface unit receive synchronization signals and control signals from the human-machine interaction area control unit, i.e., the human-machine interaction area control unit takes over the camera and camera interface unit.

[0143] S120: Use the human-machine interaction area control unit to perform human proximity detection or intrusion detection based on camera data. For example, after performing image processing using the internal ISP module of the human-machine interaction area control unit, the human-machine interaction area control unit performs human proximity detection or intrusion detection using the internal AI module.

[0144] S130: When an exception is detected, use the network switching unit to transmit the camera data obtained by the human-machine interaction area control unit to the vehicle recorder.

[0145] In some embodiments, if an exception is detected, the method further includes generating alert data using the human-machine interaction area control unit and sending the alert data to an audio device for playback or sending the alert data to a display screen for display using the second display interface unit.

[0146] Another embodiment of the present application provides a control method, which is applied to a controller system. Here, an example in which a surround view mode is applied in a vehicle is used for explanation. As shown in Figure 9, the method includes the following steps:

[0147] S210: Receive camera data through the camera interface unit.

[0148] S220: Use the human-machine interaction area control unit to perform image processing on the camera data to generate surround view image data. For example, the human-machine interaction area control unit uses its internal ISP module to perform image processing, which includes converting the image format from RAW format to RGB format, using the internal GPU module to perform image rendering, and performing processing such as 3D surround view splicing based on the image data of the multiple cameras, and overlaying the image on the vehicle model image to generate surround view image data.

[0149] S230: Using a second display interface unit, send the surround-view image data to a display screen for display.

[0150] Another embodiment of the present application provides a system startup method, which is applied to the aforementioned controller system. In this embodiment, an example is used for explanation, in which the sensor interface unit is a camera interface unit and the sensor is a camera. As shown in the flowchart of FIG. 10A and the schematic diagram shown in FIG. 10B, the method includes the following steps:

[0151] S310: Power-on initialization: includes initialization performed separately by the intelligent driving area control unit and the human-machine interaction area control unit. When the intelligent driving area control unit performs initialization, due to the existence of a functional safety monitoring mechanism, the initialization speed of the intelligent driving area control unit is slower than that of the human-machine interaction area control unit. Therefore, the human-machine interaction area control unit completes initialization first and enters the operating state.

[0152] S320: After the initialization performed by the human-machine interaction area control unit is completed, receive a synchronization signal or control signal from the human-machine interaction area control unit through the camera interface unit, and send the synchronization signal or control signal to the camera to enable the surround view mode. For this process, please refer to the control method applied to the surround view mode, and the details will not be described again.

[0153] It should be noted that if the initialization of the intelligent driving area control unit has not yet been completed, the intelligent driving area control unit may configure and activate other cameras and camera interface units unrelated to the surround view mode after the intelligent driving area control unit completes the initialization.

[0154] S330: After the initialization performed by the intelligent driving area control unit is completed, receive a synchronization signal or control signal of the intelligent driving area control unit through the camera interface unit, send the synchronization signal or control signal to the camera, and take over control of the camera interface unit or the camera based on the aforementioned priority reason.

[0155] The above startup method can be applied to the normal startup scenario of the vehicle, so that when the vehicle is powered on, the surround view mode is enabled quickly first, and then, when the vehicle is normal after powering on, the intelligent driving area control unit with a high safety level takes over the control of the camera interface unit and the camera. To ensure high safety, the above-mentioned priority control and takeover based switching is implemented.

[0156] Another embodiment of the present application provides a control method. For specific implementations of the steps, or the technical problems that can be solved, or the corresponding effects, please refer to the above-mentioned embodiments of the area control system. Here, only a brief description is provided. As shown in the schematic diagrams in Figure 11 or Figure 10B, the control method includes the following steps:

[0157] S410: Receive sensor data through the sensor interface unit.

[0158] In some embodiments, the sensor interface unit and the sensor may be a camera interface unit and a camera for collecting video data outside the vehicle. For other optional embodiments, please refer to the above-mentioned embodiments of the area control system, and the details will not be described again.

[0159] S420: Send the sensor data to the intelligent driving area control unit and the human-machine interaction area control unit.

[0160] The intelligent driving area control unit can make intelligent driving decisions based on the received data, for example, based on video data or other received data. The human-machine interaction area control unit can perform functions related to human-machine interaction based on the received data, for example, based on video data or other received data, such as performing face detection (in sentry mode) and displaying using a display screen (for example, in surround view mode).

[0161] In some embodiments, as shown in FIG. 12, the method further includes the following steps:

[0162] S510: Use two sets of interfaces of the sensor interface unit to separately receive synchronization signals or control signals sent by the intelligent driving area control unit and the human-machine interaction area control unit.

[0163] S520: The sensor interface unit selects either the intelligent driving area control unit or the human-machine interaction area control unit to send a synchronization signal or a control signal to the sensor, the synchronization signal being used as a trigger signal for the sensor or a frame rate control signal for the sensor for video capture, and the control signal being used for configuring the sensor or reading data.

[0164] In some embodiments, selecting one of the intelligent driving area control unit and the human-machine interaction area control unit includes selecting, after receiving a synchronization signal or a control signal of the intelligent driving area control unit, the sensor interface unit to send the synchronization signal or the control signal of the intelligent driving area control unit.

[0165] In some embodiments, the method further includes enabling a synchronization signal transmitted by the intelligent driving area control unit to be time synchronized with a synchronization signal transmitted by the human-machine interaction area control unit.

[0166] In some embodiments, the method comprises: generating, using the intelligent driving region control unit, content to be displayed using the instrument display unit; and using the first display interface unit to transmit the generated content to the instrument display unit for display, the generated content being displayed using the instrument display unit.

[0167] In some embodiments, the method comprises: generating content using a human-machine interaction area control unit to be displayed using the display screen; and transmitting, using the second display interface unit, the generated content to the display screen for display, the generated content being displayed using the display screen.

[0168] Those skilled in the art can realize that, in combination with the examples described in the embodiments disclosed herein, the units and algorithm steps can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether a function is performed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described functions using various methods for each specific application, but this implementation should not be considered to exceed the scope of this application.

[0169] It is clearly understood by those skilled in the art that for the detailed working processes of the aforementioned systems, devices and units, reference should be made to the corresponding processes of the aforementioned method embodiments, and details will not be described again here.

[0170] In some embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the described device embodiments are merely examples. For example, the division into units is merely a logical division of function, and actual implementation may involve other divisions. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. In addition, the shown or discussed mutual couplings or direct couplings or communication connections may be implemented using some interfaces. Indirect couplings or communication connections between devices or units may be implemented in electronic, mechanical, or other forms.

[0171] The units described as separate parts may or may not be physically separate, and the parts shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0172] In addition, the functional units of the embodiments of the present application may be integrated into one processing unit, and each of the units may exist physically alone, or two or more units may be integrated into one unit.

[0173] It should be noted that the above are merely exemplary embodiments and technical principles of the present application. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and those skilled in the art can make various obvious modifications, rearrangements, and substitutions without departing from the protection scope of the present application. Therefore, although the present application has been described in detail using the above embodiments, the present application is not limited to the above embodiments, and can include further other equivalent embodiments without departing from the concept of the present application, all of which fall within the protection scope of the present application.

Claims

1. an intelligent driving range control unit; a human-machine interaction area control unit; a sensor interface unit connected to the intelligent driving area control unit and the human-machine interaction area control unit, the sensor interface unit being connected to sensors and configured to transmit data of the sensors to the intelligent driving area control unit and the human-machine interaction area control unit; a controller system.

2. The sensor interface unit is connected to the intelligent driving area control unit and the human-machine interaction area control unit; The sensor interface unit is respectively connected to the intelligent driving area control unit and the human-machine interaction area control unit through two sets of interfaces, and signals transmitted by each set of interfaces include synchronization signals, control signals, and / or video data stream signals; The system of claim 1 , wherein the synchronization signal is used as a trigger signal for the sensor or as a frame rate control signal for the sensor for video capture, and the control signal is used to configure or read data from the sensor.

3. the sensor interface unit selects one of the intelligent driving region control unit connected to the sensor interface unit and the intelligent driving region control unit to transmit the synchronization signal and / or the control signal; 3. The system of claim 2.

4. The selection is based on the following criteria: The system of claim 3, based on at least one of the priority and safety levels of the intelligent driving area control unit and the human-machine interaction area control unit, the operating mode of the controller system, and the operating state of the controller system.

5. The system according to claim 3 or 4, wherein the synchronization signal transmitted by the intelligent driving area control unit is time-synchronized with the synchronization signal transmitted by the human-machine interaction area control unit.

6. a first display interface unit connected to the intelligent driving region control unit, the first display interface unit being connected to an instrument display unit; or a second display interface unit connected to the human-machine interaction area control unit, the second display interface unit being connected to a display screen; The system of claim 1 , further comprising at least one of:

7. a network switching unit connected to the intelligent driving area control unit and the human-machine interaction area control unit; The system of claim 1 further comprising:

8. further comprising a third area control unit connected to the network switching unit; The system of claim 7 , wherein the third zone control unit is connected to any one of a chassis system control unit, a power system control unit, or a body system control unit.

9. The network switching unit further comprises:

9. The system of claim 7 or 8, configured to connect to at least one of a laser radar sensor, a millimeter wave radar sensor, an event data recorder, a vehicle internet communication box, and an on-board recorder.

10. The intelligent driving area control unit is configured to perform an assisted driving or autonomous driving function, a part of a vehicle control function, or a part of a vehicle body control function; the third area control unit is configured to perform a chassis system control function, a power system control function, another function of the vehicle control, or another function of the vehicle body control; the human-machine interaction area control unit is configured to perform entertainment area application functions or human-machine user interface functions; 10. A system according to any one of claims 7 to 9.

11. The intelligent operating region control unit further comprises:

10. The system of claim 1, configured to connect to at least one of a millimeter wave radar, an ultrasonic radar, and an integrated positioning unit.

12. The system of claim 1 , wherein the human-machine interaction area control unit is further configured to connect to an audio device.

13. The intelligent driving area control unit or the human-machine interaction area control unit comprises: Including an image processing module, a graphics rendering module, a network / video interface module, an artificial intelligence calculation module, and a control module; The system of claim 1 , wherein the control module is configured to perform scheduling and general calculations for other modules.

14. A vehicle control system comprising a controller system according to any one of claims 1 to 13.

15. receiving sensor data through a sensor interface unit; transmitting the data of the sensors to an intelligent driving area control unit and a human-machine interaction area control unit; data processing methods, including

16. selecting one of a connection between the sensor interface unit and the intelligent driving area control unit and a connection between the sensor interface unit and the human-machine interaction area control unit for transmitting one or more of a synchronization signal and a control signal to the sensor; further comprising 16. The method of claim 15, wherein the synchronization signal is used as a trigger signal for the sensor or as a frame rate control signal for the sensor for video capture, and the control signal is used to configure or read data from the sensor.

17. The selection is based on the following criteria:

17. The method of claim 16, based on at least one of the priority and safety levels of the intelligent driving area control unit and the human-machine interaction area control unit, the operating mode of the controller system, and the operating state of the controller system.

18. 18. The method of claim 16 or 17, further comprising: enabling the synchronization signal transmitted by the intelligent driving area control unit to be time-synchronized with the synchronization signal transmitted by the human-machine interaction area control unit.

19. using the intelligent driving region control unit to generate content to be displayed using an instrument display unit; transmitting, using a first display interface unit, the generated content to an instrument display unit for display, the generated content being displayed using the instrument display unit; 16. The method of claim 15, further comprising:

20. generating content using the human-machine interaction area control unit to be displayed using a display screen; transmitting, using a second display interface unit, the generated content to a display screen for display, the generated content being displayed using the display screen; 16. The method of claim 15, further comprising:

21. A control method applied to a controller system according to any one of claims 1 to 13, said method comprising: receiving sensor data through a sensor interface unit; using a human-machine interaction area control unit to perform human proximity detection or intrusion detection based on the data of the sensor; transmitting the data of the sensor to an on-board recorder using a network switching unit when an exception is detected; A method comprising:

22. If the exception is detected, the method comprises: generating alert data using the human-machine interaction area control unit; sending the alert data to an audio device for playback or using a second display interface unit to send the alert data to a display screen for display; 22. The method of claim 21, further comprising:

23. A control method applied to a controller system according to any one of claims 1 to 13, said method comprising: receiving sensor data through a sensor interface unit; performing image processing based on the data of the sensors using a human-machine interaction area control unit to generate surround view image data; transmitting the surround view image data to a display screen for display using a second display interface unit; A method comprising:

24. 14. A system startup method adapted for a controller system according to any one of claims 1 to 13, said method comprising: performing initialization separately by an intelligent driving area control unit and a human-machine interaction area control unit, wherein the speed at which the intelligent driving area control unit performs initialization is slower than the speed at which the human-machine interaction area control unit performs initialization; After the initialization performed by the human-machine interaction area control unit is completed, receiving a synchronization signal or a control signal from the human-machine interaction area control unit through a sensor interface unit to enable a surround view mode; and transmitting the synchronization signal or the control signal to a sensor. receiving a synchronization signal or a control signal of the intelligent driving region control unit through the sensor interface unit to assume control over the sensor interface unit or the sensor after the initialization performed by the intelligent driving region control unit is completed; and transmitting the synchronization signal or the control signal to the sensor. A method comprising:

Images