Data processing device, method, storage medium and vehicle for assisted driving

Abstract

The present invention relates to a data processing device, method, storage medium and vehicle for assisted driving. The data processing device for assisted driving includes: a data processing system configured to receive and process a plurality of vehicle data; and a processing control system configured to perform degradation control in the event that the data processing system is degraded, so as to receive and process the plurality of vehicle data in a degraded mode. According to one or more embodiments of the present invention, the data processing device, method, storage medium and vehicle for assisted driving can, by reasonably dividing the work and performing degradation control on the data processing system and the processing control system, promptly and effectively allocate other modules to take over the work when some modules in the data processing device are not working, and achieve a compromise between system complexity, operation complexity, data delay, cost and backup effect, and driving safety.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology. Specifically, the present invention relates to a data processing device, method, storage medium and vehicle for assisted driving. Background Art

[0002] As vehicles become increasingly intelligent, assisted driving features such as ADAS (Advanced Driver Assistance Systems) are being incorporated into an increasing number of vehicles. As assisted driving evolves, the number and complexity of sensors and other equipment installed on vehicles is increasing to provide users with a more automated, safer, and more convenient driving experience.

[0003] Processing the data collected by these devices for assisted driving requires extensive computation, placing higher demands on the vehicle's data processing capabilities. Furthermore, given the potential for malfunction or partial failure of the modules used for assisted driving, safety is also a key consideration in vehicle design. Summary of the Invention

[0004] According to one aspect of the present invention, a data processing device for assisted driving is provided, which includes: a data processing system configured to receive and process multiple vehicle data; and a processing control system configured to perform degradation control in the event that the data processing system is degraded, so as to receive and process the multiple vehicle data in a degraded mode.

[0005] As an alternative or supplement to the above-mentioned scheme, in a data processing device for assisted driving according to an embodiment of the present invention, the data processing system includes a first data processing module, a second data processing module, a third data processing module and a fourth data processing module; wherein the first data processing module and the second data processing module are configured to receive the same first vehicle data and second vehicle data; the third data processing module is configured to receive third vehicle data and second vehicle data; and the fourth data processing module is configured to receive fourth vehicle data and second vehicle data.

[0006] As an alternative or supplement to the above-mentioned scheme, in a data processing device for assisted driving according to an embodiment of the present invention, the first vehicle data is one or more of camera data, global navigation satellite system (GNSS) data and inertial measurement unit (IMU) data; the second vehicle data is one or more of universal serial interface (USS) data, millimeter wave radar data, brake control unit (BCU) data and vehicle control unit (VCU) data; the third vehicle data is vehicle gateway data; and the fourth vehicle data is lidar data.

[0007] As an alternative or supplement to the above-mentioned solution, in a data processing device for assisted driving according to an embodiment of the present invention, the third data processing module is also configured to communicate with the vehicle gateway via the first gateway; and the fourth data processing module is also configured to receive fourth vehicle data via the second gateway.

[0008] As an alternative or supplement to the above-mentioned scheme, in a data processing device for assisted driving according to an embodiment of the present invention, the processing control system is further configured to perform the following downgraded control: when any one of the first data processing module and the second data processing module is not working, the other of the first data processing module and the second data processing module is configured to work instead; when any one of the third data processing module and the fourth data processing module is not working, the other of the third data processing module and the fourth data processing module is configured to work instead; when both the first data processing module and the second data processing module are not working, at least one of the third data processing module and the fourth data processing module is configured for assisted driving; when both the third data processing module and the fourth data processing module are not working, at least one of the first data processing module and the second data processing module is configured for assisted driving; and when none of the first to fourth data processing modules are working, assisted driving is performed.

[0009] As an alternative or supplement to the above-mentioned scheme, in a data processing device for assisted driving according to an embodiment of the present invention, the first vehicle data is transmitted via a Gigabit Serial Multimedia Link (GSML) deserializer; the second vehicle data is transmitted via a vehicle controller area network (CAN); and the third vehicle data and the fourth vehicle data are transmitted via Ethernet (ETH).

[0010] As an alternative or supplement to the above-mentioned scheme, in a data processing device for assisted driving according to an embodiment of the present invention, the first data processing module, the second data processing module, the third data processing module and the fourth data processing module are systems on a chip (SoC) and communicate through the high-speed serial computer expansion bus standard (PCIe); and the processing control system is a micro control unit (MCU) or a domain controller (DCU).

[0011] As an alternative or supplement to the above solution, in a data processing device for assisted driving according to an embodiment of the present invention, the processing control system is further configured to: transmit the second vehicle data to each data processing module in the data processing system.

[0012] As an alternative or supplement to the above solution, the data processing device for assisted driving according to an embodiment of the present invention further includes: a storage module for storing one or more of the first to fourth vehicle data.

[0013] According to another aspect of the present invention, a data processing method for assisted driving is provided, comprising: receiving and processing a plurality of vehicle data by a data processing system; and in the event that the data processing system is degraded, performing degradation control by a processing control system to receive and process the plurality of vehicle data in a degraded mode.

[0014] As an alternative or supplement to the above-mentioned scheme, in a data processing method for assisted driving according to an embodiment of the present invention, the data processing system includes a first data processing module, a second data processing module, a third data processing module and a fourth data processing module; and the data processing method for assisted driving also includes: the first data processing module and the second data processing module receive the same first vehicle data and second vehicle data; the third data processing module receives the third vehicle data and the second vehicle data; and the fourth data processing module receives the fourth vehicle data and the second vehicle data.

[0015] As an alternative or supplement to the above-mentioned scheme, in a data processing method for assisted driving according to an embodiment of the present invention, the first vehicle data is one or more of camera data, global navigation satellite system (GNSS) data and inertial measurement unit (IMU) data; the second vehicle data is one or more of universal serial interface (USS) data, millimeter wave radar data, brake control unit (BCU) data and vehicle control unit (VCU) data; the third vehicle data is vehicle gateway data; and the fourth vehicle data is lidar data.

[0016] As an alternative or supplement to the above solution, the data processing method for assisted driving according to an embodiment of the present invention also includes: the third data processing module communicates with the vehicle gateway via the first gateway; and the fourth data processing module receives fourth vehicle data via the second gateway.

[0017] As an alternative or supplement to the above-mentioned scheme, in a data processing method for assisted driving according to an embodiment of the present invention, degradation control includes: when any one of the first data processing module and the second data processing module is not working, the other of the first data processing module and the second data processing module is used instead to work; when any one of the third data processing module and the fourth data processing module is not working, the other of the third data processing module and the fourth data processing module is used instead to work; when both the first data processing module and the second data processing module are not working, at least one of the third data processing module and the fourth data processing module is used for assisted driving; when both the third data processing module and the fourth data processing module are not working, at least one of the first data processing module and the second data processing module is used for assisted driving; and when none of the first to fourth data processing modules are working, the processing control system is used for assisted driving.

[0018] As an alternative or supplement to the above-mentioned scheme, the data processing method for assisted driving according to an embodiment of the present invention also includes: transmitting the first vehicle data through a Gigabit Serial Multimedia Link (GSML) deserializer; transmitting the second vehicle data through a vehicle controller area network (CAN); and transmitting the third vehicle data and the fourth vehicle data through Ethernet (ETH).

[0019] As an alternative or supplement to the above-mentioned scheme, in a data processing method for assisted driving according to an embodiment of the present invention, the first data processing module, the second data processing module, the third data processing module and the fourth data processing module are systems on a chip (SoC) and communicate through the high-speed serial computer expansion bus standard (PCIe); and the processing control system is a micro control unit (MCU) or a domain controller (DCU).

[0020] As an alternative or supplement to the above solution, the data processing method for assisted driving according to an embodiment of the present invention further includes: receiving the second vehicle data by each data processing module in the data processing system via the processing control system.

[0021] As an alternative or supplement to the above solution, the data processing method for assisted driving according to an embodiment of the present invention further includes: storing one or more of the first to fourth vehicle data by a storage module.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, on which program instructions executable by a processor are stored. When the program instructions are executed by the processor, the data processing method for assisted driving according to any embodiment of one aspect of the present invention is executed.

[0023] According to yet another aspect of the present invention, a vehicle is provided, comprising the data processing device for assisted driving according to any one embodiment of one aspect of the present invention.

[0024] According to one or more embodiments of the present invention, the data processing device, method, storage medium and vehicle for assisted driving can, through reasonable division of labor and degradation control of the data processing system and the processing control system, promptly and effectively allocate other modules to take over the work when some modules in the data processing device are not working, and achieve a compromise between system complexity, operation complexity, data delay, cost and backup effect, and driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and / or other aspects and advantages of the present invention will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar elements are represented by the same reference numerals. The drawings include:

[0026] Figure 1 is a schematic structural diagram of a data processing device for assisted driving according to an embodiment of the present invention; and

[0027] Figure 2 FIG. 2 is a flow chart of a data processing method 200 for assisted driving according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In this specification, the present invention is described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments of the present invention. However, the present invention may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided to make this disclosure thorough and complete, and to more fully convey the scope of the present invention to those skilled in the art.

[0029] Terms such as "comprising" and "including" indicate that, in addition to the units and steps directly and explicitly stated in the specification and claims, the technical solution of the present invention does not exclude the presence of other units and steps not directly or explicitly stated. Terms such as "first" and "second" do not indicate the order of units in terms of time, space, size, etc., but are merely used to distinguish between units.

[0030] The present invention is described hereinafter with reference to the flow chart explanation, block diagram and / or flow chart of the method and equipment according to an embodiment of the present invention.It will be understood that each frame of these flow chart explanations and / or block diagrams and the combination of flow chart explanations and / or block diagrams can be realized by computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer, special-purpose computer or other programmable data processing equipment to form a machine, so that these instructions performed by the processor of computer or other programmable data processing equipment create the parts for implementing the function / operation specified in these flow charts and / or frame and / or one or more flow chart blocks.It should also be noted that in some alternative implementations, the function / operation shown in the frame can not occur in the order shown in the flow chart.For example, the two frames shown in sequence can actually be performed substantially simultaneously or these frames can sometimes be performed in reverse order, specifically depending on the function / operation involved.

[0031] Where applicable, hardware, software, or a combination of hardware and software may be used to implement the various embodiments provided by the present disclosure. Additionally, where applicable, without departing from the scope of the present disclosure, the various hardware components and / or software components set forth herein may be combined into composite components comprising software, hardware, and / or both. Where applicable, without departing from the scope of the present disclosure, the various hardware components and / or software components set forth herein may be separated into subcomponents comprising software, hardware, or both. Additionally, where applicable, it is contemplated that software components may be implemented as hardware components, and vice versa.

[0032] Now refer to Figure 1 , Figure 1 The figure shows a schematic structural diagram of a data processing device for assisted driving according to an embodiment of the present invention. Modern intelligent vehicle sensor architecture has gradually shifted from a distributed architecture to a domain-centralized architecture. In the domain-centralized architecture, five domain controllers DCU are usually used to control the five domains of the vehicle - the power domain, the chassis domain, the cockpit domain / intelligent information domain, the autonomous driving domain, and the body domain. In one embodiment, the device is used as a domain control module of the autonomous driving domain of an intelligent vehicle, so as to control the autonomous driving domain in the domain-centralized intelligent vehicle. In general, the data processing device for assisted driving receives data from various sensors and other parts of the vehicle, and processes and analyzes the data, thereby completing functions such as perception fusion, planning decision control, mapping and positioning, and redundant backup to provide instructions that can be used for assisted driving.

[0033] like Figure 1 As shown in FIG, the data processing device for assisting driving includes a data processing system 110 and a processing control system 120 (respectively composed of Figure 1The data processing system 110 may be configured to receive and process a plurality of vehicle data, such as from Figure 1 Data from various sensors and components on both sides of the vehicle can be processed. The processing control system 120 can be configured to perform degradation control in the event of degradation of the data processing system 110, thereby receiving and processing multiple vehicle data in a degraded mode. Degradation refers to the inoperability of part or all of the data processing system 110 due to various reasons, including but not limited to passive degradation due to a malfunction, and active degradation to adapt to different vehicle models.

[0034] The data processing system 110 may include a first data processing module 1101, a second data processing module 1102, a third data processing module 1103, and a fourth data processing module 1104. Where applicable, the third data processing module 1103 and the fourth data processing module 1104 may also be combined into one data processing module. In one embodiment, the first to fourth data processing modules 1101-1104 may be system-on-chip (SoCs). Specifically, the system-on-chip (SoC) may employ a chip with high computing power, such as the Orin system-on-chip (SoC) from NVIDIA, which may operate at a frequency of up to 200 trillion operations per second. The first to fourth data processing modules 1101-1104 may also exchange data via a high-bandwidth data transmission channel, for example, communicating via the high-speed Serial Computer Interconnect Express (PCIe) bus standard to achieve interconnection and interoperability. The data processing system 110 may also be configured to store the various received data in one or more storage modules 130 for possible future use.

[0035] The processing control system 120 can be a conventional microcontroller unit (MCU) or a domain controller (DCU) in the aforementioned domain control module, also known as a domain control unit. Compared with the data processing system 110, the processing control system 120 has weaker computing power and can usually only perform data processing of conventional types and amounts. Figure 1 FIG2 shows two process control modules 1201 and 1202 in the process control system 120 , but more or fewer process control modules are also feasible. The multiple process control modules in the process control system 120 can communicate with each other via a vehicle controller area network (CAN), for example.

[0036] The processing and control system 120 is used to receive the second vehicle data, perform format conversion or routine content processing on the second vehicle data, and then transmit it to the various data processing modules in the data processing system 110. The second vehicle data is usually traditional vehicle sensor data, including but not limited to universal serial interface (USS) data, millimeter wave radar data, brake control unit (BCU) data, and vehicle control unit (VCU) data. These traditional vehicle sensor data have been commonly used for vehicle control, and due to the characteristics of traditional vehicle sensor data such as small data volume and simple calculation, they can be directly received and processed by the processing and control system 120. Therefore, the second vehicle data can usually be transmitted using the vehicle CAN.

[0037] Continue to refer Figure 1 , a data processing device for assisted driving, in particular the data processing system 110, is described in more detail. As described above, the data processing system 110 may include a first data processing module 1101, a second data processing module 1102, a third data processing module 1103 and a fourth data processing module 1104. In one embodiment, the first data processing module 1101 and the second data processing module 1102 may be configured to receive the same first vehicle data and the second vehicle data as described above. The first vehicle data includes but is not limited to camera data, global navigation satellite system (GNSS) data and inertial measurement unit (IMU) data. Among them, GNSS data can be received from the outside world through a vehicle-mounted antenna. The GNSS data mainly includes the positioning data of the object where the device for receiving GNSS data is located. Speed ​​data, direction data, etc. can also be obtained by simple calculation of the original GNSS data.

[0038] like Figure 1 As shown in , when a vehicle is equipped with N cameras, namely camera 1, camera 2, ..., camera N, the camera data may be data from the N cameras. For example, the cameras may be installed at the front, rear, or side of the vehicle, and may have different shooting fields of view such as 60°, 120°, and 180°. The first data processing module 1101 and the second data processing module 1102 may perform fusion processing on the first vehicle data (e.g., camera data from multiple cameras), including pre-processing, correction, alignment, association, importance assessment, prediction, and other operations, for use in assisted driving functions.

[0039] Since the camera and the first and second data processing modules 1101 and 1102 usually need to be connected through a device such as a deserializer (for example, a Gigabit Serial Multimedia Link (GSML) deserializer) to perform operations such as generating camera synchronization signals, it is relatively cumbersome. Therefore, in this embodiment, the camera data can be provided to the first data processing module 1101 and the second data processing module 1102 for mutual backup, but not to the third data processing module 1103 and the fourth data processing module 1104, so as to achieve a compromise between system complexity, operational complexity, data latency, cost and backup effect, and driving safety. However, this does not rule out the technical solution of connecting all vehicle data to each data processing module.

[0040] In one embodiment, a specific division of labor for data processing is performed between the third data processing module 1103 and the fourth data processing module 1104. For example, the third data processing module 1103 is configured to receive third vehicle data, such as vehicle gateway data, as well as the aforementioned second vehicle data, and can communicate and interact with the vehicle gateway, for example, via the first gateway. The fourth data processing module 1104 is configured to receive fourth vehicle data (e.g., LiDAR data) as well as the aforementioned second vehicle data, for example, via the second gateway. In other words, from the perspective of vehicle data, the second vehicle data, including traditional vehicle sensor data, is actually transmitted via the CAN bus to each of the first to fourth data processing modules 1101-1104 in the data processing system 110, as well as to the processing and control system 120. Data is then communicated between the various modules in the data processing system 110 and the processing and control system 120. Therefore, as long as any module in the data processing system 110 and the processing and control system 120 is still operational, at least the second vehicle data can be received and processed in the event of degradation, thereby implementing basic assisted driving functions and ensuring vehicle safety to a certain extent. The operation during degradation will be described in detail below.

[0041] In addition, the first gateway and the second gateway can also communicate and exchange data. Due to the large amount of data in the vehicle gateway data and the lidar data, it is more appropriate to use Ethernet (ETH) to transmit such data between the first gateway, the second gateway, the processing control system 120, and the data processing system 110, thereby achieving faster data transmission.

[0042] Degradation control is described below. Degradation occurs when any one or more modules in the data processing system 110 or the processing control system 120 fail and stop working, or when any one or more modules are configured to stop working to adapt to different vehicle models.

[0043] Degradation control one (degradation mode one): When any one of the first data processing module 1101 and the second data processing module 1102 is not working, the other one of the first data processing module 1101 and the second data processing module 1102 is configured to work instead. As described above, the first data processing module 1101 and the second data processing module 1102 access the same vehicle data, including the first vehicle data and the second vehicle data, so the two are consistent in data type and quantity. Normally, the contents of the first and second data processing modules 1101 and 1102 remain consistent, but only one of them performs data processing work; when the working data processing module is not working, another processing module can be started in time to take over the data processing work, for example, so that the fusion work of the camera data can continue. In this way, the normal operation of the function using the camera data processing results can be ensured.

[0044] Degraded Control 2 (Degraded Mode 2): If either the third or fourth data processing module 1103 , 1104 , is inoperative, the other is configured to operate in its place. As previously mentioned, since the third data processing module 1103 is configured to receive the second vehicle data and communicate with the vehicle gateway, and the fourth data processing module 1104 is configured to receive the second vehicle data and lidar data, similar to the first and second data processing modules 1101 , 1102 , if either the third or fourth data processing module 1103 , 1104 , is inoperative, the other can continue to operate, for example, utilizing millimeter-wave radar data, BCU data, VCU data, and so on to implement basic assisted driving functions. If either the third or fourth data processing module 1103 , 1104 , is inoperative, data from both processing control modules 1201 and 1202 can be transferred to the other data processing module, thereby processing and utilizing each of the second through fourth vehicle data.

[0045] On the other hand, unlike the first and second data processing modules 1101 and 1102, in addition to receiving the second vehicle data, the third data processing module 1103 also communicates with the vehicle gateway via the first gateway, while the fourth data processing module 1104 also receives lidar data via the second gateway. Although data transmission between the first and second gateways can be performed via Ethernet (ETH), the third and fourth data processing modules 1103 and 1104 can also preferably process the data they receive directly, passing only necessary data through the gateway to reduce operational complexity.

[0046] Degradation Control Three (Degradation Mode Three): When both the first data processing module 1101 and the second data processing module 1102 are inoperative, at least one of the third data processing module 1103 and the fourth data processing module 1104 is configured for assisted driving. As described above, to achieve a compromise between various aspects, the first vehicle data can be connected to the first and second data processing modules 1101 and 1102. Therefore, when both 1101 and 1102 are inoperative, the first vehicle data may no longer be utilized. In this case, one or more of the third and fourth data processing modules 1103 and 1104 can be utilized to continue providing data processing support for the assisted driving function. For example, when both the third and fourth data processing modules 1103 and 1104 are operating normally, they can be used to analyze and process one or more of the second to fourth vehicle data connected thereto (e.g., lidar data and millimeter-wave radar data). When only the third data processing module 1103 is used, it can be configured to process only the second and third vehicle data directly connected thereto, or it can also process the fourth vehicle data transmitted via Ethernet. Similarly, when only the fourth data processing module 1104 is used, it can be configured to process only the second and fourth vehicle data directly connected thereto, or it can also process the third vehicle data transmitted via Ethernet. Furthermore, if the hardware processing power is insufficient, it can be configured to process only the second vehicle data to implement basic assisted driving functions.

[0047] Degradation control four (degradation mode four): When both the third data processing module 1103 and the fourth data processing module 1104 are not working, at least one of the first data processing module 1101 and the second data processing module 1102 is configured for assisted driving. When both the third and fourth data processing modules 1103 and 1104 are not working, one or more of the first and second data processing modules 1101 and 1102 can be used to continue to provide data processing support for the assisted driving function. For example, as described above, usually when both the first and second data processing modules 1101 and 1102 are working normally, one of them can be used to analyze and process the first and second vehicle data received. When one of the first and second data processing modules 1101 and 1102 is not working, the other is used to work in place. In one embodiment, the first and second data processing modules 1101 and 1102 can only process one of the first and second vehicle data for the assisted driving function.

[0048] Degraded Control Five (Degraded Mode Five): When the first through fourth data processing modules 1101-1104 are all inoperative, assisted driving is performed by the processing and control system 120. In traditional assisted driving, at least a millimeter-wave radar and an MCU are required to implement basic assisted driving functions (such as basic automatic vehicle following). Therefore, when the entire data processing system 110 is inoperative, the processing and control system 120 can be used for relatively basic assisted driving functions.

[0049] This article lists five degradation control operations, but different degradation controls can also be used based on different configurations and different requirements of the vehicle's autonomous driving domain, not limited to the above five control methods.

[0050] Next reference Figure 2 Describe a data processing method 200 for assisted driving, Figure 2 A flow chart of a data processing method 200 for assisted driving according to an embodiment of the present invention is shown.

[0051] Method 200 according to one or more embodiments of the present invention can be used for domain control of the autonomous driving domain of an intelligent vehicle. Generally speaking, in data processing method 200 for assisted driving, data is received from various sensors and other components of the vehicle, and the data is processed and analyzed to perform functions such as perception fusion, planning and decision control, mapping and positioning, and redundant backup, thereby providing instructions that can be used for assisted driving.

[0052] In step S201, the data processing system 110 may receive and process a plurality of vehicle data, such as data from various sensors and components. In step S202, if the data processing system 110 is degraded, the processing control system 120 may perform degraded control to receive and process the plurality of vehicle data in a degraded mode.

[0053] The data processing system 110 may include a first data processing module 1101, a second data processing module 1102, a third data processing module 1103, and a fourth data processing module 1104. Where applicable, the third data processing module 1103 and the fourth data processing module 1104 may be combined into one data processing module. The first to fourth data processing modules 1101-1104 may also communicate via a high-bandwidth data transmission channel, such as a high-speed PCIe bus standard, to achieve interconnection and interoperability. The method 200 further includes storing the various received data in one or more storage modules 130 for possible future use.

[0054] The processing control system 120 can be used to receive the second vehicle data, and the second vehicle data can be converted in format or processed in a conventional manner before being transmitted to the various data processing modules in the data processing system 110. The second vehicle data is typically traditional vehicle sensor data, including but not limited to universal serial interface (USS) data, millimeter wave radar data, brake control unit (BCU) data, and vehicle control unit (VCU) data. As described above, the processing control system 120 can be used to directly receive and process the second vehicle data, and the vehicle CAN can typically be used to transmit the second vehicle data. Method 200 can also include, for example, communicating between multiple processing control modules in the processing control system 120 via the vehicle CAN.

[0055] The data processing system 110 may include a first data processing module 1101, a second data processing module 1102, a third data processing module 1103, and a fourth data processing module 1104. In step S201, the first data processing module 1101 and the second data processing module 1102 may be used to receive the same first vehicle data and the second vehicle data as described above. The first vehicle data includes, but is not limited to, camera data, global navigation satellite system (GNSS) data, and inertial measurement unit (IMU) data. GNSS data may be received from the outside world via a vehicle-mounted antenna. GNSS data primarily includes positioning data of an object where a device receiving GNSS data is located. Simple calculations may also be performed on the raw GNSS data to obtain speed data, direction data, and the like.

[0056] When a vehicle is equipped with N cameras, namely, camera 1, camera 2, ..., camera N, the camera data may be data from each of the N cameras. For example, the cameras may be installed on the front, rear, or side of the vehicle and may have different shooting fields of view, such as 60°, 120°, or 180°. The first data processing module 1101 and the second data processing module 1102 may be used to perform fusion processing on the first vehicle data (e.g., camera data from multiple cameras), including preprocessing, correction, alignment, correlation, importance assessment, prediction, and other operations, for use in assisted driving functions.

[0057] Since it is usually necessary to connect the camera to the first and second data processing modules 1101 and 1102 through a device such as a deserializer (for example, a Gigabit Serial Multimedia Link (GSML) deserializer) to perform operations such as generating camera synchronization signals, it is relatively cumbersome. Therefore, in this embodiment, the camera data can be provided to the first data processing module 1101 and the second data processing module 1102 for mutual backup, but not to the third data processing module 1103 and the fourth data processing module 1104, so as to achieve a compromise between system complexity, operational complexity, data latency, cost and backup effect, and driving safety. However, this does not rule out the technical solution of connecting all vehicle data to each data processing module.

[0058] In one embodiment, a specific division of labor for data processing is performed on the third data processing module 1103 and the fourth data processing module 1104. For example, the third data processing module 1103 can be used to receive third vehicle data such as vehicle gateway data and the above-mentioned second vehicle data in step S201, and can communicate and interact with the vehicle gateway via the first gateway, for example. The fourth data processing module 1104 can be used to receive fourth vehicle data (e.g., LiDAR data) and the above-mentioned second vehicle data in step S201, for example, via the second gateway. In other words, from the perspective of vehicle data, the second vehicle data including traditional vehicle sensor data is actually sent to each of the first to fourth data processing modules 1101-1104 in the data processing system 110 via the CAN bus, and is sent to the processing control system 120, and data is communicated between the various modules in the data processing system 110 and the processing control system 120. Therefore, as long as any one module in the data processing system 110 and the processing control system 120 is still able to work, at least the second vehicle data can be received and processed when degradation occurs, so as to realize basic assisted driving functions and ensure vehicle safety to a certain extent. The operations during degradation will be described in detail below.

[0059] In addition, communication and data exchange can also be performed between the first gateway and the second gateway. Since the volume of vehicle gateway data and lidar data is large, using Ethernet (ETH) to transmit such data between the first gateway, the second gateway, the processing control system 120, and the data processing system 110 can achieve faster data transmission.

[0060] Next, we will explain degradation control. Degradation occurs when one or more modules in the data processing system 110 or the processing control system 120 malfunction or become inoperative, or when one or more modules are configured to be inoperative to accommodate different vehicle models. In the event of degradation, the following degradation control can be executed in step S202.

[0061] Degradation control one (degradation mode one): When any one of the first data processing module 1101 and the second data processing module 1102 is not working, the other one of the first data processing module 1101 and the second data processing module 1102 is used to work instead. As mentioned above, the first data processing module 1101 and the second data processing module 1102 access the same vehicle data, including the first vehicle data and the second vehicle data, so the two are consistent in data type and quantity. Normally, the contents of the first and second data processing modules 1101 and 1102 remain consistent, but only one of them is used for data processing work; when the working data processing module is not working, another processing module can be started in time to take over the data processing work, for example, so that the fusion work of the camera data can continue. In this way, the normal operation of the function using the processing results of the camera data can be ensured.

[0062] Degraded Control 2 (Degraded Mode 2): If either the third or fourth data processing module 1103 , 1104 is inoperative, the other one is used to perform its operations. On one hand, as previously described, since the third data processing module 1103 is configured to receive the second vehicle data and communicate with the vehicle gateway, and the fourth data processing module 1104 is configured to receive the second vehicle data and receive lidar data, similar to the first and second data processing modules 1101 , 1102 , if either the third or fourth data processing module 1103 , 1104 is inoperative, the other one can continue to operate, for example, utilizing millimeter-wave radar data, BCU data, VCU data, etc. to implement basic assisted driving functions. If either the third or fourth data processing module 1103 , 1104 is inoperative, data from both processing control modules 1201 and 1202 can be transferred to the other data processing module, thereby processing and utilizing each of the second through fourth vehicle data.

[0063] On the other hand, unlike the first and second data processing modules 1101 and 1102, in addition to receiving the second vehicle data, the third data processing module 1103 also communicates with the vehicle gateway via the first gateway, while the fourth data processing module 1104 also receives lidar data via the second gateway. Although data transmission between the first and second gateways can be performed via Ethernet (ETH), the third and fourth data processing modules 1103 and 1104 can also preferably process the data they receive directly, passing only necessary data through the gateway to reduce operational complexity.

[0064] Degradation Control Three (Degradation Mode Three): When both the first data processing module 1101 and the second data processing module 1102 are inoperative, at least one of the third data processing module 1103 and the fourth data processing module 1104 is configured for assisted driving. As described above, to achieve a compromise between various aspects, the first vehicle data can be connected to the first and second data processing modules 1101 and 1102. Therefore, when both 1101 and 1102 are inoperative, the first vehicle data may no longer be utilized. In this case, one or more of the third and fourth data processing modules 1103 and 1104 can be utilized to continue providing data processing support for the assisted driving function. For example, when both the third and fourth data processing modules 1103 and 1104 are operating normally, they can be used to analyze and process one or more of the second to fourth vehicle data connected thereto (e.g., lidar data and millimeter-wave radar data). When only the third data processing module 1103 is used, it can be configured to process only the second and third vehicle data directly connected thereto, or it can also process the fourth vehicle data transmitted via Ethernet. Similarly, when only the fourth data processing module 1104 is used, it can be configured to process only the second and fourth vehicle data directly connected thereto, or it can also process the third vehicle data transmitted via Ethernet. Furthermore, if the hardware processing power is insufficient, it can be configured to process only the second vehicle data to implement basic assisted driving functions.

[0065] Degradation control four (degradation mode four): When both the third data processing module 1103 and the fourth data processing module 1104 are not working, at least one of the first data processing module 1101 and the second data processing module 1102 is configured for assisted driving. When both the third and fourth data processing modules 1103 and 1104 are not working, one or more of the first and second data processing modules 1101 and 1102 can be used to continue to provide data processing support for the assisted driving function. For example, as described above, usually when both the first and second data processing modules 1101 and 1102 are working normally, one of them can be used to analyze and process the first and second vehicle data received. When one of the first and second data processing modules 1101 and 1102 is not working, the other is used to work in place. In one embodiment, the first and second data processing modules 1101 and 1102 can only process one of the first and second vehicle data for the assisted driving function.

[0066] Degraded Control Five (Degraded Mode Five): When the first through fourth data processing modules 1101-1104 are all inoperative, the processing and control system 120 is used for assisted driving. In traditional assisted driving, at least a millimeter-wave radar and an MCU are required to implement basic assisted driving functions (e.g., basic automatic vehicle following). Therefore, when the entire data processing system 110 is inoperative, the processing and control system 120 can be used for relatively basic assisted driving functions.

[0067] This article lists five degradation control operations, but different degradation controls can also be used based on different configurations and different requirements of the vehicle's autonomous driving domain, not limited to the above five control methods.

[0068] According to another aspect of the present invention, a computer-readable storage medium is provided, on which program instructions executable by a processor are stored. When the program instructions are executed by the processor, the data processing method 200 for assisted driving according to any embodiment of one aspect of the present invention is executed.

[0069] According to yet another aspect of the present invention, a vehicle is provided, comprising the data processing device for assisted driving according to any one embodiment of one aspect of the present invention.

[0070] The foregoing disclosure is not intended to limit the present disclosure to the precise form disclosed or to the particular field of use. Therefore, it is contemplated that various alternative embodiments and / or modifications of the present disclosure are possible in light of the present disclosure, whether explicitly described or implied herein. While embodiments of the present disclosure have been described thus, those skilled in the art will recognize that changes may be made in form and detail without departing from the scope of the present disclosure. Therefore, the present disclosure is limited only by the claims.

Claims

1. A data processing device for assisted driving, comprising: a data processing system configured to receive and process a plurality of vehicle data; as well as a processing control system configured to perform degradation control in the event that the data processing system is degraded, so as to receive and process the plurality of vehicle data in a degraded mode, Wherein, the data processing system includes a first data processing module, a second data processing module, a third data processing module and a fourth data processing module; wherein the first data processing module and the second data processing module are configured to receive the same first vehicle data and second vehicle data; The third data processing module is configured to receive third vehicle data and the second vehicle data; and The fourth data processing module is configured to receive fourth vehicle data and the second vehicle data, Wherein, the process control system is further configured to perform the following degradation control: When any one of the first data processing module and the second data processing module is not working, configuring the other one of the first data processing module and the second data processing module to work instead; When any one of the third data processing module and the fourth data processing module is not working, configuring the other one of the third data processing module and the fourth data processing module to work instead; When both the first data processing module and the second data processing module are not working, configuring at least one of the third data processing module and the fourth data processing module to be used for assisted driving; When both the third data processing module and the fourth data processing module are inoperative, configuring at least one of the first data processing module and the second data processing module for assisted driving; and When the first to fourth data processing modules are all inoperative, assisted driving is performed.

2. The device according to claim 1, wherein The first vehicle data is one or more of camera data, global navigation satellite system (GNSS) data, and inertial measurement unit (IMU) data; The second vehicle data is one or more of universal serial interface USS data, millimeter wave radar data, brake control unit BCU data, and vehicle control unit VCU data; The third vehicle data is vehicle gateway data; and The fourth vehicle data is lidar data.

3. The device according to claim 1, wherein The third data processing module is further configured to communicate with the vehicle gateway via the first gateway; and The fourth data processing module is further configured to receive the fourth vehicle data via a second gateway.

4. The device according to claim 1, wherein The first vehicle data is transmitted via a Gigabit Serial Multimedia Link (GSML) deserializer; The second vehicle data is transmitted via a vehicle controller area network (CAN); and The third vehicle data and the fourth vehicle data are transmitted via Ethernet ETH.

5. The device according to claim 1, wherein The first data processing module, the second data processing module, the third data processing module and the fourth data processing module are system-on-chip (SoC) and communicate via a high-speed serial computer expansion bus standard PCIe; and The processing control system is a micro control unit MCU or a domain controller DCU.

6. The device according to claim 1, wherein The process control system is further configured to: The second vehicle data is transmitted to each data processing module in the data processing system.

7. The apparatus according to claim 1, further comprising: A storage module is used to store one or more of the first to fourth vehicle data.

8. A data processing method for assisted driving, comprising: receiving and processing a plurality of vehicle data by a data processing system; as well as In the event that the data processing system is degraded, a processing control system performs degraded control to receive and process the plurality of vehicle data in a degraded mode, wherein the data processing system includes a first data processing module, a second data processing module, a third data processing module, and a fourth data processing module; and the method further includes: The first data processing module and the second data processing module receive the same first vehicle data and second vehicle data; The third data processing module receives the third vehicle data and the second vehicle data; and The fourth data processing module receives the fourth vehicle data and the second vehicle data, The degradation control includes: When any one of the first data processing module and the second data processing module does not work, the other one of the first data processing module and the second data processing module takes over the work; When any one of the third data processing module and the fourth data processing module does not work, the other one of the third data processing module and the fourth data processing module takes over the work; When both the first data processing module and the second data processing module are not working, at least one of the third data processing module and the fourth data processing module is used for assisting driving; When both the third data processing module and the fourth data processing module are not working, using at least one of the first data processing module and the second data processing module for assisted driving; and When the first to fourth data processing modules are not working, the processing control system is used for assisted driving.

9. The method according to claim 8, wherein The first vehicle data is one or more of camera data, global navigation satellite system (GNSS) data, and inertial measurement unit (IMU) data; The second vehicle data is one or more of universal serial interface USS data, millimeter wave radar data, brake control unit BCU data, and vehicle control unit VCU data; The third vehicle data is vehicle gateway data; and The fourth vehicle data is lidar data.

10. The method according to claim 8, further comprising: The third data processing module communicates with the vehicle gateway via the first gateway; as well as The fourth vehicle data is received by the fourth data processing module via the second gateway.

11. The method according to claim 8, further comprising: transmitting the first vehicle data through a Gigabit Serial Multimedia Link (GSML) deserializer; transmitting the second vehicle data via a vehicle controller area network (CAN); as well as The third vehicle data and the fourth vehicle data are transmitted via Ethernet ETH.

12. The method according to claim 8, wherein The first data processing module, the second data processing module, the third data processing module and the fourth data processing module are system-on-chip (SoC) and communicate via a high-speed serial computer expansion bus standard PCIe; and The processing control system is a micro control unit MCU or a domain controller DCU.

13. The method according to claim 8, further comprising: The second vehicle data is received by each data processing module in the data processing system via the process control system.

14. The method according to claim 8, further comprising: One or more of the first to fourth vehicle data are stored by a storage module.

15. A computer-readable storage medium having stored thereon program instructions executable by a processor, wherein when the program instructions are executed by the processor, the data processing method for assisted driving according to any one of claims 8 to 14 is executed.

16. A vehicle comprising the data processing device for assisted driving according to any one of claims 1 to 7.

Images