Control data determination method of vehicle, vehicle and storage medium

Abstract

The embodiment of the invention provides a vehicle control data determination method, a vehicle and a storage medium, and the method comprises the steps: collecting the state information of a plurality of groups of sensor suites in the vehicle in a vehicle driving process; based on the state information, a target controller used for controlling vehicle driving is determined from a main controller of the vehicle and an auxiliary controller of the vehicle, and the main controller and the auxiliary controller are used for processing environment information collected by different sensor suites; determining a target sensor suite associated with the target controller from the plurality of sensor suite, and obtaining various environmental information collected by a plurality of target sensors in the target sensor suite; and the target controller is controlled to convert the various kinds of environment information to obtain control data of the vehicle. The technical problem that the control efficiency of the vehicle is low is solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of vehicle control, in particular, to a vehicle control data determination method, a vehicle and a storage medium. BACKGROUND

[0002] At present, with the development of automobile intelligence, high-level automatic driving systems begin to be deployed on vehicles. Today, L2 level automatic driving is gradually mature, and L3 and L4 automatic driving technologies have become a hot topic, and solutions are needed to be embodied on vehicles. However, there is a big difference between L4 level automatic driving system and low-level automatic driving system. High-level automatic driving systems from L2 onwards need to have a redundant architecture, which requires backup systems to take over when the main system fails.

[0003] In related technologies, for the design of redundant architecture, two system on a chip (SOC) hardware are usually deployed in a self-driving domain control hardware, and one set of self-driving software system is deployed in each SOC. However, in this method, the sensor suite connected to the domain control hardware is the same scheme. When some sensors fail, the self-driving system of the vehicle cannot continue to maintain the self-driving state of the vehicle. Therefore, the above method cannot effectively control the vehicle, and there is a technical problem of low control efficiency of the vehicle.

[0004] At present, there is no good solution to the above problem. SUMMARY

[0005] Embodiments of the present application provide a vehicle control data determination method, a vehicle and a storage medium to at least solve the technical problem of low control efficiency of the vehicle.

[0006] According to an aspect of an embodiment of the present application, a vehicle control data determination method is provided. The method can include: collecting state information of multiple sensor suites in a vehicle during driving of the vehicle, wherein the sensor suite includes multiple sensors, and the state information is used to represent the working state of the sensors; determining a target controller for controlling the driving of the vehicle from a main controller of the vehicle and an auxiliary controller of the vehicle based on the state information, wherein the main controller and the auxiliary controller are used to process different environmental information collected by the sensor suites, and the main controller and the auxiliary controller process the environmental information in different ways, and the environmental information is used to determine at least obstacle information of the vehicle during driving; determining a target sensor suite associated with the target controller from the multiple sensor suites, and obtaining multiple environmental information collected by multiple target sensors in the target sensor suite; and controlling the target controller to convert the multiple environmental information to obtain control data of the vehicle, wherein the control data is used to control the driving state of the vehicle.

[0007] Further, based on the state information, determining a target controller for controlling the vehicle from the main controller and the auxiliary controller of the vehicle, comprising: determining the working state of the image acquisition device in the sensor suite associated with the main controller based on the state information; determining the target controller for controlling the vehicle from the main controller and the auxiliary controller based on the working state of the image acquisition device.

[0008] Further, based on the working state of the image acquisition device, determining the target controller for controlling the vehicle from the main controller and the auxiliary controller, comprising: in response to the working state of the image acquisition device being an abnormal working state, determining the auxiliary controller as the target controller; in response to the working state of the image acquisition device being a normal working state, determining the main controller as the target controller.

[0009] Further, determining a target sensor suite associated with the target controller from the plurality of sensor suites, comprising: in response to the auxiliary controller being the target controller, determining the sensor suite associated with the auxiliary controller as the target sensor suite; acquiring a plurality of environmental information collected by a plurality of target sensors in the target sensor suite, comprising: acquiring image information collected by the plurality of target sensors under a plurality of perspectives, wherein the plurality of environmental information includes the image information collected under the plurality of perspectives, the image information is used to determine obstacle information of the vehicle under the perspective and lane information of a lane in which the vehicle travels.

[0010] Further, controlling the target controller to convert the plurality of environmental information to obtain control data of the vehicle, comprising: controlling the target controller to identify the plurality of image information respectively to obtain the lane information and the obstacle information; controlling the target controller to determine a driving path of the vehicle based on the lane information and the obstacle information; controlling the target controller to convert the driving path into the control data.

[0011] Further, determining a target sensor suite associated with the target controller from the plurality of sensor suites, comprising: in response to the main controller being the target controller, determining the sensor suite associated with the main controller as the target sensor suite; acquiring a plurality of environmental information collected by a plurality of target sensors in the target sensor suite, comprising: acquiring image information collected by a plurality of image acquisition devices under a plurality of target perspectives, and a plurality of radar data collected by the plurality of target sensors, wherein the plurality of environmental information includes the plurality of image information collected under the plurality of target perspectives, and the plurality of radar data, the image information is used to determine obstacle information of the vehicle under the target perspective and lane information of a lane in which the vehicle travels, and the obstacle information is used to determine at least a position of an obstacle in a driving process of the vehicle, and the radar data is used to determine at least a distance between the vehicle and the obstacle.

[0012] Further, the control target controller converts the at least one environmental information to obtain the control data of the vehicle, including: the control target controller fuses the plurality of image information and the plurality of radar data to obtain fusion data; the control target controller predicts the fusion data to obtain a driving path of the vehicle; and the control target controller converts the driving path to the control data.

[0013] Further, the control target controller fuses the plurality of image information and the plurality of radar data to obtain fusion data, including: obtaining auxiliary image information collected by an image collection device associated with the auxiliary controller; and fusing the image information, the auxiliary image information and the radar data to obtain the fusion data.

[0014] According to an aspect of an embodiment of the present application, a control data determination apparatus of a vehicle is further provided. The apparatus can include: an acquisition unit configured to acquire state information of a plurality of sensor suites in the vehicle during driving of the vehicle, wherein the sensor suite includes a plurality of sensors, and the state information is used to represent a working state of the sensors; a first determination unit configured to determine, based on the state information, a target controller for controlling driving of the vehicle from a main controller of the vehicle and an auxiliary controller of the vehicle, wherein the main controller and the auxiliary controller are used to process different environmental information collected by the sensor suites, and a manner in which the main controller processes the environmental information is different from a manner in which the auxiliary controller processes the environmental information, and the environmental information is used to determine at least obstacle information of the vehicle during driving; a second determination unit configured to determine, from the plurality of sensor suites, a target sensor suite associated with the target controller, and acquire a plurality of environmental information collected by a plurality of target sensors in the target sensor suite; and a control unit configured to control the target controller to convert the plurality of environmental information to obtain control data of the vehicle, wherein the control data is used to control a driving state of the vehicle.

[0015] According to another aspect of an embodiment of the present application, a vehicle is further provided, including: a memory storing an executable program; and a processor configured to run the program, wherein the program is executed to perform the method in each embodiment of the present application when the program is run.

[0016] According to another aspect of an embodiment of the present application, a computer readable storage medium is further provided, including a stored executable program, wherein the computer readable storage medium is controlled to perform the method in each embodiment of the present application when the executable program is run.

[0017] According to another aspect of an embodiment of the present application, a computer program product is further provided, including a computer program, and the computer program is executed by a processor to implement the method in each embodiment of the present application.

[0018] According to a further aspect of the embodiments of the present application, a computer program product is also provided, which comprises a nonvolatile computer readable storage medium, and the nonvolatile computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the method in the various embodiments of the present application.

[0019] According to a further aspect of the embodiments of the present application, a computer program is also provided. The computer program is executed by a processor to implement the method in the various embodiments of the present application.

[0020] In the embodiments of the present application, during the driving of the vehicle, the state information of a plurality of sensor suites in the vehicle is collected, wherein the sensor suite comprises a plurality of sensors, and the state information is used to represent the working state of the sensors; based on the state information, a target controller for controlling the driving of the vehicle is determined from a main controller and an auxiliary controller of the vehicle, wherein the main controller and the auxiliary controller are used to process different environmental information collected by the sensor suites, and the main controller and the auxiliary controller process the environmental information in different ways, and the environmental information is used to determine at least the obstacle information of the vehicle during driving; a target sensor suite associated with the target controller is determined from the plurality of sensor suites, and a plurality of types of environmental information collected by a plurality of target sensors in the target sensor suite is obtained; the target controller is controlled to convert the plurality of types of environmental information to obtain control data of the vehicle, wherein the control data is used to control the driving state of the vehicle. That is, in the embodiments of the present application, the state information of a plurality of sensor suites is collected during the driving of the vehicle. The state information can be used to determine the working state of the sensors. Based on the state information, the target controller for controlling the vehicle in the vehicle can be determined. The environmental information collected by the sensors associated with the target controller is obtained. Based on the environmental information, the control information for controlling the vehicle can be determined. Thus, the technical problem of low control efficiency of the vehicle is solved, and the technical effect of control efficiency of the vehicle is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate the illustrative embodiments of the present application and the explanation of the present application, and do not constitute improper limitations on the present application. In the drawings:

[0022] Figure 1 is a flowchart of a control data determination method of a vehicle according to an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of a redundancy architecture of L4 level automatic driving software and hardware according to an embodiment of the present application;

[0024] Figure 3 is a flowchart of logic determination of self-driving domain control switching according to an embodiment of the present application;

[0025] Figure 4 is a schematic view of a vehicle control data determination apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the personnel in the art better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor should fall within the protection scope of the present application.

[0027] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, such as a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] According to the embodiments of the present application, a method embodiment of a vehicle control data determination method is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that herein.

[0029] In the present embodiment, a method of a vehicle control data determination method is provided, Figure 1 is a flowchart of a vehicle control data determination method according to an embodiment of the present application. As Figure 1 shown, the flowchart can include the following steps:

[0030] Step S102, during the driving of the vehicle, collecting state information of a plurality of sensor suites in the vehicle, wherein the sensor suite includes a plurality of sensors, and the state information is used to represent the working state of the sensors.

[0031] In the technical solution provided in the foregoing step S102 of the present application, the sensor suite can include multiple sensors, and a group of sensor suites can be associated with the main controller or the auxiliary controller, that is, the sensor suite can be used to collect data processed by the main controller or the auxiliary controller. The sensor can be a laser sensor, a front millimeter wave sensor, a corner millimeter wave sensor, an ultrasonic sensor, a surround view camera, and the like. It should be noted that this is only an example and the type of sensor is not limited.

[0032] Optionally, during the driving of the vehicle, the states of the sensors in the multiple sensor suites can be continuously monitored to obtain state information of each group of sensor suites. The state information can be used to reflect whether each sensor in the sensor suite is working normally, for example, the state information can be used to determine the online state of the sensor, the data transmission quality, the self-checking result of the sensor, and the like, and can be used to evaluate the health status and reliability of the sensor. It should be noted that this is only an example and the content of the state information is not limited.

[0033] Optionally, during the driving of the vehicle, ensuring the normal operation of the sensors is crucial to the automatic driving system. Therefore, during the driving of the vehicle, the state information of multiple groups of sensor suites in the vehicle can be collected and monitored, and based on the state information, the real-time working conditions of the sensors can be evaluated.

[0034] For example, the state information of the Lidar, the front millimeter wave radar (FRM), the corner millimeter wave radar (CR), the ultrasonic sensor system (USS), and the surround view camera in the sensor suite associated with the main controller, and the state of the front view camera and the surround view camera in the sensor suite associated with the auxiliary controller can be continuously monitored.

[0035] For another example, the state information of the sensor suite can be collected by checking whether the Lidar is online, whether the data transmission is stable, and the self-checking report of the Lidar to confirm whether it is working normally. For example, if one of the Lidars reports an abnormal data transmission, this state information can be recorded. For the front millimeter wave radar and the corner millimeter wave radar, their online state and self-checking result can be checked to evaluate whether the obstacle detection capability in their detection range is affected. If the FRM detects that its power is insufficient to reach the predetermined detection distance, this state will also be recorded to obtain the state information of the front millimeter wave radar and the corner millimeter wave radar.

[0036] In step S104, based on the status information, a target controller for controlling the vehicle's driving is determined from the vehicle's main controller and the vehicle's auxiliary controller, wherein the main controller and the auxiliary controller are used to process environmental information collected by different sensor kits, and the main controller processes the environmental information in a different way from the auxiliary controller. The environmental information is at least used to determine obstacle information during the vehicle's driving process.

[0037] In the technical solution provided in the above step S104 of the present application, the above-mentioned main controller can be the main self-driving domain controller, which can be the main control unit in the vehicle's automatic driving system and can be used to be responsible for the normal driving of the vehicle. The main controller can perform complex environmental perception, decision-making planning and vehicle control tasks based on the data provided by a series of sensors (such as lidar, millimeter-wave radar, camera, etc.). The software architecture of the main controller may include: a multi-sensor data fusion module, a decision and path planning module, a vehicle control module, etc. It should be noted that this is only an example and there is no specific restriction on the content contained in the main controller. The above-mentioned multi-sensor data fusion module can be used to process and fuse data from multiple sensors to form a comprehensive understanding of the vehicle's surrounding environment. The above-mentioned decision and path planning module can be used to perform obstacle detection, road condition analysis, path planning and behavior decision-making based on the fused data. The above-mentioned vehicle control can be used to generate control instructions such as vehicle power, steering and braking to ensure that the vehicle travels safely along the planned path.

[0038] Optionally, the auxiliary controller can be a secondary controller, also known as a redundant controller, which can serve as a backup for the primary controller and utilize a different sensor configuration and algorithm architecture than the primary controller. The auxiliary controller is designed to take over vehicle control in the event of a failure of the primary controller or its associated sensor suite, ensuring the continuity and safety of the autonomous driving system.

[0039] In this embodiment, the auxiliary controller can have an independent sensor configuration, and the auxiliary controller can be connected to a different sensor suite from the main controller. For example, the main controller may rely more on lidar and front millimeter-wave radar, while the auxiliary controller may focus more on visual sensors, such as front-view cameras and surround-view cameras. The auxiliary controller can also have a specific algorithm architecture, such as a pure visual perception algorithm, so that even when the sensors of the main system fail, the auxiliary controller can make decisions based on visual information. In an emergency, the auxiliary controller can execute a minimum risk maneuver (MRM), which can be used to ensure that the vehicle is in a minimum risk condition (MRC), such as safely parking or driving off the road.

[0040] Optionally, the presence of the auxiliary controller greatly improves the robustness and safety of the autonomous driving system. Through different sensor suites and algorithm architectures, even if partial component failures are encountered, the system can quickly switch to alternative solutions to maintain control of the vehicle and avoid potential accident risks.

[0041] Optionally, based on the state information of the sensor suite, the working states of multiple sensors in the sensor suite can be determined, and based on the working states of the multiple sensors, the target controller can be determined among the main controller and the auxiliary controller.

[0042] Optionally, the above-mentioned environmental information can be information collected by sensors, which can include image information, radar data, etc., and can be used to determine obstacle information, lane line information, and surrounding vehicle dynamic information of the vehicle during driving. The above-mentioned obstacle information can be used to identify any entity that may hinder the normal driving of the vehicle on the road, such as other vehicles, pedestrians, animals, static obstacles (such as roadblocks, trees, buildings), road debris, etc. The obstacle information not only includes the existence or non-existence of the obstacle, but also includes the position, size, shape, speed and moving direction of the obstacle. The above-mentioned road information can include the type of road where the vehicle is located, lane marking, traffic marking, road geometry (such as curves, slopes), road surface condition (such as wet, icy, potholes), etc. The above-mentioned surrounding vehicle dynamic information can be used to track the driving state of nearby vehicles, including the speed, acceleration, steering angle and future possible action prediction of the obstacle. It should be noted that this is only an example and the type and representation of the environmental information, the content of the obstacle information, and the content of the surrounding vehicle dynamic information are not limited.

[0043] Optionally, by collecting and analyzing the above-mentioned environmental information, the autonomous vehicle can construct a real-time and detailed surrounding environment model, and then make accurate driving decisions such as obstacle avoidance, following the front vehicle, adjusting speed, and planning path. This process usually involves complex algorithms, including but not limited to machine learning, computer vision, radar signal processing, and sensor data fusion, to ensure that the vehicle can drive safely and efficiently. In L4 autonomous driving, since the vehicle can drive in a specific environment without human driver intervention, accurate collection and analysis of environmental information becomes particularly important.

[0044] In this embodiment, the vehicle can be equipped with a main controller and an auxiliary controller, both of which process data collected by different sensor suites and use different algorithm architectures for environmental information processing. If a sensor in the sensor suite associated with the main controller fails, the system will evaluate the impact of the failure. If the main controller can still maintain a certain driving ability based on the data of other sensor suites, the main controller will continue to be used. However, if the failure of a key sensor severely limits the function of the main controller, the system will switch to the auxiliary controller, which will be determined as the target controller. The auxiliary controller can use different processing methods (such as pure visual algorithms) to control the vehicle based on the data of its associated sensor suite. Therefore, based on the state information of the sensors collected in step S102, the target controller that is most suitable for controlling the vehicle to travel can be determined.

[0045] Step S106, from the plurality of sensor suites, determine a target sensor suite associated with the target controller, and obtain a plurality of environmental information collected by a plurality of target sensors in the target sensor suite.

[0046] In the technical solution provided by step S106 of the present application, after the target controller is determined, the sensor suite associated with the controller can be automatically identified, and a plurality of sensors in the sensor suite can be used to collect a plurality of environmental information. The environmental information can be used to determine, including but not limited to: the position, size, speed of obstacles, road signs, and the dynamics of other vehicles and pedestrians. The selection of the above-mentioned target sensor suite ensures that the collected data can match the algorithm architecture of the target controller, so as to achieve optimal environmental perception and driving control.

[0047] In this embodiment, in L4 level automatic driving, the cooperation of the main controller and the auxiliary controller is an important part of realizing the safety and reliability of high-level automatic driving. By dynamically switching the control strategy, the system can cope with various emergencies and maintain the normal operation of the vehicle, thereby ensuring the safety of passengers and road users.

[0048] Step S108, control the target controller to convert the plurality of environmental information to obtain control data of the vehicle, wherein the control data is used to control the driving state of the vehicle.

[0049] In the technical solution provided in the above step S108 of the present application, the modules in the target controller can be used to convert the environmental information to obtain the control data of the vehicle. Through this step, the system can intelligently select the most suitable combination of controllers and sensor suites according to the state of the sensors and the needs of the vehicle's driving, to achieve safe and efficient operation of the autonomous vehicle. This design improves the redundancy and flexibility of the system, and can continue to maintain the autonomous driving function of the vehicle by switching to other sensors and controllers in the event of partial sensor failure.

[0050] Optionally, the above-mentioned control data can be instructions issued by the autonomous driving system to various components of the vehicle, and can be used to directly control the operation of the vehicle, such as including deceleration instructions, steering instructions, acceleration instructions, lane keeping instructions, emergency stopping instructions, etc. It should be noted that this is only an example and the control content and type of the control data are not limited.

[0051] Optionally, the target controller can convert various environmental information collected from the target sensor suite into control data of the vehicle. This process includes but is not limited to obstacle detection, path planning, behavior decision making, and vehicle control instruction generation. The control data can be used to determine the driving state of the vehicle, such as acceleration, deceleration, steering, stopping, etc. The above-mentioned target controller simplifies complex environmental information into specific control instructions based on its unique algorithm architecture, ensuring that the vehicle can safely and accurately drive under different driving conditions.

[0052] For example, after the autonomous driving system determines the target controller (main controller or auxiliary controller), the target controller can be used to convert the collected various environmental information into control data. The conversion of environmental information to obtain control data can include three main stages: analysis of environmental information, decision making, and control instruction generation. Among them, the target controller can analyze the environmental information collected from the target sensor suite to understand the position, speed, road conditions, traffic signals, etc. of obstacles. For example, if the front-facing camera detects a red light ahead and the laser radar reports a stationary vehicle ahead, the controller will confirm that there is a need to stop ahead. Based on the analyzed environmental information, the target controller can develop appropriate driving strategies. For example, if a pedestrian is identified crossing ahead, the vehicle can be controlled to slow down and prepare to stop to ensure safety. This decision-making process may involve complex algorithms such as artificial intelligence prediction models and behavior planning. Finally, the target controller can convert the decision into control instructions, such as adjusting the throttle, brake, and steering angle of the vehicle to execute the developed driving strategy. The control instructions can be encoded into control data and sent to components such as the power system, braking system, and steering system of the vehicle to directly control the driving state of the vehicle.

[0053] Through the steps S102-S108, in the process of vehicle driving, the state information of a plurality of sensor suites in the vehicle is collected, wherein the sensor suite comprises a plurality of sensors, and the state information is used to represent the working state of the sensors; based on the state information, a target controller for controlling the vehicle driving is determined from the main controller and the auxiliary controller of the vehicle, wherein the main controller and the auxiliary controller are used to process different environmental information collected by the sensor suite, and the main controller and the auxiliary controller process the environmental information in different ways, and the environmental information is used to determine at least the obstacle information of the vehicle in the driving process; the target sensor suite associated with the target controller is determined from the plurality of sensor suites, and a plurality of environmental information collected by a plurality of target sensors in the target sensor suite is obtained; the target controller is controlled to convert the plurality of environmental information to obtain the control data of the vehicle, wherein the control data is used to control the driving state of the vehicle. That is, the embodiment of the application collects the state information of a plurality of sensor suites in the process of vehicle driving, which can be used to determine the working state of the sensors, based on the state information, the target controller for controlling the vehicle in the vehicle is determined, the environmental information collected by the sensors associated with the target controller is obtained, and based on the environmental information, the control information for controlling the vehicle is determined, thereby solving the technical problem of low control efficiency of the vehicle, and achieving the technical effect of improving the control efficiency of the vehicle.

[0054] The above method of the application will be further described below.

[0055] In this embodiment, after obtaining the state information, the target controller for controlling the vehicle driving in the main controller and the auxiliary controller can be determined based on the state information. The method will be further described below.

[0056] As an optional implementation, in step S104, based on the state information, the target controller for controlling the vehicle driving is determined from the main controller and the auxiliary controller of the vehicle, comprising: based on the state information, the working state of the image acquisition device in the sensor suite associated with the main controller is determined; based on the working state of the image acquisition device, the target controller for controlling the vehicle driving is determined from the main controller and the auxiliary controller.

[0057] In this embodiment, the image acquisition device can be a device for acquiring images, which can be a panoramic camera.

[0058] Optionally, considering that when the panoramic camera in the sensor suite associated with the main controller fails, the vehicle loses the key information for driving to the side, and therefore the vehicle system determines that the main controller cannot be used any more, and needs to be switched to the pure vision algorithm architecture of the redundant controller for controlling the vehicle.

[0059] Optionally, during the operation of the autonomous driving system, the status information of all sensor suites can be continuously monitored, including but not limited to the working status of the image acquisition device. The status information can include the online status, fault code, data quality indicators (such as image clarity, frame rate, etc.), signal strength, and synchronization between devices of the image acquisition device. Based on the status information, the overall health status of the image acquisition device and the effectiveness of the data collected by the image acquisition device can be determined. Once it is determined that the working status of the image acquisition device in the sensor suite associated with the main controller is not good, a specific logic algorithm can be followed to determine which controller (main controller or auxiliary controller) is more suitable for vehicle control in the current situation. If the image acquisition device is a key component in the perception strategy of the main controller, and the status of the image acquisition device is below the preset threshold and is not sufficient to support the normal operation of the main controller, the target controller can be switched from the main controller to the auxiliary controller.

[0060] Optionally, the switching mechanism between the main controller and the auxiliary controller can include a series of evaluation and arbitration steps to ensure that the auxiliary controller can smoothly take over when the main controller function declines. Once the auxiliary controller is determined as the target controller, the auxiliary controller can enable the software algorithm it carries, such as the pure visual perception algorithm, to perform environment cognition. At the same time, the auxiliary controller can adjust its perception strategy to adapt to the change of the current sensor configuration, and then use the updated perception result to execute decision, planning and control logic to guide the vehicle to travel along a safe route, for example, to execute a minimum risk strategy to make the vehicle reach a minimum risk state, i.e., a safe stop or a state of maintaining in a safe lane. It should be noted that the above method of determining the working status of the target controller based on the image acquisition device is only an example, and this application is not limited to the specific method of determining the working status of the target controller based on the image acquisition device. As long as the method of determining the working status of the target controller based on the image acquisition device is within the scope of protection of the present application.

[0061] After determining the working status of the image acquisition device in the sensor suite based on the status information, the target controller for controlling the vehicle to travel can be further confirmed based on the working status. The method is further introduced as follows.

[0062] As an optional implementation, determining the target controller for controlling the vehicle to travel from the main controller and the auxiliary controller based on the working status of the image acquisition device includes: in response to the working status of the image acquisition device being an abnormal working status, determining the auxiliary controller as the target controller; and in response to the working status of the image acquisition device being a normal working status, determining the main controller as the target controller.

[0063] In this embodiment, after obtaining the state information of the image acquisition device, the working state of the image acquisition device can be determined based on the state information. If the working state of the image acquisition device is an abnormal working state, the auxiliary controller can be determined as the target controller, otherwise, if the working state of the image acquisition device is a normal working state, the main controller can be determined as the target controller.

[0064] For example, the starting process of the vehicle L4 function can include the following: after the vehicle is powered on, the main redundant controller can be first self-checked, and the main redundant controller can include a main controller and an auxiliary controller. After the self-checking is completed and no error is found, the main redundant controller can start all the functional modules inside the domain control. Because the functional modules of the main redundant controller are all pulled up, the mode of arbitration is determined as the main controller to control the vehicle. During the driving of the vehicle, if a sensor failure occurs, the channel can determine the degree of influence of the failure at this time to decide whether to continue to use the main controller to control the vehicle or to use the redundant controller to control the vehicle. Further, the judgment of the sensor failure can include the following: when the laser radar in the first sensor suite associated with the main controller fails, it can be determined to still use the main domain controller for vehicle control, mainly relying on the perception results of the omnidirectional camera and the front-view camera and other sensors, using the multi-data fusion perception algorithm inside the main controller to control the vehicle to park on the side, so that the vehicle reaches the final MRC state. The front millimeter wave, angle millimeter wave, ultrasonic wave, and front-view camera in the first sensor suite execute the same arbitration strategy. However, when the omnidirectional camera in the first sensor suite fails, because it cannot provide visual perception information of the vehicle surroundings, the vehicle loses the key information for parking on the side, at this time the vehicle system determines that the main controller cannot be used any more, and switches to the pure visual algorithm architecture of the redundant controller to control the vehicle, that is, the auxiliary controller can be determined as the target controller.

[0065] In this embodiment, a redundant scheme of two autonomous driving domain controls can be included, and different software algorithm architectures are deployed in each domain control. The sensor dependence is decoupled from the software algorithm, and different domain controls access different sensor suite schemes in the hardware access scheme of the sensor and the domain control, so as to achieve decoupling of the sensor and the domain control hardware, so that when the sensor fails, the target controller can be quickly switched without affecting the driving process of the vehicle, thereby solving the technical problem of low control efficiency of the vehicle and achieving the technical effect of improving the control efficiency of the vehicle.

[0066] For example, when the working state of the image acquisition device is determined to be an abnormal working state, such as the image acquisition device having a problem of image quality degradation, device offline, or data transmission interruption, it can be determined that the visual perception information relied on by the main controller can be unreliable or completely unavailable. In this case, it can be determined that the main controller is no longer suitable for performing the control task of the vehicle, because the visual information on which the main controller is based can be inaccurate or incomplete. In order to ensure the safety of the vehicle and the system redundancy, the transfer of control can be performed, and the auxiliary controller can be determined as the new target controller. The auxiliary controller can generally use a different perception strategy (for example, a pure visual perception algorithm, different from the multi-sensor data fusion perception algorithm of the main controller) and can use an independent sensor suite for environment perception. In this way, even if the visual perception ability of the main controller is impaired, the auxiliary controller can continue to control the vehicle and perform a minimum risk strategy to ensure that the vehicle can safely reach a minimum risk state, such as safe parking or maintaining in a safe lane.

[0067] Alternatively, when the working state of the image acquisition device is determined to be normal, that is, the image information can be stably and high-quality transmitted to the automatic driving system, it can be determined that the main controller can rely on the collected data for accurate environment perception. In the normal state, the main controller is usually a better choice due to its complex multi-sensor data fusion perception algorithm, because the main controller can provide more comprehensive and accurate perception results, thereby making more reasonable vehicle control decisions. Therefore, the main controller can be determined as the target controller for controlling the vehicle to travel, and the main controller will execute its preset algorithms, including but not limited to obstacle prediction, vehicle behavior decision, vehicle motion trajectory planning, and vehicle tracking control, etc., to realize the automatic driving function. At the same time, the auxiliary controller is in standby state, ready to take over immediately when any failure of the main controller or its associated sensors is detected.

[0068] In this embodiment, a target controller selection mechanism based on the working state of the image acquisition device is proposed, which is the key to realizing the software and hardware redundancy architecture in the automatic driving system. This mechanism monitors the sensor state in real time to ensure that there is a suitable controller to control the vehicle in any case, thereby improving the overall safety and reliability of the automatic driving system.

[0069] After determining the auxiliary controller as the target controller, the target sensor suite associated with the target controller can be determined from the plurality of sensor suites. The method is further described below.

[0070] As an optional implementation, determining a target sensor suite associated with a target controller from a plurality of sensor suites includes: in response to an auxiliary controller being the target controller, determining the sensor suite associated with the auxiliary controller as the target sensor suite; obtaining a variety of environmental information collected by a plurality of target sensors in the target sensor suite, including: obtaining image information collected by a plurality of target sensors under a plurality of perspectives, wherein the plurality of environmental information includes image information collected under a plurality of perspectives, and the image information is used to determine obstacle information of the vehicle under a perspective, as well as lane information of the lane in which the vehicle is traveling.

[0071] In this embodiment, if the auxiliary controller is the target controller, the sensor suite associated with the auxiliary controller can be determined as the target sensor suite, and image information from multiple perspectives captured by the target sensor suite can be obtained. The image information can be visual perception information that can be used to determine at least obstacle information of the vehicle from different perspectives and the lane information of the vehicle.

[0072] Optionally, after switching to the auxiliary controller (i.e., redundant controller), the sensor selection will automatically switch to the sensor suite associated with the auxiliary controller. That is, after the auxiliary controller is determined to be the target controller, the surround view camera in the sensor suite associated with the auxiliary controller is not reused with the surround view camera in the sensor suite associated with the primary controller. Therefore, it can provide complete visual perception information to the vehicle, providing a reliable basis for the vehicle to perform the corresponding pull-over operation. When the vehicle finally pulls over, it finally reaches the safe state of MRC.

[0073] Optionally, once the auxiliary controller is determined as the target controller, that is, when the image acquisition device of the main controller is operating abnormally, the target controller can be automatically switched from the main controller to the auxiliary controller, and the sensor suite associated with the auxiliary controller among the multiple sensor suites can be used as the target sensor suite. This is because the auxiliary controller is usually designed to match a specific sensor suite to support its specific perception algorithm and redundancy function. For example, the auxiliary controller may mainly rely on a pure visual perception algorithm. Therefore, the sensor suite associated with the auxiliary controller can be a sensor suite containing more visual sensors, such as a camera, to provide the necessary environmental information.

[0074] Optionally, multiple target sensors in the target sensor suite may include visual sensors such as a forward-looking camera, a surround-view camera, and a surround-view camera. These sensors can continuously collect a variety of information about the vehicle's surroundings. This information includes, but is not limited to, image information from multiple perspectives. For example, a forward-looking camera provides a view of the road ahead, a surround-view camera provides a view to the sides of the vehicle, and a surround-view camera provides a complete view of the vehicle's surroundings.

[0075] After the above-mentioned target controller collects various environmental information from the sensors in the target sensor suite corresponding to the target controller, the environmental information can be converted into control data using the following steps. The method is further described below.

[0076] As an optional implementation, the target controller converts the various environmental information into control data of the vehicle, including: the target controller identifies the multiple image information respectively to obtain lane information and obstacle information; the target controller determines a driving path of the vehicle based on the lane information and the obstacle information; and the target controller converts the driving path into control data.

[0077] In this embodiment, after the image information is collected, the target controller can be controlled to identify the multiple image information respectively to obtain lane information and obstacle information. The above-mentioned obstacle information can be various obstacles around the vehicle detected and identified by the software algorithm of the auxiliary controller through analyzing the image information from multiple target sensors, including static objects (such as road signs, buildings) and dynamic objects (such as pedestrians, other vehicles) and the like. The above-mentioned lane information can refer to a detailed description of the lane in which the vehicle is currently located, which can include the position, shape and relative relationship with the vehicle of the lane line. It should be noted that this is only an example and the representation of the obstacle information and the lane information is not specifically limited.

[0078] Optionally, the image information collected by the visual sensor can enable the auxiliary controller to accurately identify the lane line, which is of great significance for the vehicle to keep driving in the correct lane, perform lane changing operations, and judge whether to stop on the side of the road or not.

[0079] Optionally, after the image information under different perspectives is collected by the cameras at different positions of the vehicle, the target controller can be called to perform in-depth analysis and processing on the image information collected from multiple perspectives through its software algorithm. The processing process can include but is not limited to image preprocessing, feature extraction, target detection and identification and the like.

[0080] Optionally, after the image information is collected, the target controller (at this time, the auxiliary controller) can be controlled to identify the multiple image information using a pure vision perception module to obtain lane information and obstacle information. Further, the lane information and the obstacle information can be processed using a prediction and decision module and a vehicle trajectory planning module to determine a driving path of the vehicle. Finally, the target controller can generate corresponding control data based on the driving path using a control module for tracking the vehicle planning trajectory, and transmit the control data to the corresponding controller to complete the control of the vehicle.

[0081] As another optional example, after the image information is collected, the lane line detection algorithm can identify the shape and position of the lane line from the image information, which can include the type (such as solid or dashed line), width and direction of the lane line. At the same time, the obstacle recognition algorithm can detect and identify various obstacles from the image information, including pedestrians, other vehicles, static obstacles (such as roadblocks, road signs), and dynamically changing obstacles (such as animals). After obtaining the lane information and obstacle information, the target controller can use this information to determine the safe driving path of the vehicle. The driving path of the vehicle can be determined by using a path planning algorithm, which can consider the current position, speed, target position of the vehicle, the position and movement trend of the obstacles, and the geometric characteristics of the lane, etc. to generate a driving path that meets the safety requirements and is efficient. Finally, the target controller can convert the determined driving path into control data, which can be used to control the driving of the vehicle. The control data includes but is not limited to: steering control data, acceleration and deceleration control data, braking control data, lane changing control data, etc.

[0082] Optionally, the above-mentioned control data can be sent to the actuator of the vehicle's power system, braking system, steering system, etc. to perform corresponding actions, to ensure that the vehicle safely drives according to the planned path. In this way, even in the case of abnormal operation of the image acquisition device of the main controller, the auxiliary controller can ensure that the vehicle safely reaches the minimum risk state, such as safe parking or maintaining in the safe lane. It should be noted that the above method of processing image information by the target controller to obtain control data is only for illustration, and is not limited specifically here. As long as the method of determining control data based on image information is within the scope of protection of the present application.

[0083] After determining that the main controller is the target controller, the target sensor suite associated with the target sensor can be determined based on the sensor suite associated with the main controller, and various environmental data can be collected by using the target sensor suite. The method is further described below.

[0084] As an optional implementation, from the plurality of sensor kits, a target sensor kit associated with the target controller is determined, including: in response to the host controller being the target controller, determining the sensor kit associated with the host controller as the target sensor kit; and obtaining a plurality of environmental information collected by a plurality of target sensors in the target sensor kit, including: obtaining image information collected by a plurality of image collection devices under a plurality of target perspectives, and a plurality of radar data collected by a plurality of target sensors, wherein the plurality of environmental information includes the plurality of image information collected under the plurality of target perspectives, and the plurality of radar data, the image information is used to determine obstacle information of the vehicle under the target perspective, and lane information of a lane in which the vehicle travels, the obstacle information is used at least to determine a position of an obstacle in a driving process of the vehicle, and the radar data is used at least to determine a distance between the vehicle and the obstacle.

[0085] In this embodiment, if the host controller is the target controller, the image information under the target perspective collected by the plurality of image collection devices can be obtained, wherein the plurality of image collection devices can include a plurality of surround view cameras. The target perspective can be a perspective collected by the surround view cameras. The target sensors can include lidar, millimeter wave radar, ultrasonic radar, and the like. It should be noted that this is only an example and the type of target sensor is not specifically limited.

[0086] For example, the plurality of surround view cameras can be used to collect image information under a lateral perspective, and the lidar, millimeter wave radar, ultrasonic radar, and the like are used to collect radar data. The radar data can be used at least to determine the distance between the vehicle and the obstacle.

[0087] Optionally, if the system evaluation result shows that the image acquisition device of the master controller is in good working condition and has no abnormalities, the master controller can be determined as the target controller responsible for the driving control of the vehicle. In this case, the sensor suite associated with the master controller can be determined as the target sensor suite. The target sensor suite can include but is not limited to a plurality of lidars, millimeter wave radars, ultrasonic radars, image acquisition devices, etc. Image information in multiple target perspectives can be collected by image acquisition devices (such as omnidirectional cameras) installed around the vehicle. The master controller can use a multi-data fusion perception module to identify and locate various obstacles on the road from the collected image information, including but not limited to pedestrians, other vehicles, road edges, traffic signs, etc. Through image processing and deep learning technology, the master controller can accurately identify the type of obstacle and its position in the image, which corresponds to the actual position of the obstacle relative to the vehicle. In addition, in addition to image information, the target sensor suite can also include a plurality of radar sensors, such as millimeter wave radars or ultrasonic radars. These radar sensors can emit and receive electromagnetic waves or sound waves to detect obstacles in the surrounding environment. The above radar data provides accurate distance information between the obstacle and the vehicle. By analyzing the time and intensity of the radar echo, the distance between the obstacle and the vehicle can be determined.

[0088] Optionally, the master controller can process the environmental information collected from the image acquisition device and the radar sensor to obtain control data. That is, when the master controller serves as the target controller, it not only relies on image information from the image acquisition device, but also combines radar data from the radar sensor to obtain accurate control data.

[0089] After collecting various environmental information using the target sensors in the target sensor suite, the target controller can be used to convert the environmental information to obtain control data. The method is further described below.

[0090] As an optional implementation, the target controller converts at least one type of environmental information to obtain control data for the vehicle, including: controlling the target controller to fuse a plurality of image information and a plurality of radar data to obtain fused data; controlling the target controller to predict the fused data to obtain a driving path of the vehicle; and controlling the target controller to convert the driving path into control data.

[0091] In this embodiment, after a plurality of image information is collected by using an image collection device and a plurality of radar data is collected by using a plurality of target sensors, a multi-sensor data fusion perception module in the target controller can be controlled to perform fusion processing on the plurality of image information and the radar data to obtain fusion data. The target controller can be used to call an obstacle prediction and vehicle behavior decision module to predict the fusion data to obtain a moving track of the obstacle. Further, the target controller uses a vehicle motion track planning module to obtain a driving path of the vehicle based on the moving track of the obstacle. The target controller can be used to call a control module for controlling the vehicle to track the planned track to convert the driving path to obtain control data.

[0092] Optionally, the target controller can perform fusion processing on the image information and the radar data collected from a plurality of sensors to generate fusion data. The image information can include camera images from different perspectives, such as front view, surround view, and around view, which can be used to provide visual information of obstacles, lane lines, traffic signs, and the like. The radar data can include radar data provided by sensors such as front millimeter wave radar, angle millimeter wave radar, and ultrasonic radar, which can be used to determine physical attribute information of the obstacle such as distance, speed, and direction. The target controller can use advanced signal processing and machine learning techniques to align the image and radar data in time and correct them in space, and then combine the advantages of both, such as object features and color information provided by the image and accurate distance and speed information provided by the radar, to obtain the fusion data. It should be noted that the process of obtaining the fusion data based on the fusion of the image information and the radar data is not limited here.

[0093] Optionally, after obtaining the fusion data, the target controller can complete the prediction of the driving path based on the generated fusion data. The possible driving paths can be evaluated according to the position, speed, and predicted moving track of the obstacle, as well as the geometric characteristics of the lane line. Based on considering various limiting conditions (such as traffic rules, vehicle capabilities, and passenger comfort), a best driving path is selected to obtain the final driving path corresponding to the vehicle.

[0094] Optionally, if the fusion data shows that a vehicle in front is decelerating, the target controller can predict the dynamics of the vehicle in front while considering the lane line information and the positions of surrounding obstacles to determine whether the vehicle needs to change lanes or decelerate to avoid a rear-end collision. Finally, the target controller can convert the predicted driving path into control instructions for the vehicle, i.e., control data. The control data can interact with the execution system of the vehicle (such as the steering system, the braking system, and the power system) to adjust the driving state of the vehicle to achieve the planned driving path.

[0095] For example, assuming the fusion data shows that a pedestrian suddenly crosses the road on the left side of the vehicle, the target controller can plan an emergency avoidance path based on the fusion data, while slowing down the vehicle to ensure the safety of the vehicle driver. The target controller can send steering control data to the electric power steering system to make the vehicle deflect to the right, and send brake control data to the intelligent / integrated power brake system to slow down the vehicle. The generation and execution of the above control data ensure that the vehicle can respond to the sudden situation in time and safely, avoid potential collision risks, and finally guide the vehicle to a minimum risk state, such as safe parking or maintaining in a safe lane.

[0096] After the above-mentioned collection of various environmental information, the plurality of image information and the plurality of radar data in the environmental information can be fused to obtain fusion data. The method is further introduced below.

[0097] As an optional implementation, the target controller is controlled to fuse the plurality of image information and the plurality of radar data to obtain fusion data, comprising: acquiring auxiliary image information collected by an image collection device associated with the auxiliary controller; fusing the image information, the auxiliary image information and the radar data to obtain the fusion data.

[0098] In this embodiment, the image collection device associated with the auxiliary controller can be a front-view camera. In this embodiment, the front-view camera can be multiplexed in the sensor suite associated with the auxiliary controller and the sensor suite associated with the main controller, and the auxiliary image information collected by the image collection device associated with the auxiliary controller can be transmitted to the main controller through data multiplexing. The image information, the auxiliary image information and the radar data are fused to obtain the fusion data. The fusion data can be used to determine the complete obstacle information and lane information around the vehicle. The above-mentioned image information can be image data. It should be noted that this is only an example and the form of the image information is not limited.

[0099] Optionally, the auxiliary image information collected by the image collection device associated with the auxiliary controller is used to supplement the data collected by the sensors in the sensor suite associated with the main controller, so as to improve the accuracy of the control data.

[0100] As an optional implementation, the method can further comprise: in response to the existence of a fault in the computing module of the main controller, determining the auxiliary controller as the target controller.

[0101] In this embodiment, when the main self-driving domain controller (i.e., the main controller) has a fault in the module (i.e., the calculation module) of the algorithm, it can be automatically determined to enter the redundant controller, and the auxiliary controller can be determined as the target controller. Further, the MRM lane-changing strategy is executed by using the visual perception algorithm in the redundant controller and the perception results of the sensor suite associated with the auxiliary controller, so that the vehicle reaches the safe state of the MRC.

[0102] Optionally, when the auxiliary controller or the sensor suite associated with the auxiliary controller fails, the multi-data fusion algorithm architecture in the main controller can be used to control the vehicle by fusing the perception data in the sensor suite associated with the main controller, so that the vehicle is parked on the side and reaches the final MRC state.

[0103] Optionally, the above calculation module can be a main controller software architecture, which can include a multi-sensor data fusion perception module, an obstacle prediction and vehicle behavior decision module, a vehicle motion trajectory planning module, and a control module for controlling the vehicle to track the planned trajectory. It should be noted that the components included in the calculation module can be adjusted according to actual needs, and are not limited specifically herein.

[0104] The above method of this embodiment is further illustrated below.

[0105] At present, with the development of automobile intelligence, high-level automatic driving systems begin to be deployed on vehicles. Today, L2-level automatic driving is gradually mature, and L3 and L4 automatic driving technologies have become a hot topic, and solutions need to be implemented on vehicles. However, there is a big difference between L4-level automatic driving systems and low-level automatic driving systems. High-level automatic driving systems from L2 onwards need to have a redundant architecture, and require a backup system to take over when the main system fails. The automatic driving system itself has the ability to execute the minimum risk strategy (MRM) strategy, so that the vehicle eventually reaches the minimum risk state (MRC).

[0106] In the related art, for the redundant architecture design, the design of the autonomous driving domain control can include the following two schemes: in one self-driving domain control hardware, two system chips are deployed, one set of SOC is deployed in each set of SOC, and one set of self-driving software system is deployed in each set of SOC; or two autonomous driving domain control hardware are set, one SOC is deployed in each domain control hardware, one set of self-driving software system is deployed on the SOC, and the software algorithm architecture is a multi-sensor data fusion scheme. For the above two schemes, the software algorithm architecture deployed in the domain control, whether it is two hardware designs or two SOC designs in one domain control hardware, is the same, and the sensor suite connected to the domain control hardware is the same scheme. In the design of this architecture scheme, although the self-driving domain control hardware itself is redundantly designed, the software architecture and the sensor itself are not designed accordingly, and when some sensors fail, the self-driving system cannot continue to maintain the self-driving state of the vehicle, the vehicle cannot execute the minimum risk strategy of the MRM, and the minimum risk state of the vehicle cannot be guaranteed, so the above method cannot effectively control the vehicle, and there is a technical problem of low control efficiency of the vehicle.

[0107] To solve the above problems, an embodiment proposes a L4 level autonomous driving software and hardware redundant architecture design. Figure 2 is a schematic diagram of a L4 level autonomous driving software and hardware redundant architecture according to an embodiment of the present application, as Figure 2 shown, the software and hardware architecture of the domain control meeting the L4 autonomous driving in this embodiment can include: a sensor architecture connected to the main controller, i.e., the main controller sensor architecture 213; a software architecture of the main controller, i.e., the main controller software architecture 209; a sensor architecture connected to the auxiliary controller, i.e., the auxiliary controller sensor architecture 214; a software architecture of the auxiliary controller, i.e., the auxiliary controller software architecture 210; and a redundant chassis power controller architecture 211.

[0108] Optionally, the above-mentioned main controller sensor architecture 213 can be a sensor suite associated with the main controller, which can include a laser radar sensor, which can be referred to as a laser radar 201, the number of which can be three; a forward millimeter wave radar 202, the number of which can be one; an angle millimeter wave radar 203, the number of which can be four; an ultrasonic wave radar 204, the number of which can be twelve; a surround view camera 205, the number of which can be two.

[0109] Optionally, the above-mentioned auxiliary controller sensor architecture 214 can be a sensor suite associated with the auxiliary controller, which can include: a forward-looking camera 206, the number of which can be two, wherein one of the forward-looking cameras can be a wide-angle camera and one can be a long-focus camera, and the information fusion of the two provides reliable front information for the vehicle; it can also include two surround view cameras 207 and four surround view cameras 208.

[0110] Optionally, the redundant architecture of the L4 level autonomous driving software and hardware can further include a main controller software architecture 209 and an auxiliary controller software architecture 210. The main controller software architecture 209 can be a software algorithm architecture of the main controller, which can include a multi-sensor data fusion perception module 2091, an obstacle prediction and vehicle behavior decision module 2092, a vehicle motion trajectory planning module 2093, and a control module 2094 for controlling the vehicle to track the planned trajectory. The auxiliary controller software architecture 210 can be a software algorithm architecture of the auxiliary controller, which can include a pure vision perception module 2101, a prediction and decision module 2102, a vehicle trajectory planning module 2103, and a control module 2104 for tracking the planned vehicle trajectory.

[0111] Optionally, the redundant chassis power controller architecture 211 can be a topological design of associated components, which generally follows the redundancy principle. The steering assist system can be a dual-motor winding electric power steering system 2111 (EPS) and an electric power steering system 2112. The chassis brake can be a redundant backup system of an intelligent / integrated power brake system 2113 (IPB) and a redundant brake unit 2114 (RBU). The power transmission vehicle control unit 2115 can be referred to as a power system, which can be a dual-controller area network (CAN) communication vehicle control unit (VCU) electric control system.

[0112] Optionally, in the sensor architecture, the laser radar, the front millimeter wave radar, the angular millimeter wave radar, and the ultrasonic wave provide perception data of surrounding objects, the panoramic camera provides visual perception results of the side of the vehicle, the front-view camera provides visual perception results of the front of the vehicle, and the camera provides perception data for lane changing. Among them, the front-view camera sensor is multiplexed in the sensor suite 213 and the sensor suite 214, and the image information collected by the front-view camera can be transmitted to the two domain controllers through data multiplexing.

[0113] It should be noted that the types and quantities of the above-mentioned sensors are only for illustration and are not limited.

[0114] In the related art, for the redundant architecture design, whether the domain control hardware is double backup or the single domain control hardware is double SOC design, the algorithm architecture deployed in it is the same algorithm architecture, which is the algorithm architecture based on data fusion. In the sensor layer, the algorithm architecture accesses the same sensor architecture. However, in the above method, since the algorithm architecture in the autonomous driving domain control hardware is the same, and the domain control hardware is coupled with the sensor, in some sensor failure scenarios, there is a problem that the autonomous driving system cannot perform the MRM side parking behavior. In this embodiment, different software algorithm architectures are deployed in each domain control, which first decouples the sensor dependence from the software algorithm. In the hardware access scheme of the sensor and the domain control, different domain controls access different sensor suite schemes, and the decoupling of the sensor and the domain control hardware is realized.

[0115] Optionally, each domain control accesses an independent sensor suite scheme, which first completes decoupling at the hardware level. The algorithm architecture based on multi-data fusion is deployed in the SOC of the main domain control hardware, and the visual algorithm architecture is deployed in the SOC of the redundant domain control hardware. The decoupling is realized in the software algorithm architecture, and different software algorithm architectures access different sensor suite schemes, thereby realizing the double decoupling effect of hardware and software. When the sensor elements in different sensor suites fail, the arbitration logic is used to determine whether to use the data fusion algorithm of the main controller or the pure visual algorithm of the auxiliary controller, so that the software and hardware can be decoupled to the maximum extent, and the failure of a certain sensor can be avoided. The autonomous driving system is seriously degraded or crashes.

[0116] Figure 3 is a flowchart of logic determination of autonomous driving domain control switching according to an embodiment of the present application, as shown in Figure 3 The method can include the following steps:

[0117] Step S302, self-checking the main redundant controller.

[0118] In this embodiment, after the vehicle is powered on, the main redundant controller can be self-checked.

[0119] Step S304, the main controller controls the vehicle.

[0120] After the self-checking is completed and no error is found, the main redundant controller (which can also be referred to as the main auxiliary controller) can start all the functional modules in the domain control. Since all the functional modules of the main auxiliary controller are pulled up, the arbitration mode can be determined to control the vehicle by using the main controller.

[0121] Step S306, detecting a related failure.

[0122] In this embodiment, during the travel of the vehicle, a failure of the sensor occurs, and the influence degree of the failure at this time can be determined to decide whether to continue to control the vehicle using the main controller or to control the vehicle using the auxiliary controller.

[0123] Step S308: Determine whether the main controller can continue to control the vehicle.

[0124] In this embodiment, it is determined whether the main controller can continue to control the vehicle, and if yes, the main controller can be used to continue to control the vehicle, and if no, step S310 can be performed.

[0125] Optionally, when the laser radar in the sensor suite associated with the main controller fails, it can be determined to still use the main domain controller of the self-driving to control the vehicle, mainly relying on the sensing results of the surround-view camera and the front-view camera and other sensors in the sensor suite, and using the multi-data fusion perception algorithm in the main controller to control the vehicle to pull over and park, so that the vehicle reaches the final MRC state.

[0126] Step S310: The auxiliary controller controls the vehicle.

[0127] Optionally, the same arbitration strategy as described above can be performed when the front millimeter wave, angle millimeter wave, ultrasonic wave, and front-view camera in the sensor suite associated with the main controller fail. However, when the surround-view camera in the sensor suite fails, the vehicle loses the key information for pulling over because it cannot provide visual sensing information of the vehicle surroundings. At this time, the vehicle system determines that the main controller cannot be used any more, and the vehicle can be switched to the pure visual algorithm architecture of the auxiliary controller for control.

[0128] Step S312: Perform the strategy of MRM.

[0129] Optionally, after switching to the auxiliary controller, the selection of the sensor is also automatically switched to the sensor suite associated with the auxiliary controller. In this sensor suite, the surround-view camera is not multiplexed with the surround-view camera in the sensor suite associated with the main controller, and therefore, complete visual sensing information can be provided to the vehicle, and reliable basis can be provided for the vehicle to perform corresponding pull-over operations. When the vehicle finally pulls over and parks, the vehicle finally reaches the safe state of MRC.

[0130] Optionally, when the algorithm module of the main self-driving domain controller fails, the auxiliary controller can be automatically determined as the target controller, the visual perception algorithm in the auxiliary controller is used, and the sensing results of the sensor suite associated with the auxiliary controller are used to perform the pull-over lane-changing strategy of MRM, so that the vehicle reaches the safe state of MRC.

[0131] Optionally, when the auxiliary controller or the sensor in the sensor suite associated with the auxiliary controller fails, the main controller can be used as the target controller, the multi-data fusion algorithm architecture in the main controller is still used, the perception fusion data collected by the sensor in the sensor suite associated with the main control is used to control the vehicle, and the vehicle is parked on the side, so that the vehicle reaches the final MRC state.

[0132] The embodiment implements a control data determination method of a vehicle. In the method, state information of multiple sensor suites is collected during driving of the vehicle. The state information can be used to determine the working state of the sensors. Based on the state information, a target controller for controlling the vehicle can be determined. Environmental information collected by the sensors associated with the target controller can be obtained. Based on the environmental information, control information for controlling the vehicle can be determined. Thus, the technical problem of low control efficiency of the vehicle is solved, and the technical effect of high control efficiency of the vehicle is achieved.

[0133] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose authorization or refusal.

[0134] According to the embodiment of the present application, a control data determination device embodiment of a vehicle is provided. It should be noted that the device can be used to execute the control data determination method of the vehicle.

[0135] Figure 4 is a schematic diagram of a control data determination device of a vehicle according to an embodiment of the present application. As shown in Figure 4 the control data determination device of the vehicle can include an acquisition unit 42, a first determination unit 44, a second determination unit 46 and a control unit 48.

[0136] The acquisition unit 42 is configured to collect state information of multiple sensor suites in the vehicle during driving of the vehicle, wherein the sensor suite includes multiple sensors, and the state information is used to represent the working state of the sensors.

[0137] The first determining unit 44 is configured to determine, based on the state information, a target controller for controlling the vehicle from a main controller and an auxiliary controller of the vehicle, wherein the main controller and the auxiliary controller are configured to process different sensor suite collected environmental information, and the main controller and the auxiliary controller are different in processing the environmental information, and the environmental information is used at least for determining obstacle information of the vehicle in the driving process.

[0138] The second determining unit 46 is configured to determine, from a plurality of sensor suites, a target sensor suite associated with the target controller, and obtain a plurality of environmental information collected by a plurality of target sensors in the target sensor suite.

[0139] The control unit 48 is configured to control the target controller to convert the plurality of environmental information to obtain control data of the vehicle, wherein the control data is used to control the driving state of the vehicle.

[0140] Optionally, the first determining unit 44 comprises a first determining module configured to determine, based on the state information, a working state of an image collection device in a sensor suite associated with the main controller; and determine, based on the working state of the image collection device, the target controller for controlling the vehicle from the main controller and the auxiliary controller.

[0141] Optionally, the first determining module comprises a first determining submodule configured to determine the auxiliary controller as the target controller in response to the working state of the image collection device being an abnormal working state; and determine the main controller as the target controller in response to the working state of the image collection device being a normal working state.

[0142] Optionally, the first determining submodule is further configured to determine, in response to the auxiliary controller being the target controller, a sensor suite associated with the auxiliary controller as the target sensor suite; and obtain a plurality of environmental information collected by a plurality of target sensors in the target sensor suite, comprising: obtaining image information collected by the plurality of target sensors at a plurality of perspectives, wherein the plurality of environmental information comprises the image information collected at the plurality of perspectives, and the image information is used to determine obstacle information of the vehicle at the perspective and lane information of a lane in which the vehicle drives.

[0143] Optionally, the control unit 48 further comprises a first processing unit configured to control the target controller to respectively identify the plurality of image information to obtain the lane information and the obstacle information; control the target controller to determine a driving path of the vehicle based on the lane information and the obstacle information; and control the target controller to convert the driving path to the control data.

[0144] Optionally, the second determining unit 46 further comprises a second determining sub-module, configured to, in response to the main controller being the target controller, determine the sensor suite associated with the main controller as a target sensor suite; and acquire image information collected by a plurality of image collection devices under a plurality of target visual angles and radar data collected by a plurality of target sensors, wherein the plurality of environmental information comprises the plurality of image information collected under the plurality of target visual angles and the plurality of radar data, the image information is used to determine obstacle information of the vehicle under the target visual angle and lane information of a lane on which the vehicle travels, and the obstacle information is used to at least determine a position of an obstacle in the process of traveling of the vehicle, and the radar data is used to at least determine a distance between the vehicle and the obstacle.

[0145] Optionally, the control unit 48 further comprises a second processing unit, configured to control the target controller to fuse the plurality of image information and the plurality of radar data to obtain fused data; control the target controller to predict the fused data to obtain a traveling path of the vehicle; and control the target controller to convert the traveling path into control data.

[0146] Optionally, the second processing unit further comprises a processing sub-module, configured to acquire auxiliary image information collected by an image collection device associated with the auxiliary controller; and fuse the image information, the auxiliary image information and the radar data to obtain the fused data.

[0147] In the control data determination apparatus of the vehicle in this embodiment, the state information of a plurality of sensor suites in the vehicle is acquired by the acquisition unit in the process of traveling of the vehicle, wherein the sensor suite comprises a plurality of sensors, and the state information is used to represent the working state of the sensors. The target controller for controlling the traveling of the vehicle is determined from the main controller of the vehicle and the auxiliary controller of the vehicle based on the state information by the first determining unit, wherein the main controller and the auxiliary controller are used to process different environmental information collected by the sensor suite, and the main controller and the auxiliary controller process the environmental information in different ways, and the environmental information is used to at least determine obstacle information in the process of traveling of the vehicle. The target sensor suite associated with the target controller is determined from the plurality of sensor suites by the second determining unit 46, and a plurality of environmental information collected by a plurality of target sensors in the target sensor suite is acquired. The target controller is controlled by the control unit to convert the plurality of environmental information to obtain control data of the vehicle, wherein the control data is used to control the traveling state of the vehicle, thereby solving the technical problem of low control efficiency of the vehicle, and further achieving the technical effect of high control efficiency of the vehicle.

[0148] The embodiments of the present application also provide a vehicle, comprising a memory storing an executable program; and a processor configured to run the program, wherein the program is executed to perform the method in the embodiments of the present application.

[0149] The embodiment of the present application further provides a computer readable storage medium comprising a stored executable program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the method in each embodiment of the present application when the executable program is executed.

[0150] The embodiment of the present application further provides a computer program product comprising a computer program, which, when executed by a processor, implements the method in each embodiment of the present application.

[0151] The embodiment of the present application further provides a computer program product comprising a nonvolatile computer readable storage medium for storing a computer program, which, when executed by a processor, implements the method in each embodiment of the present application.

[0152] The embodiment of the present application further provides a computer program, which, when executed by a processor, implements the method in each embodiment of the present application.

[0153] In the above-described embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0154] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other manners. Among them, the apparatus embodiment described above is only schematic, for example, the division of the unit can be a logical function division, and in actual implementation, there can be another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, and can be electrical or other forms.

[0155] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0156] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0157] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0158] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method for determining control data of a vehicle, characterized in that: include: During the driving of the vehicle, status information of multiple sensor suites in the vehicle is collected, wherein the sensor suites include multiple sensors, and the status information is used to represent the working status of the sensors; determining, based on the state information, a target controller for controlling the driving of the vehicle from a main controller of the vehicle and a secondary controller of the vehicle, wherein the main controller and the secondary controller are used to process environmental information collected by different sensor suites, and the main controller processes the environmental information in a manner different from the manner in which the secondary controller processes the environmental information, and the environmental information is used to at least determine obstacle information during driving of the vehicle; Determining a target sensor suite associated with the target controller from the plurality of sensor suites, and acquiring a plurality of environmental information collected by a plurality of target sensors in the target sensor suite; The target controller is controlled to convert the plurality of environmental information to obtain control data of the vehicle, wherein the control data is used to control the driving state of the vehicle.

2. The method according to claim 1, characterized in that The determining, based on the state information, a target controller for controlling the driving of the vehicle from a main controller of the vehicle and an auxiliary controller of the vehicle, comprises: determining, based on the status information, an operating status of an image acquisition device in the sensor suite associated with the main controller; Based on the working status of the image acquisition device, the target controller for controlling the driving of the vehicle is determined from the main controller and the auxiliary controller.

3. The method according to claim 2, characterized in that The determining, based on the working state of the image acquisition device, the target controller for controlling the driving of the vehicle from the main controller and the auxiliary controller includes: In response to the working state of the image acquisition device being an abnormal working state, determining the auxiliary controller as the target controller; In response to the working state of the image acquisition device being a normal working state, the main controller is determined as the target controller.

4. The method according to claim 3, characterized in that The step of determining a target sensor suite associated with the target controller from the plurality of sensor suites includes: In response to the auxiliary controller being the target controller, determining the sensor suite associated with the auxiliary controller as the target sensor suite; The acquiring of the plurality of environmental information collected by the plurality of target sensors in the target sensor suite includes: acquiring image information collected by the plurality of target sensors under a plurality of perspectives, wherein the plurality of environmental information includes the image information collected under a plurality of perspectives, and the image information is used to determine the obstacle information of the vehicle under the perspectives, and the lane information of the lane in which the vehicle is traveling.

5. The method according to claim 4, characterized in that The controlling the target controller to convert the plurality of environmental information to obtain control data of the vehicle includes: Controlling the target controller to respectively identify the plurality of image information to obtain the lane information and the obstacle information; controlling the target controller to determine a driving path of the vehicle based on the lane information and the obstacle information; The target controller is controlled to convert the driving path into the control data.

6. The method according to claim 3, characterized in that The step of determining a target sensor suite associated with the target controller from the plurality of sensor suites includes: In response to the main controller being the target controller, determining the sensor suite associated with the main controller as the target sensor suite; The acquiring of the multiple environmental information collected by the multiple target sensors in the target sensor suite includes: acquiring image information under multiple target perspectives collected by the multiple image acquisition devices, and multiple radar data collected by the multiple target sensors, wherein the multiple environmental information includes the multiple image information collected under multiple target perspectives, and the multiple radar data, the image information is used to determine obstacle information of the vehicle under the target perspective, and lane information of the lane in which the vehicle is traveling, the obstacle information is at least used to determine the position of the obstacle while the vehicle is traveling, and the radar data is at least used to determine the distance between the vehicle and the obstacle.

7. The method according to claim 6, characterized in that The controlling the target controller to convert the plurality of environmental information to obtain control data of the vehicle includes: Controlling the target controller to fuse the plurality of image information and the plurality of radar data to obtain fused data; controlling the target controller to predict the fused data to obtain a driving path of the vehicle; The target controller is controlled to convert the driving path into the control data.

8. The method according to claim 7, characterized in that The controlling the target controller to fuse the plurality of image information and the plurality of radar data to obtain fused data includes: Acquiring auxiliary image information captured by an image capture device associated with the auxiliary controller; The image information, the auxiliary image information and the radar data are fused to obtain the fused data.

9. A vehicle, characterized in that: include: a memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 8 when running.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 8.

Images