Method, apparatus, device and storage medium for vehicle control

Abstract

According to embodiments of the present disclosure, a method, apparatus and storage medium for vehicle control are provided. The method comprises determining, in response to detecting a fault associated with an autonomous vehicle, a recovery strategy corresponding to the fault from a preset configuration table; determining, in response to determining that the recovery strategy indicates not to maintain the fault, whether a recovery operation needs to be performed based on the recovery strategy; determining, in response to determining that the recovery operation needs to be performed, at least one recovery operation to be performed based on the recovery strategy; and controlling the autonomous vehicle to perform the at least one recovery operation. Thus, embodiments of the present disclosure can perform corresponding recovery operations on the autonomous vehicle based on the recovery strategy corresponding to the fault associated with the autonomous vehicle, improve the efficiency of fault handling, and optimize the performance of the overall system of the autonomous vehicle.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, computer-readable storage media, and computer program products for vehicle control. Background Technology

[0002] With the rapid development of autonomous driving technology, people are paying increasing attention to how to handle malfunctions in autonomous vehicles. For example, when a malfunction is confirmed, relevant functions can be degraded. However, when an autonomous vehicle's functions are degraded, it can cause a loss of capability in the entire system corresponding to the vehicle. It may not possess complete autonomous driving capabilities, and the continuous loss of capability may even affect the vehicle's Operational Design Domain (ODD). How to handle malfunctions in autonomous vehicles is a key concern. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for vehicle control is provided. The method includes: in response to detecting a fault associated with an autonomous vehicle, determining a recovery strategy corresponding to the fault from a preset configuration table; in response to determining that the recovery strategy indicates that the fault should not be maintained, determining whether a recovery operation needs to be performed based on the recovery strategy; in response to determining that a recovery operation needs to be performed, determining at least one recovery operation to be performed based on the recovery strategy; and controlling the autonomous vehicle to perform the at least one recovery operation.

[0004] In a second aspect of this disclosure, an apparatus for vehicle control is provided. The apparatus includes: a first determining module configured to determine a recovery strategy corresponding to a fault from a preset configuration table in response to detecting a fault associated with an autonomous vehicle; a second determining module configured to determine whether a recovery operation needs to be performed based on the recovery strategy in response to determining that the recovery strategy indicates that the fault should not be maintained; a third determining module configured to determine at least one recovery operation to be performed based on the recovery strategy in response to determining that a recovery operation needs to be performed; and a control module configured to control the autonomous vehicle to perform at least one recovery operation.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method of the first aspect.

[0008] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A schematic diagram of a vehicle control process according to some embodiments of the present disclosure is shown;

[0012] Figure 3 This disclosure provides a schematic flowchart of vehicle control according to some embodiments of the present disclosure;

[0013] Figure 4 This disclosure provides a schematic flowchart for handling dirt malfunctions in image sensors according to some embodiments of this disclosure;

[0014] Figure 5 This disclosure illustrates a schematic diagram of a system framework for vehicle control according to some embodiments of this disclosure;

[0015] Figure 6 A schematic structural block diagram of a device for vehicle control according to certain embodiments of the present disclosure is shown;

[0016] Figure 7 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0019] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0020] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0021] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0022] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," a "machine learning network," or simply a "network," and these terms are used interchangeably.

[0023] Traditionally, when a fault is detected in an autonomous vehicle, the electronic equipment can trigger a function degradation operation. After the function degradation operation is triggered, the corresponding function of the autonomous vehicle will be unavailable. At this time, only two recovery methods are supported: system restart to make the fault disappear or continuous waiting for fault messages.

[0024] On the one hand, traditional fault handling methods only perform functional degradation operations without developing recovery strategies for the fault itself. This significantly impacts the availability of autonomous driving systems. Once a fault occurs and persists, the system remains unavailable, severely affecting the overall ODD (Original Design Domain) of the autonomous vehicle. On the other hand, in scenarios where autonomous vehicles passively wait for faults to disappear, the faults that do disappear are primarily those triggered by abnormal external inputs, neglecting faults caused by problems within the system itself. Furthermore, traditional fault handling methods are unsuitable for certain fault scenarios. For example, when sensors are dirty, the fault cannot be automatically eliminated without performing corresponding cleaning operations.

[0025] Embodiments of this disclosure provide a vehicle control scheme. According to various embodiments of this disclosure, in response to detecting a fault associated with an autonomous vehicle, a recovery strategy corresponding to the fault is determined from a preset configuration table; in response to determining that the recovery strategy indicates that the fault should not be maintained, a recovery operation is determined based on the recovery strategy; in response to determining that a recovery operation needs to be performed, at least one recovery operation to be performed is determined based on the recovery strategy; and the autonomous vehicle is controlled to perform at least one recovery operation.

[0026] Therefore, the embodiments of this disclosure can perform corresponding recovery operations on autonomous vehicles based on the recovery strategies associated with the autonomous vehicles, thereby improving the efficiency of fault handling and optimizing the overall performance of the autonomous vehicle system.

[0027] Example Environment

[0028] Figure 1 A schematic diagram of an example environment 100 in which several embodiments of the present disclosure can be implemented is shown.

[0029] An autonomous vehicle 110 is schematically illustrated in this example environment 100. This autonomous vehicle 110 can be any type of vehicle capable of carrying people and / or goods and moving via a power system such as an engine. Examples of autonomous vehicles 110 include, but are not limited to, cars, trucks, buses, electric vehicles, motorcycles, motorhomes, trains, etc. As an example, the autonomous vehicle 110 of this disclosure can be a vehicle with some level of assisted driving capability or autonomous driving capability; such vehicles are also referred to as intelligent driving vehicles.

[0030] When an electronic device 120 detects a fault associated with the autonomous vehicle 110, it can perform a predetermined type of control operation on the autonomous vehicle 110, wherein the control operation performed varies depending on the type of fault. In some embodiments, the electronic device 120 can be deployed on the autonomous vehicle 110, or it can be another device independent of the autonomous vehicle 110.

[0031] Electronic device 120 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, electronic device 120 may also support any type of user-facing interface (such as "wearable" circuitry).

[0032] Electronic device 120 can also be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Electronic device 120 may include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in cloud environments, and so on.

[0033] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0034] Example process

[0035] Figure 2 A flowchart of a vehicle control process 200 according to some embodiments of the present disclosure is shown. Process 200 can be implemented at an electronic device 120. Reference is made below. Figure 1 Describe the process 200.

[0036] In box 210, electronic device 120, in response to detecting a fault associated with autonomous vehicle 110, determines a recovery strategy corresponding to the fault from a preset configuration table.

[0037] In some embodiments, the autonomous vehicle 110 can be a vehicle that navigates and drives independently of human operation, and has certain assisted driving capabilities or autonomous driving capabilities. Such a vehicle is also referred to as an intelligent driving vehicle.

[0038] In some embodiments, the faults associated with the autonomous vehicle 110 may include any suitable fault, such as hardware faults (e.g., sensor faults), software faults, communication faults, navigation system faults, etc. The faults associated with the autonomous vehicle 110 may be identified by any suitable system or component (which may be deployed on the autonomous vehicle 110 or other remote devices), and the corresponding identified system or component may be different for different faults.

[0039] As an example, the redundant sensor system on autonomous vehicle 110 can identify the faults corresponding to the image sensors on autonomous vehicle 110.

[0040] In some embodiments, to improve the efficiency and flexibility of fault handling by performing different vehicle control operations for different faults, the electronic device 120 may have a preset configuration table that at least indicates the recovery strategy corresponding to different faults. This recovery strategy is used to guide the autonomous vehicle 110 on how to respond to and handle a specific fault when it occurs.

[0041] In some embodiments, the electronic device 120 can determine identification information corresponding to a fault. The identification information is information used to characterize the fault, and different faults correspond to different identification information.

[0042] Furthermore, the electronic device 120 can determine the data entry corresponding to the identification information from the configuration table. The data entry includes multiple parameters indicating the recovery strategy.

[0043] In some embodiments, these multiple parameters include multiple of the following: a first parameter indicating whether a recovery operation needs to be performed; a second parameter indicating whether the fault should be maintained; and a third parameter indicating at least one recovery operation to be performed.

[0044] In some embodiments, Table 1 is an example of a preset configuration table disclosed herein, and Table 1 will now be described:

[0045]

[0046] Table 1

[0047] As shown in Table 1, for the fault of moderate blindness of camera 2 on autonomous vehicle 110, the corresponding parameters include the need to perform a recovery operation (a value of 1 for the feature "whether to perform a recovery operation" indicates that a recovery operation is needed, and a value of 0 indicates that a recovery operation is not needed), the fault does not need to be maintained (a value of 1 for the feature "whether to maintain the fault" indicates that a fault needs to be maintained, and a value of 0 indicates that a fault does not need to be maintained), and at least one recovery operation to be performed is operation 200, where operation 200 represents the corresponding operation identifier, such as triggering the image sensor cleaning device to clean.

[0048] In box 220, electronic device 120, in response to determining that the recovery policy indicates that the fault should not be maintained, determines whether a recovery operation needs to be performed based on the recovery policy.

[0049] In some embodiments, the autonomous vehicle 110 that does not maintain a fault characterization should not maintain its current fault state, i.e., it needs to change its state to a non-fault state.

[0050] In some embodiments, electronic device 120 may determine whether to wait for autonomous vehicle 110 to self-recover in response to a recovery strategy indication not to maintain the fault; that is, to wait for autonomous vehicle 110 to recover on its own without performing a recovery operation. In some embodiments, whether or not to wait for autonomous vehicle 110 to self-recover may be determined based on the fault type corresponding to autonomous vehicle 110. As an example, if the external disk space on autonomous vehicle 110 is insufficient, this fault may affect data recording but not autonomous driving. In this case, the Minimum Risk Condition (MRC) system, the first redundancy system (L1 fallback), and the second redundancy system (L2 fallback) are all available, so electronic device 120 may not control autonomous vehicle 110 to perform a recovery operation. As another example, if communication between the electric automatic door of autonomous vehicle 110 and the autonomous driving control unit is lost, the system cannot control the electric door opening and closing, which affects passenger boarding and alighting but does not affect autonomous driving. In this case, electronic device 120 may determine not to control autonomous vehicle 110 to perform a recovery operation.

[0051] In some embodiments, electronic device 120 may determine that a recovery operation needs to be performed in response to determining that it is not necessary to wait for autonomous vehicle 110 to self-recover. In other embodiments, electronic device 120 may determine that a recovery operation is not necessary in response to determining that it is necessary to wait for autonomous vehicle 110 to self-recover.

[0052] In some embodiments, electronic device 120 may wait for autonomous vehicle 110 to self-recover in response to determining that a recovery operation is not required. Further, electronic device 120 may determine whether the fault self-recovers within a first time period in response to determining that a recovery operation is not required. The first time period can be any suitable duration, such as a predetermined duration or a duration parameter configured in a configuration table. Specifically, the corresponding duration parameter can be the same or different for different faults in the configuration table. Taking Table 1 as an example, the first time period is the time threshold corresponding to self-recovery.

[0053] Furthermore, the electronic device 120 may generate a message indicating that self-recovery has failed in response to a fault that has not been self-recovered within a first time period.

[0054] Returning to box 220, in some embodiments, electronic device 120 may, in response to determining a recovery strategy instruction to maintain the fault, save fault information associated with the fault. Fault information may include, but is not limited to, fault codes (fault identifiers), fault time, fault cause, etc., where fault codes are numeric or alphanumeric codes used to identify a specific problem or error. As an example, electronic device 120 may generate a resident fault code and save this fault information including the fault code; this process does not perform any fault handling on the autonomous vehicle 110 until a cold start operation is performed after a manual inspection of the autonomous vehicle 110.

[0055] In box 230, in response to determining that a recovery operation needs to be performed, electronic device 120 determines at least one recovery operation to be performed based on a recovery strategy.

[0056] In some embodiments, the at least one recovery operation includes at least one of the following: restarting a single node corresponding to the fault, restarting multiple nodes corresponding to the fault, and performing a preset action using a target device in the autonomous vehicle 110.

[0057] A node can be any appropriate node that causes a malfunction in the autonomous vehicle 110 or can resolve the corresponding malfunction of the autonomous vehicle 110. For example, it can be a hardware node (such as a sensor or controller) or a software node (such as a perception module or control module), etc.

[0058] The target device can be any suitable device, such as a sensor (image sensor, LiDAR, etc.), actuator (such as a motor, brake), controller, or other critical hardware. The preset action can be any suitable action, such as restart, shutdown, parameter adjustment, alarm, etc.

[0059] As an example, if a malfunction in autonomous vehicle 110 is associated with dirt on a sensor, the recovery operation to be performed could be to activate a cleaning device in autonomous vehicle 110 to clean the sensor. The cleaning device could be a water spray device, an air jet device, etc. As another example, if an image sensor in autonomous vehicle 110 malfunctions due to dirt, electronic equipment 120 could control the water spray device in autonomous vehicle 110 to spray water to clean the image sensor.

[0060] In frame 240, electronic device 120 controls autonomous vehicle 110 to perform at least one recovery operation.

[0061] In some embodiments, electronic device 120 may send control commands to autonomous vehicle 110 instructing at least one recovery operation, so that electronic device 120 may perform at least one recovery operation to resolve the fault corresponding to autonomous vehicle 110.

[0062] As an example, if the lidar on the autonomous vehicle 110 malfunctions, since this malfunction may cause abnormal perception output, and since the lidar is the main sensor, its abnormality has a great impact on the MRC system and the first redundancy system (L1 fallback), the electronic device 120 can control the autonomous vehicle 110 to activate the second redundancy system (L2 fallback).

[0063] As another example, if the software component or process (prediction_node) responsible for the prediction task on the autonomous vehicle 110 fails, the main system MRC of the autonomous vehicle 110 will be unavailable, but the first redundancy system (L1 fallback) and the second redundancy system (L2 fallback) will be available. Therefore, the electronic device 120 can control the autonomous vehicle 110 to start the first redundancy system (L1 fallback) or the second redundancy system (L2 fallback).

[0064] To improve the control efficiency and ensure the safety of the autonomous vehicle 110, in some embodiments, the electronic device 120 may generate an assistance request associated with the fault in response to the fault not being recovered within a second time period. The assistance request may be a request sent to a remote device to seek manual intervention. In some embodiments, the second time period may be any suitable duration, such as a predetermined duration or a second time period parameter configured in a configuration table. Specifically, the second time period parameter may be different or the same for different faults in the configuration table. Taking Table 1 as an example, this second time period is also the recovery handling timeout threshold.

[0065] Figure 3 This disclosure provides a schematic flowchart of vehicle control according to some embodiments of the present disclosure, and now addresses... Figure 3 Please provide an explanation.

[0066] In box 301, the fault detection node detects a fault in the autonomous vehicle 110. As an example, the fault can include any appropriate fault, such as hardware fault (e.g., sensor fault), software fault, communication fault, navigation system fault, etc.

[0067] In box 302, the fault detection node sets a fault code. In some embodiments, this fault code is the fault identifier corresponding to the autonomous vehicle 110. Setting this fault code by the fault detection node can be a step to be performed after a fault is detected. The fault code is a numeric or alphanumeric code used to identify a specific problem or error. In this case, the fault code can serve as a temporary marker to indicate that a specific fault has occurred in the autonomous vehicle 110.

[0068] In box 303, electronic device 120 executes a recovery strategy based on the fault code.

[0069] In some embodiments, the electronic device 120 can execute a recovery strategy based on the fault code sent by the fault detection node. Specifically, the electronic device 120 can determine the data entry corresponding to the fault code from a preset configuration table. This data entry includes multiple parameters indicating the recovery strategy, such as whether a recovery operation needs to be performed, whether the fault should be maintained, and at least one recovery operation to be performed. That is, the electronic device 120 can determine these multiple parameters (attributes) by matching the received real-time fault code information with the fault recovery attributes defined in the preset configuration table.

[0070] In box 304, electronic device 120 determines whether to maintain the fault.

[0071] In some embodiments, the electronic device 120 determines whether to maintain the fault based on the data entry corresponding to the fault code. Specifically, if the electronic device 120 determines to maintain the fault, it executes the operation in block 306. If the electronic device 120 determines not to maintain the fault, it executes the operation in block 305.

[0072] In box 305, electronic device 120 determines whether to wait for self-recovery.

[0073] In some embodiments, self-recovery refers to waiting for the autonomous vehicle 110 to recover automatically without performing a recovery operation. In some embodiments, the electronic device 120 may perform the operation of block 307 in response to determining that self-recovery needs to be waited for. The electronic device 120 may perform the operation of block 308 in response to determining that self-recovery is not required.

[0074] In box 306, electronic device 120 generates a resident fault code and saves the fault information until it is manually checked and then cold-started to recover.

[0075] In some embodiments, a resident fault code represents a fault code generated for the fault and capable of being recorded and stored long-term for subsequent diagnosis and maintenance. The electronic device 120 can also store fault information associated with the fault. This fault information may include, but is not limited to, resident fault codes, fault time, fault cause, etc., wherein the resident fault code is a numeric or alphanumeric code used for long-term recording or storage to identify a specific problem or error. In some embodiments, the electronic device 120 may not perform any fault handling on the autonomous vehicle 110 until a cold start operation is performed after a manual inspection of the autonomous vehicle 110.

[0076] In box 307, wait for self-recovery. If the self-recovery time threshold is exceeded, report self-recovery failure and record it.

[0077] In some embodiments, the self-recovery time threshold can also be referred to as the first duration. This self-recovery time threshold can be any appropriate duration, such as a predetermined duration or a duration parameter configured in the configuration table. Specifically, for different faults in the configuration table, the corresponding duration parameters can be the same or different.

[0078] In box 308, electronic device 120 determines to perform a recovery operation.

[0079] In some embodiments, the electronic device 120 may determine to perform at least one recovery operation. Specifically, it may perform at least one recovery operation corresponding to the fault configuration of the autonomous vehicle 110 in a preset configuration table. Specifically, it may perform one of the recovery operations in blocks 309, 310, and 311.

[0080] In box 309, the single node corresponding to the restart failure of electronic device 120.

[0081] A node can be any suitable node that causes a malfunction in the autonomous vehicle 110 or can resolve the corresponding malfunction in the autonomous vehicle 110. For example, it can be a hardware node (such as a sensor or controller) or a software node (such as a perception module or control module), etc. This single node can be one of these suitable nodes that is associated with the malfunction.

[0082] In box 310, multiple nodes corresponding to the restart failure of electronic device 120.

[0083] A node can be any suitable node that causes a malfunction in the autonomous vehicle 110 or can resolve the corresponding malfunction in the autonomous vehicle 110. For example, it can be a hardware node (such as a sensor or controller) or a software node (such as a perception module or control module), etc. These multiple nodes can be at least two of these suitable nodes that are associated with the malfunction.

[0084] In frame 311, electronic device 120 performs preset actions using a target device in autonomous vehicle 110.

[0085] The target device can be any suitable device, such as a sensor (image sensor, LiDAR, etc.), actuator (such as a motor, brake), controller, or other critical hardware. The preset action can be any suitable action, such as restart, shutdown, parameter adjustment, alarm, etc.

[0086] As one example, electronic device 120 can control a sound sensor (such as a car audio system) in autonomous vehicle 110 to send alarm information. As another example, electronic device 120 can control a water spray system in autonomous vehicle 110 to spray water to clean image sensors.

[0087] In box 312, electronic device 120 performs a recovery operation timing and determines whether the current time has expired.

[0088] In some embodiments, in order to improve the control efficiency of the autonomous vehicle 110 and ensure the safety of the autonomous vehicle 110, in some embodiments, the electronic device 120 may determine whether a timeout has occurred after resuming the operation timing. This timing range may be a second duration, that is, no timeout has occurred within the second duration, and a timeout has occurred if the second duration has been exceeded.

[0089] In box 313, electronic device 120 determines that recovery has failed, records the failure, and triggers manual troubleshooting.

[0090] In some embodiments, the electronic device 120 may generate a fault-related assistance request to seek human assistance in handling the fault of the autonomous vehicle 110.

[0091] In box 314, electronic device 120 confirms successful recovery and records the recovery record.

[0092] Figure 4 This disclosure provides a schematic flowchart for handling contamination faults in image sensors according to some embodiments of this disclosure, and now addresses... Figure 4 Please provide an explanation.

[0093] In box 401, the sensing node reports the dirt fault information corresponding to the image sensor to the fault detection node.

[0094] In some embodiments, the sensing node can be a node used to sense whether the image sensor is malfunctioning, which can be determined based on information from images acquired by the image sensor.

[0095] In box 402, the fault detection node receives a fault notification.

[0096] In some embodiments, the fault detection node can receive dirt fault information corresponding to the image sensor sent by the sensing node and confirm that the current image sensor has failed.

[0097] In box 403, the fault detection node sets the dirt fault code corresponding to the image sensor.

[0098] In some embodiments, this dirt fault code is an identifier indicating that the image sensor of the autonomous vehicle 110 is malfunctioning due to dirt.

[0099] In box 404, the fault recovery node executes a recovery strategy based on the dirt fault code.

[0100] In some embodiments, the electronic device 120 can execute a recovery strategy based on the dirt fault code sent by the fault detection node. Specifically, the electronic device 120 can determine the data entry corresponding to the dirt fault code from a preset configuration table. This data entry includes multiple parameters indicating the recovery strategy, such as whether a recovery operation needs to be performed, whether the fault should be maintained, and at least one recovery operation to be performed. That is, the electronic device 120 can determine these multiple parameters (attributes) by matching the dirt fault code with the fault recovery attributes defined in the preset configuration table based on the received real-time dirt fault code information.

[0101] In box 405, the fault recovery node determines whether to maintain the fault.

[0102] In some embodiments, the fault recovery node determines whether to maintain the fault based on the data entry corresponding to the dirt fault code. If the fault recovery node determines that the fault should not be maintained, the operation of block 406 is performed.

[0103] In box 406, the fault recovery node determines the node that needs to perform the recovery operation.

[0104] In some embodiments, the electronic device 120 may determine that the node needs to perform at least one recovery operation. Specifically, it may determine at least one recovery operation corresponding to the dirt fault configuration of the image sensor of the autonomous vehicle 110 in a preset configuration table.

[0105] In box 407, the fault recovery node sends a command to trigger the opening of the cleaning device to the Controller Area Network Bus (CAN Bus) node.

[0106] In some embodiments, CAN Bus is widely used in the field of vehicle control to connect and control various electronic control units, such as engine control units, anti-lock braking systems, navigation systems, airbag systems, etc.

[0107] In box 408, the CAN Bus node converts the command to trigger the opening of the cleaning device into a communication signal and sends it to the bus.

[0108] In some embodiments, the bus has a communication connection with the cleaning device and can send communication signals to the cleaning device.

[0109] In frame 409, the cleaning device receives communication signals via a bus and drives the motor to perform water spraying cleaning operations.

[0110] In box 410, the fault recovery node starts the recovery operation timer and determines whether the current time has expired.

[0111] In some embodiments, in order to improve the control efficiency of the autonomous vehicle 110 and ensure the safety of the autonomous vehicle 110, in some embodiments, the electronic device 120 may determine whether a timeout has occurred after resuming the operation timing. This timing range may be a second duration, that is, no timeout has occurred within the second duration, and a timeout has occurred if the second duration has been exceeded.

[0112] In box 411, electronic device 120 determines that recovery has failed, records the failure, and triggers manual troubleshooting.

[0113] In some embodiments, the electronic device 120 may generate a fault-related assistance request to seek human assistance in handling the fault of the autonomous vehicle 110.

[0114] In box 412, electronic device 120 confirms successful recovery and records the recovery record.

[0115] Returning to box 401, after the sensing node reports the dirt fault information corresponding to the image sensor to the fault detection node, it can execute the operation in box 413.

[0116] In box 413, the sensing node detects in real time whether the dirt fault of the image sensor has been resolved.

[0117] The sensing node can trigger the fault detection node to execute the operation in box 414 in response to detecting that the image sensor has cleared a dirt fault, i.e., there is no dirt. For example, the sensing node can send an indication message to the fault detection node that the image sensor does not have a dirt fault.

[0118] In box 414, the fault detection node determines whether the fault code has disappeared.

[0119] The fault code is the fault code saved after the fault detection node completes the execution of the 403 error.

[0120] If the fault detection node determines that the fault code has disappeared, it triggers the operation of the fault recovery node in box 410.

[0121] Figure 5 This disclosure illustrates a schematic diagram of a system framework for vehicle control according to some embodiments of this disclosure, and now focuses on... Figure 5 Please provide an explanation.

[0122] In some embodiments, the image sensor 501 can capture images and transmit them to the computing unit of the autonomous vehicle 110 via high-speed serial communication technology. The computing unit can process information from various nodes and execute corresponding control commands.

[0123] Within the computing unit, the camera image processing node 502 can process image data, such as image enhancement, feature extraction, and object recognition. This processed image data contains a preliminary understanding of the environment, such as detected pedestrians, vehicles, and traffic signs. The perception node 503 can integrate the image data from the camera image processing node 502 to obtain comprehensive environmental perception.

[0124] Fault detection node 504 can identify fault information corresponding to image sensor 501 based on environmental perception information and notify fault recovery node 505. Fault recovery node 505 can determine the corresponding recovery strategy based on this fault information and transmit the recovery operation information related to the recovery strategy to the vehicle communication bus via CAN Bus node 506. Cleaning device 507 can receive the recovery operation information transmitted by the bus and perform cleaning operations.

[0125] Therefore, the embodiments of this disclosure can perform corresponding recovery operations on the autonomous vehicle 110 based on the recovery strategy associated with the fault of the autonomous vehicle 110, thereby improving the efficiency of fault handling and optimizing the overall performance of the autonomous vehicle 110 system.

[0126] Example devices and equipment

[0127] Figure 6 A schematic structural block diagram of an apparatus 600 for processing annotation information according to certain embodiments of the present disclosure is shown. The apparatus 600 may be implemented as or included in an electronic device 120. The various modules / components in the apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0128] As shown in the figure, the device 600 includes a first determining module 610 configured to determine a recovery strategy corresponding to a fault from a preset configuration table in response to detecting a fault associated with the autonomous vehicle; a second determining module 620 configured to determine whether a recovery operation needs to be performed based on the recovery strategy in response to determining that the recovery strategy indicates that the fault should not be maintained; a third determining module 630 configured to determine at least one recovery operation to be performed based on the recovery strategy in response to determining that a recovery operation needs to be performed; and a control module 640 configured to control the autonomous vehicle to perform at least one recovery operation.

[0129] In some embodiments, process 600 further includes a storage module configured to: in response to determining that a recovery strategy indicates a sustained fault, store fault information associated with the fault.

[0130] In some embodiments, process 600 further includes a fourth determining module configured to: determine whether the fault self-recovers within a first duration in response to determining that a recovery operation is not required; and a first generating module configured to: generate a message indicating self-recovery failure in response to the fault not self-recovering within the first duration.

[0131] In some embodiments, the first duration is determined based on a configuration table.

[0132] In some embodiments, at least one recovery operation includes at least one of the following: restarting a single node corresponding to the fault; restarting multiple nodes corresponding to the fault; or performing a preset action using a target device in an autonomous vehicle.

[0133] In some embodiments, the fault is associated with dirt on the sensor, and at least one recovery operation includes activating a cleaning device in the autonomous vehicle to clean the sensor.

[0134] In some embodiments, process 600 further includes a second generation module configured to generate an assistance request associated with the fault in response to the fault not being recovered within a second duration, wherein the second duration is determined based on a configuration table.

[0135] In some embodiments, the first determining module 610 is further configured to determine identification information corresponding to the fault; and to determine a data entry corresponding to the identification information from a configuration table, the data entry including multiple parameters indicating a recovery strategy.

[0136] In some embodiments, the multiple parameters include multiple of the following: a first parameter indicating whether a recovery operation needs to be performed; a second parameter indicating whether the fault should be maintained; and a third parameter indicating at least one recovery operation to be performed.

[0137] Figure 7A block diagram of a computing device 700 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 7 The computing device 7000 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 7 The computing device 700 shown can be used to implement Figure 1 120 electronic devices.

[0138] like Figure 7 As shown, computing device 700 is in the form of a general-purpose computing device. Components of computing device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage devices 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processing unit 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of computing device 700.

[0139] Computing device 700 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to computing device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 730 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within computing device 700.

[0140] The computing device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 7 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 720 may include computer program product 725 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0141] The communication unit 740 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 700 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 700 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0142] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 700 can also communicate as needed with one or more external devices (not shown) via communication unit 740. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 700, or with any device (e.g., network card, modem, etc.) that enables computing device 700 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interface (not shown).

[0143] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0144] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0145] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0146] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0148] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for vehicle control, comprising: In response to the detection of a fault associated with the autonomous vehicle, a recovery strategy corresponding to the fault is determined from a preset configuration table; In response to determining that the recovery policy indicates that the fault should not be maintained, a determination is made based on the recovery policy whether a recovery operation needs to be performed; In response to determining that a recovery operation needs to be performed, at least one recovery operation to be performed is determined based on the recovery strategy; and Control the autonomous vehicle to perform at least one of the recovery operations.

2. The method according to claim 1, further comprising: In response to determining that the recovery strategy indicates to maintain the fault, fault information associated with the fault is saved.

3. The method according to claim 1, further comprising: In response to determining that no recovery operation is required, it is determined whether the fault self-recovers within a first time period; as well as In response to the failure to self-recover within the first time period, a message indicating self-recovery failure is generated.

4. The method according to claim 3, wherein the first duration is determined based on the configuration table.

5. The method of claim 1, wherein the at least one recovery operation comprises at least one of the following: Restart the single node corresponding to the aforementioned fault; Restart the multiple nodes corresponding to the aforementioned fault; The target device in the autonomous vehicle performs a preset action.

6. The method of claim 5, wherein the fault is associated with a fouling of a sensor, and the at least one recovery operation comprises: The cleaning device in the autonomous vehicle is activated to clean the sensors.

7. The method according to claim 1, further comprising: If the fault is not resolved within a second duration, an assistance request associated with the fault is generated, wherein the second duration is determined based on the configuration table.

8. The method of claim 1, wherein determining a recovery strategy corresponding to the fault from a preset configuration table in response to detecting a fault associated with the autonomous vehicle comprises: Determine the identification information corresponding to the fault; The data entry corresponding to the identification information is determined from the configuration table, and the data entry includes multiple parameters indicating the recovery strategy.

9. The method of claim 8, wherein the plurality of parameters includes a plurality of the following: The first parameter indicates whether a recovery operation needs to be performed; The second parameter indicates whether the fault should be maintained. The third parameter indicates the at least one recovery operation to be performed.

10. A device for vehicle control, comprising: The first determining module is configured to determine a recovery strategy corresponding to the fault from a preset configuration table in response to the detection of a fault associated with the autonomous vehicle. The second determining module is configured to determine, in response to determining that the recovery strategy indicates that the fault should not be maintained, whether a recovery operation needs to be performed based on the recovery strategy. The third determining module is configured to determine at least one recovery operation to be performed based on the recovery strategy in response to determining that a recovery operation needs to be performed. as well as The control module is configured to control the autonomous vehicle to perform the at least one recovery operation.

11. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 9.

Images