Control method, device and equipment of vehicle and road cloud integrated takeover system and medium

Abstract

The invention relates to the technical field of vehicles, in particular to a control method, device and equipment of a vehicle-road cloud integrated takeover system and a medium, and the method comprises the steps: obtaining the operation cognition information of a vehicle and the state information of a user, and judging whether the vehicle meets a first takeover request based on the operation cognition information or not when the vehicle does not meet the first takeover request; judging whether the user meets a second vehicle takeover request based on the state information of the user, if the user does not meet the second vehicle takeover request, generating a corresponding cloud takeover request based on the operation cognitive information of the vehicle, and sending the cloud takeover request to a target cloud, and the target cloud performs takeover control on the vehicle based on the operation cognitive information. Therefore, the problem that the high-concurrency scene of the takeover task cannot be processed due to the lack of a safe takeover redundancy scheme in the related technology is solved, takeover control is performed on the vehicle based on the operation cognitive information of the vehicle, the distribution vehicle end, the cloud end and other takeover modes in the takeover decision process, and the takeover task processing efficiency, comprehensiveness and safety are improved.

Description

Technical Field

[0001] The present application relates to the field of vehicles, and in particular to a control method, device, equipment and medium for a vehicle-road-cloud integrated takeover system. Background Art

[0002] The current autonomous driving technology is in the process of making a major leap from L3 to advanced autonomous driving, and it will still take a long development cycle before fully unmanned driving can be achieved. During this stage, the vehicle is still unable to handle all scenario conditions autonomously, and manual intervention is still required to take over vehicle control in abnormal traffic scenarios. Therefore, achieving a safe, smooth and reliable transition of vehicle control from the autonomous driving system to the driver has become a key issue of concern at present.

[0003] In related technologies, the current autonomous driving system mainly uses the vehicle-side perception system to complete the recognition of the surrounding scenes, and compares it with the offline calibrated boundary of the autonomous driving system's drivable working area. After exceeding the ODD (Operational Design Domain), it sends a takeover request to the user through the in-cabin vehicle system, and finally completes the transition of driving control based on the user's degree of control over the vehicle.

[0004] However, the current takeover based solely on the vehicle side has bottlenecks in safety and reliability. For example, (1) the vehicle side has a limited perception range and limited understanding of the traffic environment and traffic situation, making it impossible to effectively determine the best time to take over, and it is also difficult to handle the long-tail scenario problems common in autonomous driving systems; (2) there is a lack of redundant solutions for safe takeover. Relying solely on user takeover cannot handle scenarios with high concurrency of takeover tasks, and it is difficult to ensure safety when users do not respond effectively. This needs to be solved urgently. Summary of the invention

[0005] The present application provides a control method, device, equipment and medium for a vehicle-road-cloud integrated takeover system to solve the problem that the related technology lacks a redundant solution for safe takeover and is thus unable to handle high-concurrency scenarios of takeover tasks.

[0006] The first aspect of the present application provides a control method for a vehicle-road-cloud integrated takeover system, comprising the following steps:

[0007] Obtain vehicle operation awareness information and user status information;

[0008] Determining whether the vehicle is in a preset drivable area based on the operation recognition information;

[0009] If the vehicle is not in the preset drivable area, determine whether the vehicle satisfies the first takeover request. If the vehicle does not satisfy the first takeover request, determine whether the user satisfies the second vehicle takeover request based on the user's status information. If the user does not satisfy the second vehicle takeover request, generate a corresponding cloud takeover request based on the vehicle's operating cognition information, and send the cloud takeover request to the target cloud so that the target cloud can take over and control the vehicle based on the operating cognition information.

[0010] According to one embodiment of the present application, the obtaining of the operation cognition information of the vehicle includes:

[0011] Acquiring fault detection information of the vehicle and local scene information around the vehicle;

[0012] The operation cognition information of the vehicle is obtained according to the fault detection information and the surrounding local scene information.

[0013] According to an embodiment of the present application, after determining whether the vehicle satisfies the first takeover request, the method further includes:

[0014] If the vehicle satisfies the first takeover request, determining a current takeover state of the vehicle based on the operation recognition information of the vehicle;

[0015] A corresponding takeover task request level is generated according to the operation cognition information of the vehicle and the current takeover status of the vehicle, and a corresponding user reminder level is generated according to the takeover task request level, so as to remind the user to take over according to the user reminder level.

[0016] According to an embodiment of the present application, before generating a corresponding user reminder level according to the takeover task request level to remind the user of the takeover according to the user reminder level, the method further includes:

[0017] Determining whether the user satisfies a third vehicle takeover request based on the user's status information;

[0018] If the user satisfies the third vehicle takeover request, the third vehicle takeover request is issued to the user; otherwise, a corresponding cloud takeover request is generated based on the operation cognition information of the vehicle, and the cloud takeover request is sent to the target cloud.

[0019] According to one embodiment of the present application, after determining whether the user satisfies the second vehicle takeover request, the method further includes:

[0020] If the user satisfies the second vehicle takeover request, the second vehicle takeover request is sent to the user so that the user can take over control of the vehicle.

[0021] According to an embodiment of the present application, after sending the cloud takeover request to the target cloud, the method further includes:

[0022] receiving a takeover policy of the target cloud;

[0023] The vehicle is taken over and controlled according to the takeover strategy of the target cloud.

[0024] According to an embodiment of the present application, after sending the cloud takeover request to the target cloud, the method further includes:

[0025] If the target cloud does not satisfy the cloud takeover request, the minimum risk operation strategy of the vehicle is triggered to take over control of the vehicle according to the minimum risk operation strategy.

[0026] According to the control method of the vehicle-road-cloud integrated takeover system of the embodiment of the present application, the vehicle's operational cognition information and the user's status information are obtained. When the vehicle does not meet the first takeover request based on the operational cognition information, it is determined whether the user meets the second vehicle takeover request based on the user's status information. If the user does not meet the second vehicle takeover request, a corresponding cloud takeover request is generated based on the vehicle's operational cognition information, and the cloud takeover request is sent to the target cloud, so that the target cloud can take over and control the vehicle based on the operational cognition information. In this way, the problem of the lack of a safe takeover redundancy solution in the related technology, which makes it impossible to handle high-concurrency scenarios of takeover tasks, is solved. In the takeover decision process, based on the vehicle's operational cognition information, multiple takeover methods such as the vehicle side and the cloud side are allocated to take over and control the vehicle, thereby improving the efficiency, comprehensiveness and safety of takeover task processing.

[0027] A second aspect of the present application provides a control device for a vehicle-road-cloud integrated takeover system, including:

[0028] An acquisition module, used to acquire the vehicle's operational cognitive information and the user's status information;

[0029] A judgment module, used for judging whether the vehicle is in a preset drivable area based on the operation recognition information;

[0030] A control module is used to determine whether the vehicle satisfies a first takeover request if the vehicle is not in the preset drivable area; if the vehicle does not satisfy the first takeover request, determine whether the user satisfies a second vehicle takeover request based on the user's status information; if the user does not satisfy the second vehicle takeover request, generate a corresponding cloud takeover request based on the vehicle's operating cognition information, and send the cloud takeover request to a target cloud so that the target cloud can take over and control the vehicle based on the operating cognition information.

[0031] According to one embodiment of the present application, the acquisition module includes:

[0032] A first acquisition unit, used to acquire fault detection information of the vehicle and local scene information around the vehicle;

[0033] The second acquisition unit is used to obtain the operation cognition information of the vehicle according to the fault detection information and the surrounding local scene information.

[0034] According to an embodiment of the present application, after determining whether the vehicle satisfies the first takeover request, the control module further includes:

[0035] a judgment unit, configured to judge a current takeover state of the vehicle based on operation recognition information of the vehicle if the vehicle satisfies the first takeover request;

[0036] A reminder unit is used to generate a corresponding takeover task request level according to the operation cognition information of the vehicle and the current takeover status of the vehicle, and to generate a corresponding user reminder level according to the takeover task request level, so as to remind the user to take over according to the user reminder level.

[0037] According to an embodiment of the present application, before generating a corresponding user reminder level according to the takeover task request level to remind the user of the takeover according to the user reminder level, the reminder unit further includes:

[0038] a judging subunit, configured to judge whether the user satisfies a third vehicle takeover request based on the user's status information;

[0039] A generating subunit is used for sending the third vehicle takeover request to the user if the user satisfies the third vehicle takeover request; otherwise, generating a corresponding cloud takeover request based on the operation cognition information of the vehicle and sending the cloud takeover request to the target cloud.

[0040] According to an embodiment of the present application, after determining whether the user satisfies the second vehicle takeover request, the control module further includes:

[0041] The first control unit is used to send the second vehicle takeover request to the user if the user satisfies the second vehicle takeover request, so as to take over control of the vehicle through the user.

[0042] According to an embodiment of the present application, after sending the cloud takeover request to the target cloud, the control module further includes:

[0043] A receiving unit, configured to receive a takeover strategy of the target cloud;

[0044] The second control unit is used to take over and control the vehicle according to the takeover strategy of the target cloud.

[0045] According to an embodiment of the present application, after sending the cloud takeover request to the target cloud, the control module further includes:

[0046] The third control unit is used to trigger the minimum risk operation strategy of the vehicle if the target cloud does not satisfy the cloud takeover request, so as to take over control of the vehicle according to the minimum risk operation strategy.

[0047] According to the control device of the vehicle-road-cloud integrated takeover system of the embodiment of the present application, the vehicle's operational cognition information and the user's status information are obtained. When the vehicle does not meet the first takeover request based on the operational cognition information, it is determined whether the user meets the second vehicle takeover request based on the user's status information. If the user does not meet the second vehicle takeover request, a corresponding cloud takeover request is generated based on the vehicle's operational cognition information, and the cloud takeover request is sent to the target cloud, so that the target cloud can take over and control the vehicle based on the operational cognition information. In this way, the problem of the lack of a safe takeover redundancy solution in the related technology, which makes it impossible to handle high-concurrency scenarios of takeover tasks, is solved. In the takeover decision process, based on the vehicle's operational cognition information, multiple takeover methods such as the vehicle side and the cloud side are allocated to take over and control the vehicle, thereby improving the efficiency, comprehensiveness and safety of takeover task processing.

[0048] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the control method of the vehicle-road-cloud integrated takeover system as described in the above embodiment.

[0049] The fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement a control method for the vehicle-road-cloud integrated takeover system as described in the above embodiment.

[0050] A fifth aspect of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method described in the above embodiment.

[0051] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0053] Figure 1 A schematic diagram of the vehicle-road-cloud integrated takeover system architecture of the related technology;

[0054] Figure 2 A flow chart of a control method for a vehicle-road-cloud integrated takeover system provided according to an embodiment of the present application;

[0055] Figure 3 This is a block diagram of an example of a control device of a vehicle-road-cloud integrated takeover system according to an embodiment of the present application;

[0056] Figure 4 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0058] The following describes the control method, device, equipment and medium of the vehicle-road-cloud integrated takeover system according to the embodiment of the present application with reference to the accompanying drawings. In view of the lack of a safe takeover redundancy solution in the related art mentioned in the above background technology, which is unable to handle the problem of high concurrency scenarios of takeover tasks, the present application provides a control method for a vehicle-road-cloud integrated takeover system. In the method, the operation cognition information of the vehicle and the status information of the user are obtained. When the vehicle does not meet the first takeover request based on the operation cognition information, it is judged whether the user meets the second vehicle takeover request based on the user's status information. If the user does not meet the second vehicle takeover request, a corresponding cloud takeover request is generated based on the operation cognition information of the vehicle, and the cloud takeover request is sent to the target cloud, so that the vehicle is taken over and controlled by the target cloud based on the operation cognition information. In this way, the problem of the lack of a safe takeover redundancy solution in the related art, which is unable to handle the problem of high concurrency scenarios of takeover tasks, is solved. In the takeover decision process, based on the operation cognition information of the vehicle, multiple takeover methods such as the vehicle side and the cloud side are allocated to take over and control the vehicle, thereby improving the efficiency, comprehensiveness and safety of takeover task processing.

[0059] Specifically, before introducing the embodiments of the present application, the vehicle-road-cloud integrated takeover system architecture of the embodiments of the present application is first introduced. The present application integrates the vehicle-side, roadside unit and cloud computing platform to construct a vehicle-road-cloud integrated takeover system architecture to improve the redundant security of the autonomous driving system takeover, such as Figure 1As shown, it mainly includes four components: cognition, takeover decision, takeover control and minimum risk operation. Among them, the specific functions of the above-mentioned components are respectively distributed on the vehicle side and the cloud side. The information input of the vehicle-road-cloud integrated takeover system mainly includes vehicle status information, user (driver) status information, roadside status information, historical data and other relevant information. The data stream is transmitted based on the mobile communication of the Internet of Vehicles. The cloud-side cognitive part functions are mainly deployed in the edge cloud, and the cloud-side takeover decision and cloud-side takeover control part functions can be deployed in the regional cloud platform or the enterprise cloud platform. The specific takeover process of the vehicle-road-cloud integrated takeover system is discussed through the following specific embodiments.

[0060] Specifically, Figure 2 A flow chart of a control method for a vehicle-road-cloud integrated takeover system provided in an embodiment of the present application.

[0061] like Figure 2 As shown, the control method of the vehicle-road-cloud integrated takeover system includes the following steps:

[0062] In step S201, the vehicle's operational awareness information and the user's status information are acquired.

[0063] According to one embodiment of the present application, obtaining the operation cognition information of the vehicle includes: obtaining the fault detection information of the vehicle and the surrounding local scene information of the vehicle; and obtaining the operation cognition information of the vehicle based on the fault detection information and the surrounding local scene information.

[0064] Specifically, in an embodiment of the present application, first, the vehicle's fault detection information, the user's status information, i.e., personnel status cognition, the vehicle's surrounding local scene information, and other different dimensions of ODD (Operational Design Domain, drivable working area) judgment are obtained, and the vehicle's operation cognition information is obtained based on the fault detection information and the surrounding local scene information.

[0065] Among them, the vehicle's fault detection information may include vehicle software fault status diagnosis and vehicle hardware fault status diagnosis. Vehicle fault diagnosis may be based on fault monitoring information of various component subsystems such as vehicle communication and chassis; user status information may include user (e.g. driver) behavior status monitoring, passenger behavior status detection, and dynamic driving task backup user status, etc. The user and passenger behavior status monitoring channel may use perception information such as in-vehicle cameras to detect the user and passenger behavior status. At the same time, the steering wheel torque may be collected or the user's steering wheel control status may be judged through an angle sensor to detect the user's behavior status; the vehicle's surrounding local scene information may be based on on-board cameras, millimeter-wave radars, laser radars and other sensors to collect surrounding obstacle targets, road conditions and traffic conditions, and may specifically include natural environment conditions (such as rainy and snowy weather, visibility, light levels, etc.), road environment quality (such as lane marking quality, road surface covering, whether there is a longitudinal section, a cross section, etc.) and traffic environment (such as temporary control areas, accident sites, height and width restrictions, etc.);

[0066] Secondly, this application constructs a cloud-side and vehicle-side collaborative cognition unit for taking over the autonomous driving system. The constructed cloud-side cognition unit utilizes the advantages of edge cloud multi-source information fusion, and includes a beyond-visual-range fusion cognition module and a traffic situation and scene cognition module to form an effective supplement to the vehicle-side cognition, improve cognitive limitations, and enhance the reliability of takeover decisions. The target cloud completes beyond-visual-range fusion cognition and traffic situation cognition based on the road test unit, vehicle-mounted perception results, and cloud-side historical data, and completes cognitive information sharing with the vehicle-side.

[0067] In step S202, it is determined whether the vehicle is in a preset drivable area based on the operational recognition information.

[0068] Among them, the preset drivable area can be the judgment result of the vehicle-side takeover decision unit based on the collaborative cognition results of the vehicle-side and the cloud.

[0069] Specifically, after acquiring the vehicle's operating cognition information, the embodiment of the present application can determine whether the vehicle's autonomous driving range exceeds a preset drivable area, namely, ODD, through the vehicle's operating cognition information and the collaborative cognition results of the cloud, and then further determine the vehicle's takeover status based on the vehicle's operating cognition information and ODD.

[0070] It should be noted that the vehicle side, as the takeover decision-making entity, makes vehicle-cloud collaborative decisions guided by the cloud takeover decision. It can issue cloud takeover requests and vehicle-side takeover requests based on the vehicle's operating cognitive information and ODD judgment. For working conditions that generally exceed the ODD range, priority is given to issuing vehicle-side takeover requests. For scenarios where the vehicle side cannot effectively take over (such as sudden abnormal behavior or abnormal physical condition of the user), a cloud takeover decision output should be made at this time. When the cognitive result is in a normal state, the decision output is no takeover, and the vehicle's automatic driving system operates normally.

[0071] In step S203, if the vehicle is not in the preset drivable area, it is determined whether the vehicle satisfies the first takeover request. If the vehicle does not satisfy the first takeover request, it is determined whether the user satisfies the second vehicle takeover request based on the user's status information. If the user does not satisfy the second vehicle takeover request, a corresponding cloud takeover request is generated based on the vehicle's operating cognition information, and the cloud takeover request is sent to the target cloud so that the target cloud can take over and control the vehicle based on the operating cognition information.

[0072] Specifically, in an embodiment of the present application, if it is determined based on the vehicle's operating cognition information that the vehicle is not in a preset drivable area, that is, the vehicle is beyond the ODD range, then it is determined whether the vehicle satisfies the first takeover request, that is, whether the vehicle can be taken over is determined based on the vehicle's current fault detection information and the surrounding local scene information. If the vehicle does not meet the first takeover request, then it is determined based on the user's status information whether the user meets the second vehicle takeover request, that is, whether the user can take over the vehicle is determined based on the user's current behavior status. If the user cannot take over the vehicle due to the current behavior status (for example, the user's sudden abnormal behavior or abnormal physical condition), that is, the user does not meet the second vehicle takeover request, then a corresponding cloud takeover request is generated based on the vehicle's operating cognition information, and the cloud takeover request is sent to the target cloud, so that the vehicle can be taken over and controlled through the target cloud based on the operating cognition information.

[0073] According to one embodiment of the present application, after determining whether the vehicle satisfies the first takeover request, it also includes: if the vehicle satisfies the first takeover request, determining the current takeover status of the vehicle based on the vehicle's operating cognition information; generating a corresponding takeover task request level based on the vehicle's operating cognition information and the vehicle's current takeover status, and generating a corresponding user reminder level based on the takeover task request level, so as to remind the user to take over based on the user reminder level.

[0074] Specifically, in an embodiment of the present application, if it is determined that the vehicle satisfies the first takeover request, that is, the vehicle can be taken over based on the current fault detection information and the surrounding local scene information, during the takeover process, the takeover decision system will synchronously receive the output of the vehicle's surrounding scene operation cognitive information and judge the vehicle's current takeover status, such as the degree to which the vehicle exceeds the ODD and the takeover waiting time, so as to generate a corresponding takeover task request level based on the vehicle's current takeover status, and generate a corresponding user reminder level based on the takeover task request level, so as to remind the user to take over based on the user reminder level.

[0075] According to one embodiment of the present application, before generating a corresponding user reminder level according to the takeover task request level to remind the user to take over according to the user reminder level, it also includes: judging whether the user meets the third vehicle takeover request based on the user's status information; if the user meets the third vehicle takeover request, issuing a third vehicle takeover request to the user, otherwise, generating a corresponding cloud takeover request based on the vehicle's operating cognition information, and sending the cloud takeover request to the target cloud.

[0076] Specifically, in an embodiment of the present application, before the user is reminded to take over according to the user reminder level, it is necessary to further judge the user's status information, that is, to judge whether the user meets the third vehicle takeover request based on the user's status information. If the user meets the third vehicle takeover request, that is, the user does not have sudden abnormal behavior or abnormal physical condition, then a third vehicle takeover request is issued to the user, and the vehicle is taken over and controlled by the user. If the user has sudden abnormal behavior or abnormal physical condition, then a corresponding cloud takeover request is generated based on the vehicle's operation cognition information, and the cloud takeover request is sent to the target cloud.

[0077] It should be noted that for vehicle-side takeover, the multi-modal interactive signals of the smart cockpit are used to gradually enhance the reminder level to the driver, and for cloud-side takeover, the priority of task processing is increased.

[0078] According to one embodiment of the present application, after determining whether the user satisfies the second vehicle takeover request, it also includes: if the user satisfies the second vehicle takeover request, sending the second vehicle takeover request to the user so that the user can take over control of the vehicle.

[0079] Specifically, if the vehicle does not meet the first takeover request, it is determined based on the user's status information whether the user meets the second vehicle takeover request, that is, it can be determined through the above-mentioned user's sudden abnormal behavior or abnormal physical condition and other states. If the user does not have sudden abnormal behavior or abnormal physical condition and other states, it is determined that the user meets the second vehicle takeover request. At this time, the second vehicle takeover request is sent to the user so that the vehicle can be taken over and controlled by the user; if the user has sudden abnormal behavior or abnormal physical condition and other states, it is determined that the user does not meet the second vehicle takeover request. At this time, a corresponding cloud takeover request is generated based on the vehicle's operation cognition information, and the cloud takeover request is sent to the target cloud so that the vehicle can be taken over and controlled by the target cloud based on the operation cognition information.

[0080] According to an embodiment of the present application, after sending a cloud takeover request to a target cloud, it also includes: receiving a takeover strategy of the target cloud; and taking over and controlling the vehicle according to the takeover strategy of the target cloud.

[0081] Specifically, in the embodiment of the present application, the cloud-based cognition part includes a beyond-visual-range fusion cognition unit and a traffic situation cognition unit, wherein the beyond-visual-range fusion cognition unit collects information from various vehicle nodes and roadside cameras in the interval through the roadside unit, performs decision-level matching and fusion on the edge cloud computing platform, maps the fusion result with the interval map, and completes the understanding of each target attribute based on the offline training model, thereby completing the fusion cognition within the beyond-visual-range range; the traffic situation cognition unit uses traffic characteristic states such as vehicle flow and average driving speed to describe the traffic situation, constructs a traffic situation data set based on cloud-based historical data, and constructs machine learning models such as long- and short-time neural networks to predict and diagnose the future traffic situation trend within the long-term domain and traffic anomalies within the medium- and short-term domains, so that different takeover objects can be judged through the cognitive results of different scenarios or behaviors beyond ODD.

[0082] It should be noted that in the designed vehicle-road-cloud integrated takeover system architecture, both the target cloud and the vehicle side can issue takeover requests based on the vehicle's driving conditions, but the takeover subject is still the vehicle-side takeover decision-making part. The target cloud can issue takeover recommendation instructions to the vehicle side based on the cognitive results. At the same time, in extreme emergency situations, it can directly enter the target cloud takeover control mode in the form of adventuring takeover, so that the target cloud can take over after the vehicle side is unable to take over.

[0083] Furthermore, the target cloud takeover control includes two layers: cloud AI Agent takeover model and cloud operator takeover, to adapt to the high-concurrency scenario where a large number of intelligent connected vehicles simultaneously send cloud takeover requests. Several virtual cockpit remote takeover hardware platforms are built on the cloud. The remote takeover platform includes a wide-view virtual display, a high-performance computing host, an electric adjustable seat, a virtual pedal, a virtual gear lever, and a virtual steering wheel kit. The high-performance computing host integrates the cloud AI Agent takeover model. The AI ​​Agent takeover model can be constructed using a large model method to give full play to the advantages of computing resources and improve the ability to cope with high-concurrency takeover tasks.

[0084] At the same time, for takeover tasks that cannot be handled, the AI ​​Agent takeover model has a dynamic task allocation function, which can prioritize emergency remote takeover tasks to appropriate remote operators based on task priority and remote operator status. The virtual cockpit integrates BEV (Bird's Eye View) scene twin mapping function, signal delay compensation function and steering torque feedback function. Among them, the BEV scene twin mapping function reconstructs the scene from the BEV perspective based on the traffic scene and obstacle information obtained by cloud-based fusion perception, so that the remote operator can intuitively grasp the surrounding traffic conditions; the signal delay compensation function captures the acceleration of the pedal and steering wheel operation actions, and predicts the action sequence based on the transmission link delay, thereby realizing signal delay compensation for cloud control; the steering torque feedback function establishes a vehicle steering system dynamics model, takes the vehicle driving status and remote operator control output as input, and designs Kalman filtering and other state estimation algorithms to estimate the steering feedback torque, which is then fed back to the steering motor of the virtual cockpit to achieve high-precision simulation of the feedback torque, so as to improve the safety of the cloud-based takeover control process.

[0085] That is to say, after the cloud takeover control module receives the cloud takeover request, the offline trained AI Agent will first perform remote takeover. If the AI ​​Agent cannot handle the remote takeover task, the task priority will be divided according to the urgency of the takeover, and the task will be dynamically assigned to the operator according to the readiness status of the remote operator.

[0086] According to one embodiment of the present application, after sending a cloud takeover request to a target cloud, it also includes: if the target cloud does not satisfy the cloud takeover request, triggering the minimum risk operation strategy of the vehicle to take over control of the vehicle according to the minimum risk operation strategy.

[0087] Specifically, in an embodiment of the present application, if the vehicle-side user, the cloud-based AI Agent, and the cloud-based remote operator are all unable to effectively take over the vehicle, the MRM (Minimal Risk Maneuver) operation of the autonomous driving system is triggered, that is, the minimum risk operation, to take over and control the vehicle according to the minimum risk operation strategy. In other words, at this time, the vehicle takes safe and smooth parking by the side of the road as its primary driving goal, so that after the vehicle-side user, the cloud-based AI Agent, and the cloud-based remote operator are all unable to effectively take over the vehicle, the MRM operation of the autonomous driving system is triggered to ensure the safety of the vehicle takeover and control.

[0088] In order to enable those skilled in the art to better understand the present application, the following will be described in conjunction with the specific process:

[0089] S1, obtains vehicle fault detection information, user status information, vehicle surrounding local scene information and other ODD judgments in different dimensions to form a local understanding of the vehicle's operating status and driving scene;

[0090] S2: Complete beyond-visual-range fusion cognition and traffic situation cognition based on the road test unit, vehicle perception results, and cloud historical data on the target cloud, and share cognitive information with the vehicle side;

[0091] S3: The cloud-based takeover decision unit determines the current vehicle driving status based on the cloud-based cognitive output, and provides takeover decision guidance to the vehicle-side decision unit, while issuing a cloud-based remote takeover request in an emergency;

[0092] S4: The vehicle-side takeover decision unit determines the current vehicle driving state and whether it exceeds the ODD range of the autonomous driving system based on the collaborative cognition results of the vehicle-side and the cloud-side, and issues a vehicle-side takeover control request or a cloud-side takeover control request according to the type of dimension that exceeds the ODD range. At the same time, the takeover request level is gradually increased according to the degree to which the vehicle exceeds the ODD and the waiting time for takeover.

[0093] S5, after the cloud takeover control module receives the cloud takeover request, the offline trained AI Agent first performs remote takeover. If the AI ​​Agent cannot handle the remote takeover task, the task priority is divided according to the urgency of the takeover, and the task is dynamically assigned to the operator according to the readiness status of the remote operator;

[0094] S6: If the vehicle-side user, cloud AI Agent, and cloud remote operator are unable to effectively take over the vehicle, the MRM operation of the autonomous driving system is triggered. At this time, the vehicle's primary driving goal is to safely and smoothly pull over to the side of the road.

[0095] In summary, this application can achieve the following beneficial effects:

[0096] (1) This application expands the cognitive process of the overall traffic scene through vehicle-road-cloud integration, broadens the cognitive scope, and reduces the limitations of the single-vehicle cognitive process by allocating and communicating different types of cognitive tasks between the vehicle and the cloud, which is conducive to a more comprehensive, safe, and reliable judgment of the takeover timing and method in the takeover decision-making process;

[0097] (2) This application divides the takeover decision process into a vehicle-side takeover decision part and a cloud-side takeover recommendation part. The vehicle-side takeover decision is used as the decision-making subject. Different takeover objects can be determined according to the cognitive results of different scenarios or behaviors that exceed the ODD, thereby realizing a backup takeover solution after the driver fails to take over, thereby improving the safety of autonomous driving after failure.

[0098] (3) This application addresses the problem of parallel processing of high-concurrency tasks in cloud takeover. The specific takeover tasks are first assigned to the AI ​​agent model for takeover, and the tasks that cannot be effectively processed are further assigned to appropriate remote operators according to the urgency of the tasks. In this way, the efficiency of takeover task processing is improved when the task volume is large, and the operating burden of the operator is reduced.

[0099] According to the control method of the vehicle-road-cloud integrated takeover system of the embodiment of the present application, the vehicle's operational cognition information and the user's status information are obtained. When the vehicle does not meet the first takeover request based on the operational cognition information, it is determined whether the user meets the second vehicle takeover request based on the user's status information. If the user does not meet the second vehicle takeover request, a corresponding cloud takeover request is generated based on the vehicle's operational cognition information, and the cloud takeover request is sent to the target cloud, so that the target cloud can take over and control the vehicle based on the operational cognition information. In this way, the problem of the lack of a safe takeover redundancy solution in the related technology, which makes it impossible to handle high-concurrency scenarios of takeover tasks, is solved. In the takeover decision process, based on the vehicle's operational cognition information, multiple takeover methods such as the vehicle side and the cloud side are allocated to take over and control the vehicle, thereby improving the efficiency, comprehensiveness and safety of takeover task processing.

[0100] Secondly, the control device of the vehicle-road-cloud integrated takeover system proposed in the embodiment of the present application is described with reference to the accompanying drawings.

[0101] Figure 3 It is a block diagram of the control device of the vehicle-road-cloud integrated takeover system of an embodiment of the present application.

[0102] like Figure 3 As shown, the control device 10 based on the vehicle-road-cloud integrated takeover system includes: an acquisition module 100, a judgment module 200 and a control module 300.

[0103] The acquisition module 100 is used to acquire the vehicle's operational cognitive information and the user's status information;

[0104] A determination module 200, for determining whether the vehicle is in a preset drivable area based on the operation recognition information;

[0105] The control module 300 is used to determine whether the vehicle satisfies a first takeover request if the vehicle is not in a preset drivable area. If the vehicle does not satisfy the first takeover request, the control module 300 is used to determine whether the user satisfies a second vehicle takeover request based on the user's status information. If the user does not satisfy the second vehicle takeover request, the control module 300 generates a corresponding cloud takeover request based on the vehicle's operating cognition information, and sends the cloud takeover request to the target cloud so that the target cloud can take over and control the vehicle based on the operating cognition information.

[0106] According to one embodiment of the present application, the acquisition module 100 includes:

[0107] A first acquisition unit, used to acquire vehicle fault detection information and vehicle surrounding local scene information;

[0108] The second acquisition unit is used to obtain the vehicle's operation cognition information based on the fault detection information and the surrounding local scene information.

[0109] According to an embodiment of the present application, after determining whether the vehicle satisfies the first takeover request, the control module 300 further includes:

[0110] a judgment unit, configured to judge a current takeover state of the vehicle based on the operation recognition information of the vehicle if the vehicle satisfies the first takeover request;

[0111] The reminder unit is used to generate a corresponding takeover task request level according to the vehicle's operating cognition information and the vehicle's current takeover status, and to generate a corresponding user reminder level according to the takeover task request level, so as to remind the user to take over according to the user reminder level.

[0112] According to an embodiment of the present application, before generating a corresponding user reminder level according to the takeover task request level to remind the user of the takeover according to the user reminder level, the reminder unit further includes:

[0113] A judging subunit, configured to judge whether the user satisfies the third vehicle takeover request based on the user's status information;

[0114] A generating subunit is used to send a third vehicle takeover request to the user if the user satisfies the third vehicle takeover request; otherwise, a corresponding cloud takeover request is generated based on the vehicle's operation cognition information, and the cloud takeover request is sent to the target cloud.

[0115] According to one embodiment of the present application, after determining whether the user satisfies the second vehicle takeover request, the control module 300 further includes:

[0116] The first control unit is used to send a second vehicle takeover request to the user if the user satisfies the second vehicle takeover request, so as to take over control of the vehicle through the user.

[0117] According to an embodiment of the present application, after sending the cloud takeover request to the target cloud, the control module 300 further includes:

[0118] A receiving unit, used for receiving a takeover strategy of a target cloud;

[0119] The second control unit is used to take over and control the vehicle according to the takeover strategy of the target cloud.

[0120] According to an embodiment of the present application, after sending the cloud takeover request to the target cloud, the control module 300 further includes:

[0121] The third control unit is used to trigger the minimum risk operation strategy of the vehicle if the target cloud does not meet the cloud takeover request, so as to take over control of the vehicle according to the minimum risk operation strategy.

[0122] According to the control device of the vehicle-road-cloud integrated takeover system of the embodiment of the present application, the vehicle's operational cognition information and the user's status information are obtained. When the vehicle does not meet the first takeover request based on the operational cognition information, it is determined whether the user meets the second vehicle takeover request based on the user's status information. If the user does not meet the second vehicle takeover request, a corresponding cloud takeover request is generated based on the vehicle's operational cognition information, and the cloud takeover request is sent to the target cloud, so that the target cloud can take over and control the vehicle based on the operational cognition information. In this way, the problem of the lack of a safe takeover redundancy solution in the related technology, which makes it impossible to handle high-concurrency scenarios of takeover tasks, is solved. In the takeover decision process, based on the vehicle's operational cognition information, multiple takeover methods such as the vehicle side and the cloud side are allocated to take over and control the vehicle, thereby improving the efficiency, comprehensiveness and safety of takeover task processing.

[0123] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0124] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0125] When the processor 402 executes the program, the control method of the vehicle-road-cloud integrated takeover system provided in the above embodiment is implemented.

[0126] Furthermore, the electronic device further comprises:

[0127] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0128] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0129] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0130] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0131] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0132] The processor 402 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0133] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the control method of the vehicle-road-cloud integrated takeover system as described above is implemented.

[0134] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the method in the above embodiment when executed by a processor.

[0135] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0136] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0137] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A control method for a vehicle-road-cloud integrated takeover system, characterized in that: The following steps are involved: Obtain vehicle operation awareness information and user status information; Determining whether the vehicle is in a preset drivable area based on the operation recognition information; If the vehicle is not in the preset drivable area, determine whether the vehicle satisfies the first takeover request. If the vehicle does not satisfy the first takeover request, determine whether the user satisfies the second vehicle takeover request based on the user's status information. If the user does not satisfy the second vehicle takeover request, generate a corresponding cloud takeover request based on the vehicle's operating cognition information, and send the cloud takeover request to the target cloud so that the target cloud can take over and control the vehicle based on the operating cognition information.

2. The method according to claim 1, characterized in that The obtaining of the operation cognition information of the vehicle includes: Acquiring fault detection information of the vehicle and local scene information around the vehicle; The operation cognition information of the vehicle is obtained according to the fault detection information and the surrounding local scene information.

3. The method according to claim 1, characterized in that After determining whether the vehicle satisfies the first takeover request, the method further includes: If the vehicle satisfies the first takeover request, determining a current takeover state of the vehicle based on the operation recognition information of the vehicle; A corresponding takeover task request level is generated according to the operation cognition information of the vehicle and the current takeover status of the vehicle, and a corresponding user reminder level is generated according to the takeover task request level, so as to remind the user to take over according to the user reminder level.

4. The method according to claim 3, characterized in that Before generating a corresponding user reminder level according to the takeover task request level to remind the user of the takeover according to the user reminder level, the method further includes: Determining whether the user satisfies a third vehicle takeover request based on the user's status information; If the user satisfies the third vehicle takeover request, the third vehicle takeover request is issued to the user; otherwise, a corresponding cloud takeover request is generated based on the operation cognition information of the vehicle, and the cloud takeover request is sent to the target cloud.

5. The method according to claim 1, characterized in that After determining whether the user satisfies the second vehicle takeover request, the method further includes: If the user satisfies the second vehicle takeover request, the second vehicle takeover request is sent to the user so that the user can take over control of the vehicle.

6. The method according to claim 1, characterized in that After sending the cloud takeover request to the target cloud, the method further includes: receiving a takeover policy of the target cloud; The vehicle is taken over and controlled according to the takeover strategy of the target cloud.

7. The method according to claim 1, characterized in that After sending the cloud takeover request to the target cloud, the method further includes: If the target cloud does not satisfy the cloud takeover request, the minimum risk operation strategy of the vehicle is triggered to take over control of the vehicle according to the minimum risk operation strategy.

8. A control device for a vehicle-road-cloud integrated takeover system, characterized in that: include: An acquisition module, used to acquire the vehicle's operational cognitive information and the user's status information; A judgment module, used for judging whether the vehicle is in a preset drivable area based on the operation recognition information; A control module is used to determine whether the vehicle satisfies a first takeover request if the vehicle is not in the preset drivable area; if the vehicle does not satisfy the first takeover request, determine whether the user satisfies a second vehicle takeover request based on the user's status information; if the user does not satisfy the second vehicle takeover request, generate a corresponding cloud takeover request based on the vehicle's operating cognition information, and send the cloud takeover request to a target cloud so that the target cloud can take over and control the vehicle based on the operating cognition information.

9. The device according to claim 8, characterized in that The acquisition module comprises: A first acquisition unit, used to acquire fault detection information of the vehicle and local scene information around the vehicle; The second acquisition unit is used to obtain the operation cognition information of the vehicle according to the fault detection information and the surrounding local scene information.

10. The device according to claim 8, characterized in that After determining whether the vehicle satisfies the first takeover request, the control module further includes: a judgment unit, configured to judge a current takeover state of the vehicle based on operation recognition information of the vehicle if the vehicle satisfies the first takeover request; A reminder unit is used to generate a corresponding takeover task request level according to the operation cognition information of the vehicle and the current takeover status of the vehicle, and to generate a corresponding user reminder level according to the takeover task request level, so as to remind the user to take over according to the user reminder level.

11. The device according to claim 10, characterized in that Before generating a corresponding user reminder level according to the takeover task request level to remind the user of the takeover according to the user reminder level, the reminder unit further includes: a judging subunit, configured to judge whether the user satisfies a third vehicle takeover request based on the user's status information; A generating subunit is used for sending the third vehicle takeover request to the user if the user satisfies the third vehicle takeover request; otherwise, generating a corresponding cloud takeover request based on the operation cognition information of the vehicle and sending the cloud takeover request to the target cloud.

12. The device according to claim 8, characterized in that After determining whether the user satisfies the second vehicle takeover request, the control module further includes: The first control unit is used to send the second vehicle takeover request to the user if the user satisfies the second vehicle takeover request, so as to take over control of the vehicle through the user.

13. The device according to claim 8, characterized in that After sending the cloud takeover request to the target cloud, the control module further includes: A receiving unit, configured to receive a takeover strategy of the target cloud; The second control unit is used to take over and control the vehicle according to the takeover strategy of the target cloud.

14. The device according to claim 8, characterized in that After sending the cloud takeover request to the target cloud, the control module further includes: The third control unit is used to trigger the minimum risk operation strategy of the vehicle if the target cloud does not satisfy the cloud takeover request, so as to take over control of the vehicle according to the minimum risk operation strategy.

15. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the control method of the vehicle-road-cloud integrated takeover system as described in any one of claims 1 to 7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a control method for a vehicle-road-cloud integrated takeover system as described in any one of claims 1-7.

17. A computer program product comprising a computer program, characterized in that The computer program is executed to implement the control method of the vehicle-road-cloud integrated takeover system as described in any one of claims 1-7.

Images