A control method, system and medium for remote takeover of an unmanned vehicle

Abstract

The present application relates to a kind of unmanned vehicle remote takeover control method, system and medium, the method comprises: M1. Vehicle travels on road, the running data information of vehicle and the data information of the automatic driving failure of vehicle are acquired in real time, and the data information of the surrounding traffic participants of vehicle is collected;M2. Based on the running data information of vehicle and the data information of the automatic driving failure of vehicle, the diffusion model of the severity of the fault of vehicle based on deep learning is constructed, the severity of the fault of vehicle is characterized, and the data information of the severity of the fault of vehicle is obtained.The present application not only can analyze the data of vehicle itself and other traffic participants, facilitate to improve self-driving technology, but also can classify and manage fault type, facilitate safety officer to make judgment in time, reduce vehicle takeover rate, improve vehicle driving smoothness.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a control method, system and medium for remote takeover of an autonomous vehicle. Background Technology

[0002] Current intelligent driving technology cannot fully cover all road scenarios. When the relevant scenario is exceeded or a risk is encountered, the vehicle will take over. Takeover is divided into manual takeover and remote takeover. Generally, the owner or safety officer decides whether to take over. There is no classification of the faults reported by the vehicle or risk level prompts, which means that the vehicle needs to be taken over at any time, resulting in poor comfort and user experience. Summary of the Invention

[0003] In view of the above problems, the present invention provides a control method, system and medium for remote takeover of unmanned vehicles. It can not only analyze the data of the vehicle itself and other traffic participants to facilitate the improvement of its own autonomous driving technology, but also classify and manage fault types to facilitate timely judgment by safety officers, reduce the vehicle takeover rate and improve the smoothness of vehicle driving.

[0004] To achieve the above and other related objectives, the present invention provides the following technical solution: a control method for remote takeover of an unmanned vehicle, the method comprising:

[0005] M1. When the vehicle is driving on the road, it acquires real-time data on vehicle operation and autonomous driving malfunctions, and collects data on traffic participants around the vehicle.

[0006] M2. Based on the vehicle's operational data and autonomous driving fault data, a deep learning-based diffusion model of the severity of vehicle faults is constructed to characterize the severity of vehicle faults and obtain data on the severity of vehicle faults.

[0007] M3. Based on the vehicle's operational data and the data of traffic participants around the vehicle, a dynamic traffic learning prediction model based on time and space parameters is constructed to predict the vehicle's traffic accident rate, thereby obtaining the predicted traffic accident rate data.

[0008] M4. Based on the data information of the severity of the vehicle's malfunction and the data information of the predicted vehicle's traffic accident rate, a classification prediction algorithm based on black-winged kites is used to predict and classify the vehicle's emergency response level, thereby obtaining classified vehicle emergency response level data information.

[0009] Furthermore, the method also includes:

[0010] M5. Based on the classified vehicle emergency response data, a remote takeover control function P is established.

[0011]

[0012] Where x represents the emergency response data of the classified vehicles, and α1, α2 and α3 are the remote takeover control factors of the vehicles, which control and adjust the remote takeover of the vehicles and output the remote takeover control data of the vehicles.

[0013] Furthermore, the remote takeover control factors α1, α2, and α3 of the vehicle are,

[0014]

[0015] Where x represents the emergency response data of the categorized vehicles.

[0016] Furthermore, in step M2, constructing a deep learning-based diffusion model of vehicle fault severity to characterize the severity of vehicle faults includes:

[0017] M21. Based on the vehicle's operational data and autonomous driving fault data, a vehicle fault fusion function Q is established.

[0018]

[0019] Where y1 represents the vehicle's operational data, y2 represents the vehicle's autonomous driving fault data, and β1, β2, and β3 are feature fusion factors that fuse the vehicle's operational data and autonomous driving fault data to obtain the fused vehicle fault data.

[0020] M22. Input the fused vehicle fault data into a deep learning-based vehicle fault severity diffusion model for training and learning, and determine the representation function W of the vehicle fault severity.

[0021]

[0022] Where z represents the fused vehicle fault data, and δ1, δ2, and δ3 are the model weight factors, resulting in a well-trained deep learning-based diffusion model of vehicle fault severity.

[0023] M23. Based on the trained deep learning-based vehicle fault severity diffusion model, input the fused vehicle fault data information to characterize the vehicle fault severity and obtain the vehicle fault severity data information.

[0024] Furthermore, the constraints on the weight factors δ1, δ2, and δ3 of the model are as follows:

[0025]

[0026] Furthermore, in step M3, the construction of a dynamic traffic learning prediction model based on time and space parameters to predict the traffic accident rate of vehicles includes:

[0027] M31. Based on the vehicle's operational data and the data of traffic participants surrounding the vehicle, establish a vehicle traffic mapping function R based on time and spatial parameters.

[0028] Where a represents the vehicle's operational data, b represents the data of traffic participants around the vehicle, η represents the time parameter, and γ represents the spatial parameter. The correlation between the vehicle's operational data and the data of traffic participants around the vehicle is characterized to obtain the correlation data information between the vehicle's operational data and the data of traffic participants around the vehicle.

[0029] M32. The correlation data between the vehicle's operating data and the data of surrounding traffic participants is input into a dynamic traffic learning and prediction model for training and learning, to determine the vehicle's traffic accident rate prediction function P.

[0030]

[0031] Where c represents the correlation information between the vehicle's operating data and the data of traffic participants around the vehicle, and λ1, λ2 and λ3 are the constant parameters for predicting the vehicle's traffic accident rate, thus obtaining the trained dynamic traffic learning prediction model.

[0032] M33. Based on the trained dynamic traffic learning prediction model, input the correlation information between the vehicle's operating data and the data of traffic participants around the vehicle, predict the vehicle's traffic accident rate, and obtain the predicted vehicle traffic accident rate data.

[0033] Furthermore, the time parameter η is,

[0034]

[0035] The spatial parameter γ is,

[0036]

[0037] Where a represents the vehicle's operational data, and b represents the data of traffic participants around the vehicle.

[0038] Furthermore, in step M4, the prediction and classification of vehicle emergency response level using a classification prediction algorithm optimized based on black-winged kites includes:

[0039] M41. Input the data information on the severity of the vehicle's malfunction and the data information on the predicted traffic accident rate of the vehicle into the vehicle emergency response classification prediction model for training and learning, initialize the model weights and biases, and obtain the initialized model weights and biases data information.

[0040] M42. Based on the initialized model weights and bias data, initialize the black-winged kite population, determine the population parameters and the maximum number of iterations, and obtain the initialized black-winged kite population data.

[0041] M43. Based on the data information of the initialized black-winged kite population, establish the fitness function S for individuals in the black-winged kite population.

[0042]

[0043] Where r represents the data information of the initialized black-winged kite population, and μ1 and μ2 are the fitness determinants of the individuals in the population. The fitness values ​​of the individuals in the population are calculated to obtain the fitness value data information of the individuals in the black-winged kite population.

[0044] M44. Based on the fitness data of individuals in the black-winged kite population, an objective optimization function F is established.

[0045]

[0046] Where g represents the fitness value data of individuals in the black-winged kite population, and θ1, θ2 and θ3 are the target optimization factors. The initialized model weights and biases are optimized to obtain the optimized vehicle emergency classification prediction model.

[0047] M45. Based on the optimized vehicle emergency response classification prediction model, input the vehicle's fault severity data and the predicted vehicle traffic accident rate data to predict and classify the vehicle's emergency response, and obtain the classified vehicle emergency response data.

[0048] To achieve the above and other related objectives, the present invention also provides a control system for remote takeover of an unmanned vehicle, including a computer device programmed or configured to perform the steps of the control method for remote takeover of an unmanned vehicle as described in any one of the claims.

[0049] To achieve the above and other related objectives, the present invention also provides a computer-readable storage medium storing a computer program programmed or configured to perform the control method for remote takeover of any of the unmanned vehicles described herein.

[0050] The present invention has the following positive effects:

[0051] 1. This invention constructs a deep learning-based diffusion model of vehicle fault severity to characterize the severity of vehicle faults, and combines it with a dynamic traffic learning prediction model based on time and space parameters to predict the traffic accident rate of vehicles. This not only allows for the analysis of data on the vehicle itself and other traffic participants, facilitating the improvement of autonomous driving technology, but also allows for the analysis and research of traffic participants' habits, leading to better adaptation of technical solutions.

[0052] 2. This invention uses a classification and prediction algorithm based on black-winged kites to predict and classify the emergency response level of vehicles. Combined with the establishment of a remote takeover control function P, it controls and adjusts the remote takeover of vehicles and outputs remote takeover control data information. This not only allows for the classification and management of fault types, facilitating timely judgment by safety officers, reducing the vehicle takeover rate, and improving vehicle driving smoothness, but also enables the issuance of emergency work orders to avoid untimely takeover due to safety officer misjudgment, thereby improving safety redundancy. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0054] Figure 2 This is a schematic diagram illustrating the process of constructing a deep learning-based diffusion model for vehicle fault severity according to the present invention.

[0055] Figure 3 This is a schematic diagram illustrating the process of constructing a dynamic traffic learning and prediction model based on time and space parameters according to the present invention.

[0056] Figure 4 This is a flowchart illustrating the classification and prediction algorithm based on black-winged kite optimization according to the present invention. Detailed Implementation

[0057] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0058] Example 1: As Figure 1 As shown, a control method for remote takeover of an unmanned vehicle includes:

[0059] M1. When the vehicle is driving on the road, it acquires real-time data on vehicle operation and autonomous driving malfunctions, and collects data on traffic participants around the vehicle.

[0060] M2. Based on the vehicle's operational data and autonomous driving fault data, a deep learning-based diffusion model of the severity of vehicle faults is constructed to characterize the severity of vehicle faults and obtain data on the severity of vehicle faults.

[0061] M3. Based on the vehicle's operational data and the data of traffic participants around the vehicle, a dynamic traffic learning prediction model based on time and space parameters is constructed to predict the vehicle's traffic accident rate, thereby obtaining the predicted traffic accident rate data.

[0062] M4. Based on the data information of the severity of the vehicle's malfunction and the data information of the predicted vehicle's traffic accident rate, a classification prediction algorithm based on black-winged kites is used to predict and classify the vehicle's emergency response level, thereby obtaining classified vehicle emergency response level data information.

[0063] In this embodiment, the method further includes:

[0064] M5. Based on the classified vehicle emergency response data, a remote takeover control function P is established.

[0065]

[0066] Where x represents the emergency response data of the classified vehicles, and α1, α2 and α3 are the remote takeover control factors of the vehicles, which control and adjust the remote takeover of the vehicles and output the remote takeover control data of the vehicles.

[0067] In this embodiment, the remote takeover control factors α1, α2, and α3 of the vehicle are,

[0068]

[0069] Where x represents the emergency response data of the categorized vehicles.

[0070] In this embodiment, as Figure 2 As shown, in step M2, constructing a deep learning-based diffusion model of vehicle fault severity to characterize the severity of vehicle faults includes:

[0071] M21. Based on the vehicle's operational data and autonomous driving fault data, a vehicle fault fusion function Q is established.

[0072]

[0073] Where y1 represents the vehicle's operational data, y2 represents the vehicle's autonomous driving fault data, and β1, β2, and β3 are feature fusion factors that fuse the vehicle's operational data and autonomous driving fault data to obtain the fused vehicle fault data.

[0074] M22. Input the fused vehicle fault data into a deep learning-based vehicle fault severity diffusion model for training and learning, and determine the representation function W of the vehicle fault severity.

[0075]

[0076] Where z represents the fused vehicle fault data, and δ1, δ2, and δ3 are the model weight factors, resulting in a well-trained deep learning-based diffusion model of vehicle fault severity.

[0077] M23. Based on the trained deep learning-based vehicle fault severity diffusion model, input the fused vehicle fault data information to characterize the vehicle fault severity and obtain the vehicle fault severity data information.

[0078] In this embodiment, the constraints on the weight factors δ1, δ2, and δ3 of the model are as follows:

[0079] In this embodiment, as Figure 3 As shown, in step M3, the construction of a dynamic traffic learning prediction model based on time and space parameters to predict the traffic accident rate of vehicles includes:

[0080] M31. Based on the vehicle's operational data and the data of traffic participants surrounding the vehicle, establish a vehicle traffic mapping function R based on time and spatial parameters.

[0081] Where a represents the vehicle's operational data, b represents the data of traffic participants around the vehicle, η represents the time parameter, and γ represents the spatial parameter. The correlation between the vehicle's operational data and the data of traffic participants around the vehicle is characterized to obtain the correlation data information between the vehicle's operational data and the data of traffic participants around the vehicle.

[0082] M32. The correlation data between the vehicle's operating data and the data of surrounding traffic participants is input into a dynamic traffic learning and prediction model for training and learning, to determine the vehicle's traffic accident rate prediction function P.

[0083]

[0084] Where c represents the correlation information between the vehicle's operating data and the data of traffic participants around the vehicle, and λ1, λ2 and λ3 are the constant parameters for predicting the vehicle's traffic accident rate, thus obtaining the trained dynamic traffic learning prediction model.

[0085] M33. Based on the trained dynamic traffic learning prediction model, input the correlation information between the vehicle's operating data and the data of traffic participants around the vehicle, predict the vehicle's traffic accident rate, and obtain the predicted vehicle traffic accident rate data.

[0086] In this embodiment, the time parameter η is,

[0087]

[0088] The spatial parameter γ is,

[0089]

[0090] Where a represents the vehicle's operational data, and b represents the data of traffic participants around the vehicle.

[0091] Example 2: Based on the control method for remote takeover of an unmanned vehicle in Example 1, the present invention will be further explained and described below.

[0092] like Figure 1 As shown, a control method for remote takeover of an unmanned vehicle includes:

[0093] M1. When the vehicle is driving on the road, it acquires real-time data on vehicle operation and autonomous driving malfunctions, and collects data on traffic participants around the vehicle.

[0094] M2. Based on the vehicle's operational data and autonomous driving fault data, a deep learning-based diffusion model of the severity of vehicle faults is constructed to characterize the severity of vehicle faults and obtain data on the severity of vehicle faults.

[0095] M3. Based on the vehicle's operational data and the data of traffic participants around the vehicle, a dynamic traffic learning prediction model based on time and space parameters is constructed to predict the vehicle's traffic accident rate, thereby obtaining the predicted traffic accident rate data.

[0096] M4. Based on the data information of the severity of the vehicle's malfunction and the data information of the predicted vehicle's traffic accident rate, a classification prediction algorithm based on black-winged kites is used to predict and classify the vehicle's emergency response level, thereby obtaining classified vehicle emergency response level data information.

[0097] In this embodiment, as Figure 4 As shown, in step M4, the prediction and classification of vehicle emergency response level using the black-winged kite-based optimized classification prediction algorithm includes:

[0098] M41. Input the data information on the severity of the vehicle's malfunction and the predicted traffic accident rate data information into the vehicle emergency response classification prediction model for training and learning, initialize the model weights and biases, and obtain the initialized model weights and biases data information.

[0099] M42. Based on the initialized model weights and bias data, initialize the black-winged kite population, determine the population parameters and the maximum number of iterations, and obtain the initialized black-winged kite population data.

[0100] M43. Based on the data information of the initialized black-winged kite population, establish the fitness function S for individuals in the black-winged kite population.

[0101]

[0102] Where r represents the data information of the initialized black-winged kite population, and μ1 and μ2 are the fitness determinants of the individuals in the population. The fitness values ​​of the individuals in the population are calculated to obtain the fitness value data information of the individuals in the black-winged kite population.

[0103] M44. Based on the fitness data of individuals in the black-winged kite population, an objective optimization function F is established.

[0104]

[0105] Where g represents the fitness value data of individuals in the black-winged kite population, and θ1, θ2 and θ3 are the target optimization factors. The initialized model weights and biases are optimized to obtain the optimized vehicle emergency classification prediction model.

[0106] M45. Based on the optimized vehicle emergency response classification prediction model, input the vehicle's fault severity data and the predicted vehicle traffic accident rate data to predict and classify the vehicle's emergency response, and obtain the classified vehicle emergency response data.

[0107] The categorized vehicle emergency response data is then classified into red, orange, yellow, and green levels.

[0108] The platform transmits red, orange, yellow, and green level work orders to the remote cockpit test bench via the RCU. When the work order conditions are triggered, the remote test bench interface will display a prompt with work order information. The safety officer can choose to take over or refuse based on the work order information.

[0109] The platform tracks the takeover status based on feedback from the safety officer. When the platform administrator determines that the safety officer's takeover decision is wrong or believes that takeover is necessary, an emergency work order can be manually issued to the platform, and the safety officer will be notified to take over.

[0110] In this embodiment, the present invention provides a control system for remote takeover of an unmanned vehicle, including a computer device programmed or configured to perform the steps of any of the control methods for remote takeover of an unmanned vehicle as described above.

[0111] In this embodiment, the present invention provides a computer-readable storage medium storing a computer program programmed or configured to perform the control method for remote takeover of any of the unmanned vehicles described herein.

[0112] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0113] In summary, this invention can not only analyze data from the vehicle itself and other traffic participants to facilitate the improvement of its autonomous driving technology, but also classify and manage fault types to facilitate timely judgment by safety officers, reduce vehicle takeover rate, and improve vehicle driving smoothness.

[0114] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A control method for remote takeover of an unmanned vehicle, characterized in that, The method includes: M1. When the vehicle is driving on the road, it acquires real-time data on vehicle operation and autonomous driving malfunctions, and collects data on traffic participants around the vehicle. M2. Based on the vehicle's operational data and autonomous driving fault data, a deep learning-based diffusion model of the severity of vehicle faults is constructed to characterize the severity of vehicle faults and obtain data on the severity of vehicle faults. M3. Based on the vehicle's operational data and the data of traffic participants around the vehicle, a dynamic traffic learning prediction model based on time and space parameters is constructed to predict the vehicle's traffic accident rate, thereby obtaining the predicted traffic accident rate data. M4. Based on the data information of the severity of the vehicle's malfunction and the data information of the predicted vehicle's traffic accident rate, the emergency response level of the vehicle is predicted and classified using a classification prediction algorithm based on black kite optimization, and the classified vehicle emergency response level data information is obtained. The method further includes: M5. Based on the classified vehicle emergency response data, a remote takeover control function P is established. ， Where x represents the emergency response data of the classified vehicles, and α1, α2 and α3 are the remote takeover control factors of the vehicles, which control and adjust the remote takeover of the vehicles and output the remote takeover control data of the vehicles.

2. The control method for remote takeover of an unmanned vehicle according to claim 1, characterized in that: The remote takeover control factors α1, α2, and α3 of the vehicle are, ， ， ， Where x represents the emergency response data of the categorized vehicles.

3. The control method for remote takeover of an unmanned vehicle according to claim 1, characterized in that, In step M2, constructing a deep learning-based diffusion model of vehicle fault severity to characterize the severity of vehicle faults includes: M21. Based on the vehicle's operational data and autonomous driving fault data, a vehicle fault fusion function Q is established. ， Where y1 represents the vehicle's operational data, y2 represents the vehicle's autonomous driving fault data, and β1, β2, and β3 are feature fusion factors that fuse the vehicle's operational data and autonomous driving fault data to obtain the fused vehicle fault data. M22. Input the fused vehicle fault data into a deep learning-based vehicle fault severity diffusion model for training and learning, and determine the representation function W of the vehicle fault severity. ， Where z represents the fused vehicle fault data, and δ1, δ2, and δ3 are the model weight factors, resulting in a well-trained deep learning-based diffusion model of vehicle fault severity. M23. Based on the trained deep learning-based vehicle fault severity diffusion model, input the fused vehicle fault data information to characterize the vehicle fault severity and obtain the vehicle fault severity data information.

4. The control method for remote takeover of an unmanned vehicle according to claim 3, characterized in that: The constraints for the weight factors δ1, δ2, and δ3 of the model are as follows: 。 5. The control method for remote takeover of an unmanned vehicle according to claim 1, characterized in that, In step M3, constructing a dynamic traffic learning prediction model based on time and space parameters to predict vehicle traffic accident rates includes: M31. Based on the vehicle's operational data and the data of traffic participants surrounding the vehicle, establish a vehicle traffic mapping function R based on time and spatial parameters. ， Where a represents the vehicle's operational data, b represents the data of traffic participants around the vehicle, η represents the time parameter, and γ represents the spatial parameter. The correlation between the vehicle's operational data and the data of traffic participants around the vehicle is characterized to obtain the correlation data information between the vehicle's operational data and the data of traffic participants around the vehicle. M32. The correlation data between the vehicle's operating data and the data of surrounding traffic participants is input into a dynamic traffic learning and prediction model for training and learning, to determine the vehicle's traffic accident rate prediction function P. ， Where c represents the correlation information between the vehicle's operating data and the data of traffic participants around the vehicle, and λ1, λ2 and λ3 are the constant parameters for predicting the vehicle's traffic accident rate, thus obtaining the trained dynamic traffic learning prediction model. M33. Based on the trained dynamic traffic learning prediction model, input the correlation information between the vehicle's operating data and the data of traffic participants around the vehicle, predict the vehicle's traffic accident rate, and obtain the predicted vehicle traffic accident rate data.

6. The control method for remote takeover of an unmanned vehicle according to claim 5, characterized in that: The time parameter η is, ， The spatial parameter γ is, ， Where a represents the vehicle's operational data, and b represents the data of traffic participants around the vehicle.

7. The control method for remote takeover of an unmanned vehicle according to claim 1, characterized in that, In step M4, the prediction and classification of vehicle emergency response level using a classification prediction algorithm optimized based on black-winged kites includes: M41. Input the data information on the severity of the vehicle's malfunction and the predicted traffic accident rate data information into the vehicle emergency response classification prediction model for training and learning, initialize the model weights and biases, and obtain the initialized model weights and biases data information. M42. Based on the initialized model weights and bias data, initialize the black-winged kite population, determine the population parameters and the maximum number of iterations, and obtain the initialized black-winged kite population data. M43. Based on the data information of the initialized black-winged kite population, establish the fitness function S for individuals in the black-winged kite population. ， Where r represents the data information of the initialized black-winged kite population, and µ1 and µ2 are the fitness determinants of the individuals in the population. The fitness values ​​of the individuals in the population are calculated to obtain the fitness value data information of the individuals in the black-winged kite population. M44. Based on the fitness data of individuals in the black-winged kite population, an objective optimization function F is established. ， Where g represents the fitness value data of individuals in the black-winged kite population, and θ1, θ2 and θ3 are the target optimization factors. The initialized model weights and biases are optimized to obtain the optimized vehicle emergency classification prediction model. M45. Based on the optimized vehicle emergency response classification prediction model, input the vehicle's fault severity data and the predicted vehicle traffic accident rate data to predict and classify the vehicle's emergency response, and obtain the classified vehicle emergency response data.

8. A control system for remote takeover of an unmanned vehicle, comprising computer equipment, characterized in that, The computer device is programmed or configured to perform the steps of the control method for remote takeover of an unmanned vehicle as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is programmed or configured to perform the control method for remote takeover of an unmanned vehicle as described in any one of claims 1 to 7.

Images