Driving control right transfer method, transfer system and device

Abstract

The invention discloses a method, system and device for transferring a driving control right, and belongs to the technical field of automatic driving. The method comprises the following steps: collecting historical data in a cab; receiving or generating a driving control right transfer request; recognizing the historical data through a state recognition model to obtain a preparation state; matching a corresponding first duration threshold value according to the preparation state; generating at least one operation task according to the first duration threshold and the completion duration of the operation task; obtaining the completion duration and the accuracy rate of the operation task; and if the second condition is met, transferring the driving control right to the driver. Judging the preparation state of the driver through historical data before the control right transfer request; a corresponding first duration threshold value is matched; awakening and testing the reaction state of the driver through the operation task; the number of operation tasks can be dynamically adjusted, and a good preparation state of a driver is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a driving control right transfer method, a transfer system and a device. BACKGROUND

[0002] The progress of automatic driving vehicle technology enables vehicles to operate without the continuous intervention of a driver. However, in some cases, it is necessary to transfer control to the driver.

[0003] CN119389229A discloses an information interaction method of an automatic driving vehicle, when human intervention is needed or the vehicle detects an anomaly, the interaction module automatically transfers vehicle control to the driver according to the takeover mode to ensure driving safety. However, the emotional state of the driver is not considered when the vehicle control is transferred.

[0004] CN115140092A discloses a co-driving type intelligent automobile man-machine control permission personalized transfer method, a flexible permission transfer strategy based on a spline curve method, and individualized preview time and reaction time that affect the driver's permission transfer process are used to optimize and adjust the designed permission transfer strategy, making it more consistent with the manipulation preferences of different drivers. Although the matching degree of the external environment and driving ability is considered, the correlation between driving ability and driver emotional state is not considered.

[0005] The traditional scheme fails to consider that the driver's reaction delay caused by the driver's preparation state may increase the risk of accidents and reduce the experience during the control transfer process. For example, the vehicle actively transfers control to the driver 0.5-2 seconds before an accident occurs, which actually increases the likelihood of an accident.

[0006] Therefore, it is necessary to design a scheme for evaluating and restoring the driver's preparation state before the vehicle control right is transferred. SUMMARY

[0007] In view of the above technical problems existing in the prior art, the present application provides a driving control right transfer method, a transfer system and a device, which evaluate and restore the driver's preparation state before the vehicle control right is transferred.

[0008] The application discloses a driving control right transfer method, comprising the following steps: collecting historical data in a cab; receiving or generating a request for driving control right transfer; identifying the historical data through a state identification model to obtain a preparation state of a driver; matching a corresponding first duration threshold according to the preparation state; generating at least one operation task according to the first duration threshold and a completion duration of the operation task; obtaining the completion duration and accuracy of the operation task; and transferring the driving control right to the driver if a second condition is met, wherein the second condition comprises that the completion duration is not more than a second threshold and the accuracy is more than a third threshold.

[0009] Preferably, if the completion duration is between the second threshold and a sixth threshold and the accuracy is between the third threshold and an eighth threshold, another operation task is generated. If the completion duration is greater than the sixth threshold and the accuracy is less than the eighth threshold, the driver is prompted and the control right is refused to be transferred or an alternative control scheme is entered.

[0010] Preferably, the historical data is selected from a driver video or image and physiological data. Alert information is issued through audio or vision when the operation task is generated or completed.

[0011] Preferably, the preparation state comprises any one of the following states or a combination thereof: reading, dozing, sleeping, watching a video and being awake. A first duration threshold is preset for the state.

[0012] Preferably, if the second condition is not met, another operation task is generated; until the number of operation tasks exceeds a fourth threshold, the control right transfer is refused, and the driving speed is gradually reduced, double flashing is turned on and the vehicle is parked on the roadside if the external environment allows.

[0013] Preferably, the second condition further comprises that a request duration exceeds the first duration threshold or a number of completed operation tasks exceeds a fifth threshold, and an external environment safety state is good.

[0014] Preferably, a training method of the state identification model comprises: A data set is collected, and labels are established for images or videos in the data set; the labels comprise reading, dozing, sleeping, watching a video and being awake. The training set is trained based on a neural network to obtain the state identification model.

[0015] Preferably, a formula for dynamically adjusting the first duration threshold is represented as: T1= T1*(1+a*Ts t / Ts0-b*Acc / Acc0); Wherein, T1 represents a first duration threshold, a and b are weight coefficients respectively, Ts represents a standard duration, Acc represents a total completion accuracy, and Acc0 represents a standard accuracy. t represents a total completion duration of an operation task, Ts0 represents a standard completion duration, Acc represents a total completion accuracy, and Acc0 represents a standard accuracy.

[0016] The second aspect of the present application provides a handover system for implementing the above-mentioned handover method, comprising a collection module, a request module, a preparation state recognition module, a calibration module and a handover module, The collection module is configured to collect historical data in the cab. The request module is configured to receive or generate a request for handover of driving control. The preparation state recognition module is configured to recognize the historical data based on a state recognition model to obtain a preparation state. The calibration module is configured to match a corresponding first duration threshold according to the preparation state, generate at least one operation task according to the first duration threshold and the completion duration of the operation task, and obtain the completion duration and accuracy of the operation task. The handover module is configured to prompt the driver and refuse to hand over the control or enter an alternative control scheme if the second condition is met.

[0017] The third aspect of the present application provides a device, and the storage medium stores codes which, when executed, implement the above-mentioned driving control handover method.

[0018] Compared with the prior art, the present application has the following advantages: the preparation state of the driver is judged through the historical data before the control handover request, a corresponding first duration threshold is matched, the reaction state of the driver is tested through the operation task, the number of operation tasks can be dynamically adjusted to ensure that the preparation state of the driver is good. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a driving control handover method flowchart of Example 1; Figure 2 is a driving control handover method flowchart of Example 2; Figure 3 is a state recognition model training accuracy curve; Figure 4 is a state recognition model training recall rate curve; Figure 5 is a logic block diagram of the handover system of the present application. DETAILED DESCRIPTION

[0020] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0021] The present application will be further described in detail below with reference to the accompanying drawings: Embodiment 1. The embodiment provides a driving control transfer method, as shown in the following steps: Figure 1 Step 101: Collecting historical data in the cab.

[0022] Step 102: Receiving or generating a request for driving control transfer.

[0023] Step 103: According to the request, identifying the historical data by a state identification model to obtain a preparation state of the driver.

[0024] Step 104: According to the preparation state, matching a corresponding first duration threshold.

[0025] Step 105: According to the first duration threshold and a historical completion duration of the operation task, generating at least one operation task.

[0026] The historical completion duration of the operation task can be obtained by testing, and the historical average value of the operation task can be taken as a standard completion duration.

[0027] Step 106: Obtaining a completion duration and an accuracy rate of the operation task of the driver.

[0028] Step 107: According to the completion duration and the accuracy rate, obtaining a completion state of the operation task.

[0029] The completion state includes good, general and poor, etc. For example, when the completion duration is less than a second threshold, such as less than 1 second, and the accuracy rate is more than a third threshold, such as more than 90%, it is considered that the execution is good. When the completion duration is between the second threshold and a sixth threshold, such as 1-2 seconds, and the accuracy rate is between the third threshold and an eighth threshold, such as 50-90%, it is considered that the execution is general. When the completion duration is more than the sixth threshold, such as more than 2 seconds, and the accuracy rate is less than the eighth threshold, such as less than 50%, it is considered that the completion state is poor.

[0030] Step 108: If the completion state is good, prompting the driver and transferring the control. That is, a second condition is met: the completion duration is less than the second threshold, and the accuracy rate is more than the third threshold.

[0031] ​Step 109: If the completion state is generally, generate operation tasks, and execute step 106.

[0032] Step 110: If the completion state is not good, prompt the driver, and refuse to transfer control or enter an alternative control scheme.

[0033] The driver's preparation state is judged through historical data before the control transfer request, and a corresponding recovery duration is matched. The driver's reaction state is awakened and tested through operation tasks. The number of operation tasks and their completion state can be dynamically adjusted to ensure that the driver's preparation state is good. In the case of poor preparation state, the control transfer is refused or an alternative control scheme is entered.

[0034] Embodiment 2. The present embodiment provides an improved driving control transfer method, as shown in Figure 2 comprising the following steps: Step 201: Collect historical data in the cab.

[0035] The historical data includes driver video or image, and can also collect physiological data such as pulse characteristics, breathing characteristics, eye characteristics, etc. More specifically, heart rate, breathing rate, breathing depth, blink rate, eye movement frequency, etc. However, it is not limited thereto. It should be pointed out that the collection of historical data should be authorized by the user / driver or through fully authorized information and data by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0036] Step 202: Receive or generate a request for driving control transfer.

[0037] Step 203: According to the request, identify the historical data through a state identification model to obtain a preparation state.

[0038] Step 204: According to the preparation state, match a first duration threshold of the corresponding authority transfer.

[0039] The first duration threshold, as the recovery duration of the preparation state, can be obtained through specific tests, for example, the recovery duration is 2-5 seconds in the reading state, the recovery duration is 2-6 seconds in the dozing state, the recovery duration is 10-20 seconds in the sleep state, and the recovery duration is 0.5-2 seconds in the patrol state.

[0040] Step 205: Generate at least one operation task according to the preparation state and the completion duration of the operation task.

[0041] The operation tasks include, but are not limited to, turning the steering wheel in the indicated direction, stepping on the brake pedal, stepping on the accelerator pedal, pressing the specified operation button, and clicking the specified area. More specifically, the driver can be instructed to step on the brake pedal and the accelerator pedal with a specified force or for a specified duration. In addition, the driver can be prompted to regain attention through audio or visual cues.

[0042] Step 206: Obtain the historical completion duration and accuracy of the operation tasks.

[0043] Step 207: Determine whether the completion state is good according to the completion duration and accuracy, i.e., the completion duration does not exceed a second threshold, and the accuracy exceeds a third threshold.

[0044] Step 208: If not, generate another operation task; until the number of operation tasks exceeds a fourth threshold, refuse to hand over control, and start an alternative control scheme, such as gradually reducing the driving speed, turning on the double flash, and pulling over if the external environment allows.

[0045] Step 209: If yes, determine whether the first condition is met: the request duration exceeds a first duration threshold or the number of completed operation tasks exceeds a fifth threshold, and the external environment is in a good safety state.

[0046] The request duration is the difference between the current time and the time when the request is made.

[0047] Step 210: If yes, prompt the driver and hand over control to the driver.

[0048] Step 211: If not, generate an operation task and execute step 207.

[0049] The driver's preparation state is determined through historical data before the control handover request, and the corresponding recovery duration is matched. The driver's preparation state is awakened and tested through operation tasks. The number of operation tasks is dynamically adjusted to ensure that the driver's preparation state is good. In the case of poor preparation state, the control handover is refused or the alternative control scheme is entered.

[0050] In step 103, the preparation state can be identified based on a neural network method. For example, a state identification model is trained using RNN, CNN, LSMT, and YOLO methods, and the preparation state is obtained by identifying the video or image in the historical data through the state identification model. The training method of the state identification model includes: Step 301: Collect a data set and label the images or videos in the data set.

[0051] The specific labels are: reading, dozing, sleeping, watching videos, patrolling, and being awake. In a specific data set, 5260 pictures or videos are collected and contain all label types, of which the training set includes 3700 pictures and the test set includes 1560 pictures.

[0052] Step 302: Preprocessing the data set.

[0053] The preprocessing includes operations such as compression and denoising.

[0054] Step 303: Training the training set based on YOLOv10 to obtain a state recognition model.

[0055] Step 304: Testing the state recognition model through the test set.

[0056] The number of floating point operations of the state recognition model is 6.6, the mAPval 50-95 is 38.3, and the mAPval 50-95 calculates the average precision of the model in the range from IoU of 0.5 to 0.95. The IoU is the degree of overlap between the predicted bounding box and the true bounding box. Figure 3 The training precision curve of the state recognition model is shown in FIG. 2, Figure 4 The recall curve is shown in FIG. 3, and the specific precision is 0.926, the recall is 0.923, the F1 score is 0.924, and the precision of mAPval 50-95 is 0.621. The precision on the test set is 0.926, the recall is 0.921, and the precision of mAPval 50-95 is 0.611. It should be pointed out that the method that can be used is not limited to YOLOv10, but also other versions of YOLO such as YOLOv11, YOLOv8, and other neural network methods.

[0057] In step 206, the total completion time length of the operation task Ts t is represented as: ; wherein, ts i is represented as the completion time length of the operation task i , the total number of operation tasks is represented as n , and the total completion accuracy is represented as Acc : Acc = j / n , j is represented as the number of correct operation tasks.

[0058] The first time length threshold can also be dynamically adjusted according to the completion time length and the accuracy: T1= T1* (1 + a*Ts / Ts0 - b*Acc / Acc0) t Ts0 ; wherein, T1 is a first duration threshold, a and b are coefficients, Acc0 is a standard completion duration, Figure 5 is a standard accuracy rate, which can be obtained by testing.

[0059] Embodiment 3. In this embodiment, in the reading state, the first duration threshold is preset as 2 seconds, and two operation tasks are generated: pressing the "+" button of the steering wheel and slightly turning the steering wheel along the front road conditions. After successfully completing the operation task, the driver is prompted to take over the control of the vehicle.

[0060] If the operation task is not completed within 2 seconds or the accuracy rate of the operation task is between 30-80%, another operation task is generated: slightly turning the steering wheel along the front road conditions. After completing the operation task, the driver is prompted to take over the control of the vehicle; if the total accuracy rate is less than the eighth threshold, such as less than 30%, and the completion duration exceeds the sixth threshold, such as more than 5 seconds, the control right is refused to be transferred or an alternative control scheme is entered.

[0061] If the operation task is not completed within 2 seconds or the accuracy rate of the operation task is less than 30%, the control right is refused to be transferred or an alternative control scheme is entered. Wait for the request for taking over the control right again.

[0062] Embodiment 4. The second aspect of the present application provides a transfer system for implementing the above-mentioned driving control right transfer method, such as ​ , comprising a collection module 1, a request module 2, a preparation state identification module 3, a calibration module 4 and a transfer module 5, The collection module 1 is used to collect historical data in the cab.

[0063] The request module 2 is used to receive or generate a request for transferring the driving control right.

[0064] The preparation state identification module 3 is used to identify the historical data based on a state identification model to obtain a preparation state.

[0065] The calibration module 4 is used to: match a corresponding first duration threshold according to the preparation state; generate at least one operation task according to the first duration threshold and the completion duration of the operation task; obtain the completion duration and accuracy rate of the operation task; and obtain the completion state of the operation task according to the completion duration and the accuracy rate. The transfer module 5 is used to: if the completion state is good, prompt the driver and transfer the control right to the driver; if the completion state is general, call the calibration module 4 to generate another operation task; and if the completion state is poor, prompt the driver and refuse to transfer the control right or enter an alternative control scheme.

[0066] The handover system further comprises a training module 6 configured to train the state recognition model.

[0067] Embodiment 5. This embodiment provides a storage medium for storing code, which when executed, implements the above-mentioned driving control right handover method.

[0068] This embodiment also provides an apparatus comprising a processor and the above-mentioned storage medium, wherein the processor, when executing the code, implements the above-mentioned driving control right handover method.

[0069] The memory can include various types of storage units, such as system memory, read-only memory (ROM), and a permanent storage device. Among them, the ROM can store static data or instructions required by the processor or other modules of the computer. The permanent storage device can be a read-write storage device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device can be a removable storage device (such as a floppy disk, an optical drive). The system memory can be a read-write storage device or a volatile read-write storage device, such as a dynamic random access memory. The system memory can store some or all of the instructions and data required by the processor during runtime. In addition, the memory can include a combination of any computer readable storage media, including various types of semiconductor storage chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory can include a read and / or write removable storage device, such as a compact disc (CD), a read-only digital versatile disc (such as DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a min SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc.

[0070] In some of the flowcharts described in the specification and claims of the present application and in the above description of the drawings, a plurality of operations are included in a particular order, but it should be clearly understood that these operations can be performed in the order in which they appear in this text or in parallel, and the serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these flowcharts can include more or fewer operations, and these operations can be performed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this text are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types. In addition, it should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0071] The present application detects the driver's preparation state before the transfer of driving control, and designs and prompts the driver to operate the vehicle accordingly, calibrates the driver's state, helps the driver recover the cognitive ability of the external environment, and ensures that the driver regains full awareness of the driving environment. And fully consider the different activities of the driver before receiving the control right, such as watching movies, reading or sleeping, design the corresponding recovery time for these different activities, reduce the reaction delay of the driver, and reduce the risk of accidents. The present application aims to ensure that the new driver is associated with the driving environment again, and to ensure that the driver is focused and ready to take over the control of the vehicle, which is an effective method of situational awareness recovery.

[0072] The above is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for transferring driving control rights, characterized in that: The following steps are involved: Collect historical data in the cab; receiving or generating a request to transfer driving control; Identify historical data through the state recognition model to obtain the readiness state; According to the preparation state, matching a corresponding first duration threshold; generating at least one operation task according to the first duration threshold and the historical completion duration of the operation task; Obtain the completion time and accuracy of the operation task; If the second condition is met, driving control is handed over to the driver, where the second condition includes: the completion time does not exceed the second threshold and the accuracy rate exceeds the third threshold.

2. The handover method according to claim 1, wherein: If the completion time is between the second threshold and the sixth threshold, and the accuracy is between the third threshold and the eighth threshold, another operation task is generated; If the completion time is greater than the sixth threshold and the accuracy is less than the eighth threshold, the driver is prompted and the control is refused to be transferred or an alternative control plan is entered.

3. The handover method according to claim 1, wherein: The historical data is selected from: driver video or image, and physiological data; When an operation task is generated or completed, an audio or visual warning message is issued.

4. The handover method according to claim 1, wherein: The readiness state includes any one of the following states or a combination thereof: reading, dozing, sleeping, watching a video, and being awake; A first duration threshold is preset for the state.

5. The handover method according to claim 1, wherein: If the second condition is not met, another operation task is generated; until the number of operation tasks exceeds the fourth threshold, the control transfer is refused, and the driving speed is gradually reduced, the hazard lights are turned on, and the vehicle is parked on the side of the road if the external environment permits. The handover method according to claim 1 , wherein: The second condition also includes: the request duration exceeds a first duration threshold or the number of completed operation tasks exceeds a fifth threshold, and the external environment is in a good security state.

7. The handover method according to claim 1, characterized in that: The training methods for the state recognition model include: Collecting a data set and creating labels for images or videos in the data set; the labels include: reading, dozing, sleeping, watching a video, and awake; The training set is trained based on the neural network to obtain a state recognition model.

8. The handover method according to claim 1, wherein: The formula for dynamically adjusting the first duration threshold is expressed as: T1= T1*(1+a*Ts t / Ts0-b*Acc / Acc0); Among them, T1 represents the first duration threshold, a and b are weight coefficients, Ts t It represents the total completion time of the operation task, Ts0 represents the standard completion time, Acc represents the total completion accuracy, and Acc0 represents the standard accuracy.

9. A handover system, characterized in that: Used to implement the handover method according to any one of claims 1 to 8, comprising a collection module, a request module, a readiness status identification module, a calibration module and a handover module, The acquisition module is used to collect historical data in the cab; The request module is used to receive or generate a request for handing over driving control; The readiness state recognition module is used to identify historical data based on a state recognition model to obtain a readiness state; The calibration module is configured to: match a corresponding first duration threshold according to the preparation state; generate at least one operation task according to the first duration threshold and the completion duration of the operation task; and obtain the completion duration and accuracy of the operation task; The handover module is configured to: if the second condition is met, prompt the driver and refuse to hand over the control right or enter an alternative control scheme.

10. A device, characterized in that: It includes a storage medium, which stores a code, and when the code is executed, it implements the driving control right transfer method according to any one of claims 1 to 8.

Images