A method, device, equipment and storage medium for taking over driving of a commercial vehicle

Abstract

This application discloses a method, apparatus, device, and storage medium for driver takeover of a commercial vehicle. The method includes: acquiring sensor data; determining the current work scenario type based on a defined work area; determining a risk level for the target commercial vehicle based on the work scenario type and a warning level for the target commercial vehicle based on driver status information; triggering autonomous driving takeover when the risk level meets a first preset condition and / or the warning level meets a second preset condition; after autonomous driving takeover, determining whether the target commercial vehicle has moved from the current work area to a safe area within a set takeover time; if so, determining that the driver takeover was successful; otherwise, remote driver takeover. This application can trigger autonomous driving takeover with different preset conditions, making the response strategy more flexible and diverse. Moreover, it can ensure the timeliness and safety of autonomous driving takeover in complex work environments.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, specifically to the fields of autonomous driving, vehicle safety, and driver takeover, and particularly to a method, device, equipment, and storage medium for driver takeover of a commercial vehicle. Background Technology

[0002] In the field of vehicle safety, sudden illness of the driver (such as cardiac arrest or stroke) is an extremely dangerous emergency, and traditional vehicles have a significant safety shortcoming in such emergency scenarios—the lack of proactive emergency mechanisms. This means that when a driver loses the ability to drive due to a sudden illness, the vehicle cannot take effective measures to avoid risks on its own, which can easily lead to serious traffic accidents.

[0003] Meanwhile, the lack of deep integration between existing health monitoring equipment and vehicle control systems directly leads to the absence of critical safety functions. Even if health monitoring equipment can detect abnormalities in the driver's body in a timely manner, it cannot effectively transmit this crucial information to the vehicle control system, thus failing to trigger emergency operations such as autonomous driving takeover. This leaves the vehicle in a dangerous situation of complete loss of control after the driver becomes incapacitated.

[0004] In the unique working environment of mining areas, the above problems are even more pronounced, and the limitations of existing systems are further amplified, mainly in the following two aspects:

[0005] First, the system suffers from limited scene recognition. The mining environment is complex and varied, encompassing diverse and complex operational processes such as waiting in loading / unloading areas, steep slope ascents / descents, traversing narrow mine tunnels, operating near cliff edges, and maintenance areas. Different scenarios are accompanied by drastically different levels of extremely high risk. However, existing systems typically only recognize simple vehicle states such as "moving" or "stationary," failing to accurately identify these complex and high-risk specific scenarios in mining areas. This results in a severe lack of risk perception in the actual working environment.

[0006] Secondly, the response strategy is rigid. In the face of emergencies such as driver incapacitation, existing systems often employ a "one-size-fits-all" response strategy, most commonly by immediately applying emergency braking upon detecting incapacitation. However, in a mining environment, this rigid response strategy is extremely dangerous: applying emergency braking on a steep slope could cause a heavily loaded vehicle to lose control and overturn due to excessive inertia; braking below the loading / unloading area could leave the vehicle directly under falling rocks or loading / unloading machinery, posing a significant risk of secondary accidents; and in narrow, cliff-side mine tunnels, improper emergency braking or steering could directly lead to catastrophic consequences such as the vehicle plunging off the cliff.

[0007] In summary, existing technologies have significant shortcomings in emergency response for vehicles in mining areas, failing to meet the safety requirements of complex mining environments. Therefore, there is an urgent need for a driver takeover solution specifically designed for mining areas, capable of accurately sensing the environment, intelligently identifying unique scenarios, dynamically assessing risks, and executing scenario / risk-adaptive graded responses. This solution would fill the technological gap in emergency safety for mining vehicles and ensure the safety of personnel and equipment in mining operations. Summary of the Invention

[0008] This application provides a method, device, equipment, and storage medium for driver takeover of commercial vehicles to solve the problems of existing driver takeover solutions, such as limited scene recognition and rigid response strategies, which cannot meet the safety requirements of complex operating environments in mining areas.

[0009] The technical solution is as follows:

[0010] Firstly, a method for driver takeover in commercial vehicles is provided, including:

[0011] Sensor data is acquired from multiple sensors configured inside and outside the target commercial vehicle. The sensor data includes vehicle driving information, driver status information, and dedicated map information.

[0012] The positioning information in the vehicle driving information is matched with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and the current operating scenario type is determined based on the operating area;

[0013] Based on the type of the operation scenario, a matching risk level is determined for the target commercial vehicle, and based on the driver status information, a matching warning level is determined for the target commercial vehicle.

[0014] When the risk level meets the first preset condition, and / or the warning level meets the second preset condition, the autonomous driving takeover is triggered.

[0015] After the autonomous driving system takes over, it is determined whether the target commercial vehicle has moved from the current operating area to a safe area within the set takeover time.

[0016] If so, then the driver takeover has been successfully completed.

[0017] Otherwise, trigger remote driving takeover.

[0018] In one possible implementation, the dedicated map information is a high-precision electronic map of the mining area, and the sensor data also includes: driving environment information;

[0019] The location information in the vehicle's driving information is matched with the dedicated map information to determine the target commercial vehicle's operating area in the mining area, and the current operating scenario type is determined based on the operating area, specifically including:

[0020] The location information of the target commercial vehicle is extracted from the vehicle driving information;

[0021] The location information is matched with the high-precision electronic map of the mining area to determine the operating area of ​​the target commercial vehicle in the mining area.

[0022] Key operational parameters are determined based on the driving environment information and the vehicle driving information; the key operational parameters include at least some or all of the following: terrain risk, distance to cliff, operational status, and dynamic obstacles.

[0023] The target commercial vehicle's current operating scenario type is assessed based on key operating parameters and the operating area.

[0024] In one possible implementation, a matching risk level is determined for the target commercial vehicle based on the type of the operational scenario, specifically including:

[0025] The preset risk level library is traversed using the aforementioned job scenario type. The risk level library contains multiple risk levels of different levels, and each risk level corresponds to one or more job scenario types. The job scenario types corresponding to different risk levels are different.

[0026] If a matching work scenario type is found, the risk level of the work scenario type is taken as the risk level that the target commercial vehicle matches.

[0027] If no matching operation scenario type is found, the highest level is selected from the risk level library as the risk level that matches the target commercial vehicle.

[0028] In one possible implementation, the driver status information includes at least: the grip force applied to the steering wheel, the driver's seating posture, the driver's facial image, and the driver's vital signs;

[0029] Based on the driver status information, a matching warning level is determined for the target commercial vehicle, specifically including:

[0030] The feature vectors of the grip strength, driver's sitting posture, driver's facial image, and driver's vital signs are extracted separately, fused according to the set weights, and then input into the prediction model. The prediction model is obtained by repeated training based on the driver's historical state information as samples and physical state as labels.

[0031] The output is the driver's predicted physical condition and the corresponding warning level.

[0032] In one possible implementation, when the risk level meets a first preset condition, and / or the warning level meets a second preset condition, an autonomous driving takeover is triggered, specifically including:

[0033] When the risk level reaches the first threshold level and the target commercial vehicle is not empty, and / or when the warning level reaches the second threshold level, the autonomous driving takeover is triggered.

[0034] In one possible implementation, determining whether the target commercial vehicle has moved from the current work area to a safe area within a set takeover time period specifically includes:

[0035] Based on the operating scenario type of the target commercial vehicle, determine at least one safe area that meets a set distance threshold from the operating area of ​​the target commercial vehicle;

[0036] The location information of the target commercial vehicle is acquired in real time, and the target commercial vehicle is moved from the current working area to the safe area based on whether the location information overlaps with the at least one safe area within a set takeover time.

[0037] Secondly, a driver takeover device for a commercial vehicle is provided, comprising:

[0038] The acquisition module is used to acquire sensor data from multiple sensors configured inside and outside the target commercial vehicle. The sensor data includes vehicle driving information, driver status information, and dedicated map information.

[0039] The first determining module is used to match the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determine the current operating scenario type based on the operating area;

[0040] The second determining module is used to determine a matching risk level for the target commercial vehicle based on the type of the operation scenario, and to determine a matching warning level for the target commercial vehicle based on the driver status information;

[0041] The triggering module is used to trigger the execution of autonomous driving takeover when the risk level meets a first preset condition and / or the warning level meets a second preset condition.

[0042] The judgment module is used to determine, after the autonomous driving takes over, whether the target commercial vehicle has moved from the current working area to a safe area within a set takeover time.

[0043] The third determining module is used to determine that the driving takeover was successful if the judgment result is yes.

[0044] The triggering module is also used to trigger remote driving takeover if the judgment result is negative.

[0045] Thirdly, an electronic device is provided, comprising:

[0046] At least one processor; and

[0047] A memory communicatively connected to the at least one processor; wherein,

[0048] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0049] Fourthly, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the aspects described above and any possible implementation thereof.

[0050] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the aspects and any possible implementations described above.

[0051] Sixthly, a commercial vehicle is provided, including the electronic equipment described above.

[0052] The beneficial effects of the technical solution provided in this application include at least the following:

[0053] As can be seen from the above technical solution, this application embodiment acquires sensor data from multiple sensors configured inside and outside the target commercial vehicle, matches the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determines the current operating scenario type based on the operating area; then, it determines a matching risk level for the target commercial vehicle based on the operating scenario type, and a matching warning level for the target commercial vehicle based on the driver status information; when the risk level meets a first preset condition, and / or the warning level meets a second preset condition, it triggers the execution of autonomous driving takeover; after autonomous driving takeover, it is determined whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time; if so, it is determined that the driving takeover was successful. This application can determine the operating scenario type based on the operating area of ​​the target commercial vehicle in the mining area, and match the risk level and warning level with the operating scenario type and driver status, deeply linking complex operating scenarios and driver status, thereby triggering the execution of autonomous driving takeover with different conditions through different preset conditions, making the response strategy more flexible and diverse. Furthermore, using the set takeover time to measure the performance after driver takeover ensures the timeliness and safety of autonomous driving takeover in complex operating environments.

[0054] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram of the steps of the driving takeover method for commercial vehicles provided in the embodiments of this application.

[0057] Figure 2 This is a flowchart illustrating step 104 of the driving takeover method for commercial vehicles provided in this application embodiment.

[0058] Figure 3 This is a schematic diagram of the driver takeover process for a commercial vehicle provided in an embodiment of this application.

[0059] Figure 4 This is a structural block diagram of a driver takeover device for a commercial vehicle provided in another embodiment of this application.

[0060] Figure 5 This is a block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0061] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0062] Obviously, the described embodiments are only some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0063] It should be noted that the terminal devices involved in the embodiments of this application may include, but are not limited to, smart devices such as mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers; the display devices may include, but are not limited to, personal computers, televisions, and other devices with display functions.

[0064] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0065] Given that existing driver takeover schemes suffer from limitations such as limited scene recognition and rigid response strategies, failing to meet the safety requirements of complex mining environments, this application proposes a novel driver takeover scheme, primarily targeting the autonomous driving takeover of heavy-duty commercial trucks operating in complex environments such as mining areas. The main inventive concept involves: acquiring sensor data from multiple sensors located inside and outside the target commercial vehicle; matching the vehicle's location information with dedicated map information to determine the target commercial vehicle's operating area within the mining area; and determining the current operating scenario type based on the operating area. Subsequently, a matching risk level is determined for the target commercial vehicle based on the operating scenario type, and a matching warning level is determined based on the driver's status information. When the risk level meets a first preset condition, and / or the warning level meets a second preset condition, autonomous driving takeover is triggered. After autonomous driving takeover, it is determined whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time; if so, the driver takeover is considered successful. This application can determine the type of work scenario based on the target commercial vehicle's operating area in the mining area, and match the risk level and warning level with the type of work scenario and the driver's state. This deeply links complex work scenarios and driver state, thereby triggering different autonomous driving takeovers based on different preset conditions, making the response strategy more flexible and diverse. Moreover, it uses a set takeover time to measure the execution after driver takeover, ensuring the timeliness and safety of autonomous driving takeover in complex working environments.

[0066] Reference Figure 1 The diagram illustrates the steps of a commercial vehicle driver takeover method provided in this application embodiment. It should be noted that the executing entity of this commercial vehicle driver takeover method can be a commercial vehicle driver takeover device. This device can be a hardware device with data calculation, processing, and storage functions, or a software module or component integrated into a hardware device. Specifically, the device can be a computer, tablet computer, smartphone, etc., acting as a server to establish a communication connection with the commercial vehicle to process the commercial vehicle's driver takeover scheme. Alternatively, the device can be an on-board module installed and integrated on the target commercial vehicle, possessing the aforementioned calculation and processing functions, and also capable of processing the commercial vehicle's driver takeover scheme.

[0067] like Figure 1 As shown, the driving takeover method for this commercial vehicle may include the following steps:

[0068] Step 102: Obtain sensor data from multiple sensors configured inside and outside the target commercial vehicle. The sensor data includes vehicle driving information, driver status information, and dedicated map information.

[0069] In this application, the target commercial vehicle may be equipped with multiple sensors inside (cabin or passenger compartment) or outside, such as vehicle status sensors, including: high-precision GPS, inertial measurement units (accelerometers, gyroscopes), vehicle CAN bus data (vehicle speed, gear position, accelerator / brake pedal position, steering wheel angle, motor speed, load sensor data, retarder status, etc.); environmental perception sensors, such as: forward / lateral millimeter-wave radar (penetrating dust), short-range ultrasonic radar (side / rear obstacles), optional dust-resistant infrared camera (for auxiliary verification), lidar (for high-precision terrain modeling); and driver status sensors, such as vibration / dust-optimized steering wheel grip sensors, seat sensors, etc. Sensors (detecting seat collapse), millimeter-wave radar for vital sign monitoring (penetrating clothing to detect abnormal breathing / heartbeat); mining-specific data sources, such as high-precision digital maps of mining areas, including road networks, slope information, curve curvature, coordinates of preset high-risk areas (loading and unloading areas, steep slope start and end points, narrow road sections, cliff-side road section markings, avoidance zone locations), speed limit information, work area divisions, etc.; to monitor and collect information on the vehicle's external driving environment, vehicle interior status, and driver status in real time. At the same time, it can also obtain specific map information for the target commercial vehicle's work area, such as an electronic map of the mining area, which can be a map information constructed by real-time acquisition of external driving information, vehicle interior status, etc.

[0070] In practice, cameras, radars, and in-vehicle status sensors deployed throughout the target commercial vehicle can monitor and record the vehicle's internal and external driving status and environment. For example, it can collect information on the vehicle's external environment (obstacle detection, visual perception at night or in adverse weather conditions), vehicle information, and driver status information (fatigue driving monitoring, health status monitoring, natural language processing).

[0071] Step 104: Match the location information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determine the current operating scenario type based on the operating area.

[0072] In this application, when making a takeover decision on a vehicle, the type of operating scenario of the target commercial vehicle is also taken into account, that is, the different operating scenarios faced by the target commercial vehicle in complex operating environments are taken into account.

[0073] Optionally, the dedicated map information is a high-precision electronic map of the mining area, and the sensor data also includes driving environment information; then, when matching the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determining the current operating scenario type based on the operating area, refer to... Figure 2 As shown, the specific steps may include:

[0074] Step 202: Extract the location information of the target commercial vehicle from the vehicle driving information.

[0075] Step 204: Match the location information with the high-precision electronic map of the mining area to determine the operating area of ​​the target commercial vehicle in the mining area.

[0076] Step 206: Determine key operational parameters based on the driving environment information and the vehicle driving information; the key operational parameters include at least some or all of the following: terrain risk, distance to cliff, operational status, and dynamic obstacles.

[0077] Step 208: Evaluate the current operating scenario type of the target commercial vehicle based on the key operating parameters and the operating area.

[0078] In practice, the system can locate the vehicle in real time and match it with a high-precision mining area map to accurately determine the vehicle's work area. This work area can include: main transport road areas, loading areas, climbing areas, cliff-side areas, curve areas, repair shop entrance areas, and repair shop areas. Then, key operational parameters can be extracted from the driving environment information and the vehicle's driving information. These key operational parameters have a significant impact on the target commercial vehicle's operational performance. Furthermore, terrain risk is quantified by querying slope values ​​on the map based on the current GPS location and verifying and dynamically updating terrain risk in conjunction with measured real-time pitch angles; the distance to the cliff is calculated (using map information and lateral radar / camera perception); road surface risk is assessed (based on vibration sensor data, historical data, or map markers); operational status is inferred by combining vehicle status (vehicle speed ≈ 0, gear in neutral, engine idling, etc., classified as "loading / unloading waiting"; low-speed creep, specific gear, etc., classified as "loading / unloading in progress"; continuous uphill / downhill, throttle / brake depth, etc., classified as "climbing / downhill") and scheduling task information (if the destination is a spoil heap, it is judged that it may enter the "soil heap operation" state) for comprehensive judgment; dynamic obstacle risk is assessed by using radar / cameras to perceive the proximity and trajectory of nearby mobile equipment (electric shovels, bulldozers, other mining trucks), personnel, and falling rocks. Furthermore, based on key operational parameters and operational area assessments, the current operational scenario type of the target commercial vehicle can be determined. For example, it may include: the main transport lane, below the loading area, the middle section of the 15% uphill section leading to the spoil heap, a cliffside bend, the entrance to the maintenance workshop, etc.

[0079] It should be noted that the types of work scenarios in this application are not limited to those mentioned above, and may also include other types of work scenarios that may occur in complex work scenarios, which will not be listed here.

[0080] Step 106: Determine the matching risk level for the target commercial vehicle based on the operation scenario type, and determine the matching warning level for the target commercial vehicle based on the driver status information.

[0081] In this application, different risk levels can be pre-set for target commercial vehicles operating in complex work scenarios based on the different types of work scenarios, thus establishing a risk level database. Specifically, a mine risk level database can be established for mining work scenarios, which can be set into four risk levels according to their risk level: extreme risk level, high risk level, general risk level, and low risk level. Each risk level corresponds to a different type of work scenario. Among them, work scenario types such as directly below the loading and unloading area, operating position at the edge of the spoil heap, middle / downhill section of a steep slope (>10%), and narrow road sections near cliffs (lateral safety distance <2 meters) correspond to the extreme risk level; work scenario types such as the start and end points of steep slopes, narrow road sections not near cliffs, curves, and areas close to work equipment correspond to the high risk level; work scenario types such as driving straight on the main road and driving at low speed in a safe area correspond to the general risk level; and work scenario types such as parking lots, maintenance areas, and designated safe parking points correspond to the low risk level.

[0082] Furthermore, when determining the matching risk level for the target commercial vehicle based on the operation scenario type, the operation scenario type can be used to traverse a preset risk level library. The risk level library contains multiple risk levels of different levels, each corresponding to one or more operation scenario types, and the corresponding operation scenario types differ between different risk levels. If a matching operation scenario type is found, the risk level corresponding to that operation scenario type is used as the matching risk level for the target commercial vehicle. If no matching operation scenario type is found, the highest level is selected from the risk level library as the matching risk level for the target commercial vehicle. In other words, after determining the operation scenario type, the risk levels in the risk level library can be traversed sequentially according to the operation scenario type until a matching operation scenario type is found. If a matching risk level is found, the found risk level is used as the matching risk level for the target commercial vehicle; otherwise, the highest risk level is used as the matching risk level for the target commercial vehicle.

[0083] Optionally, the driver state information includes at least: grip force applied to the steering wheel, driver's seating posture, driver's facial image, and driver's vital signs. Therefore, when determining a matching warning level for the target commercial vehicle based on the driver state information, the feature vectors of the grip force, driver's seating posture, driver's facial image, and driver's vital signs can be extracted separately, fused according to set weights, and input into a prediction model. This prediction model is obtained through repeated training based on the driver's historical state information as samples and physical state as labels. The output is the driver's predicted physical state and the matching warning level.

[0084] In this application, driver status is assessed based on multi-source sensor data. Therefore, driver status information reflects various aspects of the driver's condition, including grip strength, posture, and vital signs. Specifically, multiple types of status information can be collected from a steering wheel grip strength sensor, a seat sensor (detecting posture collapse), and a millimeter-wave radar vital sign monitor (penetrating clothing to detect abnormal breathing / heartbeat). A matching fusion algorithm is then used to aggregate this multi-type status information, and the aggregated result is used to characterize the driver's status. For example, different status information can be assigned values ​​and set reasonable weights, and then a weighted sum can be obtained to represent the driver's status. Alternatively, different status information can be used as different feature vector branches, input into a trained model for score prediction, and the total status can be calculated based on the weight ratio of each feature vector branch in the model to characterize the driver's status.

[0085] Then, different warning levels can be set for different driver states, establishing a warning level library. This warning level library can be set from highest to lowest as follows: severe disability alarm, general alarm, pre-alarm, and normal (i.e., no alarm). Specifically, for a driver in a severely disabled state (comatose or completely incapacitated), the warning level is set to severe disability alarm; for a driver showing obvious signs of disability (such as pre-semi-comatose symptoms - slow reaction, frequent microsleep), the warning level is set to general alarm; for a driver in a mild fatigue / inattention state (which may only trigger a warning), the warning level is set to pre-alarm; and for a driver in a normal state, the warning level can be set to normal.

[0086] It should be understood that this warning level database can be modified according to drivers' evolving safety needs, such as by adding warning levels or changing the driver status associated with each warning level. This allows for flexible adjustments and updates to the warning level database, improving the user experience.

[0087] Step 108: When the risk level meets the first preset condition and / or the warning level meets the second preset condition, trigger the execution of autonomous driving takeover.

[0088] In this application, the first preset condition is a condition for meeting the risk level, and the second preset condition is a condition for meeting the warning level. If either condition is met, it indicates that the target commercial vehicle in the current operating scenario has a driving safety problem, and the autonomous driving takeover can be triggered.

[0089] One feasible solution has the following first preset condition: the risk level reaches a first threshold level and the target commercial vehicle is not empty. If it is empty, it means that no goods are being loaded, and the vehicle weight has little impact on the safety of the operating area where the target commercial vehicle is located. Therefore, the risk level can be lowered by one level. If it is not empty, the safety risk will be related to the load, so a threshold level needs to be set as the criterion for judging the risk level. The second preset condition is: the warning level reaches a second threshold level. Then, when the risk level reaches the first threshold level and the target commercial vehicle is not empty, and / or the warning level reaches the second threshold level, autonomous driving takeover is triggered.

[0090] Specifically, a tiered response strategy library is established: Risk level + Warning level = Response strategy.

[0091] Early warning + low risk = Level 1 warning: Seat vibration + dashboard icon flashing, a gentle reminder;

[0092] Early warning + general risk / high risk = Level 2 strong warning: strong seat vibration + rapid beeping sound + red flashing warning on HUD + slight braking pulse (reminder-like light braking), strongly alerting and drawing attention;

[0093] Early warning + extreme risk = Level 3 emergency warning + pre-preparation: strongest sound, light and vibration alarm + vehicle pre-deceleration (such as retarder pre-start, transmission pre-shift to low gear) + send "driver abnormal condition, high risk scenario" warning to the dispatch center to maximize the warning, prepare for possible immediate takeover, and notify the dispatch center in advance;

[0094] General alarm + low risk = Level 1 intervention: continuous strong alarm + smooth deceleration of the vehicle to a safe stop (select a safe area) + hazard lights on, safe stop, wait for manual intervention;

[0095] General alarm + general risk = Level 2 intervention: continuous strong alarm + active vehicle control to smoothly decelerate and stop along the current safe lane + hazard lights activated + send "driver incapacitated, vehicle under control" message to the dispatch center, and stop the vehicle as quickly and controllably as possible in a relatively safe environment;

[0096] General alarm + high risk / extreme risk = Level 3 emergency takeover: strongest alarm + immediate implementation of safety-first emergency takeover control strategy + broadcasting precise alarm information (location, status, risk level) to the dispatch center and nearby vehicles. At all costs, get out of the current extremely dangerous environment in the safest way to prevent major accidents.

[0097] Severe incapacity alarm + any scenario = highest priority emergency takeover: immediately implement the highest level of emergency takeover control strategy (even in low-to-medium risk scenarios, implement the high-risk strategy) + broadcast an emergency alarm that the driver has completely lost his ability and must be taken over immediately.

[0098] It is evident that when the risk level is moderate risk, the vehicle is not unloaded, and the warning level is moderate alarm, automatic takeover can be triggered to ensure operational and driving safety as much as possible.

[0099] Step 110: After the autonomous driving takeover, determine whether the target commercial vehicle has moved from the current operating area to a safe area within the set takeover time.

[0100] After autonomous driving takes over, the safety of the target commercial vehicle may not be guaranteed due to different response strategies. Therefore, based on the operating scenario type of the target commercial vehicle, at least one safe area that meets a set distance threshold from the operating area of ​​the target commercial vehicle can be determined; the location information of the target commercial vehicle can be acquired in real time, and it can be determined whether the target commercial vehicle has moved from the current operating area to the safe area based on whether the location information overlaps with the at least one safe area within a set takeover time.

[0101] If yes, proceed to step 112; otherwise, proceed to step 114.

[0102] Step 112: Confirm successful takeover of the driving position.

[0103] Step 114: Trigger remote driving takeover.

[0104] The primary goal of driver takeover is not to stop the vehicle, but to move it quickly to the nearest safe area in the most controllable way (such as a clearance zone, a straight flat road, or away from a loading / unloading area / cliff edge), and then to stop it safely, avoiding parking at high-risk points. Safe path planning is based on high-precision maps, real-time positioning, and environmental perception (radar) to plan the shortest, straightest, and least obstructed path to the nearest safe area (either preset on the map or calculated in real time) in a very short time.

[0105] Reference Figure 3 The diagram shown is a schematic representation of the autonomous driving takeover process in this application.

[0106] After sensor data is input, it is processed through different threads: scene risk identification and driver status assessment.

[0107] Among them, scenario risk identification can be based on sensor data to determine the work area and classify the types of work scenarios, and then match the appropriate risk level according to the determined types of work scenarios.

[0108] Driver condition assessment can be based on sensor data, using a fusion algorithm to determine a comprehensive driver condition index, and then using the comprehensive driver condition index to match an appropriate warning level.

[0109] Subsequently, based on the tiered response strategy, corresponding execution strategies are assigned to the risk level and warning level of the target commercial vehicle. This, combined with appropriate vehicle execution and communication scheduling, determines whether it is necessary to switch the target commercial vehicle to autonomous driving takeover mode, remote driving takeover mode, or remain in the current manual driving mode. Simultaneously, the system status can be recorded in real time.

[0110] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0111] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0112] Figure 4 This application provides a structural block diagram of a driver takeover device for a commercial vehicle according to one embodiment. Figure 4As shown. The driver takeover device 400 for a commercial vehicle in this embodiment may include an acquisition module 401, a first determination module 402, a second determination module 403, a trigger module 404, a judgment module 405, and a third determination module 406. The acquisition module 401 is used to acquire sensor data from multiple sensors configured inside and outside the target commercial vehicle. The sensor data includes vehicle driving information, driver status information, and dedicated map information. The first determination module 402 is used to match the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and to determine the current operating scenario type based on the operating area. The second determination module 403 is used to determine a matching risk level for the target commercial vehicle based on the operating scenario type, and a matching warning level for the target commercial vehicle based on the driver status information. The trigger module 404 is used to trigger automatic driving takeover when the risk level meets a first preset condition, and / or the warning level meets a second preset condition. The judgment module 405 is used to determine, after automatic driving takeover, whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time. The third determining module 406 is used to determine that the current driving takeover is successful if the determination result is yes. The triggering module 404 is also used to trigger remote driving takeover if the determination result is no.

[0113] It should be noted that the driving takeover device of the commercial vehicle in this embodiment may be part or all of an application located on a local terminal, or it may be a plugin or software development kit (SDK) or other functional unit set in an application located on a local terminal, or it may be a processing engine located on a network-side server, or it may be a distributed system located on the network side, such as a processing engine or distributed system in a network-side autonomous driving platform, etc. This embodiment does not impose any particular limitations on this.

[0114] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0115] Optionally, in one possible implementation of this embodiment, the dedicated map information is a high-precision electronic map of the mining area, and the sensor data further includes: driving environment information; when the first determining module 402 matches the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determines the current operating scenario type based on the operating area, it is specifically used to: extract the positioning information of the target commercial vehicle from the vehicle driving information; match the positioning information with the high-precision electronic map of the mining area to determine the operating area of ​​the target commercial vehicle in the mining area; determine key operating parameters based on the driving environment information and the vehicle driving information; the key operating parameters include at least some or all of the following: terrain risk, distance to cliff, operating status, dynamic obstacles; and evaluate the current operating scenario type of the target commercial vehicle based on the key operating parameters and the operating area.

[0116] Optionally, in one possible implementation of this embodiment, when the second determining module 403 determines a matching risk level for the target commercial vehicle based on the operation scenario type, it is specifically used to traverse a preset risk level library using the operation scenario type; the risk level library has multiple risk levels of different levels, each risk level corresponds to one or more operation scenario types, and the operation scenario types corresponding to different risk levels are different; if a matching operation scenario type is found, the risk level matched by the operation scenario type is taken as the risk level matched by the target commercial vehicle; if no matching operation scenario type is found, the highest level is selected from the risk level library as the risk level matched by the target commercial vehicle.

[0117] Optionally, in one possible implementation of this embodiment, the driver state information includes at least: the grip force applied to the steering wheel, the driver's seating posture, the driver's facial image, and the driver's vital signs; when the second determining module 403 determines a matching warning level for the target commercial vehicle based on the driver state information, it is specifically used to extract the feature vectors of the four items respectively: the grip force, the driver's seating posture, the driver's facial image, and the driver's vital signs, and then fuse them according to the set weights and input them into the prediction model; the prediction model is obtained by repeated training based on the driver's historical state information as samples and body state as labels; the output is the driver's predicted body state and the matching warning level.

[0118] Optionally, in one possible implementation of this embodiment, when the triggering module 404 triggers the execution of autonomous driving takeover when the risk level meets the first preset condition and / or the warning level meets the second preset condition, it is specifically used to trigger the execution of autonomous driving takeover when the risk level reaches the first threshold level and the target commercial vehicle is not empty, and / or the warning level reaches the second threshold level.

[0119] Optionally, in one possible implementation of this embodiment, when determining whether the target commercial vehicle has moved from the current work area to a safe area within a set takeover time, the determination module 405 is specifically used to determine at least one safe area that meets a set distance threshold from the work area of ​​the target commercial vehicle based on the work scenario type of the target commercial vehicle; to obtain the location information of the target commercial vehicle in real time, and to determine whether the target commercial vehicle has moved from the current work area to a safe area based on whether the location information overlaps with the at least one safe area within the set takeover time.

[0120] The device can be a server, cloud platform, distributed server, etc., that establishes a communication connection with the target commercial vehicle, receives sensor data uploaded by the target commercial vehicle, and executes the aforementioned autonomous driving takeover scheme. Alternatively, the device can be an onboard module, integrated and installed on the target commercial vehicle, acquiring sensor data collected by the target commercial vehicle and executing the aforementioned autonomous driving takeover scheme.

[0121] In this embodiment, sensor data can be acquired from multiple sensors configured inside and outside the target commercial vehicle. The positioning information in the vehicle's driving information is matched with the dedicated map information to determine the target commercial vehicle's operating area in the mining area, and the current operating scenario type is determined based on the operating area. Then, a matching risk level is determined for the target commercial vehicle based on the operating scenario type, and a matching warning level is determined for the target commercial vehicle based on the driver's status information. When the risk level meets a first preset condition, and / or the warning level meets a second preset condition, autonomous driving takeover is triggered. After autonomous driving takeover, it is determined whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time. If so, the takeover is considered successful. This application can determine the operating scenario type based on the target commercial vehicle's operating area in the mining area, and match the risk level and warning level with the operating scenario type and driver status, deeply linking complex operating scenarios and driver status. This allows for triggering different autonomous driving takeovers based on different preset conditions, resulting in a more flexible and diverse response strategy. Moreover, using a set takeover time to measure the execution after driver takeover ensures the timeliness and safety of autonomous driving takeover in complex operating environments.

[0122] One embodiment of this application provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the driving takeover method for a commercial vehicle as described above.

[0123] One embodiment of this application provides an electronic device including a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the driving takeover method for a commercial vehicle as described above.

[0124] One embodiment of this application provides a commercial vehicle including the electronic equipment described above. Specifically, the autonomous driving vehicle can be a Level 2 or higher autonomous vehicle.

[0125] It should be understood that the commercial vehicles involved in this application are mainly electric heavy-duty trucks driven in special operating scenarios such as mining areas, and have the aforementioned autonomous driving functions.

[0126] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0127] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0128] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0129] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0130] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as a driving takeover method for a commercial vehicle. For example, in some embodiments, the driving takeover method for a commercial vehicle can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the driving takeover method for a commercial vehicle described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the driving takeover method for a commercial vehicle by any other suitable means (e.g., by means of firmware).

[0131] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, at least one input device, and at least one output device.

[0132] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0135] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0136] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0137] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0138] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for taking over driving in a commercial vehicle, characterized in that, include: Sensor data is acquired from multiple sensors configured inside and outside the target commercial vehicle. The sensor data includes vehicle driving information, driver status information, and dedicated map information. The dedicated map information is a high-precision electronic map of the mining area. The sensor data also includes: driving environment information. The system matches the positioning information from the vehicle's driving information with the dedicated map information to determine the target commercial vehicle's operating area in the mining area, and determines the current operating scenario type based on the operating area. Specifically, this includes: extracting the target commercial vehicle's positioning information from the vehicle's driving information; matching the positioning information with the high-precision electronic map of the mining area to determine the target commercial vehicle's operating area in the mining area; determining key operating parameters based on the driving environment information and the vehicle's driving information; the key operating parameters include at least some or all of the following: terrain risk, distance to cliffs, operating status, and dynamic obstacles; and evaluating the target commercial vehicle's current operating scenario type based on the key operating parameters and the operating area. Determining a matching risk level for the target commercial vehicle based on the operation scenario type specifically includes: traversing a preset risk level library using the operation scenario type; the risk level library contains multiple risk levels of different levels, each risk level corresponding to one or more operation scenario types, and the operation scenario types corresponding to different risk levels are different; if a matching operation scenario type is found, the risk level corresponding to the operation scenario type is used as the risk level matching the target commercial vehicle; if no matching operation scenario type is found, the highest level is selected from the risk level library as the risk level matching the target commercial vehicle; and determining a matching warning level for the target commercial vehicle based on the driver status information. When the risk level meets the first preset condition, and / or the warning level meets the second preset condition, the autonomous driving takeover is triggered. After autonomous driving takes over, it is determined whether the target commercial vehicle has moved from the current working area to a safe area within a set takeover time. Specifically, this includes: determining at least one safe area that is at least a set distance threshold from the target commercial vehicle based on the working scenario type of the target commercial vehicle; acquiring the location information of the target commercial vehicle in real time, and determining whether the target commercial vehicle has moved from the current working area to a safe area based on whether the location information overlaps with the at least one safe area within the set takeover time. If so, then the driver takeover has been successfully completed. Otherwise, trigger remote driving takeover.

2. The method as described in claim 1, characterized in that, The driver status information includes at least: the grip force applied to the steering wheel, the driver's seating posture, the driver's facial image, and the driver's vital signs; Based on the driver status information, a matching warning level is determined for the target commercial vehicle, specifically including: The feature vectors of the grip strength, driver's sitting posture, driver's facial image, and driver's vital signs are extracted separately, fused according to the set weights, and then input into the prediction model. The prediction model is obtained by repeated training based on the driver's historical state information as samples and physical state as labels. The output is the driver's predicted physical condition and the corresponding warning level.

3. The method as described in claim 1, characterized in that, When the risk level meets the first preset condition, and / or the warning level meets the second preset condition, the automatic driving takeover is triggered, specifically including: When the risk level reaches the first threshold level and the target commercial vehicle is not empty, and / or when the warning level reaches the second threshold level, the autonomous driving takeover is triggered.

4. A driver takeover device for a commercial vehicle, characterized in that, include: The acquisition module is used to acquire sensor data from multiple sensors configured inside and outside the target commercial vehicle. The sensor data includes vehicle driving information, driver status information, and dedicated map information. The dedicated map information is a high-precision electronic map of the mining area, and the sensor data also includes: driving environment information; The first determining module, when matching the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determining the current operating scenario type based on the operating area, specifically performs the following: extracting the positioning information of the target commercial vehicle from the vehicle driving information; matching the positioning information with the high-precision electronic map of the mining area to determine the operating area of ​​the target commercial vehicle in the mining area; determining key operating parameters based on the driving environment information and the vehicle driving information; the key operating parameters include at least some or all of the following: terrain risk, distance to cliff, operating status, and dynamic obstacles; and evaluating the current operating scenario type of the target commercial vehicle based on the key operating parameters and the operating area. The second determining module, when determining a matching risk level for the target commercial vehicle based on the operation scenario type, is specifically used to traverse a preset risk level library using the operation scenario type; the risk level library contains multiple risk levels of different levels, each risk level corresponding to one or more operation scenario types, and the operation scenario types corresponding to different risk levels are different; if a matching operation scenario type is found, the risk level matched by the operation scenario type is taken as the risk level matched by the target commercial vehicle; if no matching operation scenario type is found, the highest level is selected from the risk level library as the risk level matched by the target commercial vehicle; and is also used to determine a matching warning level for the target commercial vehicle based on the driver status information; The triggering module is used to trigger the execution of autonomous driving takeover when the risk level meets a first preset condition and / or the warning level meets a second preset condition. The judgment module, after autonomous driving takeover, determines whether the target commercial vehicle has moved from the current work area to a safe area within a set takeover time. Specifically, it is used to determine at least one safe area that meets a set distance threshold from the work area of ​​the target commercial vehicle based on the work scenario type of the target commercial vehicle; and to obtain the location information of the target commercial vehicle in real time, and to determine whether the target commercial vehicle has moved from the current work area to a safe area based on whether the location information overlaps with the at least one safe area within the set takeover time. The third determining module is used to determine that the driving takeover was successful if the judgment result is yes. The triggering module is also used to trigger remote driving takeover if the judgment result is negative.

5. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-3.

6. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-3.

7. A commercial vehicle including the electronic equipment as described in claim 5.

Images