Driving takeover method, device and equipment for commercial vehicle and storage medium

Abstract

The invention discloses a driving takeover method, device and equipment for a commercial vehicle and a storage medium. The method comprises the following steps: acquiring sensor data, and determining a current operation scene type based on a determined operation area; determining a risk level for the target commercial vehicle based on the operation scene type, and determining an early warning level for the target commercial vehicle based on the driver state information; when the risk level meets a first preset condition and / or the early warning level meets a second preset condition, automatic driving takeover is triggered and executed; after automatic driving takeover, whether the target commercial vehicle is transferred from the current operation area to the safe area within the set takeover time or not is judged; and if yes, determining that the driving is successfully taken over, otherwise, taking over the remote driving. Through different preset conditions, different automatic driving takeover meeting the conditions can be triggered and executed, and response strategies are more flexible and diversified. In addition, timeliness and safety of automatic driving takeover in a complex operation environment can be guaranteed.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, specifically to technical fields such as autonomous driving, vehicle safety, and driving takeover, and in particular to a driving takeover method, device, equipment, and storage medium for a commercial vehicle. Background Art

[0002] In the field of vehicle safety, sudden driver illness (such as cardiac arrest or stroke) is an extremely dangerous emergency. Traditional vehicles have a significant safety shortcoming in such emergency scenarios: a lack of proactive emergency response mechanisms. This means that when a driver loses the ability to drive due to a sudden illness, the vehicle cannot take effective measures to mitigate the risk, which can easily lead to serious traffic accidents.

[0003] At the same time, existing health monitoring devices lack deep integration with vehicle control systems, a technological gap that directly leads to the loss of key safety features. Even if health monitoring devices can promptly detect a driver's physical abnormality, they cannot effectively transmit this critical information to the vehicle control system, thereby failing to trigger emergency actions such as autonomous driving takeover. This can lead to a dangerous loss of vehicle control if the driver becomes incapacitated.

[0004] In the special operating environment of mining areas, the above problems are more prominent, and the limitations of the existing system are further magnified, mainly in the following two aspects:

[0005] First, scenario recognition is limited. Mining environments are complex and diverse, encompassing diverse and unique operational processes, such as waiting in loading and unloading areas, climbing and descending steep slopes, navigating narrow mine tunnels, operating near cliff edges, and working in maintenance areas. Each scenario carries distinct and extremely high risk levels. However, existing systems typically only recognize simple vehicle states, such as "moving" or "stationary," and are unable to accurately identify these complex and high-risk scenarios unique to mining areas, resulting in a significant lack of risk perception in the actual operating environment.

[0006] Second, the response strategy is rigid. Existing systems often employ a one-size-fits-all response strategy for emergencies like driver incapacitation. The most common approach is to immediately initiate emergency braking upon detecting incapacitation. However, in mining environments, this rigid response strategy is extremely dangerous: Emergency braking on steep slopes can cause a heavily loaded vehicle to lose control and overturn due to excessive inertia. Braking below loading and unloading areas can cause the vehicle to become trapped directly beneath fallen rocks or loading and unloading machinery, posing a significant risk of secondary accidents. And in narrow, cliff-side mine tunnels, improper emergency braking or steering can lead to catastrophic consequences, such as a vehicle falling off the cliff.

[0007] In summary, existing technologies have obvious technical deficiencies in the emergency response of mining vehicles and cannot meet the safety needs of the complex operating environment of mining areas. Therefore, there is an urgent need for a driver takeover solution designed specifically for mining areas that can accurately perceive the environment, intelligently identify unique scenarios, dynamically assess risks, and implement scenario / risk-adaptive hierarchical responses to fill the technical gap in the field of emergency safety of mining vehicles and ensure the safety of personnel and equipment in mining operations. Summary of the Invention

[0008] The present application provides a driving takeover method, device, equipment and storage medium for a commercial vehicle to solve the problems of the existing driving takeover solution, such as single scene recognition and rigid response strategy, which cannot meet the safety requirements of the complex operating environment in the mining area.

[0009] The technical solution is as follows:

[0010] In a first aspect, a method for taking over driving of a commercial vehicle is provided, comprising:

[0011] Acquire sensor data from multiple sensors installed inside and outside the target commercial vehicle, the sensor data including vehicle driving information, driver status information, and dedicated map information;

[0012] Matching the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determining the current operating scene type based on the operating area;

[0013] Determining a matching risk level for the target commercial vehicle based on the operation scenario type, and determining a matching warning level for the target commercial vehicle based on the driver status information;

[0014] When the risk level meets a first preset condition, and / or the warning level meets a second preset condition, triggering the execution of the automatic driving takeover;

[0015] After the automatic driving takes over, determine whether the target commercial vehicle has moved from the current operating area to the safe area within the set takeover time;

[0016] If yes, it is determined that the driving is successfully taken over;

[0017] Otherwise, remote driving takeover is triggered.

[0018] In one possible implementation, the dedicated map information is a high-precision electronic map of the mining area, and the sensor data further includes: driving environment information;

[0019] Matching the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determining the current operating scene type based on the operating area, specifically including:

[0020] Extracting positioning information of the target commercial vehicle from the vehicle driving information;

[0021] Matching the positioning information with the high-precision electronic map of the mining area to determine the operating area of ​​the target commercial vehicle in the mining area;

[0022] Determining key operating parameters based on the driving environment information and the vehicle travel information; the key operating parameters include at least some or all of the following: terrain risk, distance to cliff, operating status, and dynamic obstacles;

[0023] The current operating scenario type of the target commercial vehicle is evaluated based on key operating parameters and the operating area.

[0024] In one possible implementation, determining a matching risk level for the target commercial vehicle based on the operation scenario type specifically includes:

[0025] Using the operation scenario type, traversing a preset risk level library; the risk level library is provided with a plurality of different risk levels, each risk level corresponds to one or more operation scenario types, and different risk levels correspond to different operation scenario types;

[0026] If a matching operation scene type is found, the risk level of the operation scene type is used as the risk level matched by the target commercial vehicle;

[0027] If no matching operation scenario type is found, the highest level is selected from the risk level library as the risk level that matches the target commercial vehicle.

[0028] In a possible implementation, the driver status information includes at least: grip force applied to the steering wheel, driver's sitting posture, driver's facial image, and driver's vital signs;

[0029] Determining a matching warning level for the target commercial vehicle based on the driver status information specifically includes:

[0030] Extracting feature vectors of the grip strength, the driver's sitting posture, the driver's facial image, and the driver's vital signs, respectively, fusing them according to set weights and inputting them into a prediction model; the prediction model is repeatedly trained based on the driver's historical status information as samples and the driver's physical condition as labels;

[0031] The output is the driver's predicted physical condition and the corresponding warning level.

[0032] In one possible implementation, when the risk level satisfies a first preset condition and / or the warning level satisfies a second preset condition, triggering the execution of the autonomous driving takeover specifically includes:

[0033] When the risk level reaches a first threshold level and the target commercial vehicle is not empty, and / or the warning level reaches a second threshold level, automatic driving takeover is triggered.

[0034] In one possible implementation, determining whether the target commercial vehicle has moved from the current operating area to the safe area within the set takeover time specifically includes:

[0035] Determining, based on the operating scene type of the target commercial vehicle, at least one safe area that meets a set distance threshold from the operating area of ​​the target commercial vehicle;

[0036] The position information of the target commercial vehicle is obtained in real time, and whether the target commercial vehicle has been transferred from the current operating area to the safe area is determined based on whether the position information overlaps with the at least one safe area within a set takeover time.

[0037] In a second aspect, a driving takeover device for a commercial vehicle is provided, comprising:

[0038] An acquisition module, configured to acquire sensor data from multiple sensors configured inside and outside the target commercial vehicle, wherein the sensor data includes vehicle driving information, driver status information, and dedicated map information;

[0039] A first determination module is configured to match the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determine the current operating scene type based on the operating area;

[0040] a second determination module, configured to determine a matching risk level for the target commercial vehicle based on the operation scenario type, and to determine a matching warning level for the target commercial vehicle based on the driver status information;

[0041] a triggering module, configured to trigger the execution of the automatic driving takeover when the risk level satisfies a first preset condition and / or the warning level satisfies a second preset condition;

[0042] A judgment module, configured to judge whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time after the automatic driving takes over;

[0043] The third determining module is used to determine that the driving is successfully taken over if the judgment result is yes;

[0044] The trigger module is further configured to trigger remote driving takeover if the judgment result is no.

[0045] According to a third aspect, an electronic device is provided, including:

[0046] at least one processor; and

[0047] a memory communicatively connected to the at least one processor; wherein,

[0048] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0049] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the method of the above-mentioned aspect and any possible implementation manner.

[0050] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the above-mentioned aspects and any possible implementation method.

[0051] In a sixth aspect, a commercial vehicle is provided, comprising the electronic device as described above.

[0052] The beneficial effects of the technical solution provided by this application include at least:

[0053] According to the technical solution, the sensor data is obtained from a plurality of sensors arranged inside and outside the target commercial vehicle, the positioning information in the vehicle driving information is matched with the special map information, the working area of the target commercial vehicle in the mining area is determined, and the current working scene type is determined according to the working area. Then, the matching risk level of the target commercial vehicle is determined based on the working scene type, and the matching warning level of the target commercial vehicle is determined based on the driver state information. When the risk level meets the first preset condition, and / or the warning level meets the second preset condition, the automatic driving takeover is triggered. After the automatic driving takeover, it is judged whether the target commercial vehicle is transferred from the current working area to the safe area within the set takeover time. If yes, it is determined that the driving takeover is successful. The application can determine the working scene type based on the working area of the target commercial vehicle in the mining area, and match the risk level and the warning level based on the working scene type and the driver state. The complex working scene and the driver state are deeply linked, so that the automatic driving takeover meeting different conditions is triggered by different preset conditions, and the response strategy is more flexible and various. Moreover, the set takeover time is used to measure the execution after the driving takeover, so as to ensure the timeliness and safety of the automatic driving takeover in the complex working environment.

[0054] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0056] Figure 1 is a step schematic diagram of the driving takeover method of the commercial vehicle provided by the embodiments of the application.

[0057] Figure 2 is a flowchart of step 104 in the driving takeover method of the commercial vehicle provided by the embodiments of the application.

[0058] Figure 3 is a driving takeover flowchart of the commercial vehicle provided by the embodiments of the application.

[0059] Figure 4 is a structure block diagram of the driving takeover device of the commercial vehicle provided by another embodiment of the application.

[0060] Figure 5 This is a block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following description of exemplary embodiments of the present application is provided in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0062] Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0063] It should be noted that the terminal devices involved in the embodiments of the present application may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0064] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0065] Given that existing driver takeover solutions suffer from issues such as single-scenario recognition and rigid response strategies, failing to meet the safety requirements of complex mining environments, this application proposes a new driver takeover solution specifically for autonomous driving takeover of heavy-duty commercial trucks operating in complex environments such as mining areas. The main inventive concept is to obtain sensor data from multiple sensors installed inside and outside a target commercial vehicle, match positioning information contained in the vehicle's driving information with dedicated map information, determine the target commercial vehicle's operating area within the mining area, and determine the current operating scenario type based on the operating area; then, determine a corresponding risk level for the target commercial vehicle based on the operating scenario type and a corresponding warning level for the target commercial vehicle based on the driver status information; trigger an autonomous driving takeover when the risk level meets a first preset condition and / or the warning level meets a second preset condition; and after the autonomous driving takeover, determine whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time; if so, determine that the driver takeover was successful. This application can determine the type of operating scenario based on the target commercial vehicle's operating area in the mining area, and combine the operating scenario type with the driver's status to match the risk level and warning level, deeply linking complex operating scenarios and driver status. This allows for different preset conditions to trigger the execution of autonomous driving takeovers that meet different conditions, making the response strategy more flexible and diverse. In addition, the set takeover time is used to measure the execution status after the driver takes over, ensuring the timeliness and safety of autonomous driving takeovers in complex operating environments.

[0066] Reference Figure 1 As shown, it is a schematic diagram of the steps of the driving takeover method for a commercial vehicle provided in an embodiment of the present application. It should be noted that the executor of the driving takeover method for a commercial vehicle may be a driving takeover device for a commercial vehicle, and the device may be a hardware device with functions such as data calculation, processing, and storage, or a software module or component integrated and installed on the hardware device. Specifically, the device may be a computer, tablet computer, smart phone, etc. that establishes a communication connection with the commercial vehicle as a server to process the driving takeover plan of the commercial vehicle. Alternatively, the device may be an on-board module installed and integrated on the target commercial vehicle, which has the above-mentioned calculation and processing functions, and can also process the driving takeover plan of the commercial vehicle.

[0067] like Figure 1 As shown, the driving takeover method of the commercial vehicle may include the following steps:

[0068] Step 102: Acquire sensor data from multiple sensors configured inside and outside the target commercial vehicle, wherein the sensor data includes vehicle driving information, driver status information, and dedicated map information.

[0069] In the present application, a plurality of sensors may be configured inside (inside the cabin or compartment) or outside the target commercial vehicle, such as vehicle status sensors, such as high-precision GPS, inertial measurement unit (accelerometer, gyroscope), vehicle CAN bus data (vehicle speed, gear position, throttle / brake pedal position, steering wheel angle, motor speed, load sensor data, retarder status, etc.); environmental perception sensors, such as forward / lateral millimeter-wave radar (penetrating dust), short-range ultrasonic radar (side / rear obstacles), optional dust-resistant infrared camera (auxiliary verification), lidar (for high-precision terrain modeling); driver status sensors, such as anti-vibration / dust optimized steering wheel grip sensor, seat sensor, etc. Sensors (to detect sitting collapse), millimeter-wave radar vital signs monitoring (to penetrate clothing and detect abnormal breathing / heartbeat); dedicated data sources for mining areas, such as high-precision digital maps of mining areas, including road networks, slope information, curve curvature, coordinates of preset high-risk areas (loading and unloading areas, starting and ending points of steep slopes, narrow sections, cliff section signs, avoidance zone locations), speed limit information, work area divisions and other information; to monitor and collect the vehicle's external driving environment, vehicle internal status and driver status information in real time. At the same time, it is also possible to obtain dedicated map information for the target commercial vehicle's operating area, such as an electronic map of the mining area. The electronic map can be map information constructed through real-time acquisition of external driving information, vehicle internal status, etc.

[0070] In specific implementation, cameras, radars, and in-vehicle status sensors deployed throughout the target commercial vehicle can be used to monitor and record the vehicle's internal and external driving status and environment. For example, information on the vehicle's external environment (obstacle detection, visual perception at night or inclement weather), vehicle information, and driver status information (fatigue monitoring, health status monitoring, and natural language processing) can be collected.

[0071] Step 104: Match the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determine the current operating scene type based on the operating area.

[0072] In this application, when making a takeover judgment on a vehicle, the operating scenario type of the target commercial vehicle is also taken into consideration, that is, the different operating scenarios faced by the target commercial vehicle in a complex operating environment are taken into account.

[0073] Optionally, the dedicated map information is a high-precision electronic map of the mining area, and the sensor data further includes: driving environment information; then, the positioning information in the vehicle driving information is matched with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and when determining the current operating scene type based on the operating area, reference is made to Figure 2 As shown, the following steps may be specifically included:

[0074] Step 202: extracting the positioning information of the target commercial vehicle from the vehicle driving information.

[0075] Step 204: Match the positioning information with the high-precision electronic map of the mining area to determine the operating area of ​​the target commercial vehicle in the mining area.

[0076] Step 206: Determine key operating parameters based on the driving environment information and the vehicle travel information; the key operating parameters include at least some or all of the following: terrain risk, distance to cliff, operating status, and dynamic obstacles.

[0077] Step 208: Evaluate the current operating scenario type of the target commercial vehicle based on key operating parameters and the operating area.

[0078] In specific implementations, real-time positioning and matching with high-precision mining area maps can accurately determine the vehicle's operating area. These operating areas can include: main haul roads, loading areas, climbing areas, cliff areas, curves, maintenance workshop entrances, and maintenance workshop areas. Key operating parameters, which significantly impact the target commercial vehicle's operational performance, can then be extracted from the driving environment information and vehicle travel information. Furthermore, terrain risk is quantified by querying the slope value in the map based on the current GPS location, and verifying and dynamically updating the terrain risk in combination with the measured real-time pitch angle; calculating the distance to the cliff (through map information + lateral radar / camera perception); assessing the road condition risk (based on vibration sensor data, historical data or map markers); inferring the operation status, combining the vehicle status (vehicle speed ≈ 0, neutral gear, engine idling, etc. are combined to be divided into "loading / unloading waiting"; low-speed creep, specific gear, etc. are combined to be divided into "loading / unloading in progress"; continuous uphill / downhill, throttle / brake depth, etc. are combined to be divided into "climbing / downhill") and scheduling task information (if the destination is a spoil dump, it is judged that it may enter the "soil dumping operation" state) to make a comprehensive judgment; dynamic obstacle risk is evaluated, and radar / cameras are used to perceive the proximity and trajectory of nearby mobile equipment (electric shovels, bulldozers, other mining trucks), personnel, and falling rocks. Furthermore, the current operating scenario type of the target commercial vehicle is obtained based on the key operating parameters and operating area evaluation. For example, it may include: the main transport road, under the loading area, the middle section of the 15% climbing section leading to the spoil dump, the cliff bend, the entrance to the maintenance workshop, etc.

[0079] It should be noted that the operation scene types of the present application may not be limited to those mentioned above, but may also include other operation scene types that may appear in other complex operation scenes, which are not listed one by one here.

[0080] Step 106: determining a matched risk level for the target commercial vehicle based on the job scene type, and determining a matched early warning level for the target commercial vehicle based on the driver state information.

[0081] In the scheme of the present application, different risk levels can be set for the target commercial vehicle operating in the complex job scene according to different job scene types, and a risk level library is established. Specifically, a mine risk level library can be established for a mine job scene, and four risk levels can be set in the risk level library according to the risk level from high to low: an extreme risk level, a high risk level, a general risk level, and a low risk level. Each risk level corresponds to different job scene types, wherein the job scene types such as the loading and unloading area directly below, the edge operation position of the dump site, the middle section of steep slope (>10%), downhill, and narrow cliff road section (lateral safety distance <2 meters) correspond to the extreme risk level; the job scene types such as the start and end points of steep slope, non-cliff narrow road section, curve, and area close to the operation equipment correspond to the high risk level; the job scene types such as straight driving on the main road, low-speed driving in the safety area correspond to the general risk level; and the job scene types such as the parking lot, maintenance area, and designated safe stopping point correspond to the low risk level.

[0082] Further, when determining a matched risk level for the target commercial vehicle based on the job scene type, the job scene type can be used to traverse the preset risk level library; the risk level library is provided with a plurality of risk levels of different levels, each risk level corresponds to one or more job scene types, and the job scene types corresponding to different risk levels are different; if a matched job scene type is traversed, the risk level hit by the job scene type is taken as the matched risk level of the target commercial vehicle; if a matched job scene type is not traversed, the highest level is selected from the risk level library as the matched risk level of the target commercial vehicle. In other words, after determining the job scene type, the risk level library can be traversed according to the job scene type, until a matched job scene type is traversed, and the hit risk level is taken as the matched risk level of the target commercial vehicle, otherwise, the highest risk level is taken as the matched risk level of the target commercial vehicle.

[0083] Optionally, the driver status information includes at least: the grip force applied to the steering wheel, the driver's sitting posture, the driver's facial image and the driver's vital signs; then, when determining a matching warning level for the target commercial vehicle based on the driver status information, the feature vectors of the grip force, the driver's sitting posture, the driver's facial image and the driver's vital signs can be extracted respectively, and then fused according to the set weights and input into a prediction model; the prediction model is obtained by repeated training based on the driver's historical status information as a sample and the physical condition as a label; the output is the driver's predicted physical condition and the matching warning level.

[0084] In the present application, the driver's state is evaluated based on multi-source sensor data. Therefore, the driver's state information reflects the driver's grip strength, sitting posture, vital signs and other aspects of the state information. In specific implementation, multiple types of state information collected by the steering wheel grip sensor, seat sensor (to detect sitting posture collapse), and millimeter wave radar vital signs monitor (to penetrate clothing and detect breathing / heartbeat abnormalities) can be collected separately, and then a matching fusion algorithm is used to summarize these multiple types of state information, and the summary results are used to characterize the driver's state. For example, different state information can be assigned values, and reasonable weights can be set, and then a weighted sum can be performed to obtain the state sum to characterize the driver's state. For another example, different state information can be used as different feature vector branches and input into a trained model for score prediction, and the state sum can be calculated based on the weight ratio of each feature vector branch in the model to characterize the driver's state.

[0085] Afterwards, different warning levels can be set for different driver states, and a warning level library can be established. The warning level library can be set according to the level: severe disability alarm, general alarm, pre-alarm and normal (i.e. no alarm). Specifically, for a driver with severe disability (coma or complete loss of operating ability), the warning level is set to severe disability alarm; for a driver with obvious signs of disability (such as semi-coma precursors - slow reaction, frequent micro-sleep), the warning level is set to general alarm; for a driver with mild fatigue / distraction (only warning is possible), the warning level is set to pre-alarm; for a driver with normal condition, the warning level can be set to normal.

[0086] It should be understood that the warning level library can be modified based on the driver's increased safety needs for warnings, for example, by adding warning levels or changing the driver status matched to the warning level. This allows for flexible adjustment and updating of the warning level library to enhance the user experience.

[0087] Step 108: When the risk level meets the first preset condition, and / or the warning level meets the second preset condition, trigger the execution of automatic driving takeover.

[0088] In the scheme of the present application, the first preset condition is a satisfaction condition set for the risk level, and the second preset condition is a satisfaction condition set for the early warning level. As long as one of the two conditions is met, it means that the target commercial vehicle in the current work scene has a driving safety problem, and the automatic driving takeover is triggered.

[0089] In an implementable scheme, the first preset condition is that the risk level reaches a first threshold level and the target commercial vehicle is not empty. If it is empty, it means that there is no cargo loaded at this time, and the weight of the vehicle body has little effect on the safety of the work area where the target commercial vehicle is located. Therefore, it can be considered to reduce one risk level. If it is not empty, the safety risk at this time is related to the load, so a threshold level needs to be set as the judgment standard of the risk level. The second preset condition is that the early warning level reaches a second threshold level. Therefore, when the risk level reaches the first threshold level and the target commercial vehicle is not empty, and / or the early warning level reaches the second threshold level, the automatic driving takeover is triggered.

[0090] Specifically, a hierarchical response strategy library is established: risk level + early warning level = response strategy.

[0091] Pre-alarm + low risk = first-level early warning: seat vibration + instrument panel icon flashing, gentle reminder;

[0092] Pre-alarm + general risk / high risk = second-level strong early warning: strong seat vibration + urgent buzzer + HUD red flashing warning + slight brake pulse (reminder point brake), strong warning, attention;

[0093] Pre-alarm + extreme risk = third-level emergency early warning + pre-preparation: strongest sound and light vibration alarm + vehicle pre-deceleration (such as pre-starting of the retarder, pre-hanging of the low gear of the gearbox) + sending the "driver state abnormal, high risk scene" early warning to the dispatch center, maximum warning, preparation for possible immediate takeover, and advance notification to the dispatch;

[0094] General alarm + low risk = first-level intervention: continuous strong alarm + vehicle smooth deceleration to safe parking (select safe area) + double flashing on, safe parking, and waiting for manual processing;

[0095] General alarm + general risk = second-level intervention: continuous strong alarm + vehicle actively controlled to smoothly decelerate and park along the current safe lane + double flashing on + sending the "driver disabled, parked by takeover" information to the dispatch center, to controllably park as soon as possible in a relatively safe environment;

[0096] General alarm + high risk / extreme risk = Level 3 emergency takeover: The highest alarm + immediate execution of a safety-first emergency takeover control strategy + broadcasting of precise alarm information (location, status, risk level) to the dispatch center and nearby vehicles. At all costs, escape the current extremely dangerous environment in the safest way possible to prevent a major accident.

[0097] Severe disability alarm + any scenario = highest priority emergency takeover: immediately execute the highest level of emergency takeover control strategy (even in medium and low risk scenarios, the high risk strategy is executed) + broadcast emergency alarm, the driver has completely lost his ability and must take over immediately.

[0098] It can be seen that when the risk level is general risk and there is no empty load, and the warning level is general alarm, the automatic driving takeover can be triggered to ensure the safety of operation and driving as much as possible.

[0099] Step 110: After the automatic driving takes over, determine whether the target commercial vehicle is transferred from the current operating area to the safe area within the set takeover time.

[0100] After autonomous driving takes over, the safety of the target commercial vehicle may not be guaranteed due to different response strategies. Therefore, based on the target commercial vehicle's operating scenario type, at least one safe area that meets a set distance threshold from the target commercial vehicle's operating area can be determined. The target commercial vehicle's position information can be acquired in real time, and whether the position information overlaps with the at least one safe area within a set takeover time can be used to determine whether the target commercial vehicle has moved from its current operating area to a safe area.

[0101] If yes, execute step 112 ; otherwise, execute step 114 .

[0102] Step 112: Determine that the driving is successfully taken over.

[0103] Step 114: Trigger remote driving takeover.

[0104] The primary goal of driver takeover is not to stop the vehicle, but to quickly and controllably move the vehicle to the nearest safe area (such as an avoidance zone, a straight, flat road, or away from a loading / unloading area / cliff edge), then park safely to avoid stopping at high-risk locations. Safe path planning, based on high-precision maps, real-time positioning, and environmental perception (radar), quickly plots the shortest, straightest, and least obstructed path to the nearest safe area (pre-set on the map or calculated in real time).

[0105] Reference Figure 3 As shown, this is a flow chart of the automatic driving takeover process of this application.

[0106] After the sensor data is input, the sensor data is processed by different threads: scene risk identification and driver state evaluation.

[0107] Among them, the scene risk identification can be the determination of the work area based on the sensor data, and the division of the work scene type, and then the matching of the appropriate risk level according to the determined work scene type.

[0108] The driver state evaluation can be based on sensor data, using a fusion algorithm to determine a driver state comprehensive index, and then using the driver state comprehensive index to match an appropriate warning level.

[0109] Then, based on the hierarchical response strategy, the risk level and the warning level of the target commercial vehicle are determined, and the corresponding execution strategy is allocated, and the corresponding vehicle execution and communication scheduling are matched, to determine whether it is necessary to switch the target commercial vehicle to an automatic driving takeover mode, or to a remote driving takeover mode, or to remain in the current manual driving mode. At the same time, the system state can also be recorded in real time.

[0110] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0111] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0112] Figure 4 The structure block diagram of the driving takeover device of the commercial vehicle provided by an embodiment of the present application is shown in FIG. 1. Figure 4As shown. The commercial vehicle driving takeover device 400 of this embodiment may include an acquisition module 401, a first determination module 402, a second determination module 403, a triggering module 404, a judgment module 405, and a third determination module 406. The acquisition module 401 is configured to acquire sensor data from multiple sensors installed inside and outside the target commercial vehicle. The sensor data includes vehicle driving information, driver status information, and dedicated map information. The first determination module 402 is configured to match the positioning information in the vehicle driving information with the dedicated map information to determine the target commercial vehicle's operating area in the mining area and determine the current operating scenario type based on the operating area. The second determination module 403 is configured to determine a matching risk level for the target commercial vehicle based on the operating scenario type and a matching warning level for the target commercial vehicle based on the driver status information. The triggering module 404 is configured to trigger autonomous driving takeover when the risk level meets a first preset condition and / or the warning level meets a second preset condition. The judgment module 405 is configured to determine whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time after autonomous driving takeover. The third determining module 406 is configured to determine that the current driving is successfully taken over if the judgment result is yes. The triggering module 404 is further configured to trigger the remote driving takeover if the judgment result is no.

[0113] It should be noted that part or all of the driving takeover device of the commercial vehicle of this embodiment may be an application located in the local terminal, or may also be a functional unit such as a plug-in or software development kit (SDK) set in the application located in the local terminal, or may also be a processing engine located in a network-side server, or may also be a distributed system located on the network side, for example, a processing engine or distributed system in an autonomous driving platform on the network side, etc. This embodiment does not specifically limit this.

[0114] It is understandable that the application may be a native program (nativeApp) installed on the local terminal, or may be a webpage program (webApp) of a browser on the local terminal, which is not limited in this embodiment.

[0115] Optionally, in a possible implementation of this embodiment, the dedicated map information is a high-precision electronic map of the mining area, and the sensor data also includes: driving environment information; when the first determination module 402 matches the positioning information in the vehicle driving information with the dedicated map information, determines the operating area of ​​the target commercial vehicle in the mining area, and determines the current operating scene type based on the operating area, it is specifically used to: extract the positioning information of the target commercial vehicle from the vehicle driving information; perform position matching on the positioning information with the high-precision electronic map of the mining area to determine the operating area of ​​the target commercial vehicle in the mining area; determine key operating parameters based on the driving environment information and the vehicle driving information; the key operating parameters include at least some or all of the following: terrain risk, distance to cliff, operating status, dynamic obstacles; and evaluate the current operating scene type of the target commercial vehicle based on the key operating parameters and the operating area.

[0116] Optionally, in a possible implementation of this embodiment, when the second determination module 403 determines a matching risk level for the target commercial vehicle based on the operation scene type, it is specifically used to traverse a preset risk level library using the operation scene type; the risk level library is provided with a plurality of different levels of risk levels, each risk level corresponds to one or more operation scene types, and the operation scene types corresponding to different risk levels are different; if a matching operation scene type is traversed, the risk level hit by the operation scene type is used as the risk level matching the target commercial vehicle; if no matching operation scene type is traversed, the highest level is selected from the risk level library as the risk level matching the target commercial vehicle.

[0117] Optionally, in a possible implementation of this embodiment, the driver status information includes at least: the grip force applied to the steering wheel, the driver's sitting posture, the driver's facial image and the driver's vital signs; when the second determination module 403 determines the matching warning level for the target commercial vehicle based on the driver status information, it is specifically used to extract the feature vectors of the four items, namely, the grip force, the driver's sitting posture, the driver's facial image and the driver's vital signs, respectively, and fuse them according to the set weights and input them into the prediction model; the prediction model is obtained by repeated training based on the driver's historical status information as a sample and the physical state as a label; the output is the predicted physical state of the driver and the matching warning level.

[0118] Optionally, in a possible implementation of this embodiment, the trigger module 404 triggers the execution of automatic driving takeover when the risk level meets the first preset condition and / or the warning level meets the second preset condition. Specifically, it is used to trigger the execution of automatic driving takeover when the risk level reaches a first threshold level and the target commercial vehicle is not empty, and / or the warning level reaches a second threshold level.

[0119] Optionally, in a possible implementation of this embodiment, when judging whether the target commercial vehicle has been transferred from the current operating area to the safe area within the set takeover time, the judgment module 405 is specifically used to determine, based on the operating scene type of the target commercial vehicle, at least one safe area that meets a set distance threshold from the operating area of ​​the target commercial vehicle; obtain the position information of the target commercial vehicle in real time, and judge whether the target commercial vehicle has been transferred from the current operating area to the safe area based on whether the position information overlaps with the at least one safe area within the set takeover time.

[0120] The device can be a server, cloud, or distributed server, establishing a communication connection with the target commercial vehicle, receiving sensor data uploaded by the target commercial vehicle, and executing the aforementioned autonomous driving takeover solution. Alternatively, the device can be an onboard module, integrated into the target commercial vehicle, acquiring sensor data collected by the target commercial vehicle, and executing the aforementioned autonomous driving takeover solution.

[0121] In this embodiment, sensor data can be obtained from multiple sensors installed inside and outside the target commercial vehicle. The positioning information in the vehicle driving information is matched with the dedicated map information to determine the target commercial vehicle's operating area in the mining area, and the current operating scenario type is determined based on the operating area. Then, a matching risk level is determined for the target commercial vehicle based on the operating scenario type, and a matching warning level is determined for the target commercial vehicle based on the driver status information. When the risk level meets a first preset condition and / or the warning level meets a second preset condition, autonomous driving takeover is triggered. After autonomous driving takeover, it is determined whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time. If so, the driving takeover is determined to be successful. This application can determine the operating scenario type based on the target commercial vehicle's operating area in the mining area, and match the risk level and warning level based on the operating scenario type and driver status. This deeply links complex operating scenarios and driver status, thereby triggering the execution of autonomous driving takeovers that meet different conditions through different preset conditions, making the response strategy more flexible and diverse. In addition, the set takeover time is used to measure the execution status after the driver takeover, ensuring the timeliness and safety of autonomous driving takeover in complex operating environments.

[0122] One embodiment of the present application provides a computer readable storage medium, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the driving takeover method of the commercial vehicle.

[0123] One embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the driving takeover method of the commercial vehicle.

[0124] One embodiment of the present application provides a commercial vehicle, comprising the electronic device as described above. Specifically, the autonomous vehicle can be a vehicle of L2 level and above.

[0125] It should be understood that the commercial vehicle involved in the present application is mainly an electric heavy truck driving in a special working scene such as a mine, and has the above-mentioned autonomous driving function.

[0126] In the technical solution of the present application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the present application comply with the relevant laws and regulations, and do not violate public order and good customs.

[0127] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0128] As shown in Figure 5 The electronic device 500 includes a computing unit 501 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 502 or a computer program loaded into a random access memory (RAM) 503 from a storage unit 508. Various programs and data required for the operation of the electronic device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0129] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0130] The computing unit 501 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the method for taking over driving a commercial vehicle. For example, in some embodiments, the method for taking over driving a commercial vehicle can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method for taking over driving a commercial vehicle described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the method for taking over driving a commercial vehicle by any other suitable means (e.g., via firmware).

[0131] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0132] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be retrieved from a machine-readable medium or device and executed by a processor to produce a machine for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed as a stand-alone program, or in combination with other program codes, on the machine to produce a machine that implements the functions / acts specified in the flowcharts and / or block diagrams.

[0133] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage media can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include, but are not limited to, an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0135] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0137] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0138] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for taking over driving of a commercial vehicle, characterized in that: include: Acquire sensor data from multiple sensors installed inside and outside the target commercial vehicle, the sensor data including vehicle driving information, driver status information, and dedicated map information; Matching the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determining the current operating scene type based on the operating area; Determining a matching risk level for the target commercial vehicle based on the operation scenario type, and determining a matching warning level for the target commercial vehicle based on the driver status information; When the risk level meets a first preset condition, and / or the warning level meets a second preset condition, triggering the execution of the automatic driving takeover; After the automatic driving takes over, determine whether the target commercial vehicle has moved from the current operating area to the safe area within the set takeover time; If yes, it is determined that the driving is successfully taken over; Otherwise, remote driving takeover is triggered.

2. The method according to claim 1, wherein The dedicated map information is a high-precision electronic map of the mining area, and the sensor data also includes: driving environment information; Matching the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determining the current operating scene type based on the operating area, specifically including: Extracting positioning information of the target commercial vehicle from the vehicle driving information; Matching the positioning information with the high-precision electronic map of the mining area to determine the operating area of ​​the target commercial vehicle in the mining area; Determining key operating parameters based on the driving environment information and the vehicle travel information; the key operating parameters include at least some or all of the following: terrain risk, distance to cliff, operating status, and dynamic obstacles; The current operating scenario type of the target commercial vehicle is evaluated based on key operating parameters and the operating area.

3. The method according to claim 1, wherein Determining a matching risk level for the target commercial vehicle based on the operation scenario type specifically includes: Using the operation scenario type, traversing a preset risk level library; the risk level library is provided with a plurality of different risk levels, each risk level corresponds to one or more operation scenario types, and different risk levels correspond to different operation scenario types; If a matching operation scene type is found, the risk level of the operation scene type is used as the risk level matched by the target commercial vehicle; If no matching operation scenario type is found, the highest level is selected from the risk level library as the risk level that matches the target commercial vehicle.

4. The method according to claim 3, wherein The driver status information includes at least: grip force applied to the steering wheel, driver's sitting posture, driver's facial image and driver's vital signs; Determining a matching warning level for the target commercial vehicle based on the driver status information specifically includes: Extracting feature vectors of the grip strength, the driver's sitting posture, the driver's facial image, and the driver's vital signs, respectively, fusing them according to set weights and inputting them into a prediction model; the prediction model is repeatedly trained based on the driver's historical status information as samples and the driver's physical status as labels; The output is the driver's predicted physical condition and the corresponding warning level.

5. The method according to claim 3, wherein When the risk level meets the first preset condition, and / or the warning level meets the second preset condition, triggering the execution of the automatic driving takeover specifically includes: When the risk level reaches a first threshold level and the target commercial vehicle is not empty, and / or the warning level reaches a second threshold level, automatic driving takeover is triggered.

6. The method according to claim 3, wherein Determining whether the target commercial vehicle has moved from the current operating area to the safe area within the set takeover time specifically includes: Determining, based on the operating scene type of the target commercial vehicle, at least one safe area that meets a set distance threshold from the operating area of ​​the target commercial vehicle; The position information of the target commercial vehicle is obtained in real time, and whether the target commercial vehicle has been transferred from the current operating area to the safe area is determined based on whether the position information overlaps with the at least one safe area within a set takeover time.

7. A driving takeover device for a commercial vehicle, characterized in that: include: An acquisition module, configured to acquire sensor data from multiple sensors configured inside and outside the target commercial vehicle, wherein the sensor data includes vehicle driving information, driver status information, and dedicated map information; A first determination module is configured to match the positioning information in the vehicle driving information with the dedicated map information to determine the operating area of ​​the target commercial vehicle in the mining area, and determine the current operating scene type based on the operating area; a second determination module, configured to determine a matching risk level for the target commercial vehicle based on the operation scenario type, and to determine a matching warning level for the target commercial vehicle based on the driver status information; a triggering module, configured to trigger the execution of the automatic driving takeover when the risk level satisfies a first preset condition and / or the warning level satisfies a second preset condition; A judgment module, configured to judge whether the target commercial vehicle has moved from the current operating area to a safe area within a set takeover time after the automatic driving takes over; The third determining module is used to determine that the driving is successfully taken over if the judgment result is yes; The trigger module is further configured to trigger remote driving takeover if the judgment result is no.

8. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

10. A commercial vehicle comprising the electronic device according to claim 7.

Images