Intelligent driving cooperative control terminal based on visual language action model

Abstract

The invention discloses an intelligent driving cooperative control terminal based on a visual language action model, which comprises an instruction optimization processing unit, an execution management analysis unit and a dynamic adaptive cooperative processing unit, relates to the technical field of intelligent cooperative control, and solves the problem of how to realize strategy dynamic switching based on real-time scene classification. In order to solve the technical problem of improving the self-adaptive capability of the system to complex traffic scenes, according to the invention, through a dynamic self-adaptive cooperative processing unit, coherent actions are disassembled and then conflict analysis is carried out, the grading standards of emergency actions and non-emergency actions are determined, and a pause-recovery mechanism is adopted for the non-emergency actions, so that the self-adaptive capability of the system to the complex traffic scenes is improved. The method comprises the following steps: solving conflicts for emergency actions through parameter adjustment or equivalent security substitution, improving action execution coherence and security, realizing scene fine-grained classification based on a multi-modal fusion model, triggering strategy switching through scene confidence, and starting corresponding strategies for a composite scene after confidence of all sub-scenes reaches the standard.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent collaborative control, in particular to an intelligent driving collaborative control terminal based on a visual language action model. BACKGROUND

[0002] With the development of intelligent driving technology, multi-sensor fusion and human-computer interaction become the core foundation of realizing collaborative control. In the intelligent driving system, perception, decision-making, and control are key technologies.

[0003] According to the patent application with publication number CN120348312A, an intelligent driving collaborative control method and system based on cloud-edge collaboration are disclosed. Through Attention GAN, behavior decision optimization of cloud-end and vehicle-end collaboration is realized. This framework first uses attention mechanism to make the model adaptively focus on key information in the environment, and introduces GAN to optimize the driving strategy, improving the accuracy and robustness of decision-making. On this basis, a cloud-edge heterogeneous collaborative system architecture is built, and distributed collaborative optimization is realized through federated learning, which can learn general rules from massive data using the powerful computing power of the cloud, and can make personalized decisions using the real-time processing capability of the vehicle. At the same time, strategies such as model incremental updating and selective communication uploading are designed to save communication overhead while protecting user privacy.

[0004] However, the existing intelligent driving system needs to handle multiple-link tasks such as environment perception, instruction analysis, and action execution in complex traffic scenarios, but has the following limitations: The instruction execution condition verification is single, relying on a single dimension, lacking comprehensive judgment of environmental adaptability and vehicle state, and prone to problems of instruction feasibility but scene inadaptability; The conflict processing mechanism of consecutive actions is imperfect, and the hierarchical strategy of emergency and non-emergency actions is not clear, which may lead to execution confusion due to action conflicts; The dynamic scene self-adaptation ability is weak, and the strategy switching for urban roads, highways, and bad weather scenes relies on artificial preset, lacking a dynamic triggering mechanism based on real-time scene confidence. SUMMARY

[0005] To overcome the shortcomings of the prior art, the present application provides an intelligent driving collaborative control terminal based on a visual language action model, which solves the problem of how to realize dynamic switching of strategies based on real-time scene classification and improves the self-adaptation ability of the system to complex traffic scenarios.

[0006] To achieve the above purpose, the present application realizes the following technical scheme: an intelligent driving collaborative control terminal based on a visual language action model, comprising: an instruction optimization processing unit for optimizing voice instructions, generating structured instructions, and transmitting them to an execution management analysis unit; An execution management analysis unit is configured to analyze the structured instruction, acquire a real-time environment, and analyze an instruction execution condition, generate an execution action if the execution condition is met, determine a specific reason if the execution condition is not met, and generate and transmit the execution action to a collaborative control information output unit; A dynamic adaptive collaborative processing unit is configured to receive the execution action and perform multi-target collaborative analysis and environment perception collaborative analysis, the multi-target collaborative analysis is performed in a manner that the execution action is decomposed into consecutive actions, if there is a conflict, the conflicting actions are classified and processed respectively, if there is no conflict, invalid waiting time is compressed, hardware resource allocation is optimized to improve execution efficiency, and multi-target collaborative processing information is generated; The environment perception collaborative analysis is performed in a manner that the scene is classified, and the scene classification result and a risk index are used to trigger strategy switching, when the risk index exceeds a threshold, the strategy is switched to a safety backup strategy, and environment perception collaborative processing information is generated.

[0007] As a further scheme of the present application, a sensor data acquisition unit is configured to acquire sensor data of a camera, a laser radar, a millimeter wave radar, and a microphone, and acquire a voice instruction, and transmit the sensor data and the voice instruction to an instruction optimization processing unit; A collaborative control information output unit is configured to collaboratively process and execute the multi-target collaborative processing information and the environment perception collaborative processing information.

[0008] As a further scheme of the present application, the optimization processing includes: A microphone array is used to directionally acquire the driver's voice and suppress external noise in combination with a beam forming technology, and a deep learning noise reduction model is used for noise reduction processing; a voice activity detector is used to distinguish valid instructions from noise; environment visual features, vehicle motion parameters, and positioning information are bound to each instruction to construct instruction-scene pairing data; an acoustic model and a pre-trained language model in the driving field are used to analyze the core intent of the instruction, and the structured instruction is output after disambiguating the ambiguous instruction.

[0009] As a further scheme of the present application, the analysis of the structured instruction determines the vehicle control module and the key parameters corresponding to the instruction; a safety-feasibility-scene adaptation three-dimensional verification system is constructed based on multi-source real-time environment data to determine whether the execution condition is met; if the execution condition is met, the structured instruction is converted into a parameterized execution signal; if the execution condition is not met, the reason is located through root cause analysis, and a corresponding execution action is generated.

[0010] As a further scheme of the present application, the safety-feasibility-scene adaptation three-dimensional verification system based on multi-source real-time environment data includes safety constraint verification, including verification of time to collision TTC, minimum safety distance, and compliance with traffic rules; The feasibility check includes verification of the vehicle power, braking, steering system capacity and road condition adaptability; The scene adaptation check includes verification of the matching of the instruction and the current road type and traffic flow state.

[0011] As a further scheme of the present application, the conflict action is classified and processed respectively, the continuous action with conflict is recorded as a conflict action, the conflict type of the conflict action is obtained, and the non-emergency action and the emergency action are classified, for the non-emergency action, the execution is paused and then restored after the conflict is removed, for the emergency action, the parameter is adjusted to avoid the conflict or the equivalent safe action replaces the conflict action, and the conflict cooperative processing information is generated.

[0012] As a further scheme of the present application, if there is no conflict, the time window of the continuous action is analyzed, the dependency relationship is identified by means of the time sequence alignment algorithm, the rigid constraint and the flexible window are marked, and the flexible window redundancy is compressed based on the real-time environment; at the same time, the occupation of the action on the hardware resources such as the computing power, the sensor and the actuator is analyzed, the resource demand is marked according to the demand, the on-demand allocation and the priority preemption mechanism are adopted, and the normal cooperative processing information is generated.

[0013] As a further scheme of the present application, the environment perception cooperative analysis is performed in the following manner: the scene is classified, the cooperative strategy is automatically switched for different scenes, the real-time scene is obtained, the corresponding scene classification result is determined, then the trigger condition is set based on the scene classification result, the trigger condition includes the scene confidence, when the classification confidence of the target scene is greater than or equal to the switching threshold, the strategy conversion is started, for a composite scene, all scene confidences must meet the standard before the corresponding composite strategy is triggered.

[0014] As a further scheme of the present application, the environment perception cooperative processing information is generated in the following manner: The risk indicators of the current scene classification result are monitored, the risk indicators include the following: the following safety distance, the lateral intrusion risk and the intersection conflict risk, and the risk indicators are compared with the risk threshold, the specific value of the risk threshold is set by the operator, when the risk indicators are greater than the risk threshold, the safe bottom strategy is forcibly switched, the priority is higher than that of the regular scene strategy, the real-time monitoring is performed, the risk indicators are monitored for three frames in succession, the risk indicators are returned to the safety interval, the emergency braking state is gradually restored to the regular control, after the transition is completed, the original scene strategy is automatically switched back, and the environment perception cooperative processing information is generated.

[0015] As a further scheme of the present application, the safe bottom strategy includes the following: Vehicle control: trigger ABS+ESP cooperative braking, reduce the vehicle speed to a safety value or stop stably, and simultaneously perform small-amplitude steering when there is space in the lateral direction; Global warning: turn on double flash, horn, high brake light frequency flash and vehicle external voice broadcast; System lock: freeze non-safety functions, full allocation of computing power and executor authority to risk control, and smooth recovery to the original strategy after risk removal.

[0016] The application provides an intelligent driving cooperative control terminal based on a visual language action model. The application improves the voice gain of the driving position and suppresses external noise by adopting an 8-microphone ring array combined with an MVDR adaptive beamforming technology; a high recognition rate is still maintained in extreme noise scenes by combining a wavelet transform with a Transformer hybrid noise reduction model; a safety-feasibility-scene adaptation three-dimensional verification system is established; through comprehensive analysis of multi-source environmental data, vehicle state and scene characteristics, it is ensured that the instruction execution meets safety constraints and adapts to vehicle capabilities and real-time scenes, thereby avoiding the risk of blind execution.

[0017] The application improves the continuity and safety of action execution by analyzing conflicts after decomposing consecutive actions through a dynamic adaptive cooperative processing unit, determining the classification standard of emergency actions and non-emergency actions, adopting a pause-resume mechanism for non-emergency actions, and resolving conflicts through parameter adjustment or equivalent safety substitution for emergency actions; scene fine-grained classification is realized based on a multi-modal fusion model; strategy switching is triggered through scene confidence, and the corresponding strategy is started after all sub-scene confidences of a composite scene meet the standard. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The system block diagram of the application is shown. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0020] Embodiment one Please refer to Figure 1 The application provides an intelligent driving cooperative control terminal based on a visual language action model, which includes a sensor data acquisition unit, an instruction optimization processing unit, an execution management analysis unit, a dynamic adaptive cooperative processing unit and a cooperative control information output unit, and the information between the above-mentioned functional units is unidirectional transmission as can be known from the accompanying drawings.

[0021] The sensor data acquisition unit is used to acquire sensor data corresponding to a camera, a laser radar, a millimeter wave radar and a microphone, and to obtain corresponding voice instructions, which are transmitted to the instruction optimization processing unit.

[0022] An instruction optimization processing unit is configured to optimize the acquired voice instruction, and the specific optimization processing manner is as follows: The microphone array + beam forming technology is adopted to directionally collect the voice instruction of the driver, external noise is suppressed, a noise reduction model based on deep learning is combined to reduce the voice instruction, the voice activity detection is used to distinguish the effective instruction and the noise, the visual scene feature is bound for each instruction, the instruction-scene pairing data is constructed, the obtained voice instruction is structured, and then the voice instruction is transmitted to the execution management analysis unit.

[0023] The 8-microphone ring array combined with the adaptive beam forming technology, such as the MVDR algorithm, is adopted to lock the driver direction through the sound source positioning, the spatial directivity pickup beam is formed, the voice signal gain of the driver seat is improved by more than 15dB, the external environmental noise is suppressed by more than 25dB, and for the extreme noise scene, the hybrid noise reduction model based on wavelet transform and Transformer is introduced. The environmental visual features, vehicle motion parameters and positioning information at the instruction triggering moment are captured in real time, specifically, the environmental visual features are the lane line, traffic light and obstacle detection results of the front-view camera, the vehicle motion parameters are the speed, gear position and steering light state, the above information is packaged with the voice instruction as the instruction-scene pairing data through the time sequence stamp alignment, and the instruction-scene pairing data is stored in the dynamic database. The voice instruction features are extracted by using the acoustic model, the CTC-Attention hybrid decoding is combined to realize the basic voice-to-text, the language model introduces the BERT model pre-trained in the driving field to analyze the core intention in the instruction, the instruction-scene pairing database is searched to disambiguate the ambiguous instruction, the structured instruction is finally output, and the structured instruction is transmitted to the execution management analysis unit.

[0024] The execution management analysis unit is configured to analyze the acquired structured instruction, then acquire the current real-time environment and analyze the execution condition of the structured instruction, judge whether the current real-time environment directly meets the execution condition of the structured instruction, if yes, execute according to the structured instruction as the standard, generate an execution action and transmit the execution action to the coordination control information output unit, otherwise, if not, analyze the condition not met by the current real-time environment, determine the specific reason, monitor the real-time environment, generate an execution action according to the standard that the real-time environment meets the execution condition, and then transmit the execution action to the coordination control information output unit.

[0025] The corresponding vehicle control module is determined, key parameters are extracted, multi-source real-time environment data are synchronously fused, a safety-feasibility-scene adaptation three-dimensional verification system is constructed, it is judged whether the current environment satisfies the instruction execution condition, and safety constraint verification, feasibility verification and scene adaptation verification are specifically performed, according to the condition verification result, an execution strategy is generated according to the scene: When the condition is satisfied, the structured instruction is converted into a parameterized signal that can be directly executed by the control module, the instruction ID, execution time and environment parameters are recorded, an execution action is generated and transmitted to the cooperative control information output unit, when the condition is not satisfied, the core factors that do not satisfy the condition are located through a root cause analysis model, which are divided into three categories, including environment not meeting the standard, vehicle state limitation and safety risk not being removed, and real-time monitoring is performed; For the environment not meeting the standard scene, real-time monitoring is started, a trigger threshold is preset, and a transition action is generated, such as keeping the current lane and reducing the speed to 30km / h to wait; For the vehicle state limitation scene, constraint information is fed back to the human-computer interaction unit, and an alternative solution is provided; For the safety risk not being removed scene, an active risk avoidance priority mechanism is triggered, such as maintaining a vehicle distance and continuously monitoring the dynamic of pedestrians, and the original instruction is automatically executed after the risk is removed; The cooperative control information output unit is used for executing the generated execution action.

[0026] Embodiment two As embodiment two of the application, it is implemented on the basis of embodiment one, and the difference from embodiment one is as follows: The execution management analysis unit transmits the generated execution action to the dynamic adaptive cooperative processing unit and processes it, specifically, the dynamic adaptive processing is performed according to the obtained execution action, including multi-target cooperative analysis and environment perception cooperative analysis; The specific way of multi-target cooperative analysis is to obtain the execution action and disassemble it into consecutive actions, and here the disassembly is performed according to the execution order, and the obtained consecutive actions are analyzed for conflicts, if there is a conflict between the consecutive actions, a conflict analysis signal is generated, otherwise if there is no conflict between the consecutive actions, a normal analysis signal is generated; The generated conflict analysis signal is processed, the consecutive actions with conflicts are obtained and marked as conflict actions, the conflict types of the conflict actions are obtained, and are classified into non-emergency actions and emergency actions, for the non-emergency actions, the execution is suspended and is resumed after the conflict is removed, for the emergency actions, the parameters are adjusted to avoid the conflict or an equivalent safety action is used to replace the conflict action, and conflict cooperative processing information is generated; Based on the urgency, directly associate the safety core module, and if the conflict is not handled, it will cause a safety risk within 3 seconds, then it is classified as an urgent action, the action impact is limited to comfort / efficiency class functions, or the conflict resolution window is ≥5 seconds, then it is classified as a non-urgent action; For non-urgent actions, send a pause flag to the execution module, which includes action ID, current progress, pause timestamp, freeze target parameters, synchronize start low-priority transition actions, real-time monitor conflict source state, when conflict resolution conditions are met, for urgent actions, dynamically adjust longitudinal parameters, lateral parameters or timing parameters, specific longitudinal parameters such as adjusting speed / acceleration, lateral parameters such as optimizing trajectory / angle, timing parameters such as delaying action start time, if parameter adjustment cannot completely avoid conflict, trigger a substitute action that is functionally equivalent and has a higher safety level, prioritize core intent, ensure that the substitute action is functionally equivalent to the original action, and generate conflict coordination processing information; For the generated normal analysis signal, improve the execution efficiency of the action sequence through parameter optimization, specifically, analyze the time window of consecutive action execution, and compress invalid waiting time, specifically, identify the dependency relationship between consecutive actions through timing alignment algorithm, and mark the rigid time constraints and flexible time windows of each consecutive action. For flexible time windows, dynamically compress redundant waiting based on real-time environmental data; At the same time, analyze the occupation of vehicle hardware resources by consecutive actions, such as computing power, sensors, actuators, to avoid resource idling or excessive allocation, specifically, mark resource requirements for different consecutive actions, including computing power requirements, sensor requirements, and actuator requirements, and use on-demand allocation and priority preemption to handle them, for urgent actions, temporarily preoccupy computing power, for non-urgent actions, allocate low-priority computing power, and generate normal coordination processing information; Integrate conflict analysis processing information and normal analysis processing information to generate multi-target coordination processing information, and transmit it to the collaborative control information output unit.

[0027] The specific way of environmental perception collaborative analysis is to classify the scene, such as urban road, highway, bad weather, congested road section, automatically switch the collaborative strategy for different scenes, obtain real-time scene, and determine its corresponding scene classification result, then set the trigger condition based on the scene classification result, and the trigger condition includes scene confidence and risk situation, when the classification confidence of the target scene ≥ switching threshold, start strategy conversion, for example: the confidence of urban intersection increases from 70% to 90%, the switching threshold is 85%, when it exceeds 85% threshold, trigger the conversion from straight driving strategy on main road to turning strategy at intersection, for composite scenes, all sub-scene confidence must meet the standard before triggering the corresponding composite strategy; Specifically, in highway (smooth / clear weather) scenarios, regarding safety constraints, the following distance must be maintained at more than 1.5 seconds, and the minimum safe distance must be no less than 50 meters; regarding efficiency targets, the cruising speed should be set to the speed limit of the road segment, and the overtaking response time must be controlled within 1 second; regarding actuator control details, the acceleration amplitude should not exceed 1.2 m / s. 2 Steering angular velocity not exceeding 5° / s, braking delay not exceeding 200ms; In urban intersections (signal-controlled / congested) scenarios, safety constraints require pedestrians to have a collision time of at least 3 seconds (TTC) and to slow down 5 meters before the stop line. The efficiency target is a green light passage efficiency of at least 80%, while emphasizing avoiding overtaking and balancing traffic efficiency with rule compliance. For actuator control, the starting acceleration should not exceed 0.8 m / s². 2 The turning radius shall not be less than 15 meters. For severe weather scenarios (heavy rain / high-speed driving), safety constraints are more stringent. The following distance must be no less than 3 seconds, and the maximum speed is limited to 60 km / h. The efficiency target is that the traffic efficiency should not be less than 50% of the speed limit of the road section, and traffic capacity should be guaranteed as much as possible under the premise of safety priority. The actuator control parameters are that the brake pressure change rate should not exceed 0.3 bar / s to prevent slippage, and the steering angle is limited to within 10°.

[0028] In the construction area (urban road) scenario, the safety constraints require that the distance from the construction area be no less than 3 meters and the vehicle speed not exceed 30km / h; the efficiency target is that the detour time should not exceed 1.2 times that of the original route, reducing the loss of travel time while safely avoiding the construction area; in terms of actuator control, the interval between continuous lane changes should be no less than 3 seconds, and the turn signal should be turned on more than 5 seconds in advance.

[0029] Simultaneously, the risk indicators of the current scenario classification results are monitored. These indicators include following safety distance, lateral intrusion risk, and intersection conflict risk. These are compared with risk thresholds, the specific values ​​of which are set by the operator. When a risk indicator exceeds the threshold, a forced switch to a safety fallback strategy is initiated, prioritizing it over the regular scenario strategy. Specific safety fallback strategies include: vehicle control, triggering ABS+ESP coordinated braking to reduce the vehicle to a safe speed or bring it to a complete stop within the shortest distance; if there is lateral avoidance space, simultaneously executing a small emergency turn, but prioritizing longitudinal braking; full-area warning, activating high-mounted brake light flashing and external voice announcements in addition to hazard warnings and hazard hazard warnings; system lock, temporarily freezing non-safe functions and allocating all computing power and actuator permissions to hazard avoidance control until the risk is eliminated. Real-time monitoring is also performed, with three consecutive frames monitoring the risk indicators to return to the safe range. The system gradually recovers from emergency braking to regular control. After the transition is complete, it automatically switches back to the original scenario strategy, generates environmental perception and collaborative processing information, and transmits it to the collaborative control information output unit.

[0030] The collaborative control information output unit is used to perform corresponding collaborative processing on the acquired multi-target collaborative processing information and environmental perception collaborative processing information.

[0031] Example 3 As a third embodiment of the present invention, the focus is on combining the implementation processes of the first and second embodiments.

[0032] The data in the above formulas are all calculated using numerical values, without substituting the units of the parameters. In addition, the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0033] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An intelligent driving cooperative control terminal based on a visual language action model, characterized in that, include: The instruction optimization processing unit is used to optimize voice instructions, generate structured instructions, and transmit them to the execution management and analysis unit. The execution management and analysis unit is used to analyze structured instructions, obtain the real-time environment and analyze the execution conditions of the instructions. If the conditions are met, an execution action is generated; if not, the specific reasons are determined, an execution action is generated and transmitted to the collaborative control information output unit. The dynamic adaptive collaborative processing unit is used to receive execution actions and perform multi-objective collaborative analysis and environmental perception collaborative analysis. The multi-objective collaborative analysis method is to decompose the execution actions into coherent actions. If there are conflicts, the conflicting actions are classified and processed separately. If there are no conflicts, the execution efficiency is improved by compressing invalid waiting time and optimizing hardware resource allocation, and multi-objective collaborative processing information is generated. The environmental perception collaborative analysis method involves classifying scenarios, triggering strategy switching based on scenario classification results and risk indicators, and forcibly switching to a safety fallback strategy when risk indicators exceed the limit, thereby generating environmental perception collaborative processing information.

2. The intelligent driving cooperative control terminal based on a visual language action model according to claim 1, characterized in that, It also includes a sensor data acquisition unit, which is used to acquire sensor data from cameras, lidar, millimeter-wave radar and microphones, and obtain voice commands, and transmit the sensor data and voice commands to the command optimization and processing unit. The collaborative control information output unit is used to collaboratively process and execute multi-target collaborative processing information and environmental perception collaborative processing information.

3. The intelligent driving cooperative control terminal based on a visual language action model according to claim 1, characterized in that, The optimization process includes: The system employs a microphone array combined with beamforming technology to directionally acquire driver speech and suppress external noise, and then uses a deep learning noise reduction model for noise reduction processing. It distinguishes between valid commands and noise through voice activity detection, binds environmental visual features, vehicle motion parameters and positioning information to each command, and constructs command-scene pairing data. Based on acoustic models and pre-trained language models in the driving domain, it parses the core intent of commands, disambiguates ambiguous commands, and outputs structured commands.

4. The intelligent driving cooperative control terminal based on a visual language action model according to claim 1, characterized in that, The analysis of structured instructions clarifies the corresponding vehicle control module and key parameters; a three-dimensional verification system of safety, feasibility, and scenario adaptation is constructed based on multi-source real-time environmental data to determine whether the execution conditions are met; if met, the structured instructions are converted into parameterized execution signals; if not met, root cause analysis is used to locate the factors that do not meet the conditions and generate corresponding execution actions.

5. The intelligent driving cooperative control terminal based on a visual language action model according to claim 4, characterized in that, A three-dimensional verification system based on multi-source real-time environmental data is constructed, which includes safety constraint verification, such as verification of collision time (TTC), minimum safe distance, and traffic rule compliance. Feasibility verification includes verifying the vehicle's power, braking, and steering system capabilities as well as its adaptability to road conditions. Scene adaptation verification includes verifying the matching of instructions with the current road type and traffic flow status.

6. The intelligent driving cooperative control terminal based on a visual language action model according to claim 1, characterized in that, The process involves classifying and processing conflicting actions, identifying consecutive conflicting actions as conflicting actions, and identifying the conflict type of each action. These actions are then classified into non-urgent and urgent actions. For non-urgent actions, execution is paused until the conflict is resolved. For urgent actions, parameters are adjusted to avoid conflict or replace the conflicting action with an equivalent safe action, and conflict coordination information is generated.

7. The intelligent driving cooperative control terminal based on a visual language action model according to claim 1, characterized in that, If there is no conflict, the time window of consecutive actions is analyzed, and the dependency relationship is identified by using the time alignment algorithm. Rigid constraints and flexible windows are marked, and the redundancy waiting of flexible windows is compressed based on the real-time environment. At the same time, the occupation of hardware resources such as computing power, sensors, and actuators by actions is analyzed, resource requirements are marked according to demand, and on-demand allocation and priority preemption mechanisms are adopted to generate normal collaborative processing information.

8. The intelligent driving cooperative control terminal based on a visual language action model according to claim 1, characterized in that, The environmental perception collaborative analysis method is as follows: the scene is classified, the collaborative strategy is automatically switched for different scenes, the real-time scene is obtained, and the corresponding scene classification result is determined. Then, the triggering condition is set based on the scene classification result, and the triggering condition includes the scene confidence. When the classification confidence of the target scene is greater than or equal to the switching threshold, the strategy conversion is initiated. For composite scenes, the corresponding composite strategy is triggered only when the confidence of all scenes meets the standard.

9. The intelligent driving cooperative control terminal based on a visual language action model according to claim 1, characterized in that, The generated environment-aware collaborative processing information includes: The system monitors risk indicators based on the current scenario classification results. These indicators include following safety distance, lateral intrusion risk, and intersection conflict risk. The system compares these indicators with risk thresholds, the specific values ​​of which are set by the operator. When a risk indicator exceeds the risk threshold, the system forcibly switches to a safety fallback strategy, which has higher priority than the regular scenario strategy. Simultaneously, the system monitors the risk indicator in real time. After three consecutive frames, the system returns the risk indicator to the safe range and gradually recovers from emergency braking to regular control. Once the transition is complete, the system automatically switches back to the original scenario strategy and generates environmental perception collaborative processing information.

10. The intelligent driving cooperative control terminal based on a visual language action model according to claim 9, characterized in that, The safety fallback strategy includes: Vehicle control: Trigger ABS+ESP coordinated braking to reduce the vehicle speed to a safe value or bring it to a stop, and simultaneously make a small lateral turn when there is space. Full-area warning: Activate hazard lights, honk the horn, flash the high-mounted brake light, and provide external voice announcements; System Lockdown: Freeze non-safe functions, allocate all computing power and executor permissions to risk avoidance control, and smoothly restore to the original strategy after the risk is eliminated.

Images