Multi-modal sensing vehicle software flashing intention planning method and system

Abstract

The invention relates to a multi-modal perception vehicle software flashing intention planning method and system. The method comprises the following steps: acquiring multi-modal data of a vehicle; based on the multi-modal data, through a preset intention dictionary and driving behavior analysis, calculating an explicit intention, an explicit intention confidence coefficient, an implicit intention and an implicit intention confidence coefficient of the user; fusing all the intentions and the confidence coefficients thereof to obtain a deliberate map; configuring flashing resource allocation and a flashing sequence based on flashing priority and hardware information indicated by the deliberate diagram; determining user risk configuration and user experience configuration based on a preset risk rule base and the user preference setting; generating a plurality of candidate flashing plans based on the resource allocation, the flashing sequence, the user risk configuration and the user experience configuration; and determining a recommended flashing plan from the plurality of candidate flashing plans by using a preset user preference weight. According to the method, through intention reasoning and personalized recommendation, the writing efficiency, the success rate, the resource utilization rate and the user experience are improved.

Description

Technical Field

[0001] This invention belongs to the field of vehicle software maintenance technology, specifically relating to a multimodal perception-based vehicle software flashing intent planning method and system. Background Technology

[0002] Current vehicle software flashing decisions primarily employ methods based on fixed rules or simple triggering conditions, resulting in the following technical solutions: 1. Timed Trigger Mechanism: Software flashing is executed according to a preset schedule, such as at night or during vehicle idle periods. This method is simple and reliable, but lacks flexibility and cannot adapt to changing user needs. 2. Manual Trigger Method: Relies on the user actively initiating the flashing operation or confirming the flashing prompt. This method ensures user control but increases the user's burden and is prone to delaying important updates due to user negligence. 3. Conditional Trigger Strategy: Based on simple conditional judgments, such as network connection status and battery level, it determines whether to execute the flashing. This method is an improvement over timed triggering, but still cannot fully consider complex real-world usage scenarios.

[0003] The existing technology has the following main technical defects: 1. Lack of intent understanding: The system cannot deeply understand the user's true needs and preferences, and can only execute mechanical predefined rules. 2. Incomplete scenario awareness: The system lacks a comprehensive understanding of vehicle usage scenarios, user status, and environmental conditions, leading to a disconnect between flashing decisions and actual needs. 3. Low personalization: The system cannot provide customized flashing solutions based on individual user differences and usage habits. 4. Weak proactive service capabilities: The system can only passively respond to preset conditions and cannot proactively identify the optimal flashing time and method. 5. Insufficient human-machine collaboration: The system lacks an effective mechanism for capturing and confirming user intent, making it difficult to achieve true human-machine collaborative decision-making. Summary of the Invention

[0004] To address the problems raised in the background art, one aspect of the present invention provides a multimodal perception vehicle software flashing intent planning method, comprising: acquiring multimodal data of the vehicle, the multimodal data including user state perception data, vehicle CAN bus data, vehicle operating system hardware information, and user preference settings; based on the multimodal data, calculating the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence through a preset intent dictionary and driving behavior analysis; fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence to obtain an intent graph; configuring flashing resource allocation and flashing sequence based on the flashing priority and hardware information indicated by the intent graph; determining user risk configuration and user experience configuration based on a preset risk rule base and user preference settings; generating multiple candidate flashing plans based on the resource allocation, flashing sequence, user risk configuration, and user experience configuration; and determining a recommended flashing plan from the multiple candidate flashing plans using preset user preference weights.

[0005] In some embodiments of the present invention, the step of calculating the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence based on the multimodal data, through a preset intent dictionary and driving behavior analysis, includes: extracting text from the speech text and touch events in the multimodal data; cleaning and segmenting the extracted text; matching the segmentation results with a preset intent dictionary, and outputting explicit intent and explicit intent confidence based on the matching results; calculating a score for each driving style based on the cumulative duration for vehicle speed, steering angle, and following distance to reach their respective thresholds in the multimodal data; and fusing the driving styles using a weighted method and the scores of each driving style to obtain implicit intent and implicit intent confidence.

[0006] In some embodiments of the present invention, the process of fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence to obtain the idea graph includes: determining multiple factors of the user's idea graph, including security urgency, environmental stability and user attention, historical immediate execution rate and historical delayed execution rate; calculating scores for three types of intents—immediate, delayed, and canceled—based on the multiple factors; adjusting the score of each intent by fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence; and selecting the intent with the highest score as the idea graph based on the adjusted score for each intent.

[0007] In some embodiments of the present invention, determining the user risk configuration and user experience configuration based on a preset risk rule base and user preference settings includes: determining the user's partition flashing configuration and interrupt handling configuration in response to risk triggering conditions in the preset risk rule base; and determining the user experience configuration based on user preference settings.

[0008] In some embodiments of the present invention, determining the recommended flashing plan from multiple candidate flashing plans using preset user preference weights includes: calculating the evaluation score of each candidate flashing plan based on the preset user preference weights; selecting the N candidate flashing plans with the lowest evaluation scores as the recommended flashing plan in order of evaluation scores from low to high, where N is an integer greater than 1.

[0009] In some embodiments of the present invention, the method further includes: receiving user interaction information through a visual interface; and adjusting the recommended refresh plan based on the interaction information.

[0010] A second aspect of the present invention provides a multimodal perception vehicle software flashing intent planning system, comprising: an acquisition module for acquiring multimodal data of the vehicle, the multimodal data including user state perception data, vehicle CAN bus data, vehicle operating system hardware information, and user preference settings; a fusion module for calculating, based on the multimodal data, the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence by using a preset intent dictionary and driving behavior analysis; fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence to obtain an intent graph; a configuration module for configuring flashing resource allocation and flashing sequence based on the flashing priority and hardware information indicated by the intent graph; a determination module for determining user risk configuration and user experience configuration based on a preset risk rule base and user preference settings; and a recommendation module for generating multiple candidate flashing plans based on the resource allocation, flashing sequence, user risk configuration, and user experience configuration; and determining a recommended flashing plan from the multiple candidate flashing plans using preset user preference weights.

[0011] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multimodal perception vehicle software flashing intent planning method provided in the first aspect of the present invention.

[0012] In a fourth aspect, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the multimodal perception vehicle software writing intent planning method provided in the first aspect of the present invention.

[0013] The beneficial effects of this invention are: This invention improves user satisfaction by employing multimodal perception fusion, a hierarchical intent understanding mechanism, a context-aware intent reasoning algorithm, and a personalized planning generation engine. It reduces user interference during the refresh process by over 60%; increases user satisfaction with refresh timing to 90%; reduces user-initiated cancellations or delays by 70%; increases refresh success rate from 88% to 96%; shortens average refresh completion time by 25%; improves resource utilization efficiency by 30%; reduces user complaints due to inappropriate refresh timing by 50%; reduces customer service center inquiries by 35%; and enhances user loyalty and brand satisfaction. Attached Figure Description

[0014] Figure 1 This is a basic flowchart illustrating the multimodal perception vehicle software writing intent planning method in some embodiments of the present invention. Figure 2This is a schematic diagram illustrating the specific process of the multimodal perception vehicle software writing intent planning method in some embodiments of the present invention. Figure 3 This is a schematic diagram of the structure of a multimodal perception vehicle software writing intent planning system in some embodiments of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. Detailed Implementation

[0015] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0016] Example 1 refer to Figure 1 and Figure 2 In a first aspect, the present invention provides a multimodal perception method for planning vehicle software flashing intent, comprising: S100. acquiring multimodal data of the vehicle, the multimodal data including user state perception data, vehicle CAN bus data, vehicle operating system hardware information, and user preference settings; S200. based on the multimodal data, calculating the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence through a preset intent dictionary and driving behavior analysis; fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence to obtain an intent graph; S300. configuring flashing resource allocation and flashing sequence based on the flashing priority and hardware information indicated by the intent graph; S400. determining user risk configuration and user experience configuration based on a preset risk rule base and user preference settings; S500. generating multiple candidate flashing plans based on the resource allocation, flashing sequence, user risk configuration, and user experience configuration; and determining a recommended flashing plan from the multiple candidate flashing plans using preset user preference weights.

[0017] In step S100 of some embodiments of the present invention, multimodal data of the vehicle is acquired, including user state perception data, vehicle CAN bus data, vehicle operating system hardware information and user preference settings; Specifically, it includes the following steps: S101. User state perception data includes facial key point detection, eye movement opening and closing calculation, head posture and attention analysis, and voice feature and emotion analysis; among which, voice feature and emotion analysis is performed through endpoint detection (VAD): using the webrtcvad library, with aggressiveness=2 set, to detect voice activity.

[0018] Framing: The detected speech segment is framed in 20ms increments with a step size of 10ms.

[0019] S102. Feature Extraction: MFCC uses librosa.feature.mfcc(y=frame,sr=16000,n_mfcc=13,hop_length=160,n_fft=512) to calculate delta and delta-delta, forming a 39-dimensional feature.

[0020] Fundamental frequency: Extracted using the dio and stonemask algorithms from the pyworld library.

[0021] Volume: volume=np.sqrt(np.mean(frame 2)) and convert it to dB: volume_db=20 np.log10(volume / ref_volume).

[0022] S103. Emotion Classification: The MFCC features (39 dimensions x 200) of a speech segment (e.g., 2 seconds, 200 frames) are processed through a pre-trained 1D-CNN model. The model structure is: Conv1D(64)->MaxPool->Conv1D(128)->MaxPool->GlobalAvgPool->Dense(64)->Dense(3,activation='softmax'). The output is a probability distribution of [calm, agitated, questioning]. emotion_label=np.argmax(probabilities); S104. Urgency Calculation: Calculate the average speech rate (syllables / second) and average volume (dB) of the speech segment. Perform Min-Max normalization on the speech rate and volume to obtain norm_speed and norm_volume. urgency = 0.6 norm_speed + 0.4 norm_volume.

[0023] S105. Final output: emotion label, emotion confidence, and urgency score. CAN bus data includes vehicle speed, longitudinal acceleration, brake pressure, and steering angle. S106. Other multimodal data includes driving environment complexity and flashing compatibility assessment: S1061. Map Matching: Call the Gaode / Baidu Map API, input latitude and longitude, and get the road_type of the current road (0: highway, 1: urban road, 2: rural road).

[0024] S1062. Complexity Calculation: speed_var_coeff=can_signal_stats['speed']['std'] / (can_signal_stats['speed']['mean']+1e-5); / / Vehicle speed data acquisition; steering_var=can_signal_stats['steering_angle']['std'] / 100.0 / / Coarse normalization of steering angle; brake_ratio=can_signal_stats['brake_pressure']['count_above_20'] / 60.0 / / Assume the statistics include the threshold brake count; complexity_raw = (speed_var_coeff + steering_var + brake_ratio) / 3.0; / / Complexity calculation based on vehicle speed, steering angle, and braking. complexity_score=min(1.0,complexity_raw) / / Truncation of complexity score; S1063. Flash compatibility check: ifcomplexity_score<0.3androad_type!=1andstability>0.7: flash_suitable=True; else: flash_suitable=False / / If the complexity score is less than 0.3, the road type is not 1, and the stability is greater than 0.7, then perform the flash adaptation operation; S1064. Output: road type, environment complexity score, and flash suitability flag.

[0025] In step S200 of some embodiments of the present invention, the step of calculating the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence based on the multimodal data, through a preset intent dictionary and driving behavior analysis, includes: extracting text from the voice text and touch events in the multimodal data; cleaning and segmenting the extracted text; matching the segmentation results with the preset intent dictionary, and outputting explicit intent and explicit intent confidence based on the matching results; calculating the score for each driving style based on the cumulative duration of vehicle speed, steering angle, and following distance reaching their respective thresholds in the multimodal data; and fusing the driving styles using a weighted method and the scores of each driving style to obtain implicit intent and implicit intent confidence.

[0026] Specifically, it includes the following steps: S201. Explicit Intent Recognition Input: ASR text asr_text, HMI touch event touch_event.

[0027] The processing steps include: Intent dictionary: Maintain an intent dictionary intent_keywords: {'IMMEDIATE':{'Update immediately','Refresh immediately','OK','Confirm','Start'},'DEFER':{'Later','We'll talk about it later','Delay','Next time','Later'},'CANCEL':{'Cancel','Don't update','Don't'}}; S202. Text cleaning: Segment the asr_text and remove stop words.

[0028] S203. Keyword Matching: Iterate through the segmentation results and count the number of keywords matching each intent class (keyword_count). Confidence calculation: confidence = keyword_count[top_intent] / len(cleaned_words). If touch_event is valid, the confidence is forced to 1.0.

[0029] S204. Intent Decision: Select the intent with the highest keyword_count as the explicit_intent. If the confidence of the highest intent is less than 0.3, it is determined as UNCLEAR. Output: explicit_intent, explicit_confidence.

[0030] S205. Driving Behavior Pattern Analysis Input: Time-series data of vehicle speed, steering angle, and following distance over the past hour. Processing Logic: Sliding window calculation: Calculate using a 5-second window with a 1-second step: speed_stability_5s=np.std(window_speed) / / Number of times speed is maintained for 5 seconds; lane_keeping_5s = np.mean(np.abs(window_steering)) / / Number of times the lane is kept in 5 seconds; safe_following_5s = np.mean(window_following_distance > safe_threshold(window_speed)) / / Safe distance is dynamically calculated based on vehicle speed; Duration percentage calculation: speed_stability_ratio = sum(speed_stability_5s<5) / total_windows; lane_discipline_ratio=sum(lane_keeping_5s<10) / total_window; / / Lane keeping ratio calculation; `ssafe_following_ratio=sum(safe_following_5s>0.8) / total_windows` / / Maintain a safe following distance 80% of the time; Style score: driving_style_score=0.3 speed_stability_ratio + 0.3 lane_discipline_ratio + 0.4 safe_following_ratio; Tagging: ifdriving_style_score>=0.7:style="CALM"; Elifdriving_style_score<=0.4:style="AGGRESSIVE"; else:style="NORMAL" / / Driving style: calm, aggressive, and normal; S206. Output: Driving style label, feature vector [speed_stability_ratio, lane_discipline_ratio, safe_following_ratio].

[0031] In step S200 of some embodiments of the present invention, fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence to obtain the idea graph includes: determining multiple factors of the user idea graph, the multiple factors including security urgency, environmental stability and user attention, historical immediate execution rate and historical delayed execution rate; calculating scores for three types of intents—immediate, delayed, and canceled—based on the multiple factors; adjusting the score of each intent by fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence; and selecting the intent with the highest score as the idea graph based on each adjusted intent.

[0032] Specifically, it includes the following steps: S207. Input: explicit_intent, explicit_confidence, safety_critical, env_stability, user_attention, user history (user_history). / / Explicit intent, explicit intent confidence, safety judgment, device stability, and user attention; S208. Processing steps: Factor preparation: S=1.0ifsafety_criticalelse0.3 / / Calculation of safety urgency; Env_S=env_stability / / Environment stability calculation; U_A=user_attention / / User attention calculation; H_I = user_history.get('immediate_ratio', 0.5) / / Immediate ratio of historical data; H_D=user_history.get('defer_ratio',0.3) / / Historical deferral ratio; S209. Basic Score Calculation: Score_Imm=0.4 S+0.3 U_A+0.2 Env_S+0.1 H_I; / / Immediately execute the score; Score_Def=0.4 (1-S)+0.3 (1-U_A)+0.2 (1-Env_S)+0.1 H_D; / / Delayed execution score; Score_Cond = 1.0 - max(Score_Imm, Score_Def); This combines the scores from immediate and deferred execution. S210. Explicit Intent Fusion: if explicit_confidence > 0.8: if explicit_intent=='IMMEDIATE':Score_Imm=(Score_Imm+explicit_confidence) / 2 / / If the explicit intent is to execute immediately, then adjust the immediate execution score to the average of the immediate execution score and the intent confidence. elif explicit_intent=='DEFER':Score_Def=(Score_Def+explicit_confidence) / 2 / / If the explicit intent is not to be executed immediately, the delayed execution score is adjusted to the average of the delayed execution score and the intent confidence. S211. Final Decision: scores=[Score_Imm,Score_Def,Score_Cond]; primary_intent_index=np.argmax(scores); primary_intent=['IMMEDIATE','DEFER','CONDITIONAL'][primary_intent_index]; confidence=scores[primary_intent_index] / (sum(scores)+1e-5); Output: primary_intent, intent_scores, final_confidence.

[0033] In step S300 of some embodiments of the present invention, the flashing resource allocation and flashing sequence are configured based on the flashing priority indicated by the idea graph and hardware information. Specifically, it includes the following steps: S301. Input: primary_intent, vehicle hardware manifest hw_config. Processing steps: Resource manifest: hw_config={'cpu_cores':8,'gpu_percent':100,'ram_gb':16,'storage_gb':256,'network_5g':True}; S302. Resource Allocation Strategy: Network: If primary_intent == 'IMMEDIATE', set the network priority to "HIGH" and reserve 80% of the bandwidth. Configure Linux traffic control using the tc command.

[0034] Computation: Reserve 4 CPU cores (bound via cpuset), 50% GPU computing power (configured via GPU driver API), and 2GB of memory (managed via CGroup). Storage: Check available space and reserve flash_package_size. A temporary space of 1.5. Output: Resource allocation plan (resource_plan).

[0035] S303. Flash sequence priority configuration Input: ECU dependency graph (ecu_dependency_graph), user function usage frequency (usage_freq).

[0036] Processing steps: Topological sorting: Sort the `ecu_dependency_graph` using the Kahn algorithm to obtain the base sequence `base_sequence`. User preference adjustment: Obtain the list of ECUs (high-priority ECUs) corresponding to the three most frequently used functions by the user (`high_priority_ecus`). In the `base_sequence`, prioritize `high_priority_ecus` as much as possible without violating dependencies. Adjustments are made using bubble sort or insertion sort within the dependency constraints.

[0037] S304. Output: Optimized_sequence, the flush sequence.

[0038] In step S400 of some embodiments of the present invention, determining the user risk configuration and user experience configuration based on a preset risk rule base and user preference settings includes: determining the user's partition flashing configuration and interrupt handling configuration in response to risk triggering conditions in the preset risk rule base; and determining the user experience configuration based on user preference settings.

[0039] Specifically, it includes the following steps: S401.A / B Partition Policy: Configure the flashing tool, specifying BANK_B as the target for this flash. After successful flashing, set the next boot flag to SWITCH_TO_BANK_B.

[0040] S402. Interrupt handling rules: Define a state machine. When critical_function_latency > 500ms is detected, trigger the PAUSE state, save the progress, and roll back to BANK_A.

[0041] S403. User Experience: If user_prefs.get('minimal_intrusion') == True: then only a small progress bar is displayed on the instrument panel. Otherwise: display detailed progress (percentage, remaining time, currently flashing ECU) on the central control screen. A sleep mode disabling flag is set via the system API.

[0042] S404. Output: Risk configuration risk_config, Experience configuration ux_config.

[0043] In step S500 of some embodiments of the present invention, multiple candidate flashing plans are generated based on the resource allocation, flashing sequence, user risk configuration and user experience configuration; Specifically, it includes the following steps: Input: N generated schemes scheme_list, user preference weights user_weights={'efficiency':0.3,'safety':0.5,'experience':0.2}.

[0044] In step S500 of some embodiments of the present invention, determining the recommended flashing plan from multiple candidate flashing plans using preset user preference weights includes: calculating the evaluation score of each candidate flashing plan based on the preset user preference weights; selecting the N candidate flashing plans with the lowest evaluation scores as the recommended flashing plan in order of evaluation scores from low to high, where N is an integer greater than 1.

[0045] Specifically, normalization: for each evaluation dimension, perform Min-Max normalization across all scenarios.

[0046] norm_efficiency_i=(max_duration-scheme_i.duration) / (max_duration-min_duration); / / Calculate the efficiency score based on the scheme duration and normalize the efficiency score; norm_safety_i=scheme_i.safety_score / / Assuming the safety score is already 0-1, normalize the safety score; norm_experience_i=scheme_i.experience_score / / Assuming the experience score is already 0-1, normalize the experience score; Weighted score: total_score_i=user_weights['efficiency']norm_efficiency_i+user_weights['safety']norm_safety_i+user_weights['experience'] norm_experience_i; Output: The overall score for each scheme, scheme_scores.

[0047] In some embodiments of the present invention, the method further includes: S600. Receiving user interaction information through a visual interface; adjusting the recommended refresh plan based on the interaction information. Specifically, this includes the following steps: S601. View Adaptation and UI Configuration Input: recommended_schemes, user history view selection view_history. Processing logic: View selection: default_view = view_history.get('preferred_view', 'DETAILED'). Data mapping: Simplified view: Create a data structure containing scheme type, estimated duration, risk level (high / medium / low), and core benefits (e.g., fastest completion / minimum disruption). Detailed view: Inherit from the simplified view, adding fields such as resource_plan, optimized_sequence, and risk_config.rollback_strategy. UI rendering: Based on the selected data structure, call a UI framework (e.g., Qt / QML) component for rendering. Output: UI data model ui_data_model.

[0048] S602. Interactive Timeline Generation Input: stage durations of the plan, user calendar.

[0049] Processing Logic: Timeline Construction: Define a timeline component, `TimelineComponent`. Milestone Markers: Accumulate `stage_durations`, calculate the start and end times of each stage, mark them as Milestone nodes on the timeline, and attach explanatory text (e.g., "ECU1 flashing, steering wheel temporarily unavailable"). Schedule Association: If the plan is to be executed with a delay, retrieve the user's future free time period (e.g., "18:00-22:00") from `user_calendar` and position the timeline to that time period. Output: A `timelineComponent` instance, `timeline_component`.

[0050] S603 Interactive Confirmation S6031. Parameter Adjustment and Real-time Preview Input: User UI operation events (ui_events, such as slider dragging, checkbox clicks). Processing Logic: Event Listening: Bind event listeners to sliders, checkboxes, etc. Scheme Recalculation: Adjust Time: When the user drags the refresh time slider, recalculate network bandwidth availability based on the new time period and reassess the scheme duration and risk. Call the planning algorithm from Phase 3, passing in the new time constraints. Adjust Priority: When the user checks "Prioritize refreshing navigation," modify the usage_freq weight and re-execute; S6032. Flash Sequence Optimization Real-time feedback: The recalculated key parameters of the solution (duration, risk level) are immediately displayed on the UI.

[0051] Output: Updated scheme preview (updated_scheme_preview). Confirmation command generation and storage; Input: User's final confirmation operation (final_confirmation).

[0052] Processing Logic: Generate a unique ID: scheme_id = uuid.uuid4().hex. Data Storage: Insert a record into the confirmed_schemes table of the local SQLite database flash_schemes.db, containing scheme_id, scheme_data (JSON serialization), and scheduled_time. Command Issuance: Send scheme_id and scheduled_time to the underlying FlashController service via D-Bus or gRPC. Reminder Settings: If scheduled_time is a future time, use a system timer to trigger a user reminder at scheduled_time - 5 minutes. Output: Scheme ID (scheme_id), storage status (storage_status), command issuance status (command_status).

[0053] S6033. Multi-dimensional Metrics Input: Real-time signals published by the FlashController service via D-Bus. Processing Logic: Signal Subscription: In the monitoring service, subscribe to the following D-Bus signals: com.example.FlashController.ProgressUpdate (progress); com.example.FlashController.ResourceUsage (resource); com.example.FlashController.HealthStatus (functionality). S6034. Data Parsing: Parse the following metrics from the signal load: resource_metrics: cpu_percent, memory_mb, network_rx_tx; flash_metrics: current_ecu, progress_percent, checksum_result; functional_metrics: can_response_latency_ms, display_latency_ms. Data Buffering: Store the parsed data in a circular buffer of size 100 (corresponding to 10 seconds of data). Output: Real-time monitoring data stream monitoring_stream.

[0054] S6035. Anomaly Detection and Classification Input: monitoring_stream, thresholds library.

[0055] Processing Logic: Threshold Checks: if cpu_percent > 90: anomaly="CPU_HIGH", severity="MEDIUM"; if can_response_latency_ms > 500: anomaly="FUNC_HIGH_LATENCY", severity="HIGH"; if progress_percent remains unchanged within 10 seconds: anomaly="STALL", severity="HIGH". Logging: Exception events are written to the system log flash_anomaly.log in structured format (JSON).

[0056] Output: Anomaly_event.

[0057] S6036. Dynamic Adjustment: Handling High-Severity Anomalies Input: anomaly_eventwithseverity="HIGH".

[0058] Processing Logic: State Machine Transition: The FlashController's state changes from FLASHING to PAUSED. Send Pause Command: Call the com.example.FlashController.Pause() method via D-Bus. Trigger Rollback: Call the com.example.FlashController.Rollback() method to revert the current ECU and successfully flashed ECUs to a previous version. Resource Release: Call the ResourceManager to release the exclusive resources reserved for flashing. User Notification: Display a modal dialog on the UI to notify the user that flashing has been paused and rolled back due to an error.

[0059] Output: Control commands pause_cmd, rollback_cmd, user notification user_alert.

[0060] S6037. Management of Moderate to Severe Abnormalities: Enter: anomaly_eventwithseverity="MEDIUM".

[0061] Processing Logic: Strategy Query: Query the mitigation strategy table based on anomaly_event.type. "CPU_HIGH"->{"action":"REDUCE_CONCURRENCY","params":{"max_ecu":1}} Execute Adjustment: Call FlashController.AdjustConcurrency(1). Degrade Non-Core Tasks: Temporarily reduce the navigation map rendering frame rate and pause background music playback through system services. Recovery Monitoring: Start a timer to check CPU usage every 200ms. After three consecutive checks below 80%, perform a recovery operation.

[0062] Output: Adjustment command `adjustment_cmd`, degradation configuration `degradation_config`.

[0063] S6038. Predictive Adjustment: Input: Time-series data of memory usage (memory_usage_1min).

[0064] Processing logic: Trend prediction: Use LinearRegression to fit memory_usage_1min to predict memory usage predicted_usage 10 minutes later.

[0065] S6039. Decision-making: if predicted_usage>90: trigger_garbage_collection() / / Calls the GC interface of the flushing tool; cleanup_temp_files() / / Cleans up temporary files; ifpredicted_usage_after_clean>90: allocate_temporary_memory(1 GB) / / Request temporary memory from the system; S60310. Output: prediction result (prediction_result), cleanup command (cleanup_cmd), memory request (memory_request).

[0066] In some embodiments of the present invention, the method further includes: S700. Effect evaluation and model optimization; S701. Calculation of Writing Performance Indicators Input: flash_logs (logs to be written), user feedback (user_feedback).

[0067] Processing logic: Data aggregation: Filter data from flash_logs for the past week.

[0068] KPI Calculation: success_rate=count(status=='SUCCESS') / total_count / / Statistics on the success rate of the write operation; avg_efficiency_ratio=mean(actual_duration / planned_duration) / / The closer to 1, the better; user_satisfaction_score=mean(user_feedback.rating) / / 1-5 points; Output: A dictionary of performance metrics, kpi_dict.

[0069] S702. Evaluation of the effectiveness of intention reasoning Input: intent_inference_logs.

[0070] Processing logic: Matching judgment: Traverse the logs, if the primary_intent matches the user's final confirmed intent... Figure 1 If the result is correct, it is recorded as correct = 1; otherwise, it is recorded as 0.

[0071] Indicator Calculation: accuracy=sum(correct) / total_count; confidence_bias=mean(confidence[correct==1])-mean(confidence[correct==0]); Output: inference metrics.

[0072] S703. Model Parameter Optimization S7031. Data Preprocessing and Feature Engineering Input: The newly collected raw data set (10000 records).

[0073] Processing logic: Data cleaning: Remove obvious outliers with EAR < 0 or EAR > 0.5. Perform Z-score processing on numerical features to remove outliers with |z| > 3.

[0074] Feature engineering: Continuous features: MinMaxScaler normalization to [0,1]. Discrete features: OneHotEncoder one-hot encoding. Feature combination: Creating complexity. Cross-features such as stability.

[0075] Dataset partitioning: The training set and validation set are divided in a 7:3 ratio.

[0076] Output: The preprocessed dataset, processed_dataset.

[0077] S7032. Bayesian parameter tuning Input: processed_dataset, current_params of the current model.

[0078] Processing logic: Define the search space: param_space={ 'learning_rate':(1e-4,1e-1,'log-uniform'), 'n_layers':(1,3), 'layer_size':(64,256), 'dropout_rate':(0.1,0.5) / / Define the model's learning rate, number of layers, size of each layer, and dropout rate; Initialize the optimizer: optimizer=BayesianOptimization(f=objective_function,pbounds=param_space,random_state=1) Iterative optimization: optimizer.maximize(init_points=5, n_iter=25); To obtain the optimal parameters: best_params = optimizer.max['params']; Output: Best hyperparameters combination.

[0079] S7033. Model Validation and Replacement Determination Input: best_hyperparameters, test set test_set. Processing logic: Train new model: Train a new intent inference model new_model on the full training set using best_hyperparameters. Performance evaluation: Evaluate the accuracy acc_new and acc_old of new_model and old_model on test_set.

[0080] Replacement rule: if (acc_new - acc_old) / acc_old >= 0.05: then perform model replacement.

[0081] Stability Validation: Perform 5-fold cross-validation to ensure that the standard deviation of the accuracy of new_model at each fold is <3%. Model Deployment: If the validation passes, package the weight file of new_model, deploy it to the vehicle's infotainment system via OTA, and update the model loading path. Output: Model replacement decision result (model_replacement_decision), and cross-validation report (cv_report).

[0082] S7034. Anomaly Attribution and Strategy Adjustment: Input: anomaly_logs, user_feedback. Processing logic: Attribution analysis: Group and aggregate anomaly_logs to calculate the root cause percentage for each type of anomaly. For example, in the "CPU_HIGH" anomaly, 60% of the cause is concurrent ECUs = 2.

[0083] Policy Generation: Based on the attribution results, update the policy configuration: reduce `max_concurrent_ecus` from 2 to 1. Based on user feedback, if "navigation lag" is reported in more than 30% of users, add a new rule: `reserve_navigation_cpu_cores=2`. Output: Attribution report `root_cause_report`, policy adjustment schemes `updated_policies`.

[0084] S7035.A / B Testing and Full Deployment: Input: updated_policies, vehicle grouping information.

[0085] Processing Logic: Grouping: Randomly select 200 vehicles from the fleet, assigning 100 vehicles to Group A (new strategy) and 100 vehicles to Group B (old strategy). Test Execution: Continue for one week, collecting success_rate, user_satisfaction, and anomaly_rate for both groups. Hypothesis Testing: Use a t-test to compare the mean differences between the two groups, setting a significance level of p_value < 0.05. Promotion Decision: If the new strategy significantly improves success_rate and user_satisfaction by ≥5%, and significantly decreases anomaly_rate by ≥10%, then decide to promote it to all fleets. Full Deployment: Push the new strategy to all fleets via OTA configuration update. Output: A / B test report (ab_test_report), promotion decision (rollout_decision).

[0086] S7036. Full-process closed-loop verification: Data Link and Compliance Check: Input: Data logs output from each stage of the entire process. Processing Logic: Data Chaining: Generate a globally unique session_id for each flush session. Link the data from all stages through the session_id to form a complete data chain.

[0087] Compliance checks: Data security: Check logs for sensitive information such as plaintext CANID and GPS coordinates. Ensure all logs have been anonymized according to policy.

[0088] Process Compliance: Check for sessions with a status of "CONFIRMED" but no subsequent "EXECUTED" record, and mark them as process breakpoints. Root Cause Analysis: Diagnose process breakpoints; common causes include network connection interruption, user manual cancellation, and vehicle power off. Output: Data trace report (data_trace_report), compliance report (compliance_report), and breakpoint analysis report (breakpoint_analysis).

[0089] S7037. Industry Benchmarking and Continuous Improvement: Input: kpi_dict, industry standard industry_benchmarks (e.g., ISO24089, AUTOSAR).

[0090] Processing Logic: Gap Analysis: Compare kpi_dict with industry_benchmarks and calculate the gap. For example: the industry average flashing success rate is 98%, while ours is 96%, a difference of -2%. Improvement Plan: Goal: Increase the flashing success rate to 98.5%. Measures: Optimize the dynamic adjustment algorithm in Phase 5 to reduce failures caused by resource contention. Plan: Complete algorithm development in Q3, and complete A / B testing and full rollout in Q4. Knowledge Base Update: Update the experience, optimization strategies, and models from this closed-loop verification to the "Vehicle Software Flashing Technology Manual".

[0091] Output: Benchmarking report, improvement plan, updated manual.

[0092] Example 2 refer to Figure 3In a second aspect, the present invention provides a multimodal perception vehicle software flashing intent planning system 1, comprising: an acquisition module 11, configured to acquire multimodal data of the vehicle, the multimodal data including user state perception data, vehicle CAN bus data, vehicle operating system hardware information, and user preference settings; a fusion module 12, configured to calculate, based on the multimodal data and through a preset intent dictionary and driving behavior analysis, the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence; and to fuse the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence to obtain an intent graph; a configuration module 13, configured to configure flashing resource allocation and flashing sequence based on the flashing priority and hardware information indicated by the intent graph; a determination module 14, configured to determine user risk configuration and user experience configuration based on a preset risk rule base and user preference settings; and a recommendation module 15, configured to generate multiple candidate flashing plans based on the resource allocation, flashing sequence, user risk configuration, and user experience configuration; and to determine a recommended flashing plan from the multiple candidate flashing plans using preset user preference weights.

[0093] Furthermore, the fusion module 12 includes: a determination unit, used to determine multiple factors of the user's intention graph, the multiple factors including security urgency, environmental stability and user attention, historical immediate execution rate and historical delayed execution rate; a calculation unit, used to calculate the scores of three types of intentions—immediate, delayed and canceled—based on the multiple factors; an adjustment unit, used to adjust the score of each intention by fusing the explicit intention, explicit intention confidence, implicit intention and implicit intention confidence; and an output unit, used to take the intention with the highest score as the intention graph according to each adjusted intention.

[0094] Example 3 refer to Figure 4 A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multimodal perception vehicle software flashing intent planning method of the first aspect of the present invention.

[0095] Electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0096] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 shows an electronic device 500 with various devices, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Each box shown in Figure 9 may represent one device, or multiple devices may be represented as needed.

[0097] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, it performs the functions defined in the methods of embodiments of this disclosure. It should be noted that the computer-readable medium described in embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0098] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to: Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, C++, and Python—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal perception method for planning vehicle software writing intent, characterized in that, include: Acquire multimodal data of the vehicle, including user state perception data, vehicle CAN bus data, vehicle operating system hardware information, and user preference settings; Based on the multimodal data, the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence are calculated through a preset intent dictionary and driving behavior analysis. By fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence, a concept graph is obtained. Configure the flash resource allocation and flash sequence based on the flash priority indicated by the idea graph and hardware information; Based on a preset risk rule base and user preference settings, determine the user risk configuration and user experience configuration; Based on the resource allocation, flashing sequence, user risk configuration, and user experience configuration, multiple candidate flashing plans are generated. By using preset user preference weights, a recommended flashing plan is determined from multiple candidate flashing plans.

2. The multimodal perception vehicle software writing intent planning method according to claim 1, characterized in that, The calculation of the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence based on the multimodal data, through a preset intent dictionary and driving behavior analysis, includes: Text is extracted from the speech text and touch events in the multimodal data; the extracted text is cleaned and segmented; the segmentation results are matched with a preset intent dictionary, and the explicit intent and explicit intent confidence are output based on the matching results; Based on the cumulative duration of vehicle speed, steering angle, and following distance reaching their respective thresholds in the multimodal data, a score for each driving style is calculated; by weighting and combining the scores of each driving style, the driving styles are fused to obtain implicit intent and implicit intent confidence.

3. The multimodal perception vehicle software writing intent planning method according to claim 1, characterized in that, The process of fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence to obtain the idea graph includes: Several factors are used to determine the user's idea graph, including security urgency, environmental stability and user attention, historical immediate execution rate and historical delayed execution rate; Based on the aforementioned factors, scores are calculated for three types of intentions: immediate, delayed, and canceled. The score for each intent is adjusted by fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence. Based on each adjusted intention, the intention with the highest score is selected as the main idea.

4. The multimodal perception vehicle software writing intent planning method according to claim 1, characterized in that, The process of determining user risk configuration and user experience configuration based on a preset risk rule base and user preference settings includes: In response to risk triggering conditions in the preset risk rule base, determine the user's partition flashing configuration and interrupt handling configuration; Determine the user experience configuration based on user preference settings.

5. The multimodal perception vehicle software writing intent planning method according to claim 1, characterized in that, The step of determining the recommended flashing plan from multiple candidate flashing plans using preset user preference weights includes: The evaluation score of each candidate flashing plan is calculated by weighting based on preset user preference weights. Based on the evaluation scores from low to high, the N candidate write plans with the lowest evaluation scores are selected as the recommended write plans, where N is an integer greater than 1.

6. The multimodal perception vehicle software writing intent planning method according to claim 1, characterized in that, Also includes: The system receives user interaction information through a visual interface. Based on the interaction information, the recommended refresh plan is adjusted.

7. A multimodal perception vehicle software writing intent planning system, characterized in that, include: The acquisition module is used to acquire multimodal data of the vehicle, including user status perception data, vehicle CAN bus data, vehicle operating system hardware information, and user preference settings. The fusion module is used to calculate the user's explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence based on the multimodal data and through a preset intent dictionary and driving behavior analysis. By fusing the explicit intent, explicit intent confidence, implicit intent, and implicit intent confidence, a concept graph is obtained. The configuration module is used to configure the flash resource allocation and flash sequence based on the flash priority indicated by the idea graph and hardware information; The determination module is used to determine the user's risk configuration and user experience configuration based on the preset risk rule base and user preference settings; The recommendation module is used to generate multiple candidate flushing plans based on the resource allocation, flushing sequence, user risk configuration, and user experience configuration. By using preset user preference weights, a recommended flashing plan is determined from multiple candidate flashing plans.

8. The multimodal perception vehicle software writing intent planning system according to claim 7, characterized in that, The fusion module includes: A determining unit is used to determine multiple factors of the user's idea graph, including security urgency, environmental stability and user attention, historical immediate execution rate and historical delayed execution rate; A calculation unit is used to calculate scores for three types of intentions: immediate, delayed, and canceled, based on the aforementioned multiple factors. The adjustment unit is used to adjust the score of each intention by fusing the explicit intention, explicit intention confidence, implicit intention, and implicit intention confidence. The output unit is used to select the highest-scoring intention as the intention graph based on each adjusted intention.

9. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the multimodal perception vehicle software flashing intent planning method as described in any one of claims 1 to 6.

10. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the multimodal perception vehicle software writing intent planning method as described in any one of claims 1 to 6.

Images