Driving scene prediction method and system based on vehicle-cloud combination, product and vehicle

Abstract

The invention provides a driving scene prediction method and system based on vehicle cloud combination, a product and a vehicle, and relates to the technical field of Internet of Vehicles. The method comprises the following steps: cooperatively receiving first scene information pushed by a cloud end and locally generated second scene information at a vehicle machine end, dynamically selecting information characteristics according to a vehicle networking state and a detection result of an emergency event, and executing preset operations corresponding to a current scene, a sub-scene, a vehicle using state and a service event according to the information characteristics; according to the invention, the advantages of cloud global perception and local real-time response of the vehicle machine are effectively fused, and the reliability and safety of driving guidance are enhanced.

Description

Technical Field

[0001] This application relates to the field of vehicle networking technology, and in particular to driving scenario prediction methods, systems, products and vehicles based on vehicle-cloud integration. Background Technology

[0002] Against the backdrop of the rapid development of intelligent connected vehicles, driving scenario prediction and personalized driving guidance have become important directions for improving user experience. In existing technologies, some systems rely on the cloud for scene recognition and decision-making. While these systems possess advantages such as abundant data and powerful models, they suffer from problems such as network latency, network outages, and delayed responses, making it difficult to meet real-time and security requirements. On the other hand, purely localized solutions, while offering rapid responses, are limited by onboard computing power and data dimensionality, making it difficult to achieve accurate recognition of complex scenarios. Summary of the Invention

[0003] This application provides a driving scenario prediction method, system, product, and vehicle based on vehicle-cloud integration to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.

[0004] On the one hand, this application provides a driving scenario prediction method based on vehicle-cloud integration, including the following steps: The vehicle-mounted system receives first scene information pushed from the cloud, which includes the current scene and sub-scenes, vehicle usage status, triggered business events and subsequent actions. The vehicle-mounted system generates second scene information locally, which is a scene and business event that can be determined by relying only on local data or cached data. Based on the vehicle's network connectivity status and the detection results of emergency events, the system dynamically selects either the first scenario information or the second scenario information as the information feature. Based on the information characteristics, execute preset operations corresponding to the current scenario, sub-scenario, vehicle usage status, and business events.

[0005] Furthermore, the step of dynamically selecting either the first scenario information or the second scenario information as information features based on the vehicle's network connectivity status and the detection results of emergency events specifically includes: If the network is currently connected, the information from the first scenario is used as the information feature. If the current state is offline or an emergency event is detected, the second scenario information is used as the information feature.

[0006] Furthermore, the emergency event includes at least one of the following: the vehicle speed exceeds the safe range, there is a risk of collision ahead, a pedestrian suddenly enters the drivable area, and a dynamic obstacle appears on the pedestrian crossing after the traffic light changes.

[0007] Furthermore, when an emergency event is detected in which a pedestrian suddenly enters a drivable area, the vehicle-mounted terminal predicts the pedestrian's trajectory using a self-supervised learning model and immediately generates a braking control command based on the prediction result to perform an emergency braking operation. This operation does not depend on cloud response or the first scenario information.

[0008] Furthermore, the self-supervised learning model acquires a set of overlapping pedestrian images through dual vehicle cameras, obtains training samples by in-camera sampling or cross-camera sampling, extracts pedestrian features and constructs a similarity matrix, generates a cyclic matching matrix through forward and backward matching, and trains the model with the identity matrix as the ideal target, thereby achieving pedestrian feature learning and trajectory prediction without labeled data.

[0009] Furthermore, when the vehicle is connected to the network, the first scenario information is acquired and updated to cover the second scenario information; when the vehicle is not connected to the network, the second scenario information remains effective until it is covered by the updated first scenario information after the vehicle is connected to the network.

[0010] Furthermore, the step of executing preset operations corresponding to the current scenario, sub-scenario, vehicle usage status, and business event based on the information characteristics includes the following steps: Obtain the scene, sub-scene, vehicle usage status, and triggered business events selected through the information features; Based on the type of the business event, its security priority is determined; whereby security priorities are divided into emergency security events and routine service events. If the business event is an emergency safety event, the vehicle-mounted local execution module is invoked to directly trigger preset safety operations; the preset safety operations include: performing emergency braking, lane departure correction, or collision warning based on local sensor data; If the business event is a routine service event, then a corresponding personalized service strategy will be matched according to the current scenario, sub-scenario, and vehicle usage status; the personalized service strategy includes: playing preset media content, recommending music resources, planning navigation routes, or pushing life information reminders; The personalized service strategy is executed, and the operation results are fed back through the human-machine interface on the vehicle terminal. If a new business event is detected during the operation, repeat the above steps to override the current operation process with the priority of the new business event.

[0011] On the other hand, this application provides a driving scenario prediction system based on vehicle-cloud integration, including: The cloud push module is used to receive the first scenario information pushed by the cloud on the vehicle terminal. The first scenario information includes the current scenario and sub-scenarios, vehicle usage status, triggered business events and subsequent execution actions. The vehicle-mounted local generation module is used to generate second scene information locally on the vehicle-mounted terminal. The second scene information is a scene and business event that can be determined by relying only on local data or cached data. The decision module is used to dynamically select either the first scenario information or the second scenario information as information features based on the vehicle's network connectivity status and the detection results of emergency events. The operation execution module is used to execute preset operations corresponding to the current scenario, sub-scenario, vehicle usage status, and business events based on the information characteristics.

[0012] On the other hand, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned vehicle-cloud-based driving scenario prediction method.

[0013] On the other hand, this application provides a vehicle that integrates the aforementioned vehicle-cloud-based driving scenario prediction system.

[0014] This application includes at least the following beneficial effects: It provides a driving scenario prediction method based on vehicle-cloud integration. This method collaboratively receives first scenario information pushed from the cloud and second scenario information generated locally on the vehicle's infotainment system. It dynamically selects information features based on the vehicle's network connectivity status and emergency event detection results, and executes preset operations corresponding to the current scenario, sub-scenario, vehicle usage status, and business events based on these features. This application effectively integrates the advantages of cloud-based global perception and vehicle-based real-time response, enhancing the reliability and safety of driving guidance. This application also provides corresponding systems, products, and vehicles. The beneficial effects of the systems, products, and vehicles are similar to those of the method and will not be elaborated upon here.

[0015] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0016] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0017] Figure 1 This is a flowchart of the driving scenario prediction method based on vehicle-cloud integration provided in this application; Figure 2 This is a structural diagram of the vehicle-cloud-based driving scenario prediction system provided in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] With the rapid development of intelligent connected vehicle technology, users are increasingly demanding higher levels of intelligence, personalization, and safety from in-vehicle systems. Traditional in-vehicle information systems primarily focus on basic functions such as navigation, entertainment, and remote control. Their decision-making logic is mostly static or based on simple rule triggers, lacking a comprehensive understanding and dynamic response capability regarding the driving environment, user status, and life scenarios. In recent years, the industry has begun exploring the introduction of technologies such as cloud computing, artificial intelligence, and big data analytics into vehicle systems to achieve higher levels of scenario perception and service recommendation. Among these, driving scenario prediction has become a key direction for improving user experience.

[0023] Existing technologies still have many shortcomings in predicting driving scenarios: On the one hand, most systems adopt a single cloud-dominated or local decision-making mode, lacking a flexible mechanism to dynamically switch decision sources based on network status and the urgency of events, resulting in service interruptions or cloud response delays affecting real-time performance when the network is down. On the other hand, in emergency situations, the vehicle-mounted system is not given "absolute control," making it difficult to achieve millisecond-level safety responses and posing safety risks. At the same time, the vehicle-mounted system usually retains a large amount of judgment logic that relies on network data, without optimizing and simplifying locally judgeable scenarios, resulting in resource redundancy. In addition, scenario modeling is mostly limited to a single dimension of environment or user, failing to integrate multi-source contexts such as time, location, emotion, and life information to achieve deep personalized services, and personalized functions are disconnected from safety controls. Existing systems also generally lack local continuous learning capabilities based on advanced technologies such as self-supervised learning, making it difficult to cope with complex and sudden scenarios. Finally, the state synchronization mechanism between the cloud and the device is imperfect, which can easily lead to decision conflicts or chaotic information coverage, affecting system stability and user experience.

[0024] To address the aforementioned issues, this application proposes a driving scenario prediction method, system, product, and vehicle based on vehicle-cloud collaboration. Its core lies in constructing a fusion architecture of "precise cloud-based recognition + rapid vehicle-mounted response," and achieving intelligent dynamic decision-making through a "scenario management and decision-making module" deployed on the vehicle-mounted terminal. When connected to the network, the system prioritizes high-precision scenario information (including scenarios, sub-scenarios, vehicle status, and business events) pushed from the cloud to improve judgment accuracy and service intelligence. When the network is down or an emergency event is detected (such as a pedestrian suddenly entering, high-speed driving, or collision risk), it automatically switches to scenario information generated locally on the vehicle-mounted terminal as the information feature, ensuring continuous system availability and millisecond-level response, and explicitly granting the vehicle-mounted terminal absolute control over safety-related events. Simultaneously, the vehicle-mounted terminal only retains scenarios and business events that can be judged using local sensor data or cached information (such as birthday reminders and rain mode), optimizing resource utilization and improving operational efficiency. Newly received recognition results can overwrite the current state, achieving seamless state synchronization and a continuous experience between the cloud and the terminal. In particular, in emergency scenarios such as pedestrian intrusion, the vehicle-mounted system uses a self-supervised learning model to predict pedestrian trajectories in an unsupervised manner, supporting autonomous emergency braking without cloud intervention, which significantly enhances the system's safety, robustness, and intelligence.

[0025] First, the driving scenario prediction method based on vehicle-cloud collaboration provided in this application will be described in detail below with reference to the accompanying drawings.

[0026] Reference Figure 1 The implementation process of the vehicle-cloud collaborative driving scenario prediction method provided in this application embodiment includes, but is not limited to, the following steps.

[0027] Step S110: Receive the first scene information pushed from the cloud on the vehicle's infotainment system.

[0028] The first scenario information includes the current scenario and sub-scenarios, vehicle usage status, triggered business events, and subsequent execution actions.

[0029] Step S110 is the key entry point for building a vehicle-cloud collaborative intelligent driving service system. The cloud, as a data aggregation and intelligent hub, can integrate multi-dimensional big data such as high-precision maps, real-time traffic flow, weather information, urban events, user profiles, and historical behavior. Combined with complex AI models for deep reasoning, it can accurately identify the vehicle's macro-driving scenario (such as commuting, long-distance travel, picking up children, and nighttime driving) and its sub-scenarios (such as congested roads during morning rush hour, highway driving in rain, and slow traffic in school zones), and determine the current vehicle status (such as first start, long-distance fatigue, and low battery). Simultaneously, the cloud can trigger corresponding business events (such as birthday reminders, upcoming meetings, and nearby restaurant recommendations) based on user calendars, geographical location, and social relationships, and pre-set subsequent actions (such as playing specific music, adjusting seat position, and pushing navigation suggestions). By pushing this high-value "first-scenario information" to the vehicle's infotainment system, the system can fully utilize the cloud's global perspective and powerful computing power when network conditions are good, significantly improving the accuracy, comprehensiveness, and intelligence of scenario understanding, providing users with more precise and personalized driving guidance services.

[0030] Step S120: Generate the second scene information locally on the vehicle's infotainment system.

[0031] The second scenario information refers to scenarios and business events that can be determined solely by local or cached data.

[0032] In step S120, the system maintains basic functional integrity and real-time response even in environments with no network connection or limited communication. The vehicle-mounted system independently identifies and judges certain key scenarios and events using built-in sensors (such as cameras, radar, microphones, GPS, IMU, and the DMS driver monitoring system) and data downloaded from the cloud and cached locally in advance (such as user birthdays, work calendars, frequently used routes, preset media content, and regional speed limit information). This "second scenario information" focuses on two typical scenarios: first, real-time environmental perception related to safety, such as detecting a pedestrian suddenly appearing ahead, a vehicle braking suddenly, lane departure, and automatically recognizing rainfall in rainy weather; second, personalized triggers related to services, such as determining whether it is the user's birthday, whether it is a workday, or whether they have entered a preset "home mode" area. By retaining this judgment logic on the vehicle-mounted system and clearly defining the principle of "only retaining judgments based on local data," the system achieves lightweight and efficient vehicle-mounted system functionality, avoiding lag or failures caused by attempting to call remote services while offline. More importantly, the design ensures that even in areas without signal, such as tunnels, mountains, and underground parking garages, the vehicle can still maintain basic intelligent services and safety protection capabilities, improving the system's robustness and the continuity of the user experience.

[0033] Step S130: Based on the vehicle's network connectivity status and the detection results of the emergency event, dynamically select either the first scenario information or the second scenario information as the information feature.

[0034] Specifically, if the device is currently connected to the internet, the first scenario information is used as the information feature. If the device is currently offline or an emergency event is detected, the second scenario information is used as the information feature.

[0035] In step S130, an intelligent and adaptive decision-making arbitration mechanism is constructed. The system dynamically assesses which information source should be used as the information feature by monitoring the vehicle's network connection status (such as 4G / 5G / V2X signal strength) and the output of the internal safety monitoring module in real time. Specifically, when the vehicle is in a stable network state and no emergency safety event is detected, the system prioritizes the "first scenario information" pushed from the cloud to obtain the most comprehensive and accurate scenario judgment. However, once a network interruption, weak signal, or excessive latency occurs, or the system detects any emergency event (such as vehicle speed exceeding a safety threshold, collision warning, pedestrian entering a drivable area, etc.), it immediately switches to the "second scenario information" generated locally on the vehicle's terminal as the information feature. This dual-track mechanism of "network status driven + safety priority" ensures that the system uses high-precision cloud judgment in most cases, while unconditionally relying on rapid local response at critical moments (network outage or emergency), achieving an optimal balance between accuracy and real-time performance. This step not only improves the overall reliability of the system but also provides clear and authoritative "information feature" input for subsequent execution modules, which is a key guarantee for achieving continuity and safety in intelligent driving guidance.

[0036] Step S140: Based on the information characteristics, execute the preset operations corresponding to the current scenario, sub-scenario, vehicle usage status, and business events.

[0037] In step S140, the system is responsible for transforming the "information features" output from the previous steps into specific vehicle behaviors or human-machine interaction actions, truly achieving closed-loop control from perception to decision-making to execution. Based on the selected information features, the system calls upon a preset operation strategy library to execute two types of operations: first, safety-related operations, which immediately trigger local safety responses when an emergency event is identified, such as Automatic Emergency Braking (AEB), Lane Keeping Assist (LKA), collision warning, and automatic activation of hazard warning lights. These operations are entirely completed independently by the vehicle-mounted control module, without relying on cloud commands, ensuring millisecond-level response. Second, service-related operations, which provide intelligent services to enhance user experience for routine or personalized business events, such as playing a pre-set birthday greeting video and recommending the song "Happy Birthday" when the vehicle is started for the first time on the user's birthday; automatically loading frequently used navigation routes when entering a commuting scenario; and playing refreshing music or suggesting a rest when fatigue driving is detected. During execution, the system also provides feedback on the operation results to the user through the human-machine interface (HMI), ensuring transparency and controllability. Furthermore, this step supports dynamic priority management: if a higher-priority emergency event is detected while performing a service operation, the current process is immediately interrupted, and safety-related operations are executed first, reflecting the "safety first" design principle. Through this step, the system achieves intelligent response to complex driving environments, significantly improving driving safety and the personalization of the driving experience.

[0038] In some embodiments of this application, the emergency event includes at least one of the following: the vehicle speed exceeds the safe range, there is a risk of collision ahead, a pedestrian suddenly enters the drivable area, and a dynamic obstacle appears on the pedestrian crossing after the traffic light changes.

[0039] Specifically, when the system detects that the vehicle's speed exceeds a preset safety threshold (such as the speed limit on urban roads +20km / h, speeding on highways, or passing through special sections such as curves or school zones at high speed), it is considered an emergency. The significance of this design is that high-speed driving significantly shortens the reaction time window for both the driver and the system. Any external disturbance (such as sudden obstacles, slippery road surfaces, or sudden braking by the vehicle in front) can escalate into a serious accident in a very short time. By defining "speeding" itself as an emergency state, the system can proactively intervene before a collision occurs, activating advanced driver assistance functions in advance, such as automatic speed limiting, enhanced lane keeping assist, and dynamic braking preparation. It can even issue strong warnings through the human-machine interface (such as audible alarms and seat vibrations) to prompt the driver to slow down. More importantly, in this state, the system defaults to local decision-making on the vehicle's system, avoiding control delays caused by waiting for cloud responses. This enhances active safety capabilities in the critical few seconds before an accident, achieving "preventive safety intervention" rather than being limited to post-accident response.

[0040] Secondly, a forward collision risk refers to a situation where sensors such as onboard radar and cameras detect a continuously decreasing distance to the vehicle in front or a static obstacle, and the relative speed exceeds a safety threshold, leading the system to determine a potential rear-end or frontal collision. This scenario is the most typical high-priority event in Advanced Driver Assistance Systems (ADAS). This application explicitly lists it as an emergency event, the core significance of which is to ensure the independence and authority of the local emergency braking system (AEB). Once the local perception module identifies a collision risk, regardless of whether the vehicle is connected to the internet or whether the cloud has received the data, the vehicle's infotainment system must immediately initiate the braking control process, without waiting for cloud confirmation or instructions. This design fundamentally solves the safety hazards caused by "cloud control latency," ensuring that the vehicle can still autonomously complete avoidance maneuvers even in extreme situations such as communication interruptions, signal delays, or excessive cloud server load. In addition, this mechanism also supports multi-sensor fusion judgment, improving false alarm rate control and ensuring that in complex traffic environments, it can respond to real threats in a timely manner while avoiding unnecessary emergency braking, thus ensuring a balance between driving safety and comfort.

[0041] Furthermore, the scenario of dynamic obstacles appearing at pedestrian crossings after a traffic light change specifically refers to situations where, when the traffic light turns from red to green and vehicles are preparing to cross, there are still pedestrians, non-motorized vehicles, or other dynamic obstacles on the pedestrian crossing that have not fully crossed, posing a risk of "rushing" or "being stuck." This situation is extremely common in actual driving, especially at busy intersections or when visibility is obstructed, and drivers may misjudge the timing of crossing due to distraction. This application defines this as an emergency event, and its design significance lies in compensating for the functional blind spots of traditional signal recognition systems. Most systems only recognize the traffic light status but do not combine it with the real-time dynamics of the pedestrian crossing for comprehensive judgment. This solution continuously monitors the pedestrian crossing area through a local perception module on the vehicle's in-vehicle system (such as a forward-facing camera + AI recognition). When a dynamic target is detected after the green light turns on, it is immediately determined to be an emergency event, preventing the vehicle from automatically starting or assisted acceleration, and triggering a warning to remind the driver to observe and confirm. This mechanism is not only applicable to autonomous or assisted driving modes but also provides critical safety redundancy for manual driving. More importantly, this judgment is made entirely based on real-time local sensor data, without relying on cloud maps or signal timing data. This ensures that even in abnormal situations such as traffic light malfunctions, communication interruptions, or outdated maps, the system can still effectively identify potential conflicts and guarantee traffic safety at intersections.

[0042] Furthermore, when an emergency event is detected where a pedestrian suddenly enters a drivable area, the vehicle's onboard system predicts the pedestrian's trajectory using a self-supervised learning model and immediately generates braking control commands based on the prediction results to execute emergency braking. This operation is independent of cloud response and initial scene information. Pedestrian behavior is highly random, and traditional rule-based systems struggle to accurately predict their trajectories. To address this, this application combines the vehicle's onboard self-supervised learning model with continuous image sequences captured by dual cameras to predict pedestrian movement trends in real time and determine whether they are likely to enter the vehicle's driving path. Once a risk is confirmed, the system immediately triggers local emergency braking, and this operation is completely independent of the cloud. This end-to-end localized design of "perception-prediction-decision-execution" minimizes response time and significantly improves the ability to handle unexpected scenarios such as "ghost pedestrians." Secondly, this mechanism demonstrates a high level of protection for vulnerable road users (VRUs) and complies with the "minimum risk state" requirement in Intelligent Vehicle Ethics and Functional Safety (ISO 26262). By granting the vehicle's infotainment system absolute control over the handling of pedestrian intrusions, the system can generate braking commands within 0.1-0.3 seconds, far faster than human reaction time, significantly reducing the probability of injury or death.

[0043] In some embodiments of this application, the self-supervised learning model acquires a set of overlapping pedestrian images through dual vehicle cameras, obtains training samples by in-camera sampling or cross-camera sampling, extracts pedestrian features and constructs a similarity matrix, generates a cyclic matching matrix through forward and backward matching, and trains the model with the identity matrix as the ideal target, thereby achieving pedestrian feature learning and trajectory prediction without labeled data.

[0044] Traditional deep learning-based pedestrian detection and trajectory prediction models heavily rely on large-scale, high-quality labeled datasets (such as Cityscapes, COCO, and KITTI). This requires manual annotation of bounding boxes, ID matching, and trajectory labeling for pedestrians in each frame of the image, a time-consuming, labor-intensive, and extremely costly process that struggles to cover all real-world road scenarios (such as extreme weather, low light, and severe occlusion). In contrast, the self-supervised learning mechanism employed in this application cleverly utilizes the natural consistency between time series and spatial perspectives as a supervisory signal, eliminating the need for any manual annotation.

[0045] Specifically, the system uses two video frames and Starting from the set of pedestrian images detected in the image, the feature extractor extracts from... and The feature matrix is ​​obtained from the middle and Because the appearance of the same person is highly consistent in adjacent frames or different viewpoints, the model constructs a similarity matrix. And using Softmax to Perform row normalization to generate the forward matching matrix. (express Each row instance in the middle is matched (The probability of each pedestrian instance). Similarly, for transpose matrix Softmax normalization is performed to obtain the backward matching matrix. The forward matching matrix With backward matching matrix Multiply to form a cyclic matching matrix Ideally, if the matching is accurate, each pedestrian should be correctly matched back to themselves, therefore It should approximate the identity matrix identity matrix As a natural supervisory objective, the feature extractor is driven to continuously optimize the model, ensuring that its output features have good consistency across frames and viewpoints. Therefore, a loss function is constructed to train the feature extractor. It satisfies the following formula (1): (1); In formula (1), Represents video frames and The number of pedestrians detected in the two frames (assuming the number of pedestrians is the same in both frames).

[0046] Formula (1), as the core loss function of the self-supervised pedestrian feature learning model, has the overall significance of driving the model to automatically learn pedestrian features with cross-frame matching capabilities by constructing a supervision signal based on cycle consistency that does not require manual annotation. Represents the forward matching matrix With backward matching matrix The resulting cyclic matching, obtained through multiplication, should ideally match each pedestrian exactly back to themselves. It should approximate the identity matrix The formula measures the difference between the two using the L1 norm, and uses... The loss value is normalized to the square of the number of pedestrians, ensuring it reflects the average matching error. This design cleverly utilizes time series data and visual consistency across multiple perspectives as a supervision objective, achieving end-to-end optimization of the pedestrian feature extractor under unlabeled data conditions. This not only reduces model training costs but also improves matching robustness in complex scenarios such as occlusion and pose changes, providing crucial technical support for high-precision trajectory prediction and autonomous braking decisions by the vehicle-mounted system in emergency situations (such as a pedestrian suddenly entering the vehicle). This mechanism enables the system to "learn while running" during daily vehicle operation, automatically extracting effective training samples from massive amounts of unlabeled driving videos, significantly reducing the data cost and engineering threshold of model training, and achieving a true "data closed loop."

[0047] Moreover, self-supervised learning does not rely on a specific annotation system, but rather learns the consistency of pedestrians' own visual features. Therefore, the model is more adaptable to new environments, new cities, and new lighting conditions. For example, on urban roads that have never been trained on, the system can still automatically learn the clothing styles, walking habits, and other features of local pedestrians through local image sequences collected by dual cameras, achieving effective predictions without the need to re-label the data.

[0048] Traditional supervised learning models, once deployed, have largely fixed capabilities unless retrained and a new version released. Self-supervised learning models, however, possess inherent online learning potential. Vehicles continuously collect new pedestrian image pairs during daily driving, which can serve as incremental samples to continuously optimize the local model. For example, the system can periodically train feature matching on recently collected data, fine-tuning the feature extractor parameters to gradually adapt to dynamic factors such as seasonal changes (e.g., pedestrians changing from short sleeves to down jackets) and regional differences (e.g., more pedestrians using umbrellas on rainy days in the south).

[0049] Although self-supervised learning involves operations such as feature extraction, similarity calculation, and Softmax normalization, its model structure is relatively lightweight and can be further compressed through model pruning and quantization techniques, making it suitable for efficient operation on in-vehicle computing platforms (such as SoC chips). Furthermore, because the prediction process is only activated when an emergency event is detected, the system adopts an "on-demand" strategy, avoiding continuous high-load computation and effectively balancing performance and power consumption. In addition, the model is only used for auxiliary decision-making in critical safety scenarios and does not participate in routine navigation, entertainment, or other non-safety functions, ensuring the priority allocation of computing resources and meeting the ASIL level requirements of functional safety standards (such as ISO 26262).

[0050] Furthermore, the inter-camera sampling design further enhances the model's robustness. When two cameras have a significant overlap in their fields of view, images of the same pedestrian from both perspectives can be used as positive sample pairs for training. This multi-view information fusion helps the model learn more discriminative features and reduces mismatches caused by occlusion, pose changes, or lighting differences. Intra-camera sampling, on the other hand, utilizes pedestrian sequences captured by the same camera in different time frames to learn the temporal continuity of motion, providing a temporal basis for trajectory prediction. By combining these two sampling methods, the model can not only identify pedestrians but also understand their spatial and temporal motion patterns, thereby achieving more accurate trajectory prediction, especially demonstrating stronger predictive capabilities in high-risk scenarios such as "ghost pedestrians" and sudden changes in direction by pedestrians.

[0051] Optionally, the self-supervised learning model can employ a contrastive learning model based on a dual-branch convolutional neural network (CNN). Its core architecture includes two convolutional neural network branches with shared parameters, each receiving overlapping pedestrian image sets captured by dual in-vehicle cameras. To adapt to the real-time requirements of the vehicle, the model adopts a lightweight network architecture such as MobileNet, combining depthwise separable convolutions and channel pruning techniques to optimize computational efficiency. Deployed on the vehicle's edge computing module, it can independently predict pedestrian trajectories in emergency events (such as a pedestrian suddenly entering a drivable area), directly generating braking control commands. It coordinates with the emergency event handling logic of the scene management and decision-making modules to ensure low-latency response and driving safety.

[0052] In some embodiments of this application, when the vehicle is connected to the network, the first scene information is acquired and updated to cover the second scene information; when the vehicle is not connected to the network, the second scene information remains effective until it is covered by the updated first scene information after the vehicle is connected to the network.

[0053] The core function of this mechanism is to achieve dynamic synchronization and seamless connection of scene states between the cloud and the vehicle's infotainment system. This design fully considers the uncertainties of the network environment during actual driving of intelligent connected vehicles (such as signal blind spots in tunnels, mountainous areas, and underground parking garages). By establishing a state management strategy of "cloud-based as the standard, local as a backup," it ensures that the system always makes decisions based on the latest and most accurate information. When the vehicle is connected to the network, the "first scene information" generated by the cloud based on global data (such as traffic flow, weather, and user profiles) has higher completeness and accuracy. Therefore, the system prioritizes and covers outdated or simplified judgment results that may exist locally on the vehicle, ensuring the level of service intelligence. However, in the event of a network outage or unstable communication, the system automatically downgrades to relying on the "second scene information" generated by the vehicle's infotainment system to maintain basic scene recognition and functional operation, avoid service interruptions, and ensure the continuity of user experience.

[0054] This overlay mechanism not only resolves the common "information conflict" and "state drift" issues in cloud-edge collaborative systems, but also constructs a closed-loop state update process, ensuring that the vehicle always maintains logical consistency with the cloud. More importantly, this design embodies a fine balance between safety and service requirements: in non-emergency situations, by periodically updating the first-scenario information, the system can dynamically adjust personalized service strategies (such as navigation routes and media recommendations); while in the event of an emergency, even if local information is overwritten, the system can immediately switch back to local decision-making mode, ensuring that the safety response is unaffected. Furthermore, this mechanism supports "breakpoint resume" information synchronization—after the vehicle reconnects to the network, it does not need to reinitialize the entire scenario state, but directly receives the latest pushed first-scenario information to complete the overlay, greatly improving the system's response efficiency and resource utilization, providing a solid foundation for building a highly reliable and robust intelligent driving guidance system.

[0055] In some embodiments of this application, step S140 involves performing a preset operation corresponding to the current scenario, sub-scenario, vehicle usage status, and business event based on information characteristics, including the following steps.

[0056] Step S210: Obtain the scene, sub-scene, vehicle usage status and triggered business events selected by information features.

[0057] In step S210, the specific content requiring a response is precisely extracted from the information features, providing a clear input basis for subsequent operations. The information features are the final decision results after dynamic selection in step S130, originating from the first scenario information pushed from the cloud or the second scenario information generated locally on the vehicle's system. Through these information features, the system can accurately identify the current driving situation, including macro-scenarios (such as commuting, long-distance travel), sub-scenarios (such as morning rush hour congestion, driving in the rain), vehicle usage status (such as first start, low battery, fatigue driving), and triggered business events (such as a pedestrian running in front, a user's birthday reminder, a meeting about to start, etc.). This information extraction process not only follows the scenario judgment result but is also a prerequisite for accurate response. Only with a clear understanding of the current multi-dimensional context information can the system correctly distinguish event types, assess priorities, and invoke corresponding execution strategies, avoiding misoperations or service mismatches, ensuring the accuracy of driving guidance and the consistency of user experience.

[0058] Step S220: Determine the security priority of the business event based on its type. Security priorities are divided into emergency security events and routine service events.

[0059] In step S220, a priority determination mechanism for the system's response logic is established. By classifying and prioritizing business events, the system can quickly identify the most urgent and critical tasks among multiple concurrent events, ensuring that resources are prioritized for high-risk scenarios involving driving safety. Emergency safety events mainly include vehicle speed exceeding safe limits, forward collision risks, pedestrians suddenly entering drivable areas, and dynamic obstacles appearing on crosswalks after traffic lights change. These events are highly time-sensitive and have serious consequences, requiring immediate response. Regular service events, on the other hand, encompass functions that enhance user experience, such as personalized recommendations, navigation optimization, and lifestyle reminders, allowing for a certain delay. This classification mechanism not only reflects the emphasis on functional safety but also provides a decision-making basis for the subsequent routing of execution paths. By clearly defining priorities, the system can maintain clear response logic in complex driving environments, avoiding conflicts such as entertainment service pop-ups interfering with safety warnings, thus ensuring driver focus and driving safety.

[0060] In step S230, if the business event is an emergency safety event, the vehicle terminal local execution module is invoked to directly trigger the preset safety operation.

[0061] The preset safety operations include: performing emergency braking, lane departure correction, or collision warning based on local sensor data.

[0062] In step S230, the vehicle's infotainment system is granted absolute control in critical moments, ensuring low-latency and highly reliable safety intervention. When the system detects an emergency safety event, it no longer relies on cloud commands or complex service processes, but immediately invokes the local execution module on the vehicle's infotainment system to directly trigger preset safety operations. These operations are completed independently based entirely on real-time data from local sensors (such as cameras, radar, and IMU), without requiring a network connection, and the response time can be controlled within milliseconds. For example, when a pedestrian is detected suddenly crossing the road ahead, the system can immediately activate automatic emergency braking (AEB); when the vehicle deviates from its lane, it automatically corrects its direction or issues a strong warning. This mechanism effectively avoids risks such as cloud communication delays and signal interruptions, ensuring that the system can still autonomously take protective measures in critical moments. In addition, since the safety operations are directly triggered locally, the uncertainty caused by multi-layer scheduling is avoided, improving the system's robustness and predictability, meeting the requirements of automotive functional safety standards such as ISO 26262, and providing a solid safety backup capability for intelligent driving systems.

[0063] Step S240: If the business event is a regular service event, then match the corresponding personalized service strategy according to the current scenario, sub-scenario and vehicle usage status.

[0064] Personalized service strategies include: playing pre-installed media content, recommending music resources, planning navigation routes, or pushing life information reminders.

[0065] In step S240, for non-urgent routine service events, the system no longer takes mandatory intervention measures. Instead, it proactively matches personalized service strategies based on the current driving scenario, sub-scenario, and vehicle usage status, achieving a shift from "passive response" to "proactive service." For example, when a user starts the vehicle for the first time on their birthday, the system can automatically play a pre-set birthday surprise video; in commuting scenarios, it plans the optimal navigation route based on real-time traffic conditions; when it detects that the driver is in a low mood, it recommends soothing music; and it pushes reminders when a meeting is approaching. These service strategies not only rely on current environmental information but also integrate contextual data such as user identity, lifestyle information, and historical behavior to achieve deep personalization. Through this step, the in-vehicle system is no longer a cold, impersonal tool but becomes an intelligent partner that understands user needs and provides emotional interaction, significantly improving the comfort and enjoyment of the driving experience.

[0066] Step S250: Execute the personalized service strategy and provide feedback on the operation results through the vehicle's human-machine interface.

[0067] In step S250, the system decision is translated into user-perceptible interactive behaviors, and the system's transparency and credibility are enhanced through a feedback mechanism. After executing the personalized service strategy, the system needs to present the operation results to the user through the vehicle's human-machine interface (HMI), such as displaying a recommended song list on the central control screen, announcing navigation changes via voice, popping up a birthday greeting animation, or vibrating to remind the user that a meeting has started. This visual or audible feedback not only lets the user know that the system is providing services for them, but also helps the user confirm the rationality of the operation and avoid the trouble caused by accidental triggering. At the same time, good interaction design can improve the user's trust and acceptance of the intelligent system. In addition, this step also allows users to intervene in or turn off the service, reflecting the design concept of "human-machine collaboration"—the system actively provides services, but the final control is still in the hands of the driver, ensuring that convenience is provided without interfering with safe driving.

[0068] In step S260, if a new business event is detected during the operation, repeat steps S210 to S250 above to override the current operation process with the priority of the new business event.

[0069] In step S260, a dynamic iteration and priority preemption mechanism for system response is established to ensure the system's ability to respond to the latest and highest priority events in complex and ever-changing driving environments. In actual driving, business events may occur frequently and overlap; for example, while the system is playing a birthday greeting video, a pedestrian may suddenly be detected running in front. Failure to promptly interrupt the current service and respond to the emergency could lead to serious consequences. Through this step, the system continuously monitors for new events. Once a higher-priority business event (especially an emergency safety event) is detected, the currently executing operation is immediately interrupted, and the system re-enters step S210 for information extraction and priority determination, executing the corresponding operation with the new event as the highest priority. This "high priority over low priority" mechanism ensures that safety events are always handled first, demonstrating the system's intelligent scheduling capabilities in multi-tasking environments. Simultaneously, this design also supports smooth switching between services, such as switching from navigation broadcast to collision warning, improving the system's flexibility and adaptability, and providing a guarantee for achieving all-weather, all-scenario intelligent driving guidance.

[0070] In summary, the driving scenario prediction method based on vehicle-cloud integration provided in this application has the following technical effects.

[0071] This method effectively integrates cloud-based big data analytics with the low-latency response of in-vehicle systems by constructing a collaborative architecture that combines vehicle and cloud capabilities. When the vehicle is connected to the network, the system prioritizes high-precision scene information pushed from the cloud to achieve a deep understanding of complex driving environments and user intentions, enhancing the intelligence level of personalized services. In the event of a network outage or an emergency, it automatically switches to local judgment results on the in-vehicle system, ensuring the continued availability of critical functions and granting the vehicle absolute control over safety-related operations, guaranteeing millisecond-level emergency response. Through dynamic selection of decision-making criteria and state coverage mechanisms, the system achieves seamless connection and continuous experience between the cloud and the device, significantly improving the reliability, safety, and robustness of intelligent driving guidance.

[0072] Furthermore, this application introduces a self-supervised learning model, enabling the vehicle-mounted system to autonomously learn pedestrian characteristics and predict movement trajectories even without labeled data, further enhancing the intelligence and adaptability of local decision-making. At the execution level, the system establishes a hierarchical response mechanism based on safety priorities, distinguishing between emergency safety events and routine service events. This ensures that local safety functions (such as emergency braking and lane correction) can be triggered independently and quickly in high-risk scenarios, while also supporting proactive push notifications and human-machine interaction feedback for personalized services (such as birthday reminders and music recommendations). When a new event occurs, the system supports dynamic overlay of high-priority tasks onto the current process, achieving intelligent scheduling in multi-event environments. The overall solution not only addresses the shortcomings of existing technologies, such as decision lag, network outages, and insufficient emergency response, but also promotes the evolution of in-vehicle systems from "passive execution" to "proactive intelligence," providing users with a safe, continuous, and personalized intelligent travel experience.

[0073] Secondly, refer to Figure 2 This application provides a driving scenario prediction system based on vehicle-cloud integration, including: The cloud push module 310 is used to receive the first scenario information pushed by the cloud on the vehicle terminal. The first scenario information includes the current scenario and sub-scenarios, vehicle usage status, triggered business events and subsequent execution actions.

[0074] The vehicle-mounted local generation module 320 is used to generate second scene information locally on the vehicle-mounted terminal. The second scene information is a scene and business event that can be determined by relying only on local data or cached data.

[0075] The decision module 330 is used to dynamically select either the first scenario information or the second scenario information as information features based on the vehicle's network connectivity status and the detection results of emergency events.

[0076] The operation execution module 340 is used to execute preset operations corresponding to the current scenario, sub-scenario, vehicle usage status and business events based on information characteristics.

[0077] Furthermore, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the aforementioned vehicle-cloud-based driving scenario prediction method.

[0078] Furthermore, this application provides a vehicle that integrates the aforementioned vehicle-cloud-based driving scenario prediction system.

[0079] Similarly, the technical effects of the above system embodiments, program product embodiments, and vehicle embodiments are consistent with the technical effects of the above method embodiments.

[0080] It should be noted that in all specific embodiments of this application, all data processing activities related to user identity or personal characteristics, such as user information, user behavior data, historical data, and location information, will be conducted in accordance with the principles of legality, legitimacy, and necessity. All data collection, use, storage, and processing will be subject to compliance with applicable national and regional laws, regulations, and industry standards, and informed consent from users will be obtained in a clear and explicit manner before processing. For the processing of sensitive personal information, separate consent from users will be obtained through prominent means such as pop-up prompts and independent confirmation pages. If any processing conflicts with laws and regulations, the laws and regulations will prevail, and necessary data processing will only be carried out within the scope permitted by laws and regulations, ensuring that all data-based applications, analyses, and technical implementations are conducted within the scope permitted by laws and regulations.

[0081] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0082] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary skill. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0083] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable programs for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can retrieve and execute a program from or in conjunction with such a program execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with a program execution system, apparatus, or device.

[0085] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Additionally, computer-readable media can even be paper or other suitable media on which programs can be printed, for example, by optically scanning the paper or other media, then editing, interpreting, or, if necessary, processing it in a suitable manner to obtain the program electronically, and then storing it in computer memory.

[0086] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable program execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0087] In the foregoing description of this specification, the reference to terms such as "one embodiment / implementation," "another embodiment / implementation," or "certain embodiments / implementations," etc., indicates that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in an embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0088] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0089] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A driving scenario prediction method based on vehicle-cloud integration, characterized in that, Includes the following steps: The vehicle-mounted system receives first scene information pushed from the cloud, which includes the current scene and sub-scenes, vehicle usage status, triggered business events and subsequent actions. The vehicle-mounted system generates second scene information locally, which is a scene and business event that can be determined by relying only on local data or cached data. Based on the vehicle's network connectivity status and the detection results of emergency events, dynamically select either the first scenario information or the second scenario information as the information feature; Based on the information characteristics, execute preset operations corresponding to the current scenario, sub-scenario, vehicle usage status, and business events.

2. The driving scenario prediction method based on vehicle-cloud integration according to claim 1, characterized in that, The step of dynamically selecting either the first scenario information or the second scenario information as information features based on the vehicle's network connectivity status and the detection results of emergency events specifically includes: If the network is currently connected, the information from the first scenario is used as the information feature. If the current state is offline or an emergency event is detected, the second scenario information is used as the information feature.

3. The driving scenario prediction method based on vehicle-cloud integration according to claim 2, characterized in that, The emergency events include at least one of the following: the vehicle speed exceeds the safe range, there is a risk of collision ahead, a pedestrian suddenly enters the drivable area, and a dynamic obstacle appears on the pedestrian crossing after the traffic light changes.

4. The driving scenario prediction method based on vehicle-cloud integration according to claim 3, characterized in that, When an emergency event is detected in which a pedestrian suddenly enters a drivable area, the vehicle-mounted system predicts the pedestrian's trajectory using a self-supervised learning model and immediately generates a braking control command based on the prediction result to perform an emergency braking operation. This operation does not depend on cloud response or the first scenario information.

5. The driving scenario prediction method based on vehicle-cloud integration according to claim 4, characterized in that, The self-supervised learning model acquires a set of overlapping pedestrian images using dual vehicle cameras, obtains training samples by in-camera sampling or cross-camera sampling, extracts pedestrian features and constructs a similarity matrix, generates a cyclic matching matrix through forward and backward matching, and trains the model with the identity matrix as the ideal target, thereby achieving pedestrian feature learning and trajectory prediction without labeled data.

6. The driving scenario prediction method based on vehicle-cloud integration according to claim 1, characterized in that, When the vehicle is connected to the network, the first scenario information is acquired and updated to overwrite the second scenario information; when the vehicle is not connected to the network, the second scenario information remains in effect until it is overwritten by the updated first scenario information after the vehicle is connected to the network.

7. The driving scenario prediction method based on vehicle-cloud integration according to claim 1, characterized in that, The step of executing preset operations corresponding to the current scenario, sub-scenario, vehicle usage status, and business event based on the information features includes the following steps: Obtain the scene, sub-scene, vehicle usage status, and triggered business events selected through the information features; Based on the type of the business event, its security priority is determined; whereby security priorities are divided into emergency security events and routine service events. If the business event is an emergency safety event, the vehicle-mounted local execution module is invoked to directly trigger preset safety operations; the preset safety operations include: performing emergency braking, lane departure correction, or collision warning based on local sensor data; If the business event is a routine service event, then a corresponding personalized service strategy will be matched according to the current scenario, sub-scenario, and vehicle usage status; the personalized service strategy includes: playing preset media content, recommending music resources, planning navigation routes, or pushing life information reminders; The personalized service strategy is executed, and the operation results are fed back through the human-machine interface on the vehicle terminal. If a new business event is detected during the operation, repeat the above steps to override the current operation process with the priority of the new business event.

8. A driving scenario prediction system based on vehicle-cloud integration, characterized in that, include: The cloud push module is used to receive the first scenario information pushed by the cloud on the vehicle terminal. The first scenario information includes the current scenario and sub-scenarios, vehicle usage status, triggered business events and subsequent execution actions. The vehicle-mounted local generation module is used to generate second scene information locally on the vehicle-mounted terminal. The second scene information is a scene and business event that can be determined by relying only on local data or cached data. The decision module is used to dynamically select either the first scenario information or the second scenario information as information features based on the vehicle's network connectivity status and the detection results of emergency events. The operation execution module is used to execute preset operations corresponding to the current scenario, sub-scenario, vehicle usage status, and business events based on the information characteristics.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the vehicle-cloud-based driving scenario prediction method as described in any one of claims 1 to 7.

10. A vehicle, characterized in that, The vehicle integrates a driving scenario prediction system based on vehicle-cloud integration as described in claim 8.

Images