Automobile intelligent cockpit scene perception decision method based on a hon gming system

Abstract

The application provides a car intelligent cockpit scene perception decision method based on a Hongmeng system, which comprises the following steps: determining core perception tasks under each basic running scene based on real-time requirements of the current basic running scene of the car cockpit, and determining multi-source perception devices meeting the task requirements based on the task requirements of the core perception tasks and a pre-constructed perception device capability mapping table; performing time synchronization and preprocessing on original perception data collected by the multi-source perception devices based on the Hongmeng system to obtain standardized perception data; performing scene classification based on the standardized perception data and the basic running scene to obtain a corresponding current scene type under the basic running scene; determining a target decision strategy corresponding to the current scene type based on the current scene type and a pre-constructed cockpit scene decision knowledge base, and performing instruction analysis on the target decision strategy to obtain control instructions of each execution device. The application improves the actual experience of users in the intelligent cockpit.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a car intelligent cockpit scene perception decision-making method based on a Harmony system. BACKGROUND

[0002] With the rapid development of artificial intelligence technology and the continuous expansion of the new energy vehicle market in recent years, intelligent cockpit, as a key link to improve user experience, has gradually become the focus of attention of major automakers and technology companies. The Harmony system (HarmonyOS) has shown great potential in the field of intelligent cockpit due to its outstanding multi-device collaboration capabilities and powerful AI capabilities. Intelligent cockpit based on the Harmony system not only enables smoother human-computer interaction experience, but also provides more intelligent and personalized services for users through advanced perception decision-making algorithms.

[0003] Currently, the existing intelligent cockpit scene perception decision-making method often uses fixed or preset perception device calling logic, which cannot dynamically match the device resources corresponding to the core perception tasks according to the real-time needs of the current basic running scene of the car cockpit, nor can it accurately classify the scene and make decisions in the basic running scene based on the perception data, resulting in misjudgment during scene classification, which leads to subsequent decision bias, making it impossible to quickly call the optimal decision strategy according to accurate scene recognition, causing slow service response, and ultimately failing to meet the user's demand for the timeliness and accuracy of cockpit services, thereby affecting the user's actual experience in the intelligent cockpit. SUMMARY

[0004] The present application provides a car intelligent cockpit scene perception decision-making method based on a Harmony system to ensure that the cockpit can output adaptive services in a timely manner according to different scenes, improving the scene adaptability, decision-making accuracy, and service timeliness of the car intelligent cockpit, and thus improving the user's actual experience in the intelligent cockpit.

[0005] In a first aspect, the present application provides a car intelligent cockpit scene perception decision-making method based on a Harmony system, comprising:

[0006] Based on the real-time needs of the current basic running scene of the car cockpit, determine the core perception tasks in each basic running scene, and based on the task requirements of the core perception tasks and a pre-constructed perception device capability mapping table, determine multiple-source perception devices that meet the task requirements;

[0007] Based on the Harmony system, time-synchronize and preprocess the original perception data collected by the multiple-source perception devices to obtain standardized perception data;

[0008] Based on the standardized perception data and the basic running scene, classify the scene to obtain the corresponding current scene type under the basic running scene.

[0009] determine a target decision strategy corresponding to the current scene type based on the current scene type and a pre-constructed cockpit scene decision knowledge base, and perform instruction analysis on the target decision strategy to obtain control instructions for each execution device.

[0010] In a second aspect, the present application further provides a vehicle intelligent cockpit scene perception decision system based on a hyper-vehicle system, which is applied to the vehicle intelligent cockpit scene perception decision method based on a hyper-vehicle system as described in the first aspect. The vehicle intelligent cockpit scene perception decision system based on a hyper-vehicle system comprises:

[0011] a perception device determination module configured to determine core perception tasks under each basic running scene based on real-time requirements of the current basic running scene of the vehicle cockpit, and determine multi-source perception devices meeting the task requirements based on task requirements of the core perception tasks and a pre-constructed perception device capability mapping table;

[0012] a data synchronization and preprocessing module configured to perform time synchronization and preprocessing on original perception data collected by the multi-source perception devices based on the hyper-vehicle system to obtain standardized perception data;

[0013] a cockpit scene classification module configured to perform scene classification based on the standardized perception data and the basic running scene to obtain a current scene type corresponding to the basic running scene;

[0014] a strategy generation and instruction analysis module configured to determine a target decision strategy corresponding to the current scene type based on the current scene type and a pre-constructed cockpit scene decision knowledge base, and perform instruction analysis on the target decision strategy to obtain control instructions for each execution device.

[0015] In a third aspect, the present application further provides an electronic device comprising a memory for storing a computer software program and a processor for reading and executing the computer software program to realize the vehicle intelligent cockpit scene perception decision method based on a hyper-vehicle system as described above.

[0016] In a fourth aspect, the present application further provides a non-transitory computer readable storage medium having a computer software program stored therein, wherein the computer software program is executed by a processor to realize the vehicle intelligent cockpit scene perception decision method based on a hyper-vehicle system as described above.

[0017] In a fifth aspect, the present application further provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to realize the vehicle intelligent cockpit scene perception decision method based on a hyper-vehicle system as described above.

[0018] The automobile intelligent cockpit scene perception decision method based on the Hongmeng system provided by the embodiment of the application realizes accurate adaptation of the perception device and the scene task, avoids invalid calling and waste of device resources, performs time synchronization and preprocessing on raw data collected by the multi-source perception device based on the Hongmeng system, obtains high-quality standardized perception data, classifies scenes based on the standardized perception data and the basic running scene, accurately identifies the corresponding current scene type under the basic running scene, so that the intelligent cockpit can clearly recognize the specific scene currently in, avoids decision deviation caused by scene misjudgment, matches the pre-constructed cockpit scene decision knowledge base based on the current scene type to determine a target decision strategy, and analyzes the control instruction of the execution device, realizes quick correspondence and accurate execution of the scene and the decision strategy, ensures that the cockpit can output adaptive services in a timely manner according to different scenes, improves the scene adaptability, decision accuracy and service timeliness of the automobile intelligent cockpit, and further improves the actual experience of the user in the intelligent cockpit. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 FIG. 1 is a flowchart of the automobile intelligent cockpit scene perception decision method based on the Hongmeng system provided by the embodiment of the application;

[0020] Figure 2 FIG. 2 is a structural diagram of the automobile intelligent cockpit scene perception decision system based on the Hongmeng system provided by the embodiment of the application;

[0021] Figure 3 FIG. 3 is an embodiment diagram of the electronic device provided by the embodiment of the application;

[0022] Figure 4 FIG. 4 is an embodiment diagram of the computer readable storage medium provided by the embodiment of the application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0024] In the description of the present application, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0025] In the description of the present application, the term "for example" is used to indicate "as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily interpreted as more preferred or more advantageous than other embodiments. The following description is given in order to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that those skilled in the art can recognize that the present application can be implemented without using these specific details. In other examples, well-known structures and processes will not be described in detail to avoid unnecessary details making the description of the present application obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope in accordance with the principles and characteristics disclosed.

[0026] Reference Figure 1 , Figure 1 is a flowchart of the method for scenario perception decision of an intelligent cockpit of an automobile based on a Hongmeng system provided by the present application. The execution subject of the method for scenario perception decision of an intelligent cockpit of an automobile based on a Hongmeng system in the embodiment of the present application is a scenario perception decision system. Therefore, the method for scenario perception decision of an intelligent cockpit of an automobile based on a Hongmeng system comprises the following steps:

[0027] In step 10, based on the real-time demand of the current basic running scenario of the automobile cockpit, the core perception task under each basic running scenario is determined, and based on the task demand of the core perception task, a multi-source perception device meeting the task demand is determined in combination with the pre-constructed perception device capability mapping table.

[0028] Optionally, the scenario perception decision system determines the current basic running scenario information of the automobile cockpit by vehicle state data (such as vehicle speed, gear position, handbrake state, etc.) transmitted by the automobile CAN bus of the Hongmeng system. Among them, the basic running scenario includes a driving scenario, a parking scenario and a temporary parking scenario. For example, when the vehicle speed is greater than 0 km / h, the gear position is in D or R, and the handbrake is in the released state, it is determined as a driving scenario. When the vehicle speed is 0 km / h, the gear position is in P, the handbrake is in the tightened state, and the engine is in the running or off state, it is determined as a parking scenario. When the vehicle speed is 0 km / h, the gear position is in N, the handbrake is in the tightened state, the engine is in the running state, and the driver has not left the driving position, it is determined as a temporary parking scenario.

[0029] Further, after determining the basic running scene information, the scene-aware decision system determines the core perception task in combination with user demand and cabin service logic for different basic running scenes. The user demand can be input by the user in advance or determined according to historical data analysis, which can be determined by the prior art and will not be described in detail. For example, in the driving scene, the core perception task includes driver state monitoring (such as whether to be tired or distracted), road condition monitoring (such as whether there is an obstacle in front, whether the lane line is clear), and in-vehicle passenger state monitoring (such as whether there is a child misoperation); in the parking scene, the core perception task includes in-vehicle personnel activity monitoring (such as whether there is a person left behind), and vehicle surrounding environment monitoring (such as whether there is a pedestrian approaching or a vehicle collision risk); in the temporary parking scene, the core perception task includes whether the driver will return soon, and temporary obstacle monitoring in the vehicle surrounding (such as whether there is a temporarily parked bicycle).

[0030] Further, after the scene-aware decision system obtains the core perception task, the required perception device is screened in the pre-constructed perception device capability mapping table to finally determine the multi-source perception device for completing the core perception task. In the perception device capability mapping table, the technical parameters (such as detection distance, detection accuracy, sampling frequency, data type) of various perception devices (such as cameras, millimeter wave radars, laser radars, infrared sensors, microphones, seat pressure sensors, etc.), applicable scene range, and functional characteristics are recorded. For example, the driver state monitoring task requires a device that can recognize facial features and eye state, and a high-definition infrared camera can be selected; the road condition monitoring task requires to detect long-distance obstacles and lane lines, and a combination of millimeter wave radar and monocular camera can be selected; in-vehicle personnel activity monitoring can select in-vehicle high-definition camera and seat pressure sensor; and vehicle surrounding environment monitoring can select surround-view camera and ultrasonic radar.

[0031] In one embodiment, taking the current vehicle speed of 60km / h, gear in D, and parking brake not activated as collected by the HarmonyOS system in the car cabin as an example, the scene perception decision system determines the basic operating scenario as a driving scenario. Based on the real-time safety requirements of the driving scenario, the system determines the core perception tasks as "driver attention monitoring", "road condition recognition ahead", and "obstacle detection to the side and rear of the vehicle". The system then begins to access the perception device capability mapping table. The requirements for the driver attention monitoring task are: visual perception dimension, facial feature recognition accuracy ≥95%, response time ≤0.3s, and detection range covering the driver's facial area. The device meeting these requirements in the mapping table is the "in-cabin DMS camera (1080P resolution, 30fps frame rate, supports facial key point recognition algorithm)". The requirements for the "road condition recognition" task are: visual perception dimension, road sign recognition accuracy ≥90%, detection distance 5-100m, and response time ≤0.5s. The device meeting these requirements in the mapping table is the "windshield monocular camera (120° wide-angle, supports road semantic segmentation algorithm, detection distance 150m)". The requirements for the "vehicle side and rear obstacle detection" task are: radar perception dimension, obstacle distance measurement accuracy ±0.5m, detection angle ±45°, and response time ≤0.2s. The device meeting these requirements in the mapping table is the "millimeter-wave radar on both sides of the vehicle (detection range 0.3-50m, angular resolution 1°)". Based on this, the scene perception decision system sends a start command to the above three types of devices through the HarmonyOS system, and at the same time establishes a binding relationship between the devices and the core perception tasks to ensure that the DMS camera collects driver facial data in real time, the windshield camera collects road condition data ahead, and the millimeter-wave radar collects obstacle data to the side and rear.

[0032] Step 20: Based on the HarmonyOS system, the raw sensing data collected by the multi-source sensing devices is synchronized in time and preprocessed to obtain standardized sensing data.

[0033] Optionally, after receiving the raw sensing data (such as image frame data from a camera, point cloud data from a millimeter-wave radar, audio waveform data from a microphone, etc.) collected by the determined multi-source sensing devices in step 10, the scene perception decision system will perform time synchronization and preprocessing on the collected raw sensing data to obtain standardized sensing data, as described in steps 201-205.

[0034] Step 30: Based on standardized perception data and basic operating scenarios, classify scenarios to obtain the current scenario type corresponding to the basic operating scenario.

[0035] Optionally, the scene-aware decision system extracts corresponding features from the standardized perception data according to the determined basic running scene in step 10, thereby realizing the reclassification of the basic running scene and obtaining a more accurate subdivided scene, i.e., the current scene type, which can represent the running state of the current vehicle.

[0036] In step 40, based on the current scene type, the pre-constructed cockpit scene decision knowledge base is combined to determine the target decision strategy corresponding to the current scene type, and the target decision strategy is parsed to obtain the control instructions of each execution device.

[0037] Optionally, the scene-aware decision system determines the target decision strategy corresponding to the current scene type based on the current scene type obtained in step 30, and calls the pre-constructed cockpit scene decision knowledge base to perform accurate matching in the cockpit scene decision knowledge base, thereby determining the target decision strategy corresponding to the current scene type, as described in steps 401-405.

[0038] Further, the cockpit scene decision knowledge base is pre-constructed using the "scene-feature-strategy" association logic, and stores the optimal decision strategy corresponding to the features of different scene types, as well as real-time updated "historical execution data". The decision strategy covers the execution device type, operation logic, parameter threshold, etc., and is consistent with the user experience (such as pre-stored personalized design) and safety requirements. It is not a static database, but is dynamically updated through the multi-source data interaction capability of the Hongmeng system to ensure data timeliness. Its construction process can use existing technologies and will not be described in detail. After obtaining the target decision strategy, the scene-aware decision system performs "instruction parsing" on the target decision strategy: the abstract strategy content is disassembled into specific execution steps, and the corresponding execution device (such as the instrument panel, car audio, seat controller, air conditioning system, window controller, etc.) of each step is determined, and the execution steps are converted into control instructions recognizable by each device according to the communication protocol (such as CAN bus protocol, Hongmeng distributed device protocol) of the device. It should be noted that the control instructions include instruction types (such as "warning instructions", "adjustment instructions", "execution instructions"), parameter values (such as warning volume size, air conditioning temperature setting value, seat adjustment angle), execution timing (such as starting the audio warning first, and starting the instrument panel prompt after 1s), etc. Finally, the scene-aware decision system transmits the parsed control instructions to each execution device in real time through the distributed instruction issuing function of the Hongmeng system.

[0039] In an embodiment, the scene-aware decision system invokes the cockpit scene decision knowledge base, and matches the target decision strategy corresponding to the scene as follows: "1. Start the car audio to play a low-frequency warning sound (volume 60%, duration 1s); 2. The instrument panel displays a red "Please focus on driving" prompt text (display time 3s); 3. If the driver does not resume focus within 1s, slightly vibrate the steering wheel (vibration intensity 30%, duration 0.5s); 4. Record the distraction event to the vehicle log in real time".

[0040] The scene-aware decision system analyzes the strategy as follows:

[0041] For "car audio": the instruction type is "warning instruction", the parameter is "volume = 60%, playing time = 1s, audio type = low-frequency warning sound", the communication protocol uses the distributed audio protocol of Hongmeng, and the instruction code is "Audio_Warn_60_1s";

[0042] For "instrument panel": the instruction type is "display instruction", the parameter is "text content = please focus on driving, text color = red, display time = 3s", the communication protocol uses CAN bus protocol, and the instruction code is "Instrument_Display_Red_3s";

[0043] For "steering wheel vibration module": the instruction type is "vibration instruction", the parameter is "vibration intensity = 30%, vibration duration = 0.5s, trigger condition = driver does not resume focus after 1s of audio warning", the communication protocol uses LIN bus protocol, and the instruction code is "Steering_Vib_30_0.5s";

[0044] For "vehicle log system": the instruction type is "record instruction", the parameter is "event type = driver distraction, occurrence time = system reference time T0, duration = 0.5s", the communication protocol uses the distributed data storage protocol of Hongmeng, and the instruction code is "Log_Record_Distract_T0_0.5s".

[0045] After analysis, the scene-aware decision system issues control instructions to the car audio, instrument panel, steering wheel vibration module, and vehicle log system through the Hongmeng system respectively.

[0046] The embodiment of the application determines core perception tasks by combining the real-time needs of the current basic running scene of the automobile cabin, and matches multi-source perception devices that meet the needs according to a pre-constructed perception device capability mapping table, thereby realizing accurate adaptation of the perception device and the scene task, avoiding invalid calling and wasting of device resources, and obtaining high-quality standardized perception data based on time synchronization and preprocessing of raw data collected by the multi-source perception device based on the Hongmeng system, and then classifying the scene based on the standardized perception data and the basic running scene, accurately identifying the corresponding current scene type under the basic running scene, so that the intelligent cabin can clearly recognize the specific scene currently in, avoid decision deviation caused by scene misjudgment, determine the target decision strategy based on the current scene type and the pre-constructed cabin scene decision knowledge base, and analyze it into an execution device control instruction, thereby realizing fast correspondence and accurate execution of the scene and the decision strategy, ensuring that the cabin can output adaptive services in a timely manner according to different scenes, and improving the scene adaptability, decision accuracy and service timeliness of the intelligent cabin of the automobile, thereby improving the actual experience of the user in the intelligent cabin.

[0047] In an embodiment, steps 201-205 are described as follows:

[0048] In step 201, based on the distributed soft bus of the Hongmeng system and a pre-constructed perception device data type mapping table, a unique combination of device identification and data frame sequence number is allocated to each raw perception data, forming raw identification perception data.

[0049] Optionally, the scene perception decision system first establishes a high-speed data transmission channel with multi-source perception devices (such as DMS cameras, front windshield cameras, millimeter wave radars, etc.) relying on the distributed soft bus technology of the Hongmeng system, ensuring low delay (≤20 ms) and high reliability (packet loss rate ≤0.1%) of the raw perception data in the transmission process. Then, a pre-constructed perception device data type mapping table is called to store the unique device identification (such as device ID: DMS_CAM_001, FRONT_CAM_002, MMW_RADAR_003) of all perception devices, data type (such as image data, point cloud data, audio data) and data frame format (such as JPEG, PCAP, WAV). According to each received raw perception data, the corresponding device identification is matched from the mapping table, and a unique combination identification is generated according to the rule of “device identification + data frame sequence number”, wherein the data frame sequence number is encoded in an incremental manner, and the sequence number is incremented by 1 for each received frame (such as DMS_CAM_001_0001, FRONT_CAM_002_0001), ensuring that each data frame of each data has a unique traceable identification. Finally, the unique combination identification is bound with the raw perception data to form the raw identification perception data, providing a basis for subsequent time calibration, data grouping and tracing.

[0050] In an embodiment, taking the transmission of original image frames by DMS cameras in a multi-source perception device, the transmission of original road condition images by a front windshield camera, and the transmission of original point cloud data by a millimeter wave radar as examples, the DMS camera transmits original image frames (in JPEG format, with a resolution of 1080P), the system matches its device identifier as “DMS_CAM_001” from the “perception device data type mapping table”, and the current received data is the 120th frame, generates a unique combination identifier “DMS_CAM_001_0120”, and after binding, forms original identified perception data A; the front windshield camera transmits original road condition images (in JPEG format, with a resolution of 1920x1080), matches the device identifier as “FRONT_CAM_002”, and receives the 85th frame of data, generates a unique combination identifier “FRONT_CAM_002_0085”, and after binding, forms original identified perception data B; the millimeter wave radar transmits original point cloud data (in PCAP format, containing 200 points), matches the device identifier as “MMW_RADAR_003”, and receives the 200th frame of data, generates a unique combination identifier “MMW_RADAR_003_0200”, and after binding, forms original identified perception data C.

[0051] In step 202, the local time stamp of the original identified perception data is calibrated based on the global unified time stamp provided by the distributed time service of the Hongmeng system, to obtain a calibrated unified time stamp.

[0052] Optionally, the scene perception decision system obtains the local time stamp (generated by the clock of each perception device, with clock deviation) carried in the original identified perception data generated in step 201, starts to call the distributed time service of the Hongmeng system, and obtains a global unified time stamp (based on a high-precision clock module in the cockpit or satellite time, with an error of ≤1 μs). Then, the deviation value of the local clock of each perception device from the global unified clock is calculated (obtained through the clock offset of the HDTS periodic synchronization), and then the original local time stamp is added with the corresponding deviation value to obtain the calibrated unified time stamp. During the calibration process, the change trend of the deviation value (such as updating the deviation value every 100 ms) is recorded, and if the deviation value exceeds a threshold value (such as 5 ms), the device clock is re-synchronized to ensure the calibration accuracy. Finally, each original identified perception data carries a calibrated unified time stamp.

[0053] Continuing the above embodiment, the local timestamp T1_local carried by the original identity perception data A (DMS_CAM_001_0120) is 1695000000.033s, the DMS camera clock bias value ΔT1 obtained by HDTS is 0.007s, the calibrated unified timestamp T1_cal is T1_local+ΔT1=1695000000.040s; the local timestamp T2_local carried by the original identity perception data B (FRONT_CAM_002_0085) is 1695000000.050s, the front windshield camera clock bias value ΔT2 is -0.010s, the calibrated unified timestamp T2_cal is T2_local+ΔT2=1695000000.040s; the local timestamp T3_local carried by the original identity perception data C (MMW_RADAR_003_0200) is 1695000000.020s, the millimeter wave radar clock bias value ΔT3 is 0.020s, and the calibrated unified timestamp T3_cal is T3_local+ΔT3=1695000000.040s. After calibration, the unified timestamps of the three groups of data are all 1695000000.040s, achieving time alignment.

[0054] In step 203, a time synchronization window is determined based on the time sensitivity of the cockpit core perception task, and the original identity perception data with calibrated timestamps is grouped according to the time window in which the calibrated unified timestamps fall, based on the time synchronization window, to obtain a multi-source data group.

[0055] Optionally, the scene-aware decision system analyzes the time sensitivity of the cockpit core perception tasks (such as driver attention monitoring and front road condition recognition determined in step 10), that is, different tasks have different requirements for data time accuracy, for example, driver attention monitoring requires real-time (time window ≤ 0.1s), and front road condition recognition can allow a slightly larger window (≤ 0.2s). According to the maximum value of the task time sensitivity, determine the "time synchronization window" (such as taking 0.1s to ensure that all task requirements are met), and use a sliding window mechanism, the window start time starts from the first calibration unified timestamp, and a new window is generated every interval window length (0.1s) (such as [1695000000.040, 1695000000.050], [1695000000.050, 1695000000.060]). Subsequently, the system traverses all original identification perception data carrying the calibration unified timestamp, judges the calibration unified timestamp of each data falling into the time synchronization window, groups the multi-source data in the same window into a group, and forms a multi-source data group. In the grouping process, if a certain window is missing data of a certain type of perception device (such as missing millimeter wave radar data), the device state monitoring function of the Hongmeng system is used to troubleshoot whether the device is abnormal, if the device is normal, then wait for the next frame of data (at most wait for 1 / 2 of the window length), if the device is abnormal, mark the group of data as "incomplete" and trigger the backup device call (if there is no backup device, skip this group).

[0056] Continue the above embodiment, the driver attention monitoring in the cockpit core perception task has the highest time sensitivity (0.1s), the scene-aware decision system determines the time synchronization window to be 0.1s, and the sliding window sequence is [1695000000.040, 1695000000.050], [1695000000.050, 1695000000.060]…

[0057] The original identification perception data A (T1_cal=1695000000.040s), B (T2_cal=1695000000.040s), and C (T3_cal=1695000000.040s) all fall into the first window [1695000000.040, 1695000000.050], and are grouped into a multi-source data group G1;

[0058] The subsequently received DMS camera 121stframe data (T_cal=1695000000.045s), front windshield camera 86thframe data (T_cal=1695000000.046s), and millimeter wave radar 201stframe data (T_cal=1695000000.048s) also fall into the window G1;

[0059] When the 122th frame data of the received DMS camera (T_cal=1695000000.051s) is received, it falls into the next window [1695000000.050, 1695000000.060), the multi-source data group G2 is started to be built. The system confirms that the G1 group contains multi-frame data of three types of equipment, and is marked as "complete data group".

[0060] In step 204, the time deviation value is obtained by calculating the deviation of the calibration unified timestamp of each piece of data in each multi-source data group from the center time of the group time window.

[0061] Optionally, the scene awareness decision system first determines the "time window center time" of each multi-source data group, and the center time is the average of the start time and the end time of the group time synchronization window (for example, the center time T_center=1695000000.045s of the window [1695000000.040, 1695000000.050]). Then, for each piece of original identification awareness data in each multi-source data group, the calibration unified timestamp is extracted, and the difference between the timestamp and the window center time is calculated to obtain the "time deviation value". The time deviation value is divided into positive and negative (a positive value indicates that the data timestamp is later than the center time, and a negative value indicates that it is earlier than the center time), and the deviation values of multi-frame data of the same device in the same group are counted (for example, the average value and the standard deviation are calculated), and if the standard deviation exceeds a threshold value (for example, 0.005s), it is determined that the device data has time jitter, and needs to be smoothed in subsequent preprocessing. The time deviation value will be an important metadata of the standardized awareness data, which is used for the judgment of data time consistency in the subsequent scene classification model.

[0062] Continue with the above embodiment, the multi-source data group G1 (window [1695000000.040, 1695000000.050), center time T_center=1695000000.045s). It is obtained that:

[0063] The time deviation value Δt1 of the original identification awareness data A (T1_cal=1695000000.040s) is 1695000000.040-1695000000.045=-0.005s;

[0064] The time deviation value Δt2 of the original identification awareness data B (T2_cal=1695000000.040s) is 1695000000.040-1695000000.045=-0.005s;

[0065] The time deviation value Δt3 of the original identification perception data C (T3_cal=1695000000.040s) is 1695000000.040-1695000000.045=-0.005s;

[0066] The deviation value Δt1' of the 121th frame data of the DMS camera in the G1 group (T_cal=1695000000.045s) is 0s, the deviation value Δt2' of the 86th frame data of the front windshield camera (T_cal=1695000000.046s) is 0.001s, and the deviation value Δt3' of the 201th frame data of the millimeter wave radar (T_cal=1695000000.048s) is 0.003s. The standard deviation of the data deviation values of each device in the G1 group is calculated: the standard deviation σ1 of the DMS camera is 0.0035s (≤0.005s), the standard deviation σ2 of the front windshield camera is 0.0038s (≤0.005s), and the standard deviation σ3 of the millimeter wave radar is 0.0042s (≤0.005s), so it is determined that there is no obvious time jitter.

[0067] In step 205, the original identification perception data, the unique combination identification, the calibrated unified timestamp and the time synchronization window are encapsulated based on the time deviation value, to obtain standardized perception data.

[0068] Optionally, the scene perception decision system obtains the standardized perception data according to the obtained time deviation value, the original identification perception data, the unique combination identification, the calibrated unified timestamp and the time synchronization window, according to a pre-defined standardized data encapsulation format, and the description of steps 2051-2054.

[0069] The embodiment of the application takes the distributed soft bus and time service of the Hongmeng system as technical support, solves the data traceability problem through the unique combination identification, solves the multi-device clock deviation problem through global time calibration, solves the data time correlation problem through the time synchronization window grouping, quantifies the data time consistency through the deviation calculation, and finally forms the standardized perception data through encapsulation. The problems of identification confusion, time asynchronization and poor data correlation in traditional data processing are effectively solved.

[0070] In an embodiment, the description of steps 2051-2054 is as follows:

[0071] In step 2051, the content of the original identification perception data is reconstructed based on the time deviation value and the adaptive difference value of the data acquisition frequency, to obtain reconstructed perception data; the data timestamp of the reconstructed perception data is the same as the window center time of the belonging multi-source data group.

[0072] Optionally, the scene-aware decision system first calculates an adaptive difference value according to the time deviation value calculated in step 204 (such as At = -0.005 s) and the acquisition frequency of the corresponding original identification perception data (such as the DMS camera acquisition frequency f = 30 fps, that is, one frame of data is generated every 33.3 ms). The adaptive difference value reflects the acquisition frame number corresponding to the time interval of the original data and the window center time of the multi-source data set, and its formula is adaptive difference value N = |At| x f (the result is an integer). Then, the original identification perception data content is reconstructed: if the original data timestamp is earlier than the window center time (At < 0), linear interpolation is performed based on the features (such as image data pixel brightness, point cloud data coordinate position) of the original data and the subsequent N frames of data to generate reconstructed data corresponding to the window center time; if the original data timestamp is later than the window center time (At > 0), interpolation reconstruction is performed based on the features of the original data and the previous N frames of data. During the reconstruction process, the continuity of the data features (such as no obvious jaggies on the edges of the reconstructed image, no distortion of the target contour after the point cloud reconstruction) is ensured, and finally the reconstructed perception data is obtained, whose data timestamp is strictly the same as the window center time of the corresponding multi-source data set, thereby eliminating the influence of time deviation on data correlation.

[0073] Continue with the above embodiment, in the multi-source data set G1 (window center time T_center = 1695000000.045 s):

[0074] Original identification perception data A (DMS_CAM_001_0120): time deviation value At1 = -0.005 s, acquisition frequency f1 = 30 fps, adaptive difference value N1 = |-0.005| x 30 = 0.15 -> rounded to 0 (that is, no cross-frame interpolation is needed, only the data features need to be fine-tuned). Therefore, based on the pixel features (such as driver face key point coordinates) of the data, the face position parameters (x coordinate + 2 pixels, y coordinate + 1 pixel) are fine-tuned to generate reconstructed perception data A1, and the timestamp is set to 1695000000.045 s;

[0075] Original identification perception data C (MMW_RADAR_003_0200): time deviation value At3 = -0.005 s, acquisition frequency f3 = 20 fps (one frame of data is generated every 50 ms), adaptive difference value N3 = |-0.005| x 20 = 0.1 -> rounded to 0. Therefore, the coordinates of the target vehicle in the point cloud data are fine-tuned (x coordinate + 0.01 m, y coordinate - 0.005 m) to generate reconstructed perception data C1, and the timestamp is set to 1695000000.045 s;

[0076] Front windshield camera 86th frame data (T_cal=1695000000.046s): time offset value Δt2'=0.001s, acquisition frequency f2=25fps (generate a frame of data every 40ms), adaptive difference N2=0.001*25=0.025→rounded to 0. Therefore, fine-tune the pixel coordinates of the lane line in the road condition image to generate reconstructed perception data B1, and the timestamp is set to 1695000000.045s.

[0077] And the timestamps of all reconstructed perception data are consistent with the window center time, ensuring data time synchronization.

[0078] Step 2052, for the reconstructed perception data, determine the abnormality degree of each road data based on the historical data distribution of the cockpit scene of the hyper- system, and replace each road data whose abnormality degree exceeds the preset abnormality degree threshold to obtain standard perception data without abnormality.

[0079] Optionally, the scene perception decision system calls the cockpit scene history database of the hyper- system, which stores the perception data distribution characteristics (such as the facial feature value distribution of the DMS camera image, the target distance distribution of the millimeter wave radar point cloud) under the same type of basic running scene (such as driving scene) in the past 3 months, including data mean μ, standard deviation σ and probability density function P(x). For each road of reconstructed perception data generated in step 2051, extract its core features (such as the driver's eye opening degree of A1, the lane line width of B1, and the target vehicle distance of C1), calculate the abnormality degree based on the historical data distribution, wherein the abnormality degree reflects the deviation degree of the data feature from the historical normal distribution, and the "Mahalanobis distance" algorithm is used for calculation (excluding the influence of feature correlation). If the abnormality degree exceeds the preset threshold (such as threshold=3, corresponding to 99.7% confidence interval), it is determined that the road data is abnormal data, and through the device redundancy management function of the hyper- system, the same period data collected by the standby perception device (such as standby DMS camera) or the substitute data generated based on the similar historical data is called to replace the abnormal data; if the abnormality degree does not exceed the threshold, the original reconstructed perception data is retained. Finally, "standard perception data without abnormality" is obtained, ensuring that the data quality meets the subsequent dimension normalization requirements.

[0080] Continuing the above example, reconstruct the perception data A1 (DMS image): extract the core feature "eye openness" x1 = 0.2 (0 is completely closed, 1 is completely open). Get the mean value μ1 = 0.8, the standard deviation σ1 = 0.15, and the covariance matrix Σ1 = 0.0225 (single feature scene) of the eye openness in the driving scene from the historical database. Calculate the anomaly degree (Mahalanobis distance) D1 = |x1 - μ1| / σ1 = |0.2 - 0.8| / 0.15 = 4 > 3 (threshold), and determine it as abnormal data. Call the image data (eye openness x1' = 0.75) collected by the backup DMS camera at the same time, and get the standard perception data A2 after replacement;

[0081] Reconstruct the perception data B1 (road condition image): extract the core feature "lane line width" x2 = 0.05 m. The mean value of the historical data is μ2 = 0.045 m, the standard deviation is σ2 = 0.008 m, and the anomaly degree D2 = |0.05 - 0.045| / 0.008 = 0.625 < 3, which is determined as normal data and retained as standard perception data B2.

[0082] Reconstruct the perception data C1 (radar point cloud): extract the core feature "target vehicle distance" x3 = 30.2 m. The mean value of the historical data is μ3 = 28 m, the standard deviation is σ3 = 3 m, and the anomaly degree D3 = |30.2 - 28| / 3 ≈ 0.733 < 3, which is determined as normal data and retained as standard perception data C2. Finally, the standard perception data set {A2, B2, C2} without anomaly is obtained.

[0083] Step 2053, for the standard perception data, based on the data dimensions of different multi-source perception devices, the data of different dimensions is converted to a unified cabin scene feature dimension space, and the dimension normalized converted perception data is obtained.

[0084] Optionally, the scene-aware decision system first analyzes the original data dimensions of different multi-source perception devices, such as DMS camera data with 2D image dimensions (width x height x channel, such as 1920 x 1080 x 3), millimeter wave radar data with 3D point cloud dimensions (point number x coordinate, such as 200 x 3), and front windshield camera data with 2D road image dimensions (1920 x 1080 x 3). And the pre-constructed "cabin scene feature dimension space" is a unified high-dimensional feature space (such as 1024 dimensions), which contains the core features of the cabin scene (such as driver state features, road condition features, and obstacle features), and each dimension corresponds to a standardized feature (such as driver distraction level, lane line integrity, and obstacle risk level). Then, a dimension conversion model (such as a feature extraction based on a convolutional neural network + fully connected layer mapping model) is called to convert the dimensions of each standard perception data. The specific process is as follows: first, extract the local features of the data (such as image edge features and point cloud clustering features) through the convolutional layer, and then map the local features to the unified cabin scene feature dimension space through the fully connected layer, to ensure that the original data of different dimensions have the same number of feature dimensions and data range after conversion (such as feature values normalized to the [0, 1] interval). And in the conversion process, through the AI computing power scheduling function of the Hongmeng system, the cabin edge computing module is called to accelerate feature extraction (time consumption ≤0.3s), and finally the converted perception data with normalized dimensions is obtained, realizing the feature alignment of multi-source data.

[0085] Continuing the above embodiment, the standard perception data set {A2, B2, C2} is obtained according to step 2052. For the standard perception data A2 (DMS image): the original dimension is 1080x1920x3 (2D image). The system extracts facial features (such as 68 features of eyes, mouth, head key points) through the CNN model, and then maps the 68-dimensional features to a 1024-dimensional cabin scene feature space through a fully connected layer, wherein the first 50 dimensions correspond to the driver state features (such as the first dimension is the normalized value 0.94 of the eye opening degree, and the second dimension is the normalized value 0.2 of the head deflection angle), to obtain the converted perception data A3 (1024 dimensions, feature values are in the interval [0, 1]); for the standard perception data B2 (road condition image): the original dimension is 1080x1920x3 (2D image). Extract road features (lane lines, traffic lights, front vehicles, a total of 80 features) through the CNN model, and map to a 1024-dimensional space, wherein the 51st-150th dimensions correspond to road features (such as the 51st dimension is the normalized value 0.98 of the lane line integrity, and the 52nd dimension is the normalized value 1.0 (green) of the traffic light color), to obtain the converted perception data B3; for the standard perception data C2 (radar point cloud): the original dimension is 200x3 (3D point cloud). Extract obstacle features (target number, distance, speed, a total of 40 features) through the PointNet model, and map to a 1024-dimensional space, wherein the 151st-200th dimensions correspond to obstacle features (such as the 151st dimension is the normalized value 0.52 of the target vehicle distance, and the 152nd dimension is the normalized value 0.17 of the relative speed), to obtain the converted perception data C3. Finally, A3, B3, and C3 are all 1024-dimensional feature vectors, realizing dimension normalization.

[0086] Step 2054, based on the converted perception data, combine the unique combination identifier, the calibrated unified timestamp, and the time synchronization window, and encapsulate according to the format requirements of the Hongmeng system cabin data interaction protocol to obtain standardized perception data.

[0087] Optionally, the scene-aware decision system first determines the format requirements of the cockpit data interaction protocol of the Hongmeng system, which specifies the field order of data encapsulation, data type (such as string, floating point number, integer), encoding method (such as UTF-8, Base64), and verification rule (such as CRC32, MD5). Among them, the core fields of the protocol include: basic information field: unique combination identifier (such as DMS_CAM_001_0120), data group number (such as G1), data generation time (window center time); Time-related fields: calibrated universal timestamp, time synchronization window, time deviation value; Data content field: converted perception data (1024-dimensional feature vector, stored in JSON array format), original data type (such as “DMS image” “Radar point cloud”); Verification field: data check code, integrity mark (such as “complete” “partially missing”). Then, according to the protocol format, the unique combination identifier of step 201, the calibrated universal timestamp of step 202, the time synchronization window of step 203, the time deviation value of step 204, and the converted perception data of step 2053 are integrated and encapsulated. Base64 encoding is performed on the data content field (to reduce transmission bandwidth occupancy), and CRC32 check code is generated (to ensure data integrity). After encapsulation, the “standardized perception data” is sent to the scene classification module through the distributed data transmission function of the Hongmeng system, and the transmission log (such as transmission time, reception status) is recorded to ensure data traceability and verifiability.

[0088] Continuing the above embodiment, for the converted perception data A3, the scene-aware decision system encapsulates according to the Hongmeng cockpit data interaction protocol:

[0089] Basic information field: unique combination identifier “DMS_CAM_001_0120”, data group number “G1”, data generation time “1695000000.045s”;

[0090] Time-related fields: calibrated universal timestamp “1695000000.040s”, time synchronization window “[1695000000.040, 1695000000.050)”, time deviation value “-0.005s”;

[0091] Data content field: converted perception data A3 (1024-dimensional feature vector, JSON array format: [0.94, 0.2, …, 0.05]), original data type “DMS image”, Base64 encoded as “eyJmZWF0dXJlcyI6WzAuOTQsMC4yLC4uLjAsMC4wNV19”;

[0092] Check field: CRC32 check code "0x9ABCDEF0", integrity mark "complete". Similarly, complete the packaging of B3 and C3 to generate standardized perception data set {SA3, SB3, SC3}, which is transmitted to the scene classification module through the distributed soft bus of the Hongmeng system, and the transmission delay is ≤15ms and the packet loss rate is ≤0.05%.

[0093] The embodiment of the application takes the historical data support of the Hongmeng system, the AI computing power scheduling and the data interaction protocol as the technical basis, eliminates the time deviation through data reconstruction, guarantees the data quality through the abnormality degree determination, realizes the multi-source data feature alignment through the dimension normalization, and finally generates the standardized perception data conforming to the interaction standard of the Hongmeng system through the protocol packaging. The core problems of time asynchronization, data abnormality and dimension non-uniformity in the traditional data processing are solved, the high-value data input with the unified format, reliable quality and aligned features is provided for the scene classification of step 30, and the scene classification precision and the decision response speed are improved.

[0094] In an embodiment, steps 301-305 are described as follows:

[0095] In step 301, the matching degree screening is performed based on the scene inherent features in the basic running scene in combination with the standardized perception data, and the anchor feature group corresponding to each basic running scene is obtained.

[0096] Optionally, the scene-aware decision system first identifies the scene-inherent characteristics of different basic running scenes, which are the essential attributes of the basic scenes with stability and uniqueness, such as the inherent characteristics of the driving scene including "vehicle speed > 0 km / h, gear position in non-P range, parking brake not activated"; the inherent characteristics of the parking scene including "vehicle speed = 0 km / h, gear position in P range, parking brake activated"; the inherent characteristics of the temporary parking scene including vehicle speed = 0 km / h, gear position in N range, parking brake not activated (or activated for ≤5 min). Therefore, after receiving the standardized perception data, the parameters related to the scene-inherent characteristics in the data (such as vehicle speed, gear position, parking brake state, etc.) are extracted, the matching degree of the standardized perception data and the inherent characteristics of each basic scene is calculated through the feature matching degree formula, the basic running scene determined in step 10 is verified again to exclude the error of the initially determined basic running scene. Among them, the matching degree calculation covers all inherent characteristic dimensions, and if the matching degree of the inherent characteristics of a basic scene and the standardized perception data ≥ 95% (preset threshold), the "core feature set" corresponding to the basic scene is determined as the "anchoring feature group". The anchoring feature group includes the key feature dimensions required for scene classification under the basic scene, for example, the anchoring feature group of the driving scene includes "driver state features (eye closure time, head deflection angle), road conditions (lane line type, traffic signal color, distance to the front vehicle), obstacle features (obstacle type, relative speed)"; the anchoring feature group of the parking scene includes "in-vehicle personnel features (number of personnel, activity state), environmental features (temperature, humidity), voice features (instruction keywords)".

[0097] Further, the feature matching degree formula is as follows:

[0098] ;

[0099] Among them, is the feature matching degree (value range , 1 represents complete matching); is the number of dimensions of the inherent characteristics of the basic scene (such as the driving scene : speed, gear, parking brake); is the weight of the th feature dimension, and the weight of the key feature is higher, such as the weight of the speed ; is the real-time value of the th feature in the standardized perception data; is the standard value set of the th feature in the inherent characteristics of the basic scene; Similarity of real-time value and standard value set (numerical features are normalized by Euclidean distance, and categorical features are matched as 1 or 0). Maximum value of similarity (ensuring the value of this item is 1, avoiding the denominator affecting the weight).

[0100] In an embodiment, the scene-related parameters extracted in the standardized perception data are taken as examples: vehicle speed = 60 km / h (> 0 km / h), gear = D gear (not P gear), parking brake = not activated. First, the matching degrees of the parameters with the inherent characteristics of each basic scene are calculated: the matching degree with the inherent characteristics of the driving scene: vehicle speed matching (100%), gear matching (100%), parking brake matching (100%), total matching degree = 100% ≥ 95%; the matching degree with the inherent characteristics of the parking scene: vehicle speed not matching (0%), gear not matching (0%), parking brake not matching (0%), total matching degree = 0%; the matching degree with the inherent characteristics of the temporary parking scene: vehicle speed not matching (0%), gear not matching (0%), total matching degree = 0%. Therefore, the basic running scene is determined to be the driving scene, and the corresponding anchor feature group is: driver state feature dimension: eye closure duration, head deflection angle; road condition feature dimension: lane line type, traffic signal light color, distance to front vehicle, speed of front vehicle; obstacle feature dimension: obstacle type (vehicle / person / no), obstacle distance, relative speed.

[0101] In step 302, for the anchor feature group, the dynamic deviation of the standardized perception data from the anchor feature group during real-time collection is calculated to obtain a dynamic perception deviation value.

[0102] Optionally, the scene perception decision system extracts the standard threshold range of each feature dimension from the anchor feature group determined in step 301, which is determined based on the industry standards stored in the Hongmeng system and a large amount of historical data, for example, in the driving scene: driver state feature: eye closure duration standard threshold range [0, 0.3s], head deflection angle standard threshold range [0, 15°]. Then, the real-time collection values of each feature dimension in the standardized perception data are extracted, and the "dynamic deviation value" of each feature real-time value from the corresponding standard threshold range is calculated. For numerical features (such as eye closure duration and head deflection angle), the deviation value is the difference between the real-time value and the upper limit of the standard threshold (if the real-time value exceeds the upper limit, it is a positive value, otherwise it is 0); for categorical features (such as traffic signal light color and obstacle type), the deviation value is the "category deviation weight" (such as the deviation weight = 0 when the current traffic signal light is green, and the deviation weight = 1 when the current traffic signal light is red and the vehicle does not slow down). Finally, the dynamic deviation values of all feature dimensions are normalized (mapped to the [0, 1] interval) to obtain a comprehensive dynamic perception deviation value, which is used to quantify the deviation degree of real-time data from the standard state.

[0103] Continuing the above embodiment, the driving scene anchoring feature group, the real-time feature value of the standardized perception data is as follows:

[0104] Driver state features: eye closure duration t = 0.5 s, head deflection angle θ = 30°;

[0105] Road conditions: current speed = 60 km / h, safe distance = 60 x 1.5 / 3.6 ≈ 25 m, front vehicle distance = 50 m, front vehicle speed = 30 km / h, traffic signal color = green;

[0106] Obstacle features: side rear obstacle type = sedan, distance = 30 m, relative speed = 10 km / h (same direction as the current vehicle).

[0107] The process of calculating the dynamic deviation value of each feature is as follows: eye closure duration deviation value: t - standard upper limit = 0.5 - 0.3 = 0.2 s, normalized = 0.2 / (maximum possible deviation 1 s) = 0.2; head deflection angle deviation value: θ - standard upper limit = 30 - 15 = 15°, normalized = 15 / (maximum possible deviation 45°) = 0.33; front vehicle distance deviation value: safe distance 25 m ≤ 50 m, no deviation, normalized = 0; front vehicle speed deviation value: current speed 60 km / h > front speed 30 km / h, there is a risk of rear-end collision, deviation value = (60 - 30) / (maximum possible deviation 60 km / h) = 0.5; traffic signal color deviation value: green corresponds to normal traffic, deviation value = 0; obstacle relative speed deviation value: 10 km / h ≤ 60 + 10 = 70 km / h, no deviation, deviation value = 0. Then the comprehensive dynamic perception deviation value = (0.2 + 0.33 + 0 + 0.5 + 0 + 0) / 6 ≈ 0.17 (normalized), reflecting that there is a certain deviation in the real-time data.

[0108] Step 303, based on the anchoring feature group, the basic running scene is logically disassembled to obtain multiple initial sub-scenes belonging to the basic running scene, and the initial sub-scenes are verified based on the historical standard perception data and historical scene classification data stored by the Hongmeng system to obtain potential sub-scenes.

[0109] Optionally, the scene-aware decision system logically disassembles the basic running scene based on the anchor feature group in step 301, that is, generates a plurality of initial sub-scenes according to the mapping relationship of feature dimension-feature state. For example, in the driving scene, based on the combination of "driver state feature" and "road condition feature": driver state feature state: normal (deviation value = 0), distraction (deviation value > 0); Road condition feature state: normal (no congestion, normal traffic light), congestion (front vehicle distance < safe vehicle distance), emergency (traffic light turns red, sudden obstacles); Combination generates initial sub-scenes: normal driving scene, driver distraction scene, front congestion scene, emergency avoidance scene, distraction + congestion scene, etc. Then, call the "historical standard perception data set" and "historical scene classification data set" stored in the Hongmeng system to verify each initial sub-scene: if the frequency of an initial sub-scene in the historical data is ≥1000 times (preset threshold) and the historical classification accuracy is ≥90%, it is determined as a "potential sub-scene"; if the frequency is <1000 times or the classification accuracy is <90%, it is determined as an "invalid sub-scene" (such as "distraction + emergency avoidance + extreme weather" which is a low-frequency scene), and is excluded. Finally, the potential sub-scene that meets the actual application requirements is obtained, providing a candidate range for subsequent scene classification.

[0110] Continue the above facts, the initial sub-scenes of the driving scene in step 301 based on the anchor feature group logical disassembly are as follows:

[0111] 1. Normal driving scene: normal driver state, normal road condition, no emergency obstacle;

[0112] 2. Driver distraction scene: driver state deviation (eyes closed / head deflection), normal road condition; 3. Front congestion scene: normal driver state, front vehicle distance < safe vehicle distance, low front vehicle speed; 4. Emergency avoidance scene: normal driver state, sudden obstacle / traffic light turns red;

[0113] 5. Distraction + congestion scene: driver state deviation, front congestion;

[0114] 6. Extreme weather + normal driving scene: normal driver state, road condition affected by extreme weather (low frequency).

[0115] Call the Hongmeng system historical data verification as follows:

[0116] Initial sub-scenes 1-5: historical frequency ≥5000 times, classification accuracy ≥92%, determined as potential sub-scenes;

[0117] Initial sub-scenario 6: Historical occurrence frequency = 800 times < 1000 times, judged as an invalid sub-scenario and removed. The potential sub-scenarios for the final driving scenario are: normal driving scenario, driver distraction scenario, traffic congestion scenario ahead, emergency avoidance scenario, and distraction + traffic congestion scenario.

[0118] Step 304: Based on the correlation between the dynamic perception deviation value and each potential sub-scenario under the basic operating scenario, determine the scenario correlation factor; the scenario correlation factor quantifies the direction of the influence of the dynamic perception deviation value on scenario classification.

[0119] Optionally, the scene perception decision-making system calculates the scene correlation factor for each dynamic perception deviation feature dimension to each potential sub-scene using a correlation strength formula. This correlation factor ranges from -1 to 1, where 1 represents a strong positive influence, -1 represents a strong negative influence, and 0 represents no influence. During the calculation, the weights of the feature dimensions in the anchored feature group (e.g., driver state feature weight = 0.4, road condition feature weight = 0.3, obstacle feature weight = 0.3) are considered to ensure the rationality of the correlation factor. Finally, a set of scene correlation factors corresponding to each potential sub-scene is obtained, providing a quantitative basis for subsequent scene selection.

[0120] Furthermore, the formula for correlation strength is as follows:

[0121] ;

[0122] in, For potential sub-scenes Total scene correlation factor (range of values) ; The number of feature dimensions for dynamically sensing deviation values; For the first The weights of each feature dimension in the anchored feature group; For the first Normalized dynamic bias values ​​for each feature dimension; For the first The maximum normalization bias value of each feature dimension (ensure that this value is 1); For the associated direction sign function (when feature With child scene The value is 1 for a positive association, -1 for a negative association, and 0 for no association. Features With child scene The correlation (obtained based on historical data training, such as positive correlation, negative correlation, no correlation).

[0123] Continuing the above example, the dynamic perception deviation value feature dimensions (eye closure duration deviation D1 = 0.2, head deflection angle deviation D2 = 0.33, front vehicle speed deviation D3 = 0.5) and the potential sub-scenarios of step 303, the scene perception decision system calculates the scene correlation factor as follows:

[0124] Potential sub-scenario "driver distraction scenario": D1 correlation factor: driver distraction is related to eye closure, positive influence, combined weight 0.4, factor F1 = 0.4 x (D1 / 0.5) = 0.4 x 0.4 = 0.16 (0.5 is the maximum deviation of D1); D2 correlation factor: head deflection is related to distraction, positive influence, factor F2 = 0.4 x (D2 / 0.67) = 0.4 x 0.49 ≈ 0.196 (0.67 is the maximum deviation of D2); D3 correlation factor: front vehicle speed deviation has no direct correlation with distraction, factor F3 = 0; finally, the total correlation factor F = 0.16 + 0.196 + 0 = 0.356 (positive, indicating that the deviation value promotes the scenario);

[0125] Potential sub-scenario "front congestion scenario": D1 correlation factor: eye closure is not related to congestion, F1 = 0; D2 correlation factor: head deflection is not related to congestion, F2 = 0; D3 correlation factor: front vehicle speed deviation is positively related to congestion, combined weight 0.3, factor F3 = 0.3 x (D3 / 1) = 0.15 (1 is the maximum deviation of D3); finally, the total correlation factor F = 0 + 0 + 0.15 = 0.15 (positive, weak influence);

[0126] Potential sub-scenario "normal driving scenario": D1, D2 correlation factors are negative (the greater the deviation value, the greater the deviation from normal), D3 correlation factor is negative (vehicle speed deviation is large, deviating from normal), total correlation factor F = -0.28 (negative, indicating that the deviation value inhibits the scenario).

[0127] Step 305, based on the scene correlation factor and the coupling relationship between different core features in the anchor feature group, a coupling degree matrix is constructed, and based on the coupling degree matrix, dynamic perception deviation value, scene correlation factor, potential sub-scenarios are screened and verified to obtain the current scene type.

[0128] Optionally, the scene-aware decision system analyzes the coupling relationship between different core features according to the anchor feature group and the scene correlation factor of step 301. For example, in the driving scene, there is a coupling relationship between "driver distraction" and "front congestion" (distraction may cause a delay in response to congestion), and there is a coupling relationship between "traffic light color" and "relative speed" (the relative speed should be reduced when the light is red). The coupling relationship is quantified as a "coupling degree matrix", where the matrix element C(i,j) represents the coupling strength between the ith feature and the jth feature (the value range is [0, 1], 1 represents complete coupling, and 0 represents no coupling). Then, combined with the coupling degree matrix, the dynamic perception bias value of step 302, and the scene correlation factor of step 304, the potential sub-scene is screened and verified to obtain the current scene type. The specific description is as steps 3051-3054.

[0129] Continuing the above embodiment, the scene-aware decision system first constructs the coupling degree matrix of the driving scene (simplified as three core features: D1-eye closing bias, D2-head deflection bias, and D3-front vehicle speed bias) based on the potential sub-scene and the correlation factor of step 304.

[0130]

[0131] Therefore, the coupling degree of D1 and D2 is high, because they both belong to the driver distraction feature; and the coupling degree of D3 is low.

[0132] The embodiment of the application is supported by the historical data of the Hongmeng system and the feature coupling relationship. The scene-aware decision system locks the classification range through the anchor feature group, quantifies the deviation degree of real-time data through the dynamic perception bias value, screens out the potential sub-scene with high feasibility through sub-scene disassembly and verification, determines the influence direction of the bias value on the scene classification through the scene correlation factor, and finally realizes accurate screening through the coupling degree matrix and the scene matching score. The embodiment of the application fully utilizes the advantages of historical data of the Hongmeng system and the multi-feature cooperation capability, effectively solves the problems of fuzzy range, unquantized bias, invalid sub-scene, and no correlation in traditional scene classification, and ensures the accuracy and reliability of the classification result.

[0133] In an embodiment, steps 3051-3055 are described as follows:

[0134] Step 3051, based on the coupling degree matrix, the core feature combination satisfying the coupling degree threshold is screened out, and based on the core feature combination, the scene correlation factor, and the potential sub-scene, the potential sub-scene corresponding to each core feature combination is determined to form a sub-scene candidate group.

[0135] Optionally, the scene-aware decision system extracts the coupling strength values between all features from the coupling degree matrix constructed in step 305, and sets a pre-defined "coupling degree threshold" (e.g. 0.6, determined based on historical scene classification accuracy verification, and coupling strength ≥ 0.6 indicates that there is a strong association between features, which can form an effective feature combination). The scene-aware decision system traverses the coupling degree matrix and filters out all feature pairs or feature sets that meet the coupling degree threshold to form a core feature combination. The core feature combination meets the principles of strong coupling between features and covering key classification dimensions of sub-scenes. For example, in the driving scene, the coupling strength between "eye closure duration deviation (D1) and head deflection angle deviation (D2)" is 0.8 ≥ 0.6, which can form a "driver state core feature combination"; the coupling strength between "front vehicle speed deviation (D3) and front vehicle distance deviation (D4)" is 0.7 ≥ 0.6, which can form a "road condition core feature combination".

[0136] Further, the scene-aware decision system determines the relevance of each core feature combination to each potential sub-scene based on the scene association factor in step 304. If the absolute value of the scene association factor corresponding to a core feature combination is ≥ 0.2 (a pre-defined association threshold, indicating that the feature combination has a significant impact on the sub-scene), the potential sub-scene and the core feature combination are bound to form a sub-scene candidate group. Each sub-scene candidate group must include the three elements of "core feature combination + associated potential sub-scene + association factor". For example, "driver state core feature combination (D1+D2) + driver distraction scene + association factor 0.356" constitutes a sub-scene candidate group, and "road condition core feature combination (D3+D4) + front congestion scene + association factor 0.15" constitutes another sub-scene candidate group. The final sub-scene candidate groups must exclude duplicate or weakly associated combinations (e.g. combinations with an association factor absolute value < 0.2) to ensure the effectiveness and relevance of the candidate groups.

[0137] Continuing with the above example, the driving scene coupling degree matrix in step 305 (supplement D4 - front vehicle distance deviation, coupling strength with D3 = 0.7), the scene-aware decision system sets the coupling degree threshold = 0.6.

[0138] Filtering core feature combinations: D1 (eye closure deviation) and D2 (head deflection deviation) coupling strength = 0.8 ≥ 0.6, forming a "driver state core feature combination (D1+D2)"; D3 (front vehicle speed deviation) and D4 (front vehicle distance deviation) coupling strength = 0.7 ≥ 0.6, forming a "road condition core feature combination (D3+D4)"; D1 and D3 coupling strength = 0.1 < 0.6, not forming a combination.

[0139] Combined with the scene association factors in step 304 (driver distraction scene association factor 0.356, front congestion scene 0.15, normal driving scene -0.28):

[0140] Driver state core feature combination (D1+D2): correlation factor 0.356≥0.2, bind "driver distraction scene", form sub-scene candidate group G1: {D1+D2, driver distraction scene, 0.356};

[0141] Road condition core feature combination (D3+D4): correlation factor 0.15<0.2, not bound for the time being;

[0142] Supplementary verification: driver state core feature combination and normal driving scene correlation factor absolute value 0.28≥0.2, bind "normal driving scene", form sub-scene candidate group G2: {D1+D2, normal driving scene, -0.28};

[0143] Road condition core feature combination and front congestion scene correlation factor 0.15<0.2, secondary verification by historical data of the Hongmeng system (the contribution of this combination to the congestion scene classification is ≥30%), adjust the correlation threshold to 0.15, bind "front congestion scene", form sub-scene candidate group G3: {D3+D4, front congestion scene, 0.15}.

[0144] Finally, the sub-scene candidate group is {G1, G2, G3}.

[0145] Step 3052, for the sub-scene candidate group, based on the fitness of each sub-scene to the dynamic perception deviation value, the optimal candidate sub-scene is obtained.

[0146] Optionally, the scene-aware decision system calculates the fitness of each sub-scene candidate group formed in step 3051 to the dynamic perception deviation value of step 302, which quantifies the matching degree of the real-time deviation value of the core feature combination to the sub-scene classification requirement. The specific calculation process is as follows: first, extract the real-time deviation value corresponding to the core feature combination in the dynamic perception deviation value, calculate the combined deviation value (such as the combined deviation value of the driver state combination D1+D2 = (D1+D2) / 2, take the average value); then, combined with the scene correlation factor of the sub-scene candidate group, the deviation-correlation synergy formula is used to calculate the fitness, if the correlation factor is positive (such as G1's 0.356), the fitness increases with the increase of the combined deviation value; if the correlation factor is negative (such as G2's -0.28), the fitness decreases with the increase of the combined deviation value. The fitness value range is [0, 1], the higher the value, the stronger the fitness. Then, the fitness of all sub-scene candidate groups is sorted, and the candidate group with the highest fitness and exceeding the fitness threshold (such as 0.5) is selected, and the corresponding sub-scene is the "optimal candidate sub-scene". If there are multiple candidate groups with the same fitness and all exceeding the threshold, the "sub-scene priority rule" stored in the Hongmeng system (such as "the priority of safety-related sub-scene is higher than that of comfort-related sub-scene" in driving scene) is used to determine the optimal candidate sub-scene, to ensure the uniqueness and rationality of the screening result.

[0147] Further, the deviation-correlation synergy formula is as follows:

[0148] ;

[0149] Wherein, is the sub-scene fitness (value range ; is the average deviation value (normalized) of the core feature combination; is the sub-scene correlation factor; is the maximum value of all sub-scene correlation factors; is the maximum value of the absolute value of all sub-scene correlation factors.

[0150] Continuing the above example, the sub-scene candidate group {G1, G2, G3}, the dynamic perception deviation values of step 302: D1=0.2, D2=0.33, D3=0.5, D4=0 (no deviation for front vehicle distance). Sub-scene candidate group G1 (D1+D2, driver distraction scene, 0.356): combined deviation value = (0.2+0.33) / 2=0.265; adaptation degree = combined deviation value x correlation factor / maximum correlation factor = 0.265 x 0.356 / 0.356=0.265 (since the correlation factor is positive, it is directly related); sub-scene candidate group G2 (D1+D2, normal driving scene, -0.28): combined deviation value = 0.265; adaptation degree = (1-combined deviation value) x |correlation factor| / |maximum negative correlation factor|= (1-0.265) x 0.28 / 0.28=0.735 (since the correlation factor is negative, the adaptation degree is negatively correlated with the combined deviation value); sub-scene candidate group G3 (D3+D4, front congestion scene, 0.15): combined deviation value = (0.5+0) / 2=0.25; adaptation degree = 0.25 x 0.15 / 0.356≈0.105 (the correlation factor is small, the adaptation degree is low). The adaptation degree ranking: G2 (0.735) > G1 (0.265) > G3 (0.105), but the normal driving scene corresponding to G2 is contradictory to the dynamic perception deviation value (the driver is distracted). The system calls the sub-scene priority rule ("the safety risk sub-scene priority is higher than the normal scene"), recalculates the adaptation degree of G1 (corrected to 0.65, because the distraction scene involves safety risk), and finally G1 adaptation degree 0.65 > G2 adaptation degree 0.735 (after correction, G2 adaptation degree is reduced to 0.3 due to scene contradiction), determines the optimal candidate sub-scene as "driver distraction scene".

[0151] Step 3053, based on the optimal candidate sub-scene at the current time, the scene classification result of the previous time is verified for time sequence consistency, and the time sequence consistency coefficient of the optimal candidate sub-scene at the current time is obtained.

[0152] Optionally, the scene perception decision system obtains the scene classification result of the previous time, the previous time refers to the previous data collection period (such as 0.1s, consistent with the time synchronization window length) of the current time, and the classification result is stored in the scene log database of the Hongmeng system (including scene type, classification time, and associated feature deviation value). Subsequently, the time sequence consistency coefficient of the optimal candidate sub-scene at the current time and the scene classification result of the previous time is calculated, which reflects the continuity of the scene in the time dimension, avoiding frequent switching of scene classification due to instantaneous data fluctuations (such as the previous time being "normal driving scene", and the current time being suddenly determined as "emergency avoidance scene", which needs to be verified whether there is real scene change). Among them, the time sequence consistency coefficient is determined according to the following two dimensions:

[0153] Scenario type relevance: If the current optimal candidate sub-scenario is the same as the previous time scenario type (e.g., both are "driver distraction scenarios"), the relevance coefficient = 1; if the types are different but there is a logical transition relationship (e.g., the previous time "normal driving scenario" -> current "driver distraction scenario", there is a logical relationship of gradual distraction occurrence), the relevance coefficient = 0.6; if the types are different and there is no logical transition (e.g., the previous time "normal driving scenario" -> current "emergency avoidance scenario"), the relevance coefficient = 0.2;

[0154] Characteristic bias value continuity: Calculate the similarity (e.g., Euclidean distance normalization) of the current core feature combination bias value and the corresponding bias value at the previous time. The higher the similarity, the closer the continuity coefficient to 1.

[0155] The final timing consistency coefficient is the product of the relevance coefficient and the continuity coefficient, with a value range of [0, 1], and the higher the value, the stronger the scenario timing consistency.

[0156] Continue with the above example, the optimal candidate sub-scenario "driver distraction scenario" (current time T = 1695000000.045s). The scenario classification result at the previous time T-1 = 1695000000.035s is "normal driving scenario", the core feature combination (D1+D2) bias value = (0.05+0.08) / 2 = 0.065; scenario type relevance: the current "driver distraction scenario" and the previous "normal driving scenario" have a logical transition (distraction from none to some), the relevance coefficient = 0.6; characteristic bias value continuity: the current combination bias value = 0.265, the previous time = 0.065, the similarity = 1- |0.265-0.065| / max(0.265, 0.065) = 1-0.2 / 0.265≈0.245, the continuity coefficient = 0.245; timing consistency coefficient = relevance coefficient x continuity coefficient = 0.6 x 0.245≈0.147. At this time, verification: the D1 bias value from the previous time to the current time rises from 0.05 to 0.2, D2 rises from 0.08 to 0.33, the bias value shows a gradually increasing trend (consistent with the logical relationship of gradual distraction occurrence), the continuity coefficient is corrected to 0.4, and the final timing consistency coefficient = 0.6 x 0.4 = 0.24.

[0157] Step 3054, if the timing consistency coefficient of the optimal candidate sub-scenario meets the consistency threshold, the optimal candidate sub-scenario is determined as the current scenario type; if the timing consistency coefficient of the optimal candidate sub-scenario does not meet the consistency threshold, the scenario relevance factor is backtracked based on the sub-scenario candidate group until the consistency threshold is met, and the optimal candidate sub-scenario that meets the consistency threshold is determined as the current scenario type.

[0158] Optionally, the scene awareness decision system first sets a timing consistency threshold (such as 0.3, determined based on historical scene switching data, and a coefficient ≥ 0.3 indicates that the scene change conforms to the timing logic and there is no abnormal fluctuation), and compares the timing consistency coefficient calculated in step 3053 with the threshold. If the coefficient meets the consistency threshold, the current optimal candidate sub-scene is directly determined as the current scene type, the classification result is stored in the Hongmeng system scene log database, and the decision strategy matching of the subsequent step 40 is triggered. If the coefficient does not meet the consistency threshold, it is determined that the timing consistency of the current optimal candidate sub-scene is insufficient (such as being caused by instantaneous data error or abnormal interference), the scene correlation factor backtracking adjustment process is started, and the scene correlation factors of each candidate group are recalculated based on the sub-scene candidate group:

[0159] Backtracking basis: core feature deviation value of previous moment scene, change trend of current moment deviation value (such as whether the deviation value continuously increases / decreases);

[0160] Adjustment rule: if the deviation value continuously increases (such as D1 from 0.05→0.2→0.3), the correlation factor of the corresponding sub-scene is increased; if the deviation value fluctuates (such as D1 from 0.05→0.2→0.1), the correlation factor is reduced;

[0161] Iterative verification: after adjusting the correlation factor, the adaptation degree and the timing consistency coefficient of the sub-scene candidate group are recalculated until the coefficient ≥ 0.3. At this time, the optimal candidate sub-scene corresponding to the coefficient is determined as the current scene type. During the backtracking process, the number of iterations is limited (such as a maximum of 3 times). If the coefficient still does not meet the threshold after 3 iterations, the cloud scene verification service of the Hongmeng system is called to assist in judgment combined with more historical data to ensure the reliability of the final scene classification result.

[0162] Continue with the above embodiment, the timing consistency coefficient of step 3053 is 0.24 < threshold 0.3, and the scene awareness decision system starts the correlation factor backtracking:

[0163] First backtracking:

[0164] Analyze the change trend of the deviation value: D1 from 0.02 at T-2=1695000000.025s→0.05 at T-1=0.05→0.2 at present (continuously increasing), D2 from 0.03→0.08→0.33 (continuously increasing), and it is determined that the distraction trend is significant; adjust the correlation factor of G1 (driver distraction scene): from 0.356 to 0.5; recalculate the adaptation degree: G1 adaptation degree=0.265×0.5 / 0.5=0.265→0.7 after correction (due to trend verification); recalculate the timing consistency coefficient: correlation coefficient=0.6 (logical transition), continuity coefficient=0.5 (trend continues, after correction), coefficient=0.6×0.5=0.3≥0.3;

[0165] Threshold judgment: the coefficient 0.3 meets the threshold, stop backtracking, determine the "driver distraction scene" as the current scene type, and store it to the scene log database, trigger the decision strategy matching of step 40 (such as starting the sound alarm). If the coefficient still does not meet after the first backtracking (such as coefficient = 0.28), adjust the correlation factor of the road condition core feature combination for the second time, until the coefficient meets the standard.

[0166] The embodiment of the application focuses on the key classification dimension through core feature combination screening, excludes weakly associated sub-scenes through adaptability calculation, avoids frequent scene switching through time sequence consistency verification, and finally solves the problems of feature dimension redundancy, poor time sequence consistency and instantaneous error interference in traditional scene classification, ensuring the accuracy of the current scene type.

[0167] In an embodiment, steps 401-405 are described as follows:

[0168] Step 401, based on the current scene type, extracting the feature parameters associated with the scene type from the standardized perception data, and constructing the current scene feature group.

[0169] Optionally, the scene perception decision system determines the core feature dimensions that need to be extracted under the scene type according to the pre-defined "scene-feature mapping rule". For example, the core feature dimensions of the "driver distraction scene" include "driver state features" (eye closure time, head deflection angle, distraction duration), "vehicle running features" (current speed, lane deviation degree, distance from the front vehicle), and "environmental features" (light intensity, whether in a highway section), which directly affect the matching direction of the decision strategy (such as stronger warning for distraction on a highway, and appropriate reduction of warning intensity on a low-speed road). Then, from the standardized perception data generated in step 20, the real-time parameters corresponding to the above core feature dimensions are filtered and extracted: for "driver state features", the eye closure time (such as 0.5s) and head deflection angle (such as 30°) are extracted from the DMS camera standardized image, and the distraction duration (such as 2s) is calculated from the time sequence data; for "vehicle running features", the current speed (such as 60km / h) is extracted from the vehicle CAN bus standardized data, the lane deviation degree (such as no deviation, deviation value is 0) is calculated from the front windshield camera standardized data, and the distance from the front vehicle (such as 50m) is extracted from the millimeter wave radar standardized data; for "environmental features", the light intensity (such as 5000lux) is extracted from the light sensor standardized data, and whether it is in a highway section (such as no, in an urban road) is determined from the navigation system standardized data.

[0170] Finally, the extracted real-time parameters are integrated in the format of "feature dimension-parameter value-data source" to construct the "current scene feature group", ensuring that the feature group covers all key information required for scene decision-making, and that the parameter values are all standardized data (such as time unit unified as s, speed unit unified as km / h), providing accurate feature input for subsequent strategy matching.

[0171] In an embodiment, in the "driver distraction scene", the scene perception decision-making system determines the core feature dimensions and corresponding parameters to be extracted based on the "scene-feature mapping rules" as follows:

[0172] Driver state features:

[0173] Eye closure duration: extracted from DMS camera standardized image, value is 0.5s (after standardization, the original data has been removed noise);

[0174] Head deflection angle: extracted from DMS camera standardized image, value is 30° (exceeds 15° distraction threshold);

[0175] Distraction duration: calculated based on time series standardized data, has lasted for 2s since the first detection of distraction;

[0176] Vehicle operating features:

[0177] Current speed: extracted from vehicle CAN bus standardized data, value is 60km / h (common speed on urban roads);

[0178] Lane deviation degree: extracted from front windshield camera lane line recognition result, value is 0 (no lane deviation, after standardization, deviation value range is [0, 1], 0 represents no deviation);

[0179] Distance from the front vehicle: extracted from millimeter wave radar standardized point cloud data, value is 50m (greater than safe distance 25m);

[0180] Environmental features:

[0181] Light intensity: extracted from in-vehicle light sensor standardized data, value is 5000lux (normal daytime light);

[0182] Whether it is a highway section: determined from the navigation system standardized data, value is "no" (currently on urban trunk road).

[0183] The above parameters are integrated to construct the current scene feature group F: F={driver state features: {eye closure duration: 0.5 s, head deflection angle: 30°, distraction duration: 2 s}, vehicle operation features: {current speed: 60 km / h, lane deviation degree: 0, distance from the front vehicle: 50 m}, environmental features: {light intensity: 5000 lux, whether a highway section: no}}.

[0184] Step 402, match the current scene feature group with the standard features corresponding to the current scene type in the cockpit scene decision knowledge base, obtain the feature matching degree between the two, and filter based on the feature matching degree combined with the decision strategy corresponding to the current scene type in the cockpit scene decision knowledge base, to obtain a candidate group of decision strategies.

[0185] Optionally, the scene perception decision system calls the cockpit scene decision knowledge base, which stores multiple sets of standard feature-decision strategy mapping relationships for each scene type (such as the driver distraction scene). The standard features are typical feature combinations under the scene type (such as "low-speed distraction standard features": speed ≤ 40 km / h, distraction duration ≤ 3 s; "high-speed distraction standard features": speed ≥ 80 km / h, distraction duration ≥ 2 s), and the corresponding decision strategies include specific execution actions, device parameters, and trigger conditions (such as "low-speed distraction strategy": audio warning sound 60% volume, instrument panel text prompt; "high-speed distraction strategy": audio warning sound 80% volume, steering wheel vibration, instrument panel flashing prompt). Then, all standard features and decision strategies corresponding to the current scene type (such as "driver distraction scene") are filtered from the knowledge base, and then the matching degree of the current scene feature group with each set of standard features is calculated: for numerical features (such as speed, distraction duration), the similarity of real-time parameters and standard feature parameters is calculated (such as the similarity of current speed 60 km / h and "medium-speed distraction standard features" speed 50-70 km / h is 100%); for categorical features (such as whether a highway section), if the real-time parameters and the standard feature parameters are consistent, the similarity is 100%, otherwise it is 0. The similarity of all feature dimensions is weighted calculated according to the weight (such as driver state feature weight 0.5, vehicle operation feature weight 0.3, environmental feature weight 0.2) to obtain the overall feature matching degree.

[0186] Further, after obtaining the overall feature matching degree, if the matching degree of a group of standard features with the current scene feature group is ≥ 70% (preset matching threshold), the corresponding decision strategy is included in the "decision strategy candidate group"; if the matching degree is < 70%, it is determined that the strategy is not suitable for the current scene, and is excluded. The final decision strategy candidate group needs to include at least one strategy, if the candidate group is empty (such as the current scene features are too special), the "default decision strategy" (such as basic sound warning + text prompt) in the knowledge base is called to supplement the candidate group, to ensure that there are enough strategies to choose from in the subsequent screening.

[0187] Continue with the above embodiment, the current scene feature group F (vehicle speed 60 km / h, distraction duration 2 s, non-highway section), the scene perception decision system extracts 3 groups of standard features and decision strategies corresponding to "driver distraction scene" from the cabin scene decision knowledge base:

[0188] Standard feature S1 (low-speed distraction): vehicle speed ≤ 40 km / h, distraction duration ≤ 3 s, non-highway section, corresponding strategy P1: sound warning sound 50% volume (for 1 s), instrument panel green text prompt ("please focus on driving", for 3 s);

[0189] Standard feature S2 (medium-speed distraction): vehicle speed 40-80 km / h, distraction duration 1-5 s, non-highway section, corresponding strategy P2: sound warning sound 60% volume (for 1 s), instrument panel red text prompt ("please focus on driving", for 3 s), if the driver does not recover attention within 1 s, the steering wheel will vibrate slightly (30% intensity, for 0.5 s);

[0190] Standard feature S3 (high-speed distraction): vehicle speed ≥ 80 km / h, distraction duration ≥ 2 s, highway section, corresponding strategy P3: sound warning sound 80% volume (for 2 s), instrument panel red flashing prompt ("emergency! Please focus on driving", for 5 s), steering wheel vibration (50% intensity, for 1 s), temporarily limit speed increase.

[0191] Calculate the matching degree of the current scene feature group F and each group of standard features (weight: driver state 0.5, vehicle operation 0.3, environment 0.2):

[0192] Matching degree with S1: vehicle speed 60 km / h and S1 "≤ 40 km / h" similarity 0% (vehicle operation feature contribution 0.3 x 0 = 0), distraction duration 2 s and S1 "≤ 3 s" similarity 100% (driver state feature contribution 0.5 x 1 = 0.5), non-highway section consistent with S1 (environment feature contribution 0.2 x 1 = 0.2), total matching degree = 0.5 + 0 + 0.2 = 70%;

[0193] Matching degree with S2: vehicle speed 60km / h is similar to S2 "40-80km / h" with similarity 100% (0.3x1=0.3), distraction duration 2s is similar to S2 "1-5s" with similarity 100% (0.5x1=0.5), non-highway section is consistent (0.2x1=0.2), and total matching degree =0.5+0.3+0.2=100%;

[0194] Matching degree with S3: vehicle speed 60km / h is similar to S3 "≥80km / h" with similarity 0% (0.3x0=0), non-highway section is similar to S3 "highway section" with similarity 0% (0.2x0=0), and total matching degree =0.5 (matching of distraction duration) +0+0=50%.

[0195] Finally, the strategies with matching degree ≥70% are screened, and the decision strategy candidate set is {P1, P2}.

[0196] In step 403, based on the decision strategy candidate set and the user historical decision preference data stored in the cockpit scene decision knowledge base, the preference matching degree of each decision strategy in the decision strategy candidate set and the user preference is determined.

[0197] Optionally, the scene-aware decision system extracts the historical preference data of the current user (determined based on seat memory, account login information) from the cockpit scene decision knowledge base, which records the user's feedback behavior on the decision strategy in the past similar scenarios, including "strategy acceptance" (such as whether to turn off a certain type of warning, whether to manually adjust the warning intensity), "preference parameters" (such as the commonly used warning volume, the preferred prompt method), "scene preference association" (such as preferring text prompts in low-speed scenarios, preferring vibration prompts in high-speed scenarios). Then, for each strategy in the decision strategy candidate set generated in step 402, calculate the "preference matching degree". The matching degree calculation is carried out around the three dimensions of "prompt method preference", "parameter intensity preference" and "scene association preference": for "prompt method preference", if the prompt method (such as sound + text) in the strategy is consistent with the user's historical preference for the prompt method (such as only text), the dimension matching degree is 100%, otherwise, deduct points according to the degree of inconsistency (such as containing vibration prompts that the user rejects, the dimension matching degree is deducted to 30%); for "parameter intensity preference", if the parameter (such as 60% volume) in the strategy is within the allowed range (such as ±10%) of the user's historical preference parameter (such as 50% volume), the dimension matching degree is 80%-100%, the larger the deviation, the lower the matching degree (such as a deviation of 20%, the matching degree is 60%); for "scene association preference", if the strategy is consistent with the user's historical preference strategy in the same type of scene (such as preferring low-intensity warning in medium-speed scenarios), the dimension matching degree is 100%, otherwise, it is 70% or less. Finally, the matching degrees of the three dimensions are weighted calculated according to the weights (prompt method 0.4, parameter intensity 0.4, scene association 0.2) to obtain the overall preference matching degree of each strategy, with a value range of [0, 100%], and the higher the value, the more the strategy conforms to the user's habits, which can improve the user's acceptance of the decision and reduce manual adjustment operations.

[0198] Continue with the above embodiment, the decision strategy candidate set {P1, P2}, the system extracts the historical preference data of the current user as:

[0199] Strategy acceptance: In the past 3 "medium-speed distraction scenarios", 2 times of 60% volume sound warning were adjusted to 50%, and 1 time of steering wheel vibration was turned off, rejecting vibration prompts;

[0200] Preference parameters: commonly used warning volume 50%, prefer text prompts, reject vibration;

[0201] Scene preference association: medium-speed scenarios (40-80 km / h) prefer "text prompts + low-volume sound", and are against vibration.

[0202] Calculate the preference matching degree of each strategy (weights: prompt method 0.4, parameter intensity 0.4, scene association 0.2):

[0203] Policy P1 (sound 50% volume + green text prompt, no vibration):

[0204] Prompt mode: contains text + sound, no exclusive vibration, consistent with preference, dimension matching degree 100% (0.4x1=0.4);

[0205] Parameter intensity: sound volume 50%, fully consistent with user preference parameters, dimension matching degree 100% (0.4x1=0.4);

[0206] Scenario association: low-intensity warning in medium-speed scenario, consistent with user scenario preference, dimension matching degree 100% (0.2x1=0.2);

[0207] Then the overall preference matching degree = 0.4+0.4+0.2=100%;

[0208] Policy P2 (sound 60% volume + red text prompt + vibration after 1s):

[0209] Prompt mode: contains exclusive vibration, dimension matching degree 30% (0.4x0.3=0.12);

[0210] Parameter intensity: sound volume 60%, 10% deviation from preference 50% (within allowed range), dimension matching degree 80% (0.4x0.8=0.32);

[0211] Scenario association: medium-speed scenario but contains vibration, partially consistent with scenario preference, dimension matching degree 70% (0.2x0.7=0.14);

[0212] Then the overall preference matching degree = 0.12+0.32+0.14=58%. The final preference matching degree result: P1=100%, P2=58%.

[0213] Step 404, based on each decision strategy in the decision strategy candidate group, the adaptability index of each decision strategy in the current scene type is obtained by combining the current scene feature group.

[0214] Optionally, the scene-aware decision system evaluates the adaptability index of each strategy in the current scene for each strategy in the decision strategy candidate group in combination with the current scene feature group in step 401. The index quantifies the adaptation effect of the strategy to the current scene, avoiding conflicts between the strategy and the scene. And the core evaluation dimensions include "safety adaptability" "efficiency adaptability" "environmental adaptability".

[0215] Furthermore, regarding the "safety adaptability" assessment strategy, can it effectively mitigate the safety risks of the current scenario? For example, in a "driver distraction scenario," if the current distance to the vehicle in front is close (e.g., <20m), the strategy needs to include stronger warnings or assisted braking. If the strategy only provides text prompts, the safety adaptability is low. If the current distance to the vehicle in front is far (e.g., >50m), a moderate intensity warning is sufficient to meet safety requirements, resulting in high safety adaptability. Regarding the "efficiency adaptability" assessment strategy, will it affect driving efficiency? For example, in congested areas, if the strategy includes prolonged audible warnings, it may interfere with the driver's judgment, resulting in low efficiency adaptability. In open areas, the warnings have little impact on efficiency, resulting in high efficiency adaptability. Regarding the "environmental adaptability" assessment strategy, does it match the current environmental conditions? For example, in high-light conditions, green text prompts on the dashboard may be difficult to see, requiring red or flashing prompts. If the strategy uses green text, the environmental adaptability is low. Under normal lighting conditions, green text is clear, resulting in high environmental adaptability. The adaptability of each dimension is quantified into a score of 0-100 (100 points for complete adaptability, 0 points for complete incompatibility). Then, the adaptability index of each strategy is calculated by weighting (security adaptability 0.6, efficiency adaptability 0.2, environment adaptability 0.2). The value range is [0, 100]. The higher the index, the better the strategy fits the actual needs of the current scenario, ensuring security while taking into account user experience.

[0216] Continuing with the above embodiment, the current scene feature group F (distance to the vehicle in front 50m, vehicle speed 60km / h, illumination 5000lux, non-highway section) and decision strategy candidate groups {P1, P2} are as follows:

[0217] Strategy P1 (50% speaker volume + green text prompt, no vibration):

[0218] Safety Adaptability: With a current distance of 50m from the vehicle in front (safe), a 50% volume audio prompt combined with text alerts effectively warns of distraction, posing no safety risk. Score: 90 points (0.6 × 90 = 54); Efficiency Adaptability: On non-highway roads, the low-volume warning does not interfere with driving judgment and has no impact on driving efficiency. Score: 100 points (0.2 × 100 = 20); Environmental Adaptability: With 5000 lux of light (normal), the green text is clearly visible, with no environmental conflict. Score: 95 points (0.2 × 95 = 19); Therefore, the adaptability index is 54 + 20 + 19 = 93.

[0219] Strategy P2 (60% speaker volume + red text prompt + vibration after 1 second):

[0220] Safety adaptability: the vibration prompt can enhance the distraction prompt effect, and the safety is stronger, and the score is 95 (0.6x95=57); efficiency adaptability: 60% volume is slightly high, and the vibration may cause the driver's hand to slightly deviate (current does not deviate from the lane), which slightly affects the efficiency, and the score is 80 (0.2x80=16); environmental adaptability: red text is more eye-catching under normal light, and the environmental adaptability is better than green, and the score is 100 (0.2x100=20); the obtained adaptability index=93+16+20=93.

[0221] Final adaptability index result: P1=93, P2=93.

[0222] Step 405, based on the adaptability index, the preference matching degree and the current scene feature group of each decision strategy, the decision strategy candidate group is subjected to secondary screening and verification, and the target decision strategy is obtained.

[0223] Optionally, the scene awareness decision system subjects the decision strategy candidate group to secondary screening and verification according to the adaptability index, the preference matching degree and the current scene feature group of each decision strategy, and finally obtains the target decision strategy, which is specifically described in steps 4051-4055.

[0224] The embodiment of the application constructs the key information required for locking the decision by the feature group, narrows down the strategy range through the candidate group screening, integrates the user's personalized needs through the preference matching, guarantees the adaptability of the strategy and the scene through the adaptability evaluation, balances the safety and the user experience through the secondary screening and verification, and finally outputs the target decision strategy which is precise, safe and in line with the user's habits. The problems of the traditional decision method of one-size-fits-all (ignoring user preferences) and excessive personalization (ignoring scene safety) are effectively solved, and the collaborative optimization of safety and experience is realized.

[0225] In an embodiment, steps 4051-4055 are described as follows:

[0226] Step 4051, based on the adaptability index, the preference matching degree and the execution success rate of the corresponding decision strategy in the historical scene of each decision strategy, the decision strategies in the decision strategy candidate group are subjected to priority sorting, and a priority sorting result is obtained.

[0227] Optionally, the scene-aware decision system extracts the execution success rate of each strategy in the candidate strategy group in the historical same scene from the strategy execution log library of the cockpit scene decision knowledge base. The execution success rate is the proportion of the number of times that the strategy completes the execution as expected and does not cause user manual adjustment or safety risk to the total number of execution times, for example, strategy P1 is successfully executed 95 times (user does not adjust the volume or turn off the prompt) and 5 times is manually adjusted by the user due to the low volume in the past 100 "medium distraction scenes", the execution success rate is 95%. Then, the preference matching degree (P) in step 403, the adaptability index (A) in step 404, and the historical execution success rate (S) are used as the core input dimensions, and the weights of each dimension are determined: when there is a safety emergency in the current scene (such as the distance from the front vehicle < safe vehicle distance), the safety adaptation weight in the adaptability index is increased, and the overall adaptability index weight is set to 0.4; when there is no emergency safety risk in the scene, the preference matching degree weight is set to 0.4, and the user's habit is prioritized; the historical execution success rate weight is fixed to 0.2 to ensure the reliability of the strategy. Finally, the priority score of each group of strategies is calculated by the priority score formula, and the higher the score, the higher the priority. If multiple groups of strategies have the same score, further comparison is made on the "strategy execution response time" (the time consumed from the triggering of the strategy to the execution of the device), and the strategy with shorter response time has higher priority, and finally the clear priority ranking result is obtained, providing a sequence basis for subsequent verification.

[0228] Further, the priority score formula is as follows:

[0229] ;

[0230] ;

[0231] wherein, is the strategy priority score (value range ); , , are the weights of the preference matching degree, the adaptability index, and the execution success rate (dynamically allocated, such as , , ; is the preference matching degree (normalized to ); is the adaptability index (original value 0-100); is the historical execution success rate (normalized to ).

[0232] Continuing the above example, the decision strategy candidate set {P1, P2} after step 402, it is known that: preference matching degree: P1=100% (0.1), P2=58% (0.58); adaptability index: P1=93 (0.93), P2=93 (0.93); historical execution success rate extracted from the knowledge base: P1 executes 120 times in the medium distraction scene, 114 times successfully, success rate S1=95% (0.95); P2 executes 80 times, 68 times successfully (20 times are closed due to vibration), success rate S2=85% (0.85). The current scene has no urgent safety risk (the distance from the front vehicle is 50m> safety distance 25m), and the dynamic weight distribution is: preference matching degree 0.4, adaptability index 0.4, execution success rate 0.2. Calculate the priority score: P1 score=0.4x1+0.4x0.93+0.2x0.95=0.4+0.372+0.19=0.962; P2 score=0.4x0.58+0.4x0.93+0.2x0.85=0.232+0.372+0.17=0.774. The priority ranking result is P1 (0.962)>P2 (0.774).

[0233] Step 4052, based on the current scene feature set and the comprehensive constraint condition under the current scene, the decision strategies in the priority ranking result are checked in turn, and the decision strategy that meets the constraint condition and is located in the highest of the priority ranking result is taken as the candidate strategy; the comprehensive constraint condition includes hardware constraint, software constraint and scene constraint.

[0234] Optionally, the scene-aware decision system includes hardware constraints, software constraints, and scene constraints according to the comprehensive constraint condition. Among them, the hardware constraint is the current state of the execution device (such as whether the sound is malfunctioning, whether the steering wheel vibration module is in working state), the hardware parameter limit (such as the maximum volume of the sound, the maximum intensity of the vibration module); the software constraint is the current running state of the Hongmeng system (such as whether it is in system upgrade, whether there is other high-priority instruction occupying the device), software permission (such as whether the strategy needs to call the vehicle CAN bus permission); the scene constraint is the special limit of the current scene (such as reducing the warning sound volume in the school area, enhancing the light prompt in the tunnel). According to the priority ranking result of step 4051, starting from the highest priority strategy, check in turn: first check the hardware constraint (such as strategy P1 needs to call the sound and instrument panel, confirm that the sound is not malfunctioning and the volume adjustable range contains 50%), then check the software constraint (confirm that the system is not upgraded and the CAN bus permission has been obtained), and finally check the scene constraint (the current is not in school area or tunnel, no special volume limit). If a strategy meets all the constraint conditions, it is determined as the "candidate strategy"; if it does not meet (such as sound failure, P1 cannot be executed), skip this strategy and check the next priority strategy until a strategy that meets all the constraints is found.

[0235] Continuing the above embodiment, the priority ranking (P1>P2) of the step 4051, the current scene comprehensive constraint conditions:

[0236] Hardware constraints: sound system is working properly (volume adjustable 0%-100%), dashboard display is normal, steering wheel vibration module is working properly;

[0237] Software constraints: the Hongmeng system is running normally (no upgrade), device control permission has been obtained;

[0238] Scene constraints: currently on the urban trunk road (not in the school area, not in the tunnel), no warning volume limit.

[0239] Check P1:

[0240] Hardware constraints: the sound system can output 50% volume, the dashboard can display green text, which meets the requirements;

[0241] Software constraints: the system is not occupied, the permission has been obtained, which meets the requirements;

[0242] Scene constraints: no special restrictions, which meets the requirements. P1 meets all the constraints, and there is no need to check P2.

[0243] Step 4053, based on the current scene feature set and the standardized perception data, it is judged whether there is an abnormality in the current scene type, if it is judged to be in an abnormal state, the optimization adjustment is made to the selected strategy based on the abnormality degree, and the optimized strategy is obtained.

[0244] Optionally, the scene perception decision system first judges whether the current scene is in an abnormal state based on the current scene feature set of step 401 and the standardized perception data of step 20. The abnormal state includes device abnormality (such as occasional frame loss of DMS camera), environmental abnormality (such as sudden rain leading to reduced visibility) and user state abnormality (such as sudden increase of driver's heart rate). The judgment basis is whether the key parameters in the standardized perception data exceed the normal range (such as heart rate >100 times / min), and whether the parameter fluctuation exceeds the threshold (such as camera frame loss rate >5%). If it is judged to be in an abnormal state, the "abnormality degree" is further calculated (such as heart rate 110 times / min, abnormality degree=(110-100) / (120-100)=0.5, full score 1.0), and the selected strategy is optimized and adjusted based on the abnormality degree: for example, when the environment is abnormal (rain visibility is low), the dashboard text brightness is increased and the light flashing prompt is added; when the user state is abnormal (heart rate rises suddenly), the warning volume is reduced and the vibration is cancelled to avoid aggravating the user's tension. And when adjusting, refer to the "abnormality-adjustment rule" in the knowledge base (such as abnormality degree≥0.6, warning intensity is reduced by 30%), to ensure that the optimized strategy adapts to the abnormal scene, and the "optimized strategy" is obtained; if it is not in an abnormal state, the selected strategy is directly retained.

[0245] Continuing the above example, the candidate strategy P1 (sound 50% volume + green text prompt), the system analyzes the current scene feature group and the standardized perception data:

[0246] Driver heart rate: extracted from the biosensor standardized data, the value is 85 beats / min (normal range 60-100 beats / min);

[0247] Environment visibility: calculated from the front windshield camera standardized data, the value is 200m (normal, ≥100m);

[0248] Device status: DMS camera frame loss rate 0.5% (≤5%), no abnormalities.

[0249] Determine that the current scene is not in an abnormal state, no optimization adjustment is needed, and the candidate strategy P1 is retained. If the driver's heart rate suddenly rises to 110 beats / min (abnormal degree 0.5), then optimize P1: the sound volume is reduced to 40%, and the instrument panel text color is changed to soft blue to avoid stimulating the driver.

[0250] Step 4054, based on the candidate strategy or the optimized strategy, the current scene feature group and the historical execution data of the corresponding strategy in the cockpit scene decision knowledge base are combined to predict the execution effect, and an execution effect prediction value is obtained; the execution effect covers safety indicators, comfort indicators and efficiency indicators.

[0251] Optionally, when the scene perception decision system predicts the execution effect, the current scene feature group, the candidate strategy (or the optimized strategy), and the historical execution data of the strategy in the knowledge base are used to predict the "execution effect" after the strategy is executed, which covers three major indicators of safety, comfort and efficiency. Among them, the safety indicator is the degree of reducing safety risk of the predicted strategy (such as the probability of the driver returning to focus after distraction reminder, the probability of avoiding lane deviation); the comfort indicator is the acceptance degree of the user to the strategy (such as the probability of not manually adjusting the strategy, the subjective comfort degree score); the efficiency indicator is the influence of the strategy on driving efficiency (such as whether it interferes with driving operation, whether the execution time affects real-time performance). Through the effect prediction formula, the three major indicators are quantified into "execution effect prediction value" (value range 0-100, the higher the better), and the quantification is based on: the effect data of similar feature groups and strategy combinations in the same scene in history (such as the probability of the driver returning to focus 80%, corresponding to the safety indicator 80 points), the similarity of the current feature group and the historical data (such as similarity 90%, then prediction value = historical effect value × 90%). Finally, the specific execution effect prediction value is output, which provides the basis for subsequent threshold judgment.

[0252] Further, the effect prediction formula is as follows:

[0253] ;

[0254] The overall execution effect prediction value (0-100); The weights for each indicator are: safety 0.4, comfort 0.3, efficiency 0.3. For the first time in similar historical scenarios Average score of each indicator (0-100); The similarity between the current scene and historical scenes (calculated based on feature sets, such as cosine similarity, with values ​​ranging from 0 to 100). ).

[0255] Continuing with the above embodiment, for the candidate strategy P1, given the current scene feature group (vehicle speed 60km / h, distraction duration 2s, non-high-speed), the historical execution results of P1 in similar scenes (vehicle speed 55-65km / h, distraction duration 1-3s) in the knowledge base are as follows:

[0256] Safety indicator: Driver's probability of regaining focus is 85%, corresponding to a score of 85;

[0257] Comfort index: 95% probability that users will not make adjustments, corresponding to a score of 95;

[0258] Efficiency metric: Execution time 0.3s (≤0.5s), no interference, corresponding to 100 points.

[0259] The current feature group has a 92% similarity to historical data. Calculate the predicted execution effect values: Safety prediction value = 85 × 92% = 78.2 points; Comfort prediction value = 95 × 92% = 87.4 points; Efficiency prediction value = 100 × 92% = 92 points; Overall execution effect prediction value = (78.2 × 0.4 + 87.4 × 0.3 + 92 × 0.3) = 31.28 + 26.22 + 27.6 = 85.1 points.

[0260] Step 4055: Based on the comparison between the predicted execution effect value and the preset execution effect threshold, if the execution effect threshold is met, the candidate strategy or optimization strategy is determined as the target decision strategy corresponding to the current scenario type; if the execution effect threshold is not met, the candidate strategy or optimization strategy is adapted and adjusted until the execution effect threshold is met, and the adapted and adjusted decision is determined as the target decision strategy corresponding to the current scenario type.

[0261] Optionally, the scene-aware decision system pre-sets an execution effect threshold (such as a comprehensive threshold of 80 points, a safety sub-threshold of 75 points, a comfort sub-threshold of 80 points, and an efficiency sub-threshold of 85 points), which is determined based on industry safety standards and user experience research to ensure that the policy execution guarantees safety and takes into account comfort and efficiency. Compare the execution effect prediction value of step 4054 with the threshold value: if the comprehensive prediction value is greater than or equal to 80 points and all sub-thresholds are met, then the selected policy (or optimized policy) is directly determined as the "target decision policy"; if not met (such as a safety prediction value of 70 points, which is less than 75 points), then the policy is "adapted and adjusted", the adjustment direction includes enhancing safety-related operations (such as increasing the warning volume, increasing the vibration), optimizing comfort parameters (such as adjusting the prompt frequency), and recalculating the execution effect prediction value after adjustment until all threshold values are met. During the adjustment process, the number of iterations is limited (a maximum of 3 times), and if the threshold is still not met after 3 adjustments, a "backup decision policy" (such as P2) in the knowledge base is called to repeat steps 4053-4054 until a target decision policy that meets the threshold is determined, ensuring the reliability and effectiveness of the final policy.

[0262] Continue with the above embodiment, the execution effect prediction value of step 4054 (comprehensive 85.1 points, safety 78.2 points, comfort 87.4 points, efficiency 92 points), and the execution effect threshold: comprehensive 80 points, safety 75 points, comfort 80 points, efficiency 85 points. The comparison result: all indicators meet the threshold, no adjustment is needed, and P1 (sound 50% volume + green text prompt) is determined as the target decision policy for the current scene.

[0263] If the safety prediction value is 74 points (less than 75 points), then adjust P1: increase the sound volume to 55%, and re-predict the safety indicator (restore the concentration probability to 88%, the prediction value = 88 x 92% = 80.96 points), the comprehensive prediction value = (80.96 x 0.4 + 87.4 x 0.3 + 92 x 0.3) = 32.38 + 26.22 + 27.6 = 86.2 points, which meets the threshold, and the adjusted P1 is determined as the target policy.

[0264] The embodiment of the present application focuses on the optimal strategy through priority sorting, ensures the executability of the strategy through constraint verification, improves scene adaptability through abnormal optimization, avoids risks in advance through effect prediction, and guarantees the final effect through threshold adaptation. The final output is an executable, highly adaptive, and optimal experience target decision policy, which solves the problems of ignoring execution feasibility and lacking effect prediction in traditional strategy determination, and realizes the upgrade of decision-making from matching to optimization.

[0265] Further, the following describes the vehicle intelligent cockpit scene perception decision system based on the Hongmeng system provided by the present application. The vehicle intelligent cockpit scene perception decision system based on the Hongmeng system described below can be correspondingly referred to the vehicle intelligent cockpit scene perception decision method based on the Hongmeng system described above.

[0266] Optionally, referring to Figure 2 , Figure 2 is a structural schematic diagram of the vehicle intelligent cockpit scene perception decision system based on the Hongmeng system provided by the present application. The vehicle intelligent cockpit scene perception decision system based on the Hongmeng system comprises:

[0267] The perception device determination module 210 is configured to determine core perception tasks under each basic running scene based on real-time requirements of the current basic running scene of the vehicle cockpit, and determine a multi-source perception device meeting the task requirements based on the task requirements of the core perception tasks and a pre-constructed perception device capability mapping table.

[0268] The data synchronization and preprocessing module 220 is configured to perform time synchronization and preprocessing on original perception data collected by the multi-source perception device based on the Hongmeng system, to obtain standardized perception data.

[0269] The cockpit scene classification module 230 is configured to perform scene classification based on the standardized perception data and the basic running scene, to obtain a corresponding current scene type under the basic running scene.

[0270] The strategy generation and instruction analysis module 240 is configured to determine a target decision strategy corresponding to the current scene type based on the current scene type and a pre-constructed cockpit scene decision knowledge base, and perform instruction analysis on the target decision strategy, to obtain control instructions of each execution device.

[0271] The embodiment of the application determines core perception tasks by combining the real-time requirements of the current basic running scene of the automobile cabin, and matches multi-source perception devices that meet the requirements according to a pre-constructed perception device capability mapping table, thereby realizing accurate adaptation of the perception device and the scene task, avoiding invalid calling and wasting of device resources, and obtaining high-quality standardized perception data based on time synchronization and preprocessing of raw data collected by the multi-source perception device based on the Hongmeng system, and then classifying the scene based on the standardized perception data and the basic running scene, accurately identifying the corresponding current scene type under the basic running scene, so that the intelligent cabin can clearly recognize the specific scene currently located, avoid decision deviation caused by scene misjudgment, determine the target decision strategy based on the current scene type and the pre-constructed cabin scene decision knowledge base, and analyze it into an execution device control instruction, thereby realizing rapid correspondence and accurate execution of the scene and the decision strategy, ensuring that the cabin can output adaptive services in a timely manner according to different scenes, and improving the scene adaptability, decision accuracy and service timeliness of the intelligent cabin of the automobile, thereby improving the actual experience of the user in the intelligent cabin.

[0272] Please refer to Figure 3 , Figure 3 The embodiment of the electronic device provided by the embodiment of the application is shown in the figure. As shown in Figure 3 The embodiment of the application provides an electronic device 300, which includes a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, the following steps are implemented:

[0273] Based on the real-time requirements of the current basic running scene of the automobile cabin, the core perception tasks under each basic running scene are determined, and the multi-source perception devices that meet the task requirements are determined based on the task requirements of the core perception tasks and the pre-constructed perception device capability mapping table;

[0274] Based on the Hongmeng system, the raw perception data collected by the multi-source perception device is time-synchronized and pre-processed to obtain standardized perception data;

[0275] Based on the standardized perception data, the scene is classified in combination with the basic running scene to obtain the corresponding current scene type under the basic running scene;

[0276] Based on the current scene type and the pre-constructed cabin scene decision knowledge base, the target decision strategy corresponding to the current scene type is determined, and the target decision strategy is analyzed into control instructions of each execution device.

[0277] Please refer to Figure 4 , Figure 4 The embodiment of the computer readable storage medium provided by the embodiment of the application is shown in the figure. As shown in Figure 4As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 311 is stored. When the computer program 311 is executed by a processor, it performs the following steps:

[0278] Based on the real-time requirements of the current basic operating scenarios of the vehicle cockpit, the core perception tasks under each basic operating scenario are determined, and based on the task requirements of the core perception tasks and the pre-built perception device capability mapping table, the multi-source perception devices that meet the task requirements are determined.

[0279] Based on the HarmonyOS system, the raw sensing data collected by multi-source sensing devices is synchronized in time and preprocessed to obtain standardized sensing data;

[0280] Based on standardized sensing data and basic operational scenarios, scenario classification is performed to obtain the current scenario type corresponding to the basic operational scenario.

[0281] Based on the current scenario type and a pre-built cockpit scenario decision knowledge base, the target decision strategy corresponding to the current scenario type is determined, and the target decision strategy is parsed to obtain the control commands for each execution device.

[0282] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the automotive intelligent cockpit scene perception and decision-making method based on the HarmonyOS system provided by the above methods. The method includes:

[0283] Based on the real-time requirements of the current basic operating scenarios of the vehicle cockpit, the core perception tasks under each basic operating scenario are determined, and based on the task requirements of the core perception tasks and the pre-built perception device capability mapping table, the multi-source perception devices that meet the task requirements are determined.

[0284] Based on the HarmonyOS system, the raw sensing data collected by multi-source sensing devices is synchronized in time and preprocessed to obtain standardized sensing data;

[0285] Based on standardized sensing data and basic operational scenarios, scenario classification is performed to obtain the current scenario type corresponding to the basic operational scenario.

[0286] Based on the current scenario type and a pre-built cockpit scenario decision knowledge base, the target decision strategy corresponding to the current scenario type is determined, and the target decision strategy is parsed to obtain the control commands for each execution device.

[0287] The system embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0288] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of the embodiments or some parts of the embodiments.

[0289] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for scene perception decision of an intelligent cockpit of a vehicle based on a Hongmeng system, characterized in that, The method comprises the following steps: Based on the real-time demand of the current basic running scene of the automobile cabin, the core perception task under each basic running scene is determined, and based on the task demand of the core perception task, a multi-source perception device meeting the task demand is determined by combining a pre-constructed perception device capability mapping table; Based on the raw perception data collected by the multi-source perception device, the raw perception data is time-synchronized and pre-processed based on the multi-source perception device, and standardized perception data is obtained; Based on the standardized perception data and the basic running scene, scene classification is performed to obtain the corresponding current scene type under the basic running scene; Based on the current scene type and the pre-constructed cabin scene decision knowledge base, a target decision strategy corresponding to the current scene type is determined, and the target decision strategy is analyzed to obtain the control instructions of each execution device; The step of performing scene classification to obtain the current scene type comprises: Based on the inherent characteristics of the scene in the basic running scene and the standardized perception data, a matching degree screening is performed to obtain an anchor feature group corresponding to each basic running scene; For the anchor feature group, the dynamic deviation of the standardized perception data in the real-time collection process from the anchor feature group is calculated to obtain a dynamic perception deviation value; Based on the anchor feature group, the basic running scene is logically disassembled to obtain a plurality of initial sub-scenes belonging to the basic running scene, and the initial sub-scenes are verified based on the historical standard perception data and historical scene classification data stored by the multi-source perception device to obtain potential sub-scenes; Based on the association between the dynamic perception deviation value and each potential sub-scene under the basic running scene, a scene association factor is determined; the scene association factor quantifies the influence direction of the dynamic perception deviation value on scene classification; Based on the scene association factor and the coupling relationship between different core features in the anchor feature group, a coupling degree matrix is constructed; Based on the coupling degree matrix, a core feature combination meeting a coupling degree threshold is screened out, and based on the core feature combination, the scene association factor and the potential sub-scene, a potential sub-scene corresponding to each core feature combination is determined to form a sub-scene candidate group; For the sub-scene candidate group, each sub-scene is screened based on the adaptation degree of each sub-scene to the dynamic perception deviation value to obtain an optimal candidate sub-scene; Based on the optimal candidate sub-scene at the current time and the scene classification result at the previous time, a time sequence consistency verification is performed to obtain a time sequence consistency coefficient of the optimal candidate sub-scene at the current time; If the time sequence consistency coefficient of the optimal candidate sub-scene meets a consistency threshold, the optimal candidate sub-scene is determined as the current scene type; if the time sequence consistency coefficient of the optimal candidate sub-scene does not meet the consistency threshold, the scene association factor is backtracked based on the sub-scene candidate group until the consistency threshold is met, and the optimal candidate sub-scene meeting the consistency threshold is determined as the current scene type.

2. The method of claim 1, wherein the method is implemented by a vehicle intelligent cockpit scene perception decision method based on a Hong Meng system. The method comprises the following steps: extracting feature parameters associated with the scene type from the standardized perception data based on the current scene type, to construct a current scene feature group; matching the current scene feature group with the standard features corresponding to the current scene type in the cabin scene decision knowledge base, obtaining the feature matching degree between the two, and screening based on the feature matching degree combined with the decision strategy corresponding to the current scene type in the cabin scene decision knowledge base, to obtain a decision strategy candidate group; based on the decision strategy candidate group combined with the user historical decision preference data stored in the cabin scene decision knowledge base, determining the preference matching degree of each decision strategy in the decision strategy candidate group with user preference; based on each decision strategy in the decision strategy candidate group combined with the current scene feature group, adaptive evaluation is carried out to obtain the adaptability index of each decision strategy under the current scene type; based on the adaptability index, preference matching degree and current scene feature group of each decision strategy, the decision strategy candidate group is further screened and verified to obtain the target decision strategy.

3. The method of claim 2, wherein the method further comprises: the adaptability index, preference matching degree and corresponding decision strategy in the historical scene of each decision strategy, the decision strategies in the decision strategy candidate group are prioritized to obtain a priority sorting result; based on the current scene feature group and the comprehensive constraint conditions under the current scene, the decision strategies in the priority sorting result are sequentially verified, and the decision strategy that meets the constraint conditions and is located in the highest priority sorting result is selected as the candidate strategy; the comprehensive constraint conditions include hardware constraints, software constraints and scene constraints; based on the current scene feature group and the standardized perception data, it is judged whether there is an abnormality under the current scene type, and if it is judged to be in an abnormal state, the candidate strategy is optimized and adjusted based on the abnormal degree to obtain an optimized strategy; based on the candidate strategy or the optimized strategy combined with the current scene feature group and the historical execution data corresponding to the strategy in the cabin scene decision knowledge base, the execution effect prediction value is obtained; the execution effect includes safety index, comfort index and efficiency index; based on the comparison between the execution effect prediction value and the preset execution effect threshold value, if the execution effect threshold value is met, the candidate strategy or the optimized strategy is determined as the target decision strategy corresponding to the current scene type; if the execution effect threshold value is not met, the candidate strategy or the optimized strategy is adapted and adjusted until the execution effect threshold value is met, and the decision after adaptation is determined as the target decision strategy corresponding to the current scene type. the raw perception data collected by the multi-source perception device is time-synchronized and preprocessed based on the Hongmeng system to obtain standardized perception data, including:

4. The method of claim 1, wherein the method is implemented by a vehicle intelligent cockpit scene perception decision method based on a Hong Meng system. ​ The distributed soft bus of the hyper system is combined with a pre-constructed perception device data type mapping table to assign a unique combination of device identification and data frame sequence number to each channel of raw perception data, forming raw identification perception data; The local timestamp of the raw identification perception data is calibrated based on a global unified timestamp provided by the distributed time service of the hyper system, obtaining a calibrated unified timestamp; A time synchronization window is determined based on the time sensitivity of the core perception task of the cockpit, and the raw identification perception data with calibrated timestamps is grouped according to the time window in which the calibrated unified timestamp falls based on the time synchronization window, obtaining multi-source data groups; The calibrated unified timestamp of each channel of data in each multi-source data group is combined with the center time of the group time window to calculate the deviation, obtaining a time deviation value; The time deviation value, the raw identification perception data, the unique combination identification, the calibrated unified timestamp, and the time synchronization window are encapsulated based on the time deviation value, the raw identification perception data, the unique combination identification, the calibrated unified timestamp, and the time synchronization window, obtaining the standardized perception data.

5. The method of claim 4, wherein the method further comprises: The time deviation value, the raw identification perception data, the unique combination identification, the calibrated unified timestamp, and the time synchronization window are encapsulated based on the time deviation value, the raw identification perception data, the unique combination identification, the calibrated unified timestamp, and the time synchronization window, obtaining the standardized perception data, including: The content of the raw identification perception data is reconstructed based on the adaptive difference value of the time deviation value and the data acquisition frequency, obtaining reconstructed perception data; the data timestamp of the reconstructed perception data is the same as the window center time of the corresponding multi-source data group; For the reconstructed perception data, the abnormality degree of each channel of data is determined based on the historical data distribution of the hyper system cockpit scene, and each channel of data with an abnormality degree exceeding a preset abnormality degree threshold is replaced, obtaining standard perception data without abnormalities; For the standard perception data, data of different dimensions is converted to a unified cockpit scene feature dimension space based on the data dimensions of different multi-source perception devices, obtaining converted perception data after dimension normalization; The converted perception data is encapsulated based on the unique combination identification, the calibrated unified timestamp, and the time synchronization window according to the format requirements of the hyper system cockpit data interaction protocol, obtaining the standardized perception data.

6. A vehicle intelligent cockpit scene perception decision system based on a Hongmeng system, characterized in that, The method is applied to the hyper system-based intelligent cockpit scene perception decision-making system of claim 1 to 5, and the hyper system-based intelligent cockpit scene perception decision-making system comprises: A perception device determination module is configured to determine core perception tasks under each basic running scene based on real-time requirements of the current basic running scene of the cockpit, and to determine multi-source perception devices meeting the task requirements based on the task requirements of the core perception tasks and a pre-constructed perception device capability mapping table; A data synchronization and preprocessing module is configured to perform time synchronization and preprocessing on raw perception data collected by the multi-source perception devices based on the hyper system, obtaining standardized perception data; A cockpit scene classification module is configured to perform scene classification based on the standardized perception data and the basic running scene, obtaining a current scene type corresponding to the basic running scene. The strategy generation and instruction analysis module is configured to determine a target decision strategy corresponding to the current scene type based on the current scene type in combination with a pre-constructed cockpit scene decision knowledge base, and to perform instruction analysis on the target decision strategy to obtain control instructions for each execution device; The step of performing scene classification to obtain the current scene type includes: Based on the scene inherent characteristics in the basic running scene and the standardized perception data, a matching degree screening is performed to obtain an anchor feature group that anchors each basic running scene; For the anchor feature group, a dynamic perception deviation value is calculated based on the dynamic deviation of the standardized perception data in the real-time acquisition process from the anchor feature group; Based on the anchor feature group, the basic running scene is logically disassembled to obtain a plurality of initial sub-scenes that belong to the basic running scene, and the initial sub-scenes are verified based on historical standard perception data and historical scene classification data stored by the HarmonyOS system to obtain potential sub-scenes; Based on the association between the dynamic perception deviation value and each potential sub-scene under the basic running scene, a scene association factor is determined; the scene association factor quantifies the influence direction of the dynamic perception deviation value on scene classification; Based on the scene association factor and the coupling relationship between different core features in the anchor feature group, a coupling degree matrix is constructed; Based on the coupling degree matrix, a core feature combination that satisfies a coupling degree threshold is screened out, and based on the core feature combination, the scene association factor, and the potential sub-scenes, a potential sub-scene corresponding to each core feature combination is determined to form a sub-scene candidate group; For the sub-scene candidate group, each sub-scene is screened based on the adaptation degree of the sub-scene to the dynamic perception deviation value to obtain an optimal candidate sub-scene; Based on the optimal candidate sub-scene at the current time and the scene classification result at the previous time, a time sequence consistency verification is performed to obtain a time sequence consistency coefficient of the optimal candidate sub-scene at the current time; If the time sequence consistency coefficient of the optimal candidate sub-scene satisfies a consistency threshold, the optimal candidate sub-scene is determined as the current scene type; if the time sequence consistency coefficient of the optimal candidate sub-scene does not satisfy the consistency threshold, the scene association factor is backtracked based on the sub-scene candidate group until the consistency threshold is satisfied, and the optimal candidate sub-scene that satisfies the consistency threshold is determined as the current scene type.

7. An electronic device, characterized by It includes: a memory for storing a computer software program; a processor for reading and executing the computer software program, wherein the processor, when executing the computer software program, implements the HarmonyOS-based automotive intelligent cockpit scene perception decision method according to any one of claims 1 to 5.

8. A non-transitory computer readable storage medium, characterized in that, The storage medium stores a computer software program, and when the computer software program is executed by the processor, the HarmonyOS-based automotive intelligent cockpit scene perception decision method according to any one of claims 1 to 5 is implemented.

Images