Model training and vehicle driving scene display method, device, equipment and medium

Abstract

The application discloses a model training and vehicle driving scene display method, which comprises the following steps: obtaining sample driving scene data; wherein the sample driving scene data comprises sample driver identity information, sample vehicle state data, sample vehicle driving environment data, sample driver operation information, sample driver state information and sample scene display information; training a neural network model by using the sample driving scene data, so as to obtain a vehicle driving scene display model matched with the sample driver identity information; the vehicle driving scene display model is used for determining a vehicle driving scene displayed in a vehicle driving scene display system; and the vehicle driving scene display model is optimized by using auxiliary driving scene data of an auxiliary driver; the auxiliary driving scene data is matched with the sample driving scene data. By the above scheme, the vehicle driving scene display efficiency and the vehicle driving scene display accuracy can be improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer science, and more particularly to a method, apparatus, device, and medium for model training and vehicle driving scene display. Background Technology

[0002] Currently, an increasing number of vehicles are equipped with vehicle driving scene display systems. When drivers require a panoramic view of the driving scene, they can use these systems to display the entire driving environment. However, at present, switching between driving scenes displayed by these systems often requires manual switching by the driver, resulting in low efficiency, a high risk of error, and an impact on the driving experience. Therefore, how to automatically switch the vehicle driving scene information displayed by the system based on driving scene data, thereby improving both the efficiency and accuracy of the display, is a problem that needs to be solved. Summary of the Invention

[0003] This invention provides a method, apparatus, device, and storage medium for model training and vehicle driving scene display, which can improve the efficiency and accuracy of vehicle driving scene display.

[0004] According to one aspect of the present invention, a model training method is provided, comprising:

[0005] Acquire sample driving scenario data; wherein, the sample driving scenario data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scenario display information;

[0006] Using the sample driving scenario data, a neural network model is trained to obtain a vehicle driving scenario display model that matches the driver identity information of the sample; the vehicle driving scenario display model is used to determine the vehicle driving scenario displayed in the vehicle driving scenario display system.

[0007] The vehicle driving scenario display model is optimized using assisted driving scenario data; the assisted driving scenario data is matched with sample driving scenario data.

[0008] According to another aspect of the present invention, a method for displaying a vehicle driving scene is provided, comprising:

[0009] Acquire target driving scenario data; the target driving scenario data includes: target driver identity information, target vehicle status data, target vehicle driving environment data, target driver operation information, and target driver status information;

[0010] Based on the vehicle driving scenario display model, the display information of the vehicle driving scenario is determined according to the target driving scenario data and the target driver identity information; wherein, the vehicle driving scenario display model is trained based on the model training method described in any embodiment of the present invention.

[0011] According to another aspect of the present invention, a model training apparatus is provided, the apparatus comprising:

[0012] The sample data acquisition module is used to acquire sample driving scenario data; wherein, the sample driving scenario data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scenario display information;

[0013] The model training module is used to train the neural network model using the sample driving scenario data to obtain a vehicle driving scenario display model that matches the driver identity information of the sample; the vehicle driving scenario display model is used to determine the vehicle driving scenario displayed in the vehicle driving scenario display system.

[0014] The model optimization module is used to optimize the vehicle driving scenario display model using assisted driving scenario data; the assisted driving scenario data is matched with sample driving scenario data.

[0015] According to another aspect of the present invention, a vehicle driving scene display device is provided, the device comprising:

[0016] The target data acquisition module is used to acquire target driving scenario data; the target driving scenario data includes: target driver identity information, target vehicle status data, target vehicle driving environment data, target driver operation information, and target driver status information;

[0017] The display information determination module is used to determine the display information of the vehicle driving scene based on the vehicle driving scene display model, according to the target driving scene data and the target driver identity information; wherein, the vehicle driving scene display model is trained based on the model training method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the model training method according to any embodiment of the present invention, or to execute the vehicle driving scene display method according to any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the model training method described in any embodiment of the present invention, or to implement the vehicle driving scene display method described in any embodiment of the present invention.

[0023] The technical solution of this invention involves acquiring sample driving scene data; training a neural network model using the sample driving scene data to obtain a vehicle driving scene display model that matches the identity information of the sample drivers; and optimizing the vehicle driving scene display model using auxiliary driving scene data from the driver. This solution provides a method for constructing a vehicle driving scene display model that matches the identity information of sample drivers based on sample driving scene data. It solves the problem that when displaying the vehicle driving scene on an in-vehicle display during vehicle operation, the driver needs to manually adjust the driving scene, resulting in low efficiency in switching driving scenes on the in-vehicle display. Constructing a vehicle driving scene display model that matches the identity information of sample drivers based on sample vehicle form data fully considers the influence of sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scene display information on the vehicle driving scene display model. This achieves the effect of training a vehicle driving scene display model that matches the identity information of different drivers, thereby improving the model accuracy of the vehicle driving scene display model. Meanwhile, optimizing the vehicle driving scenario display model based on assisted driving scenario data can enrich the training data of the model, thereby improving its accuracy and versatility. Displaying vehicle driving scenarios in the vehicle driving scenario display system using this model can improve the efficiency of the scenario display.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart of a model training method is provided for Embodiment 1 of the present invention;

[0027] Figure 2 This is a flowchart of a model training method provided in Embodiment 2 of the present invention;

[0028] Figure 3 This is a flowchart of a vehicle driving scenario display method provided in Embodiment 3 of the present invention;

[0029] Figure 4 This is a schematic diagram of the structure of a model training device provided in Embodiment 4 of the present invention;

[0030] Figure 5 This is a schematic diagram of the structure of a vehicle driving scene display device provided in Embodiment 5 of the present invention;

[0031] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment Six of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," "third," and "fourth," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "etc.", and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] Example 1

[0035] Figure 1 This document provides a flowchart of a model training method according to Embodiment 1 of the present invention. This embodiment is applicable to situations requiring accurate and efficient display of vehicle driving scenarios. The method can be executed by a model training device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0036] S110. Obtain sample driving scenario data.

[0037] The sample driving scenario data refers to the data used to train the neural network to obtain the vehicle driving scenario display model. The sample driving scenario data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scenario display information.

[0038] Driver identity information refers to information used to identify a driver's personal identity, which may include, but is not limited to, age, gender, height, and driving experience. Vehicle status data refers to data related to the vehicle's status during operation; this may include, but is not limited to, vehicle gear shift frequency, turning, driving time, turn signals during turns and lane changes, use of low beam headlights, use of high beam headlights, handbrake status, parking gear, horn use, door closure status, window closure status, and headlight closure status. Vehicle driving environment data refers to environmental information data surrounding the vehicle; sample vehicle driving environment data includes, but is not limited to, road surface conditions, weather, season, traffic lights, presence of pedestrians and obstacles, traffic signs, and road markings. Driver operation information refers to the driver's operational behaviors while driving the vehicle; this includes, but is not limited to, controlling the brake pedal, controlling headlights, using the seatbelt, and controlling the steering wheel. Driver status information includes the driver's driving behaviors and whether the driver is fatigued. Driving behaviors include making phone calls and chatting. The sample scene display information refers to the vehicle driving scene displayed on the in-vehicle display while the vehicle is in motion. The vehicle driving scene displayed on the in-vehicle display can be a panoramic view of the driving scene or the driving scene in front of the vehicle. The panoramic view of the driving scene includes all the scenery around the vehicle.

[0039] Specifically, with the driver's permission, the system acquires the sample driver's identity information input by the driver. It obtains sample vehicle driving status data and sample driver operation information through vehicle sensors, and sample vehicle driving environment data through onboard cameras and vehicle radar. Driving images captured by image acquisition devices during the driving process are used to determine the driver's state information. For example, based on driving images captured by image acquisition devices, the system can determine the driver's driving behavior; facial images within the driving images are identified, and feature extraction and image analysis are performed. Based on the image analysis results and the driver's continuous driving time, it can determine whether the driver is fatigued. The system then acquires sample scene display information corresponding to the sample driver's identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, and sample driver state information, as displayed on the onboard display. Finally, the system transmits the sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver state information, and sample scene display information to a server used to train the vehicle driving scene display model via the vehicle's internal bus.

[0040] For example, sample driving scenario data can be obtained through the following sub-steps:

[0041] S1101. Collect vehicle driving scene data when the user triggers the vehicle driving scene display system.

[0042] The vehicle driving scene display system refers to a system used to control the vehicle's display screen to show the driving scene. Based on the driver's selection of driving scene display information, the system can determine whether to display a panoramic view of the driving scene or the driving scene in front of the vehicle on the in-vehicle display.

[0043] Specifically, when a driver needs to view the vehicle's driving scenario on the in-vehicle display, they can select the information to be displayed using the in-vehicle computer, thus triggering the driving scenario display system. When the user triggers the system, driving scenario data is collected. This data includes original driver identity information, original vehicle status data, original vehicle driving environment data, original driver operation information, original driver status information, and original scenario display information. Because the driving scenario data package contains invalid data, data processing is required to obtain sample driving scenario data.

[0044] S1102. Perform data cleaning on the vehicle driving scenario data to remove invalid data and obtain valid driving scenario data.

[0045] Data cleaning refers to checking the consistency of the original driving data and removing invalid data from the original driving data.

[0046] Specifically, the acquired vehicle driving scenario data is cleaned to remove invalid data and retain valid data from the original driving data, which is then used as valid driving scenario data.

[0047] S1103. Perform data reduction processing on the effective driving scenario data to determine the sample driving scenario data.

[0048] Data reduction refers to minimizing the amount of effective driving scenario data while preserving the original driving data as much as possible, in order to ensure the training efficiency of the vehicle driving scenario display model.

[0049] Specifically, data reduction processing is performed on the valid driving scenario data to extract representative data, which is then used as sample driving scenario data.

[0050] It is understandable that performing data cleaning and data reduction on the collected vehicle driving scene data to determine the sample driving scene data can improve the reliability of the determined sample driving scene data, and at the same time improve the training efficiency of the vehicle driving scene display model.

[0051] S120. Using sample driving scenario data, train the neural network model to obtain a vehicle driving scenario display model that matches the sample driver identity information.

[0052] Among them, the vehicle driving scenario display model is used to determine the vehicle driving scenario displayed in the vehicle driving scenario display system.

[0053] Specifically, the sample driving scenario data is divided into training sample scenario data and test sample scenario data. The training sample scenario data is used to train the neural network model. The training method is as follows: the sample driver identity information, sample vehicle state data, sample vehicle driving environment data, sample driver operation information, and sample driver state information from the training sample scenario data are used as input parameters to the neural network model. The sample scenario display information from the training driving scenario data is used as supervision data to train the neural network model. The trained neural network model is then tested using test sample scenario data. Based on the test results, it is determined whether the trained neural network model can serve as a vehicle driving scenario display model that matches the sample driver identity information.

[0054] For example, the accuracy of the vehicle driving scene displayed in the vehicle driving scene display system can be determined based on the test results of the trained neural network model, and whether the trained neural network model can be used as a vehicle driving scene display model can be determined. If the accuracy meets the preset conditions, then the trained neural network model can be used as a vehicle driving scene display model that matches the sample driver's identity information.

[0055] For example, a neural network model can be trained through the following sub-steps:

[0056] S1201. Extract sample scene feature data from sample driving scene data.

[0057] The sample scene feature data includes training sample feature data and test sample feature data.

[0058] Specifically, principal component analysis can be used to analyze and extract features from effective driving scenario data to determine the characteristic data of the sample scenario.

[0059] S1202. Using the feature data of the training samples, train the neural network model to obtain the trained neural network model.

[0060] Specifically, the training sample feature data is divided into training sample feature data and test sample feature data. The neural network model is trained using the training sample feature data to obtain the trained neural network model.

[0061] S1203. Test the trained neural network model using test sample feature data, and determine the vehicle driving scenario display model that matches the driver identity information of the sample based on the test results.

[0062] Specifically, the trained neural network model is tested using test sample feature data. Based on the test results, it is determined whether the trained neural network model can be used as a vehicle driving scene display model. If the test results indicate that the trained neural network model can be used as a vehicle driving scene display model, then a vehicle driving scene display model matching the sample driver's identity information is obtained.

[0063] This method extracts sample scene feature data from sample driving scene data and divides it into training sample feature data and test sample feature data. The training sample feature data is used to train the neural network model, and the test sample feature data is used to optimize the trained neural network model, resulting in a vehicle driving scene display model. This improves the model training efficiency and accuracy of the vehicle driving scene display model.

[0064] S130. Optimize the vehicle driving scenario display model by using assisted driving scenario data.

[0065] The assisted driving scenario data is matched with the sample driving scenario data. The assisted driving scenario data includes: assisted driver identity information, assisted vehicle status data, assisted vehicle driving environment data, assisted driver operation information, assisted driver status information, and assisted scenario display information.

[0066] Specifically, the assisted driving scenario data is used as the model optimization data for the vehicle driving scenario display model, and the model parameters of the vehicle driving scenario display model are optimized through the assisted driving scenario data.

[0067] For example, the vehicle driving scene display model can be optimized as follows: determine the sample driver profile based on the sample driver's identity information, sample driver's status information, and sample driver's operation information; determine the target driver profile that matches the sample driver profile from the auxiliary driver profiles of the auxiliary driver, and optimize the vehicle driving scene display model based on the auxiliary driving scene data corresponding to the target driver profile.

[0068] Among them, driver profiles refer to data that can characterize a driver's driving habits, vehicle driving scenario information selection habits, and identity characteristics.

[0069] Specifically, a sample driver profile is determined based on the sample driver's identity information, status information, and operation information; an auxiliary driver profile is determined based on the assisted driver's identity information, status information, and operation information. The assisted driver profiles and sample driver profiles are then matched, and the assisted driver profile with a match degree greater than a matching threshold is selected as the target driver profile for matching. Furthermore, the vehicle driving scenario display model is optimized based on the assisted driving scenario data corresponding to the target driver profile.

[0070] It is understandable that identifying the target driver profile that matches the sample driver profile from the assisted driver profile, and optimizing the vehicle driving scenario display model based on the assisted driving scenario data corresponding to the target driver profile, can enrich the training data of the vehicle driving scenario display model and make the vehicle driving scenario display model better meet the needs of the vehicle driver corresponding to the sample driver profile.

[0071] The technical solution provided in this embodiment acquires sample driving scene data; uses the sample driving scene data to train a neural network model, obtaining a vehicle driving scene display model that matches the sample driver's identity information; and uses auxiliary driving scene data from the assisted driver to optimize the vehicle driving scene display model. This solution provides a method for constructing a vehicle driving scene display model that matches the sample driver's identity information based on sample driving scene data. It solves the problem that when displaying the vehicle driving scene on an in-vehicle display during vehicle operation, the driver needs to manually adjust the driving scene, resulting in low efficiency in switching driving scenes on the in-vehicle display. Constructing a vehicle driving scene display model that matches the sample driver's identity information based on sample vehicle form data fully considers the impact of sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scene display information on the vehicle driving scene display model. This achieves the effect of training a vehicle driving scene display model that matches the identity information of different drivers, thereby improving the model accuracy of the vehicle driving scene display model. Meanwhile, optimizing the vehicle driving scenario display model based on assisted driving scenario data can enrich the training data of the model, thereby improving its accuracy and versatility. Displaying vehicle driving scenarios in the vehicle driving scenario display system using this model can improve the efficiency of the scenario display.

[0072] Example 2

[0073] Figure 2 This is a flowchart of a model training method provided in Embodiment 2 of the present invention. This embodiment optimizes the above embodiment and provides a preferred implementation scheme for training a neural network model using sample driving scenario data to obtain a vehicle driving scenario display model that matches the sample driver's identity information. Specifically, as shown... Figure 2 As shown, the method includes:

[0074] S210. Obtain sample driving scenario data.

[0075] The sample driving scenario data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scenario display information.

[0076] S220. Divide the sample driving scenario data into model training sample data and model optimization sample data.

[0077] The model training sample data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scene display information; the model optimization sample data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, and sample driver status information.

[0078] S230. Divide the model training sample data into first sample data and second sample data.

[0079] The first sample data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver status information, and sample scene display information; the second sample data includes: sample driver identity information, sample driving scene data including sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scene display information.

[0080] S240. Train the neural network model using the first sample data to determine the first classifier in the neural network model.

[0081] Specifically, the neural network model is trained using the first sample data. The sample driving scene data in the first sample data, including the sample driver identity information, sample vehicle state data, sample vehicle driving environment data, and sample driver state information, are used as the input data for the first classifier in the neural network model. The sample scene display information in the first sample data is used as the supervision data for the first classifier. The first classifier in the neural network model can be trained in this way.

[0082] S250. Use the second sample data to train the neural network model and determine the second classifier of the neural network model.

[0083] Specifically, the neural network model is trained using the second sample data. The sample driver identity information, sample driving scene data including sample vehicle driving environment data, sample driver operation information, and sample driver state information in the second sample data are used as input data for the second classifier in the neural network model. The sample scene display information in the second sample data is used as supervision data for the second classifier. The second classifier in the neural network model can be trained.

[0084] S260. The model optimization sample data is filtered through the first classifier and the second classifier to determine the target optimization sample data.

[0085] Specifically, the model optimization sample data is divided into two groups, which are then input into a first classifier and a second classifier, respectively. The first and second classifiers assign pseudo-labels to the two groups of unlabeled sample data. These pseudo-labels refer to the vehicle scene display information predicted by the first and second classifiers based on the model optimization sample data. After assigning pseudo-labels, the confidence levels of the two groups of model optimization sample data are determined. Model optimization sample data with confidence levels higher than a certain threshold is selected as the target optimization sample data. The confidence threshold can be set according to actual needs.

[0086] For example, the target optimized sample data can be determined through the following sub-steps:

[0087] S2601. Divide the model optimization sample data into third sample data and fourth sample data.

[0088] The third sample data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, and sample driver status information; the fourth sample data includes: sample driver identity information, sample driving scenario data including sample vehicle driving environment data, sample driver operation information, and sample driver status information.

[0089] S2602. Based on the third sample data and the first classifier, obtain the first confidence level corresponding to the first predicted scene display information.

[0090] The predicted driver operation information refers to the driver operation information obtained by the first classifier through data analysis of the third sample data. The confidence level, determined using interval estimation in mathematical statistics, is the probability that the predicted scene display information and the actual scene display information are within an allowable error range.

[0091] Specifically, the third sample data is used as input data for the first classifier. The first classifier analyzes the third sample data to obtain the first predicted scene display information corresponding to the third sample data. An interval estimation method is then used to determine the first confidence level corresponding to the first predicted scene display information.

[0092] S2603. Based on the fourth sample data and the second classifier, determine the second confidence level corresponding to the second predicted scene display information.

[0093] Specifically, the fourth sample data is used as input data for the second classifier. The second classifier analyzes the fourth sample data to obtain the second predicted scene display information corresponding to the fourth sample data. An interval estimation method is then used to determine the second confidence level corresponding to the second predicted scene display information.

[0094] S2604. Filter the model optimization sample data based on the first confidence level and the second confidence level to determine the target optimization sample data.

[0095] Specifically, the first and second confidence levels are compared with confidence thresholds, and the third and fourth sample data are filtered based on the comparison results. Third sample data with confidence levels greater than the threshold are used as the third target data; fourth sample data with confidence levels greater than the threshold are used as the fourth target data. Model optimization sample data containing both third and fourth target data are used as target optimization sample data.

[0096] It is understandable that by filtering the model optimization sample data based on the model optimization sample data, the first classifier, and the second classifier, the target optimization sample data can be determined, which can improve the reliability of the target optimization sample data and thus improve the accuracy of the vehicle driving scene display model.

[0097] S270. Optimize the first and second classifiers based on the target optimization sample data to obtain a vehicle driving scene display model that matches the driver identity information of the sample.

[0098] Among them, the vehicle driving scenario display model is used to display the vehicle driving scenarios shown in the vehicle driving scenario display system.

[0099] Specifically, the parameters of the first and second classifiers in the target optimization sample data neural network model are optimized by adjusting the parameters of the first and second classifiers. The optimized first and second classifiers are then used as a vehicle driving scenario display model that matches the sample driver's identity information.

[0100] S280: Optimize the vehicle driving scenario display model by using assisted driving scenario data.

[0101] Among them, the assisted driving scenario data is matched with the sample driving scenario data.

[0102] The technical solution of this embodiment, after acquiring sample driving scene data, divides the sample driving scene data into model training sample data and model optimization sample data; the model training sample data is further divided into first sample data and second sample data; the first and second sample data are used to train the first and second classifiers in the neural network model; then, the trained first and second classifiers are used to filter the model optimization sample data to determine the target optimization sample data; and the parameters in the first and second classifiers are optimized using the target optimization sample data. This results in a more accurate vehicle driving scene display model.

[0103] Example 3

[0104] Figure 3 This is a flowchart of a vehicle driving scene display method provided in Embodiment 3 of the present invention. This embodiment is applicable to situations where a vehicle driving scene is displayed using a vehicle driving scene display model. This method can be executed by a vehicle driving scene display device, which can be implemented in hardware and / or software and can be configured in an electronic device. Specifically, as shown... Figure 3 As shown, the method includes:

[0105] S310, Obtain target driving scenario data.

[0106] The target driving scenario data includes: target driver identity information, target vehicle status data, target vehicle driving environment data, target driver operation information, and target driver status information.

[0107] Specifically, the vehicle that generates the target driving scenario data can transmit the target driving scenario data to the server via the vehicle bus, so that the server can obtain the target driving scenario data.

[0108] S320. Based on the vehicle driving scenario display model, determine the display information of the vehicle driving scenario according to the target driving scenario data and the target driver's identity information.

[0109] The vehicle driving scenario display model is trained based on any of the model training methods in Example 1 or Example 2.

[0110] Specifically, by inputting the target driving scene data into the vehicle driving scene display model, the output data of the vehicle driving scene display model can be obtained. Based on the output data of the vehicle driving scene display model, it is determined whether to display the vehicle driving scene in a panoramic view. If so, the vehicle driving scene is displayed in a panoramic view through the in-vehicle display; otherwise, the vehicle driving scene in front of the vehicle is displayed through the in-vehicle display.

[0111] The technical solution of this embodiment, based on a vehicle driving scenario display model, determines the display information of the vehicle driving scenario according to the target driver's identity information, target vehicle status data, target vehicle driving environment data, target driver operation information, and target driver status information in the target driving scenario data. This solution fully considers the impact of the surrounding environment information, target vehicle status information, and target driver status information on the display information of the vehicle driving scenario during vehicle operation, making the displayed information more accurate.

[0112] Example 4

[0113] Figure 4This is a schematic diagram of a model training device provided in Embodiment 4 of the present invention. This embodiment is applicable to situations requiring accurate and efficient display of vehicle driving scenarios. Figure 4 As shown, the model training device includes: a sample data acquisition module 410, a model training module 420, and a model optimization module 430.

[0114] The sample data acquisition module 410 is used to acquire sample driving scenario data. The sample driving scenario data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scenario display information.

[0115] The model training module 420 is used to train the neural network model using sample driving scenario data to obtain a vehicle driving scenario display model that matches the identity information of the sample driver; the vehicle driving scenario display model is used to determine the vehicle driving scenario displayed in the vehicle driving scenario display system.

[0116] The model optimization module 430 is used to optimize the vehicle driving scenario display model by using assisted driving scenario data; the assisted driving scenario data is matched with the sample driving scenario data.

[0117] The technical solution provided in this embodiment acquires sample driving scene data; uses the sample driving scene data to train a neural network model, obtaining a vehicle driving scene display model that matches the sample driver's identity information; and uses auxiliary driving scene data from the assisted driver to optimize the vehicle driving scene display model. This solution provides a method for constructing a vehicle driving scene display model that matches the sample driver's identity information based on sample driving scene data. It solves the problem that when displaying the vehicle driving scene on an in-vehicle display during vehicle operation, the driver needs to manually adjust the driving scene, resulting in low efficiency in switching driving scenes on the in-vehicle display. Constructing a vehicle driving scene display model that matches the sample driver's identity information based on sample vehicle form data fully considers the impact of sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scene display information on the vehicle driving scene display model. This achieves the effect of training a vehicle driving scene display model that matches the identity information of different drivers, thereby improving the model accuracy of the vehicle driving scene display model. Meanwhile, optimizing the vehicle driving scenario display model based on assisted driving scenario data can enrich the training data of the model, thereby improving its accuracy and versatility. Displaying vehicle driving scenarios in the vehicle driving scenario display system using this model can improve the efficiency of the scenario display.

[0118] For example, the model optimization module 430 is specifically used for:

[0119] Based on the sample driver's identity information, sample driver status information, and sample driver operation information, a sample driver profile is determined;

[0120] Based on the sample driver profile, a target driver profile matching the sample driver profile is determined from the assisted driver profile, and the vehicle driving scenario display model is optimized based on the assisted driving scenario data corresponding to the target driver profile.

[0121] For example, the sample data acquisition module 410 is specifically used for:

[0122] Collect vehicle driving scenario data when a user triggers the vehicle driving scenario display system;

[0123] Data cleaning is performed on vehicle driving scenario data to remove invalid data and obtain valid driving scenario data;

[0124] Data reduction processing is performed on the valid driving scenario data to determine the sample driving scenario data.

[0125] For example, the model training module 420 is specifically used for:

[0126] Extract sample scene feature data from the sample driving scene data; wherein, the sample scene feature data includes training sample feature data and test sample feature data;

[0127] The neural network model is trained using the feature data of the training samples to obtain the trained neural network model;

[0128] The trained neural network model was tested using test sample feature data. Based on the test results, a vehicle driving scenario display model matching the driver identity information of the sample was determined.

[0129] For example, model training module 420 includes:

[0130] The sample data determination unit is used to divide the sample driving scenario data into model training sample data and model optimization sample data.

[0131] The training sample data partitioning unit is used to divide the model training sample data into first sample data and second sample data. The first sample data includes: sample driving scenario data including sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver status information, and sample scenario display information; the second sample data includes: sample driver identity information, sample driving scenario data including sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scenario display information.

[0132] The first classifier training unit is used to train the neural network model using the first sample data and determine the first classifier in the neural network model.

[0133] The second classifier training unit is used to train the neural network model using the second sample data and determine the second classifier of the neural network model.

[0134] The optimized sample data filtering unit is used to filter the model optimization sample data through the first classifier and the second classifier to determine the target optimization sample data;

[0135] The model optimization unit is used to optimize the first and second classifiers based on the target optimization sample data to obtain a vehicle driving scene display model that matches the driver identity information of the sample.

[0136] For example, the optimized sample data filtering unit is specifically used for:

[0137] The model optimization sample data is divided into third sample data and fourth sample data;

[0138] Based on the third sample data and the first classifier, the first confidence level corresponding to the first predicted scene display information is obtained;

[0139] Based on the fourth sample data and the second classifier, determine the second confidence level corresponding to the second predicted scene display information;

[0140] The model optimization sample data is filtered based on the first and second confidence levels to determine the target optimization sample data.

[0141] The model training device provided in this embodiment can be applied to the model training methods provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0142] Example 5

[0143] Figure 5 This is a schematic diagram of a vehicle driving scene display device provided in Embodiment 5 of the present invention. This embodiment is applicable to situations where a vehicle driving scene is displayed using a vehicle driving scene display model. Figure 5 As shown, the vehicle driving scene display device includes: a target data acquisition module 510 and a display information determination module 520.

[0144] The target data acquisition module 510 is used to acquire target driving scenario data. The target driving scenario data includes: target driver identity information, target vehicle status data, target vehicle driving environment data, target driver operation information, and target driver status information.

[0145] The display information determination module 520 is used to determine the display information of the vehicle driving scene based on the vehicle driving scene display model, according to the target driving scene data and the target driver identity information; wherein, the vehicle driving scene display model is trained based on any of the model training methods in Embodiment 1 and Embodiment 2.

[0146] The technical solution provided in this embodiment, based on a vehicle driving scenario display model, determines the display information of the vehicle driving scenario according to the target driver's identity information, target vehicle status data, target vehicle driving environment data, target driver operation information, and target driver status information in the target driving scenario data. This solution fully considers the impact of the surrounding environment information, target vehicle status information, and target driver status information on the display information of the vehicle driving scenario during vehicle operation, making the displayed information more accurate.

[0147] The vehicle driving scene display device provided in this embodiment can be applied to the vehicle driving scene display method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0148] Example 6

[0149] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0150] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0151] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0152] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as model training and vehicle driving scene demonstration methods.

[0153] In some embodiments, the model training and vehicle driving scene demonstration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the model training and vehicle driving scene demonstration method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the model training and vehicle driving scene demonstration method by any other suitable means (e.g., by means of firmware).

[0154] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable model training and vehicle driving scenario display device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0156] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0157] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0158] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0159] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0160] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0161] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A model training method, characterized in that, include: Acquire sample driving scenario data; wherein, the sample driving scenario data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scenario display information; Using the sample driving scenario data, a neural network model is trained to obtain a vehicle driving scenario display model that matches the driver identity information of the sample; wherein, the vehicle driving scenario display model is used to determine the vehicle driving scenario displayed in the vehicle driving scenario display system. The vehicle driving scenario display model is optimized using assisted driving scenario data; wherein, the assisted driving scenario data is matched with sample driving scenario data; the assisted driving scenario data includes: assisted driver identity information, assisted vehicle status data, assisted vehicle driving environment data, assisted driver operation information, assisted driver status information, and assisted scenario display information; The optimization of the vehicle driving scenario display model using assisted driving scenario data includes: determining a sample driver profile based on sample driver identity information, sample driver status information, and sample driver operation information; determining an auxiliary driver profile based on auxiliary driver identity information, auxiliary driver status information, and auxiliary driver operation information; matching the auxiliary driver profile with the sample driver profile, selecting the auxiliary driver profile with a matching degree greater than a preset matching threshold as the target driver profile for matching with the sample driver profile, and optimizing the vehicle driving scenario display model based on the assisted driving scenario data corresponding to the target driver profile.

2. The method according to claim 1, characterized in that, Obtain sample driving scenario data, including: Collect vehicle driving scenario data when a user triggers the vehicle driving scenario display system; The vehicle driving scenario data is cleaned to remove invalid data and obtain valid driving scenario data. The valid driving scenario data is subjected to data reduction processing to determine the sample driving scenario data.

3. The method according to claim 1, characterized in that, Using the sample driving scenario data, a neural network model is trained to obtain a vehicle driving scenario display model that matches the driver identity information of the sample, including: Extract sample scene feature data from the sample driving scene data; wherein, the sample scene feature data includes training sample feature data and test sample feature data; The neural network model is trained using the training sample feature data to obtain the trained neural network model. The trained neural network model is tested using the feature data of the test samples. Based on the test results, a vehicle driving scenario display model that matches the driver identity information of the sample is determined.

4. The method according to claim 1, characterized in that, Using the sample driving scenario data, a neural network model is trained to obtain a vehicle driving scenario display model that matches the driver identity information of the sample, including: The sample driving scenario data is divided into model training sample data and model optimization sample data; The model training sample data is divided into first sample data and second sample data; wherein, the first sample data includes: sample driving scenario data including sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver status information and sample scenario display information; the second sample data includes: sample driver identity information, sample driving scenario data including sample vehicle driving environment data, sample driver operation information, sample driver status information and sample scenario display information; The neural network model is trained using the first sample data to determine the first classifier in the neural network model; The second sample data is used to train the neural network model to determine the second classifier of the neural network model; The model optimization sample data is filtered using the first classifier and the second classifier to determine the target optimization sample data; The first classifier and the second classifier are optimized based on the target optimized sample data to obtain a vehicle driving scene display model that matches the driver identity information of the sample.

5. The method according to claim 4, characterized in that, The model optimization sample data is filtered using the first classifier and the second classifier to determine the target optimization sample data, including: The model optimization sample data is divided into third sample data and fourth sample data; Based on the third sample data and the first classifier, the first confidence level corresponding to the first predicted scene display information is obtained; Based on the fourth sample data and the second classifier, determine the second confidence level corresponding to the second predicted scene display information; The model optimization sample data is filtered based on the first confidence level and the second confidence level to determine the target optimization sample data.

6. A method for displaying vehicle driving scenarios, characterized in that, include: Acquire target driving scenario data; The target driving scenario data includes: target driver identity information, target vehicle status data, target vehicle driving environment data, target driver operation information, and target driver status information; Based on the vehicle driving scenario display model, the display information of the vehicle driving scenario is determined according to the target driving scenario data and the target driver identity information; wherein, the vehicle driving scenario display model is trained based on the model training method described in any one of claims 1-5.

7. A model training device, characterized in that, include: The sample data acquisition module is used to acquire sample driving scenario data; wherein, the sample driving scenario data includes: sample driver identity information, sample vehicle status data, sample vehicle driving environment data, sample driver operation information, sample driver status information, and sample scenario display information; The model training module is used to train the neural network model using the sample driving scenario data to obtain a vehicle driving scenario display model that matches the driver identity information of the sample; wherein, the vehicle driving scenario display model is used to determine the vehicle driving scenario displayed in the vehicle driving scenario display system. The model optimization module is used to optimize the vehicle driving scenario display model using assisted driving scenario data; wherein, the assisted driving scenario data is matched with sample driving scenario data; the assisted driving scenario data includes: assisted driver identity information, assisted vehicle status data, assisted vehicle driving environment data, assisted driver operation information, assisted driver status information, and assisted scenario display information; The model optimization module is specifically used for: determining a sample driver profile based on the sample driver's identity information, sample driver's status information, and sample driver's operation information; determining an auxiliary driver profile based on the auxiliary driver's identity information, auxiliary driver's status information, and auxiliary driver's operation information; matching the auxiliary driver profile with the sample driver profile, selecting the auxiliary driver profile with a matching degree greater than a preset matching threshold as the target driver profile to match the sample driver profile, and optimizing the vehicle driving scene display model based on the assisted driving scene data corresponding to the target driver profile.

8. A vehicle driving scene display device, characterized in that, include: The target data acquisition module is used to acquire target driving scenario data; The target driving scenario data includes: target driver identity information, target vehicle status data, target vehicle driving environment data, target driver operation information, and target driver status information; The display information determination module is used to determine the display information of the vehicle driving scene based on the vehicle driving scene display model, according to the target driving scene data and the target driver identity information; wherein, the vehicle driving scene display model is trained based on the model training method described in any one of claims 1-5.

9. An electronic device, characterized in that, The device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the model training method as described in any one of claims 1-5, or implement the vehicle driving scene display method as described in claim 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the model training method as described in any one of claims 1-5, or the vehicle driving scene display method as described in claim 6.

Images