Simulated driving control method, device, equipment, medium, product and vehicle

By acquiring and processing vehicle sensor data, and generating accurate virtual driving scenarios, the problem of insufficient detailed simulation in existing simulated driving technology is solved, and a more realistic simulated driving experience is achieved.

CN119091723APending Publication Date: 2024-12-06DEEPAL AUTOMOBILE TECH CO LTD

Patent Information

Application Number
CN202411189545.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The accuracy of virtual scenes in the existing simulated driving technology is insufficient, resulting in the inadequate detail simulation during simulated driving, resulting in a gap between the simulated driving experience and the actual situation.

Method used

By obtaining vehicle sensor data, determining vehicle posture information and driving scene data, coordinator system transformation and physical attribute assignment, generating more accurate virtual driving scenes, including road markings, obstacles and vehicle motion models, to improve simulated driving scenes authenticity.

Benefits of technology

It improves the accuracy and authenticity of simulated driving scenes, providing users with a more realistic simulated driving experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091723A_ABST
    Figure CN119091723A_ABST
Patent Text Reader

Abstract

The invention relates to a simulated driving control method, device and equipment, a medium, a product and a vehicle. The method comprises the following steps: acquiring sensor data of the vehicle; determining vehicle data according to the sensor data; the pose information of the vehicle is determined according to the sensor data; according to the pose information, performing coordinate system transformation processing on the vehicle driving data and the obstacle data to obtain transformed vehicle driving data and transformed obstacle data; generating and displaying a virtual driving scene corresponding to the vehicle according to preset physical parameters, the transformed vehicle driving data and the transformed obstacle data; and controlling the vehicle to perform simulated driving based on the virtual driving scene corresponding to the vehicle. According to the mode adopted by the invention, the authenticity of simulated driving and the user experience can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of simulated driving technology, and in particular to a simulated driving control method, device, equipment, medium, product and vehicle. Background Art

[0002] With the development of vehicle technology and related science and technology, vehicles have become an indispensable means of transportation in people's daily lives; based on the auxiliary functions of vehicles, driving training, racing games and other simulated driving services can be provided to users of vehicles.

[0003] At present, the related technology adopts the construction of virtual scenes to put users in a virtual driving environment, so that users can simulate driving through a simulated driver. However, due to the insufficient accuracy of the constructed virtual scenes, the details of the simulated driving process are not simulated finely enough, resulting in a gap between the simulated driving experience and the actual situation. Summary of the invention

[0004] One of the purposes of the present invention is to provide a simulated driving control method to solve the problem in the prior art that the simulation of details in the simulated driving process is not fine enough, resulting in a gap between the simulated driving experience and the actual situation; the second purpose is to provide a simulated driving control device; the third purpose is to provide an electronic device; the fourth purpose is to provide a computer-readable storage medium; the fifth purpose is to provide a computer program product; and the sixth purpose is to provide a vehicle.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A simulated driving control method, comprising:

[0007] Acquire sensor data of the vehicle; determine vehicle data based on the sensor data; wherein the vehicle data represents a driving scene in which the vehicle is located; the vehicle data includes vehicle driving data and obstacle data; and determine the position information of the vehicle based on the sensor data;

[0008] According to the position information, coordinate system transformation processing is performed on the vehicle driving data and the obstacle data to obtain transformed vehicle driving data and transformed obstacle data;

[0009] Generate and display a virtual driving scene corresponding to the vehicle according to preset physical parameters, the transformed vehicle driving data and the transformed obstacle data; wherein the physical parameters are road physical property parameters and collision parameters during vehicle driving;

[0010] Based on the virtual driving scene corresponding to the vehicle, the vehicle is controlled to perform simulated driving.

[0011] Further, according to the preset physical parameters, the transformed vehicle driving data and the transformed obstacle data, a virtual driving scene corresponding to the vehicle is generated and displayed, including:

[0012] Drawing a road marking model according to the lane line data in the transformed vehicle driving data;

[0013] generating at least one obstacle model according to the transformed obstacle data;

[0014] Constructing a vehicle motion model based on the vehicle motion data in the transformed vehicle driving data; wherein the vehicle motion model represents the motion conditions of surrounding vehicles in the game virtual scene where the vehicle is located;

[0015] According to the physical parameters, physical value assignment processing is performed on the road marking model, each obstacle model and the vehicle motion model to generate and display a virtual driving scene corresponding to the vehicle.

[0016] Further, at least one obstacle model is generated according to the transformed obstacle data, including:

[0017] generating at least one initial obstacle model according to the transformed obstacle data;

[0018] Determining distance information between each of the initial obstacle models; wherein the distance information includes the distance between any two of the initial obstacle models;

[0019] According to the distance information, each of the initial obstacle models is corrected to obtain each of the obstacle models.

[0020] Furthermore, the method further comprises:

[0021] The virtual driving scene corresponding to the vehicle is subjected to detail enhancement processing, and the virtual driving scene corresponding to the vehicle is processed according to the weather data and the illumination data in the sensor data.

[0022] Further, based on the virtual driving scene corresponding to the vehicle, controlling the vehicle to perform simulated driving includes:

[0023] Receiving a control instruction sent by a target interactive device in the vehicle, wherein the control instruction is generated by the target interactive device in response to an operation of the target interactive device by a user;

[0024] Determine and display the motion mode information of the vehicle according to the control instruction; wherein the motion mode information represents the driving behavior of the vehicle on the road;

[0025] Based on the virtual driving scene corresponding to the vehicle and according to the motion mode information of the vehicle, the vehicle is controlled to execute the simulated driving operation indicated by the control instruction.

[0026] Further, determining vehicle data according to the sensor data includes:

[0027] Determine the road information and key road information of the vehicle according to the sensor data; wherein the road information represents a geometric model of the road surface on which the vehicle is traveling; and the key road information represents signs and markings of the road on which the vehicle is traveling;

[0028] Determine the surrounding vehicle information of the vehicle according to the sensor data; wherein the surrounding vehicle information represents the position, speed and movement trajectory of vehicles around the vehicle;

[0029] Determine the traffic flow information of the vehicle according to the surrounding vehicle information; wherein the traffic flow information represents the traffic flow conditions on the road on which the vehicle is traveling;

[0030] The vehicle driving data and obstacle data in the vehicle data are generated according to the road information, surrounding vehicle information and traffic flow information of the vehicle.

[0031] Furthermore, the vehicle data also includes fusion data, and the method further includes:

[0032] Preprocessing the sensor data to obtain processed sensor data;

[0033] The point cloud data and the image data in the processed sensor data are fused to obtain fused data in the vehicle data.

[0034] Furthermore, the method further comprises:

[0035] The vehicle driving data and the obstacle data are respectively subjected to format conversion processing to obtain processed vehicle driving data and processed obstacle data.

[0036] Furthermore, the method further comprises:

[0037] In response to a simulated driving signal sent by at least one other vehicle, a virtual driving scene corresponding to each of the other vehicles is obtained to update the virtual driving scene corresponding to the vehicle; wherein the simulated driving signal indicates that the other vehicles are in a simulated driving state.

[0038] According to the above-mentioned technical means, the vehicle posture information is determined through sensor data, and the coordinate system transformation and physical property assignment of vehicle driving data and obstacle data are performed to restore the real road conditions in the virtual scene to improve the authenticity of the simulated driving scene.

[0039] A simulated driving control device, comprising:

[0040] A determination module, used to obtain sensor data of a vehicle; determine vehicle data according to the sensor data; wherein the vehicle data represents a driving scene in which the vehicle is located; the vehicle data includes vehicle driving data and obstacle data; and determine the position information of the vehicle according to the sensor data;

[0041] A transformation module, used to perform coordinate system transformation processing on the vehicle driving data and the obstacle data respectively according to the posture information to obtain transformed vehicle driving data and transformed obstacle data;

[0042] A generation module, used to generate and display a virtual driving scene corresponding to the vehicle according to preset physical parameters, the transformed vehicle driving data and the transformed obstacle data; wherein the physical parameters are road physical property parameters and collision parameters during vehicle driving;

[0043] The control module is used to control the vehicle to perform simulated driving based on the virtual driving scene corresponding to the vehicle.

[0044] Furthermore, the generation module is specifically used for:

[0045] Drawing a road marking model according to the lane line data in the transformed vehicle driving data;

[0046] generating at least one obstacle model according to the transformed obstacle data;

[0047] Constructing a vehicle motion model based on the vehicle motion data in the transformed vehicle driving data; wherein the vehicle motion model represents the motion conditions of surrounding vehicles in the game virtual scene where the vehicle is located;

[0048] According to the physical parameters, physical value assignment processing is performed on the road marking model, each obstacle model and the vehicle motion model to generate and display a virtual driving scene corresponding to the vehicle.

[0049] Furthermore, the generation module is specifically used for:

[0050] generating at least one initial obstacle model according to the transformed obstacle data;

[0051] Determining distance information between each of the initial obstacle models; wherein the distance information includes the distance between any two of the initial obstacle models;

[0052] According to the distance information, each of the initial obstacle models is corrected to obtain each of the obstacle models.

[0053] Furthermore, the generation module is also specifically used for:

[0054] The virtual driving scene corresponding to the vehicle is subjected to detail enhancement processing, and the virtual driving scene corresponding to the vehicle is processed according to the weather data and the illumination data in the sensor data.

[0055] Furthermore, the control module is specifically used for:

[0056] Receiving a control instruction sent by a target interactive device in the vehicle, wherein the control instruction is generated by the target interactive device in response to an operation of the target interactive device by a user;

[0057] Determine and display the motion mode information of the vehicle according to the control instruction; wherein the motion mode information represents the driving behavior of the vehicle on the road;

[0058] Based on the virtual driving scene corresponding to the vehicle and according to the motion mode information of the vehicle, the vehicle is controlled to execute the simulated driving operation indicated by the control instruction.

[0059] Further, the determination module is specifically used for:

[0060] Determine the road information and key road information of the vehicle according to the sensor data; wherein the road information represents a geometric model of the road surface on which the vehicle is traveling; and the key road information represents signs and markings of the road on which the vehicle is traveling;

[0061] Determine the surrounding vehicle information of the vehicle according to the sensor data; wherein the surrounding vehicle information represents the position, speed and movement trajectory of vehicles around the vehicle;

[0062] Determine the traffic flow information of the vehicle according to the surrounding vehicle information; wherein the traffic flow information represents the traffic flow conditions on the road on which the vehicle is traveling;

[0063] The vehicle driving data and obstacle data in the vehicle data are generated according to the road information, surrounding vehicle information and traffic flow information of the vehicle.

[0064] Furthermore, the determining module is further specifically used for:

[0065] Preprocessing the sensor data to obtain processed sensor data;

[0066] The point cloud data and the image data in the processed sensor data are fused to obtain fused data in the vehicle data.

[0067] Furthermore, the device is also used for:

[0068] The vehicle driving data and the obstacle data are respectively subjected to format conversion processing to obtain processed vehicle driving data and processed obstacle data.

[0069] Furthermore, the device is also used for:

[0070] In response to a simulated driving signal sent by at least one other vehicle, a virtual driving scene corresponding to each of the other vehicles is obtained to update the virtual driving scene corresponding to the vehicle; wherein the simulated driving signal indicates that the other vehicles are in a simulated driving state.

[0071] An electronic device comprising a memory and a processor;

[0072] The memory stores computer-executable instructions;

[0073] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0074] A computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the above first aspect and / or various possible implementations of the first aspect.

[0075] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the computer program implements the above first aspect and / or various possible implementation modes of the first aspect.

[0076] A vehicle comprises an electronic device that implements the above first aspect and / or various possible implementations of the first aspect.

[0077] Beneficial effects of the present invention:

[0078] (1) The present invention uses data from real driving scenes to construct virtual scenes, thereby improving the authenticity of simulated driving scenes;

[0079] (2) The present invention is based on coordinate system conversion and physical property assignment processing of data in real driving scenes, which can improve the accuracy of simulated driving scenes and provide users with a more realistic simulated driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A schematic diagram of the structure of a vehicle provided by an embodiment of the present invention;

[0081] Figure 2 The process of the simulated driving control method provided by one embodiment of the present invention is Figure 1 ;

[0082] Figure 3 The process of the simulated driving control method provided by one embodiment of the present invention is Figure 2 ;

[0083] Figure 4 A flowchart of a real road segment augmented reality game provided by an embodiment of the present invention;

[0084] Figure 5 A schematic diagram of the structure of a simulated driving control device provided by an embodiment of the present invention;

[0085] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0086] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0087] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0088] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and corresponding operation entrances shall be provided for users to choose to authorize or refuse.

[0089] First, the nouns involved in the present invention are explained:

[0090] Augmented Reality (AR): AR technology uses optoelectronic display technology, interactive technology, multiple sensor technologies, computer graphics and multimedia technology to integrate the computer-generated virtual environment with the real environment around the user, so that the user can be convinced from the sensory effect that the virtual environment is an integral part of the real environment around him.

[0091] With the development of vehicle technology and related science and technology, vehicles have become an indispensable means of transportation in people's daily lives; based on the auxiliary functions of vehicles, driving training, racing games and other simulated driving services can be provided to users of vehicles.

[0092] In one example, traffic data is collected and simulated using real-time traffic scenes based on real roads to build a virtual scene, allowing users to be in a virtual driving environment and perform simulated driving through a simulated driver. However, due to the lack of accuracy in building the virtual scene, the details of the simulated driving process are not simulated in detail, resulting in a gap between the simulated driving experience and the actual situation.

[0093] In view of this, an embodiment of the present invention proposes a simulated driving control method, which determines the vehicle posture information through sensor data, and performs coordinate system transformation and physical property assignment on data such as vehicle driving data and obstacle data, so as to restore the real road conditions in the virtual scene and improve the authenticity of the simulated driving scene.

[0094] Figure 1 FIG. 1 is a schematic diagram of the structure of a vehicle provided by an embodiment of the present invention. Figure 1 As shown, the scene may include vehicles and users. The vehicle is used to process sensor data, control vehicle behavior, or interact with users. For example, Figure 1 As shown, in a racing game scene, the vehicle may include multiple sensors 101, electronic devices 102, interactive devices 103, and a feedback system 104; wherein, multiple sensors may be used to collect sensor data, electronic devices are used to process and display augmented reality virtual scenes, interactive devices may be used to interact with users, etc., and the feedback system is used to provide visual, auditory, and tactile feedback, allowing players to obtain a realistic driving experience.

[0095] The execution subject of the embodiment of the present invention may be an electronic device of the vehicle, such as a vehicle computer or other controller, which may be specifically configured according to actual needs.

[0096] The technical solution of the present invention is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0097] Figure 2 The process of the simulated driving control method provided by one embodiment of the present invention is Figure 1 .like Figure 2 As shown, the method includes:

[0098] 201. Acquire sensor data of a vehicle; determine vehicle data based on the sensor data; wherein the vehicle data represents a driving scene in which the vehicle is located; the vehicle data includes vehicle driving data and obstacle data; and determine the position information of the vehicle based on the sensor data.

[0099] Exemplarily, during actual driving, a vehicle collects sensor data through multiple sensors in the vehicle. For example, sensors such as lidar, camera, and radar are equipped on the vehicle to collect environmental data. Lidar is used for high-precision three-dimensional scanning of road structures to obtain point cloud data, and cameras are used to identify image data such as traffic signs, road signs, and surrounding vehicle information. The sensor data collected by the sensors are obtained through electronic equipment in the vehicle, and the collected sensor data is processed to determine vehicle data to characterize the driving scene in which the vehicle is located. The vehicle data includes vehicle driving data and obstacle data. The vehicle driving data includes lane line data and vehicle motion data. The lane line data mainly describes the geometric shape and position of the road, including the coordinates, type (solid line, dashed line, etc.) and attributes (lane width, line color, etc.) of the lane line. The vehicle motion data includes the motion status of surrounding vehicles, such as position, speed, acceleration, and direction. The obstacle data characterizes static or dynamic obstacles on the road, including their position, size, and type (such as vehicles, pedestrians, roadblocks, etc.). At the same time, the collected sensor data is also processed to obtain the vehicle's posture information, including the sensor installation position in the vehicle that collects the sensor data and the vehicle's own posture (position and direction).

[0100] For example, after obtaining sensor data, based on the data model construction method, the pre-processed and analyzed data is integrated into a unified data model, including road geometry, traffic signs, surrounding environment and other vehicle information, to obtain a data model, which includes vehicle data, and the constructed data model is verified, compared with the actual scene, and optimized based on feedback to ensure the accuracy and reliability of the model. The above data analysis steps are to convert the original sensor data into models and information that can be used to restore the game scene. These models will be used by the game engine to create a three-dimensional scene of real road conditions, providing players with a more realistic driving experience.

[0101] 202. According to the position information, coordinate system transformation processing is performed on the vehicle driving data and the obstacle data respectively to obtain transformed vehicle driving data and transformed obstacle data.

[0102] For example, as mentioned above, vehicle driving data and obstacle data usually use the local coordinate system of the vehicle. Therefore, after determining the vehicle's posture information, the vehicle driving data and obstacle data in the vehicle data are subjected to coordinate system transformation processing and converted into the global coordinate system of the target virtual scene such as a game virtual scene to obtain the transformed vehicle driving data and transformed obstacle data.

[0103] 203. Generate and display a virtual driving scene corresponding to the vehicle according to preset physical parameters, transformed vehicle driving data, and transformed obstacle data; wherein the physical parameters are road physical property parameters and collision parameters during vehicle driving.

[0104] Exemplarily, the electronic device in the vehicle uses the processed and converted data to construct a virtual driving scene in the game engine, that is, the road structure, surrounding environment and vehicle position represented by the transformed vehicle driving data and the transformed obstacle data are restored into elements in the virtual driving scene using 3D modeling software. At the same time, based on the game engine, the elements in the virtual driving scene, the transformed vehicle driving data and the transformed obstacle data can be used to restore the real road conditions in the virtual driving environment, including road geometry, vehicle movement and traffic signs, etc., and in the process of restoring the real road conditions, preset physical parameters can be obtained, such as road physical property parameters such as friction and gravity during vehicle driving, and collision parameters of the collision model. According to the physical properties of the real world, such as friction, gravity and collision model, objects in the virtual scene are assigned values ​​to achieve realistic physical effects; further, a virtual driving scene corresponding to the real driving scene in which the vehicle is located is generated, such as an augmented reality virtual scene generated in a game application scenario; and the generated virtual driving scene can be displayed through a display device in the electronic device of the vehicle to provide users with a realistic simulated driving experience.

[0105] 204. Based on the virtual driving scene corresponding to the vehicle, control the vehicle to perform simulated driving.

[0106] For example, when the electronic device completes and displays the virtual driving scene corresponding to the vehicle, the user can obtain control operations from the electronic device on the vehicle, such as designing the user's interaction method and interface to input control instructions for acceleration or deceleration or start or stop, so that the electronic device can control the vehicle to perform simulated driving to achieve a realistic driving experience.

[0107] For example, in a game scenario, players interact with the virtual environment through the game interface, such as using handles, steering wheels and other devices to perform driving operations. Players experience the game based on real road conditions, including road bends, traffic sign compliance and interaction with other vehicles. Visual, auditory and tactile feedback systems can also be designed to give players a real driving experience. Visual feedback includes real road and environment rendering, auditory feedback includes vehicle engine sound and environmental noise, and tactile feedback includes steering wheel vibration and force feedback. The game system simulates real driving experience based on real-time data to help players practice driving skills.

[0108] The simulated driving control method provided in the embodiment of the present invention determines the vehicle posture information and the vehicle driving data and obstacle data in the vehicle data through sensor data, and constructs a virtual driving scene through the transformation of the coordinate system and the assignment of physical properties, so as to restore the real road conditions so that the user can perform simulated driving. In this way, a more realistic virtual driving scene restoration and real road condition simulation can be achieved, providing users with an immersive driving experience and rich interactive feelings.

[0109] Figure 3 The process of the simulated driving control method provided by one embodiment of the present invention is Figure 2 .like Figure 3 As shown, in this embodiment Figure 2 Based on the illustrated embodiment, the simulated driving control method provided by the present invention is described in detail, and the method includes:

[0110] 301. Acquire sensor data of a vehicle; determine vehicle data based on the sensor data; wherein the vehicle data represents a driving scene in which the vehicle is located; the vehicle data includes vehicle driving data and obstacle data; and determine the position information of the vehicle based on the sensor data.

[0111] It should be noted that this step is similar to the aforementioned step 201 and will not be repeated here.

[0112] In a possible implementation, determining vehicle data according to sensor data includes:

[0113] Step 1: Determine the road information and key road information of the vehicle based on the sensor data; wherein the road information represents the geometric model of the road surface on which the vehicle is traveling; and the key road information represents the signs and markings of the road on which the vehicle is traveling.

[0114] Step 2: Determine the surrounding vehicle information of the vehicle based on the sensor data; wherein the surrounding vehicle information represents the position, speed and movement trajectory of the vehicles around the vehicle.

[0115] Step 3: Determine the traffic flow information of the vehicle based on the surrounding vehicle information; wherein the traffic flow information represents the traffic conditions on the road on which the vehicle is traveling.

[0116] Step 4: Generate vehicle driving data and obstacle data in the vehicle data based on the vehicle's road information, surrounding vehicle information, and traffic flow information.

[0117] Specifically, Figure 4 This is a flow chart of a real road segment augmented reality game provided by an embodiment of the present invention. Figure 4 As shown, after the sensor of the vehicle collects data, the electronic device can process the point cloud data collected by the laser radar in the sensor data, such as performing ground extraction, road boundary recognition, etc., to reconstruct the geometric model of the road surface. At the same time, the image data in the sensor data is processed through the image processing algorithm to identify key information such as signs and markings on the road, and map it to the road geometric model, complete the road surface reconstruction and road sign and marking recognition process, and obtain road information. Then, the image data in the sensor data is processed through the image processing algorithm to detect surrounding vehicles and track their positions, speeds and movement trajectories, and traffic flow analysis is performed based on the number, density and speed of vehicles to simulate the traffic conditions on the real road, complete vehicle detection and tracking and traffic flow statistics, and obtain surrounding vehicle information and traffic flow information; the above information obtained through processing and analysis is integrated into a unified data model, including vehicle driving data and obstacle data in the vehicle data, to characterize road geometry, traffic signs, surrounding environment and other vehicles.

[0118] In a possible implementation, the method further includes:

[0119] Step 1: pre-process the sensor data to obtain processed sensor data.

[0120] Step 2: Fuse the point cloud data and image data in the processed sensor data to obtain fused data in the vehicle data.

[0121] Specifically, Figure 4 As shown in the figure, during the data preprocessing process, the sensor data may contain noise and inaccurate information. In the data preprocessing stage, the data needs to be cleaned, outliers or erroneous data need to be removed, and filtering algorithms need to be applied to reduce the impact of noise on subsequent analysis. At the same time, during the data recording and analysis process, the data collected by different sensors need to be fused to obtain more comprehensive and accurate environmental information. The point cloud data of the lidar and the image data of the camera in the processed sensor data can be combined for fusion processing to obtain fused data in the vehicle data to enhance the understanding of the surrounding environment.

[0122] For example, outliers can be identified and deleted by setting thresholds or using statistical methods, such as the formula: data point > mean + n × standard deviation; missing data can be identified and filled or deleted, and the filling formulas are: mean value, median value, and interpolation method; based on the moving average filtering method, the signal is smoothed by calculating the average value of the data points in the window, and the formula is: Among them, N represents the number of data points considered by the filter, i represents the position of the currently processed data point in the window, and x i It represents the value of the current data point; the state of the system is estimated using state estimation and measurement update. At the same time, the information from different sensors is fused to obtain more comprehensive and accurate data through the formula: fused data = coefficient 1 × data source 1 + coefficient 2 × data source 2 + ..., and different features or feature sets such as vehicle type, lane type, etc. are fused to improve the richness of information through the formula: fused feature = feature 1 ⊕ feature 2.

[0123] 302. According to the position information, coordinate system transformation processing is performed on the vehicle driving data and the obstacle data respectively to obtain transformed vehicle driving data and transformed obstacle data.

[0124] It should be noted that this step is similar to the aforementioned step 201 and will not be repeated here.

[0125] In a possible implementation manner, the method further includes: performing format conversion processing on the vehicle driving data and the obstacle data respectively to obtain processed vehicle driving data and processed obstacle data.

[0126] Specifically, before the coordinate system transformation process is performed, it is necessary to define and use standardized data formats and structures for use in the game engine. Therefore, by writing scripts or using existing tools and plug-ins, the vehicle driving data and obstacle data can be converted into formats supported by the game engine, such as three-dimensional models, textures, maps, etc., to obtain processed vehicle driving data and processed obstacle data, and then continue to perform coordinate system transformation processing through step 202. Through this step, the collected vehicle data is converted into a format available to the game engine after processing and analysis, and efficient scene restoration and real road condition simulation can be achieved to improve the authenticity of the virtual driving scene corresponding to the constructed game, so as to provide users with a more realistic driving experience.

[0127] 303. Draw a road marking model according to the lane line data in the transformed vehicle driving data.

[0128] Exemplarily, the electronic device uses a drawing function to process the lane line data in the transformed vehicle driving data and draw a road marking model.

[0129] For example, in a racing game scenario, you can use a preset game engine and the drawing function of the game engine to draw road markings based on lane line data, and you can use a line renderer or mapping technology to achieve lane lines of different types and colors.

[0130] 304. Generate at least one obstacle model according to the transformed obstacle data.

[0131] Exemplarily, the electronic device uses simulation technology to perform simulation processing on the transformed obstacle data to generate corresponding multiple obstacle models.

[0132] For example, in a racing game scenario, corresponding object models are generated in the game scenario based on obstacle data. These models can be selected from existing resource libraries, or new models can be created as needed.

[0133] In a possible implementation, step 304 includes:

[0134] Step 1: Generate at least one initial obstacle model based on the transformed obstacle data.

[0135] Step 2: Determine the distance information between each initial obstacle model; wherein the distance information includes the distance between any two initial obstacle models.

[0136] Step 3: According to the distance information, each initial obstacle model is corrected to obtain each obstacle model.

[0137] Specifically, in order to ensure that the size and position of the model are consistent with the real data, the electronic device performs mechanical energy simulation processing on the transformed obstacle data to generate multiple corresponding initial obstacle models, and calculates the distance between each pair of initial obstacle models in the preset coordinate system, and determines whether there is overlap between each pair of initial obstacle models based on each distance. If so, the corresponding obstacle model can be corrected, for example, the overlapping area can be deleted according to the projection method, so as to obtain a more accurate obstacle model to ensure that the size and position of the model are consistent with the real data.

[0138] 305. Construct a vehicle motion model based on the vehicle motion data in the transformed vehicle driving data; wherein the vehicle motion model represents the motion conditions of surrounding vehicles in the game virtual scene where the vehicle is located.

[0139] Exemplarily, the electronic device may use a physics engine to calculate and process the vehicle motion data in the transformed vehicle driving data to obtain the motion trajectories of surrounding vehicles, and then simulate the motion of surrounding vehicles in a virtual scene.

[0140] 306. Perform physical value assignment processing on the road marking model, obstacle models and vehicle motion model according to the physical parameters to generate and display a virtual driving scene corresponding to the vehicle.

[0141] Exemplarily, the electronic device performs physical value assignment processing on the generated road marking model, obstacle models and vehicle motion model based on the acquired physical parameters, such as adding lane lines, obstacles and vehicle motion conditions to the model of the virtual scene, and applying corresponding materials and textures based on the physical parameters. During this process, collisions and interactions between vehicles can be considered, and then a virtual driving scene corresponding to the generated vehicle is constructed and displayed; Figure 4 As shown, in the game scene, that is, when restoring the game scene, the model construction and road condition construction corresponding to the game scene are carried out to obtain the game scene, so that users can perform AR game driving simulation.

[0142] In a possible implementation, the method further includes: performing detail enhancement processing on the virtual driving scene corresponding to the vehicle, and processing the virtual driving scene corresponding to the vehicle according to weather data and lighting data in the sensor data.

[0143] Specifically, in order to further improve the degree of restoration of the game scene and the simulation effect of real road conditions, for the already constructed virtual driving scene, based on image processing technology, the details of the virtual driving scene corresponding to the vehicle are enhanced. At the same time, the weather data and lighting data in the sensor data are obtained, and the virtual driving scene corresponding to the vehicle is processed to adjust the ambient lighting and weather conditions of the game scene.

[0144] For example, environmental details around the road can be added, such as buildings, trees, and street lights, to provide more realistic visual effects; high-precision 3D models and textures can be used to improve the realism of the scene; real-time weather and lighting effects can be introduced to dynamically adjust the ambient lighting and weather conditions of the game scene according to real weather data and time, such as sunny days, rainy days, snowy days, and night; more advanced physics engines can be used to simulate the interaction between vehicles and the environment, such as vehicle suspension systems, tire friction, and aerodynamics, to provide a more realistic driving experience.

[0145] 307. Based on the virtual driving scene corresponding to the vehicle, control the vehicle to perform simulated driving.

[0146] It should be noted that this step is similar to the aforementioned step 201 and will not be repeated here.

[0147] In a possible implementation, step 307 includes:

[0148] Step 1: receiving a control instruction sent by a target interactive device in a vehicle, wherein the control instruction is generated by the target interactive device in response to an operation of the target interactive device by a user.

[0149] Step 2: Determine and display the vehicle's motion mode information according to the control command; wherein the motion mode information represents the vehicle's driving behavior on the road.

[0150] Step 3: Based on the virtual driving scene corresponding to the vehicle and according to the vehicle's motion mode information, the vehicle is controlled to execute the simulated driving operation indicated by the control instruction.

[0151] Specifically, the electronic device receives the control instruction sent by the target interactive device, and analyzes and processes the acquired sensor data according to the control instruction, determines the motion mode information of the vehicle to characterize the driving behavior of the vehicle on the road, and displays it. Then, the electronic device can control the vehicle to execute the simulated driving operations indicated by the control instruction, such as acceleration, deceleration, turning, etc., according to the vehicle's motion mode information in the displayed virtual driving scene.

[0152] For example, by using sensor data such as an inertial measurement unit to track the real-time position, direction and speed of the vehicle, the motion mode can be identified based on the vehicle position and sensor data, such as acceleration, deceleration, and turning; players can turn on the racing game mode, and after turning on the mode, they can step on the accelerator, brake and other interactive devices to determine the vehicle's motion mode, and feedback to the head-up display (HUD), so that the vehicle can be controlled to perform simulated driving operations according to the vehicle's motion mode. Then, by identifying and restoring the vehicle's motion mode, the vehicle's driving behavior on a real road can be more accurately simulated. At this time, the vehicle will not actually be started, and players can immersively experience the real-scene racing game behavior, thereby improving the user's simulated driving experience.

[0153] 308. Respond to a simulated driving signal sent by at least one other vehicle, and obtain virtual driving scenes corresponding to each other vehicle to update the virtual driving scenes corresponding to the vehicle; wherein the simulated driving signal indicates that the other vehicle is in a simulated driving state.

[0154] Exemplarily, after the current vehicle enters the simulated driving state, the electronic equipment of the vehicle can monitor the vehicle status of the surrounding vehicles. If the simulated driving signals sent by multiple other vehicles are received, it means that there are other vehicles in the simulated driving state, and they can perform simulated driving with the current vehicle in the same virtual driving scene; for example, in the game scene, a multi-player mode is introduced to allow multiple players to interact and compete in the same virtual scene, increasing the fun and competitiveness of the game. At the same time, the data of the real road section can be regularly updated and expanded, and new roads and scenes can be added. As mentioned above, when multiple other vehicles are performing simulated driving with the current vehicle in the same virtual driving scene, the corresponding virtual driving scenes can be obtained from each other vehicle, and the obtained virtual driving scenes of each other can be merged with the virtual driving scene of the current vehicle to obtain a new virtual driving scene. Furthermore, new roads and scenes are added to allow users to experience different driving environments and challenges to improve the user's simulated driving experience.

[0155] The simulated driving control method provided by the embodiment of the present invention can improve the accuracy of the data by preprocessing and fusing the vehicle data obtained based on the sensor data; determine the vehicle posture information and the vehicle driving data and obstacle data in the vehicle data through the sensor data, and construct a road marking model, an obstacle model and a vehicle motion model through the transformation of the coordinate system and the assignment of physical properties, and then construct a more accurate and more realistic virtual driving scene after repairing the obtained obstacle model, so as to achieve more realistic virtual driving scene restoration and real road condition simulation, and provide users with an immersive driving experience.

[0156] Figure 5 A schematic diagram of the structure of a simulated driving control device provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the driving simulation control device of the embodiment of the present invention comprises:

[0157] The determination module 401 is used to obtain sensor data of the vehicle; determine vehicle data according to the sensor data; wherein the vehicle data represents the driving scene of the vehicle; the vehicle data includes vehicle driving data and obstacle data; and determine the position information of the vehicle according to the sensor data;

[0158] The transformation module 402 is used to perform coordinate system transformation processing on the vehicle driving data and the obstacle data according to the position information to obtain the transformed vehicle driving data and the transformed obstacle data;

[0159] The generation module 403 is used to generate and display a virtual driving scene corresponding to the vehicle according to preset physical parameters, the transformed vehicle driving data and the transformed obstacle data; wherein the physical parameters are the road physical property parameters and collision parameters during the vehicle driving process;

[0160] The control module 404 is used to control the vehicle to perform simulated driving based on the virtual driving scene corresponding to the vehicle.

[0161] In some embodiments, the generation module 403 is specifically used to: draw a road marking model based on the lane line data in the transformed vehicle driving data; generate at least one obstacle model based on the transformed obstacle data; construct a vehicle motion model based on the vehicle motion data in the transformed vehicle driving data; wherein the vehicle motion model represents the movement of surrounding vehicles in the virtual game scene where the vehicle is located; and perform physical assignment processing on the road marking model, each obstacle model and the vehicle motion model according to physical parameters to generate and display the virtual driving scene corresponding to the vehicle.

[0162] In some embodiments, the generation module 403 is specifically used to: generate at least one initial obstacle model based on the transformed obstacle data; determine the distance information between each initial obstacle model; wherein the distance information includes the distance between each pair of initial obstacle models; and perform correction processing on each initial obstacle model based on the distance information to obtain each obstacle model.

[0163] In some embodiments, the generation module 403 is further specifically used to: perform detail enhancement processing on the virtual driving scene corresponding to the vehicle, and process the virtual driving scene corresponding to the vehicle according to the weather data and lighting data in the sensor data.

[0164] In some embodiments, the control module 404 is specifically used to: receive a control instruction sent by a target interactive device in a vehicle, wherein the control instruction is generated by the target interactive device in response to an operation of the target interactive device by a user; determine and display the vehicle's motion mode information according to the control instruction; wherein the motion mode information represents the vehicle's driving behavior on the road; based on a virtual driving scene corresponding to the vehicle and according to the vehicle's motion mode information, control the vehicle to execute the simulated driving operation indicated by the control instruction.

[0165] In some embodiments, the determination module 401 is specifically used to: determine the vehicle's road information and road key information based on sensor data; wherein the road information represents the geometric model of the road surface on which the vehicle is traveling; the road key information represents the signs and markings of the road on which the vehicle is traveling; determine the vehicle's surrounding vehicle information based on the sensor data; wherein the surrounding vehicle information represents the position, speed and movement trajectory of vehicles around the vehicle; determine the vehicle's traffic flow information based on the surrounding vehicle information; wherein the traffic flow information represents the traffic flow conditions on the road on which the vehicle is traveling; generate vehicle driving data and obstacle data in the vehicle data based on the vehicle's road information, surrounding vehicle information and traffic flow information.

[0166] In some embodiments, the determination module 401 is further specifically used to: pre-process the sensor data to obtain processed sensor data; and fuse the point cloud data and image data in the processed sensor data to obtain fused data in the vehicle data.

[0167] In some embodiments, the device is further used to: perform format conversion processing on the vehicle driving data and the obstacle data respectively to obtain processed vehicle driving data and processed obstacle data.

[0168] In some embodiments, the device is also used to: respond to a simulated driving signal sent by at least one other vehicle, and obtain a virtual driving scene corresponding to each other vehicle, so as to update the virtual driving scene corresponding to the vehicle; wherein the simulated driving signal indicates that the other vehicle is in a simulated driving state.

[0169] The device of the embodiment of the present invention can be used to execute the technical solution of any of the above-mentioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0170] Figure 6 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 6 As shown, the electronic device may include: at least one processor 501 and a memory 502 .

[0171] The memory 502 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer-executable instructions.

[0172] The memory 502 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0173] The processor 501 is used to execute the computer-executable instructions stored in the memory 502 to implement the application operation method based on intelligent driving described in the aforementioned method embodiment. Among them, the processor 501 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. Specifically, when implementing the application operation method based on intelligent driving described in the aforementioned method embodiment, the electronic device can be, for example, an electronic control unit, a server or a domain controller on a vehicle and other electronic devices with processing functions.

[0174] Optionally, the electronic device may further include a communication interface 503. In a specific implementation, if the communication interface 503, the memory 502 and the processor 501 are implemented independently, the communication interface 503, the memory 502 and the processor 501 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0175] Optionally, in a specific implementation, if the communication interface 503, the memory 502 and the processor 501 are integrated on a chip, the communication interface 503, the memory 502 and the processor 501 can communicate through an internal interface.

[0176] The present invention also provides a computer-readable storage medium, in which computer program instructions are stored. When a processor executes the computer program instructions, the above-mentioned solution of the application operation method based on intelligent driving is implemented.

[0177] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned solution of the application running method based on intelligent driving.

[0178] The present invention also provides a vehicle, including an electronic device that implements the above-mentioned solution of the application running method based on intelligent driving.

[0179] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0180] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in a dedicated integrated circuit. Of course, the processor and the readable storage medium can also exist as discrete components in an application running device based on intelligent driving.

[0181] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0182] Finally, it should be noted that the above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. A simulated driving control method, characterized in that: The method comprises: Acquire sensor data of the vehicle; determine vehicle data based on the sensor data; wherein the vehicle data represents a driving scene in which the vehicle is located; the vehicle data includes vehicle driving data and obstacle data; and determine the position information of the vehicle based on the sensor data; According to the position information, coordinate system transformation processing is performed on the vehicle driving data and the obstacle data to obtain transformed vehicle driving data and transformed obstacle data; Generate and display a virtual driving scene corresponding to the vehicle according to preset physical parameters, the transformed vehicle driving data and the transformed obstacle data; wherein the physical parameters are road physical property parameters and collision parameters during vehicle driving; Based on the virtual driving scene corresponding to the vehicle, the vehicle is controlled to perform simulated driving.

2. The method according to claim 1, characterized in that Generating and displaying a virtual driving scene corresponding to the vehicle according to preset physical parameters, the transformed vehicle driving data, and the transformed obstacle data, including: Drawing a road marking model according to the lane line data in the transformed vehicle driving data; generating at least one obstacle model according to the transformed obstacle data; Constructing a vehicle motion model based on the vehicle motion data in the transformed vehicle driving data; wherein the vehicle motion model represents the motion conditions of surrounding vehicles in the game virtual scene where the vehicle is located; According to the physical parameters, physical value assignment processing is performed on the road marking model, each obstacle model and the vehicle motion model to generate and display a virtual driving scene corresponding to the vehicle.

3. The method according to claim 2, characterized in that Generating at least one obstacle model according to the transformed obstacle data, including: generating at least one initial obstacle model according to the transformed obstacle data; Determining distance information between each of the initial obstacle models; wherein the distance information includes the distance between any two of the initial obstacle models; According to the distance information, each of the initial obstacle models is corrected to obtain each of the obstacle models.

4. The method according to claim 2, characterized in that: The method further comprises: The virtual driving scene corresponding to the vehicle is subjected to detail enhancement processing, and the virtual driving scene corresponding to the vehicle is processed according to the weather data and the illumination data in the sensor data.

5. The method according to claim 1, characterized in that Based on the virtual driving scene corresponding to the vehicle, controlling the vehicle to perform simulated driving includes: Receiving a control instruction sent by a target interactive device in the vehicle, wherein the control instruction is generated by the target interactive device in response to an operation of the target interactive device by a user; Determine and display the motion mode information of the vehicle according to the control instruction; wherein the motion mode information represents the driving behavior of the vehicle on the road; Based on the virtual driving scene corresponding to the vehicle and according to the motion mode information of the vehicle, the vehicle is controlled to execute the simulated driving operation indicated by the control instruction.

6. The method according to claim 1, characterized in that Determining vehicle data according to the sensor data includes: Determine the road information and key road information of the vehicle according to the sensor data; wherein the road information represents a geometric model of the road surface on which the vehicle is traveling; and the key road information represents signs and markings of the road on which the vehicle is traveling; Determine the surrounding vehicle information of the vehicle according to the sensor data; wherein the surrounding vehicle information represents the position, speed and movement trajectory of vehicles around the vehicle; Determine the traffic flow information of the vehicle according to the surrounding vehicle information; wherein the traffic flow information represents the traffic flow conditions on the road on which the vehicle is traveling; The vehicle driving data and obstacle data in the vehicle data are generated according to the road information, surrounding vehicle information and traffic flow information of the vehicle.

7. The method according to claim 6, characterized in that The vehicle data also includes fusion data, and the method further includes: Preprocessing the sensor data to obtain processed sensor data; The point cloud data and the image data in the processed sensor data are fused to obtain fused data in the vehicle data.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: The vehicle driving data and the obstacle data are respectively subjected to format conversion processing to obtain processed vehicle driving data and processed obstacle data.

9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: In response to a simulated driving signal sent by at least one other vehicle, a virtual driving scene corresponding to each of the other vehicles is obtained to update the virtual driving scene corresponding to the vehicle; wherein the simulated driving signal indicates that the other vehicles are in a simulated driving state.

10. A simulated driving control device, characterized in that: include: A determination module for acquiring sensor data of the vehicle; Determine vehicle data according to the sensor data; wherein the vehicle data represents a driving scene in which the vehicle is located; the vehicle data includes vehicle driving data and obstacle data; and determine the position information of the vehicle according to the sensor data; A transformation module, used to perform coordinate system transformation processing on the vehicle driving data and the obstacle data respectively according to the posture information to obtain transformed vehicle driving data and transformed obstacle data; A generation module, used to generate and display a virtual driving scene corresponding to the vehicle according to preset physical parameters, the transformed vehicle driving data and the transformed obstacle data; wherein the physical parameters are road physical property parameters and collision parameters during vehicle driving; The control module is used to control the vehicle to perform simulated driving based on the virtual driving scene corresponding to the vehicle.

11. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 9 when being executed by a processor.

14. A vehicle, characterized in that: The invention comprises an electronic device for executing the simulated driving control method as claimed in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Dynamic scene information recording and simulation playback method, device, apparatus and medium

    CN107678306A

  • Testing system and method for intelligent driving vehicle

    CN117075569A

  • Real vehicle driving scene synchronous gamification method, controller and vehicle

    CN117138335A

Cited By

  • Competition management method and device

    CN119851485A