Image reconstruction method and device for fact virtual scene for intelligent automobile test

Abstract

The invention relates to the technical field of intelligent driving, in particular to an image reconstruction method and device for a fact virtual scene for intelligent automobile testing, and the method comprises the following steps: collecting the driving data, with time information, of a tested automobile; obtaining a vehicle driving track; matching the time sequence information of the vehicle and the event; real-time processing is carried out based on information such as image data of interferents, and reconstruction of a virtual scene is realized; the device comprises a first acquisition module, a second acquisition module, a scene adding module, an optimization module and a real-time reconstruction module. Compared with a traditional mode of manual image reconstruction according to the collected information, the method can make full use of the information collected by the vehicle and the road information, employs a plurality of algorithms to improve the real-time performance of data processing, carries out the information optimization processing, enables the information of the reconstructed real-time virtual scene to be more comprehensive, and improves the real-time performance of the real-time virtual scene. And the authenticity of the intelligent automobile test scene is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving, and in particular to an image reconstruction method and device for a factual virtual scenario for intelligent vehicle testing. Background Art

[0002] In recent years, the rapid development of intelligent vehicle technology has played an important role in improving road safety, reducing traffic accidents, and enhancing road traffic efficiency. Currently, in the testing of intelligent vehicles, more intelligent vehicle tests need to be carried out by building real-time virtual driving scenarios.

[0003] In current technical applications, there are implementation methods such as scene construction based on three-dimensional modeling and construction methods based on image stitching and mapping. However, in actual applications and designs, the manual modeling process requires a large amount of time, manpower, and resources. Especially when building high-precision three-dimensional models, for complex urban traffic scenarios, it is difficult to quickly expand this method to meet the diverse scenario testing needs of intelligent vehicles. At the same time, most design solutions cannot efficiently respond to dynamic changes in real time. For example, real-time simulation of dynamic factors such as vehicles, pedestrians, or weather is difficult to achieve in traditional methods. The design of some virtual scenarios lacks the support for the fusion of data from multiple sensors of intelligent vehicles (such as lidar, millimeter-wave radar, cameras, etc.), and it is difficult to meet the all-round testing needs of the autonomous driving system in the simulation environment. It is difficult to accurately build and display the simulation scenario for numerous interference information. Therefore, the current method for generating virtual test scenarios has the defects of low authenticity and poor real-time performance, and cannot meet the requirements of current intelligent vehicle testing for the diversity of working conditions and the accuracy of event occurrence. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies existing in the prior art, and to provide an image reconstruction method and device for a factual virtual scenario for intelligent vehicle testing.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] An image reconstruction method for a factual virtual scenario for intelligent vehicle testing includes the following steps:

[0007] S1. Collect the driving data of the test vehicle with time information;

[0008] S2. Obtain the vehicle driving trajectory according to the vehicle positioning information and driving data information;

[0009] S3. Perform slicing processing on the data information collected by multiple sensors, label the interference events and time information, and match the timing information of the vehicle and the events;

[0010] S4. Perform real-time processing based on information such as the image data of the interferer to achieve the reconstruction of the virtual scene.

[0011] Preferably, in S2, according to the vehicle positioning information and driving data information, obtaining the vehicle driving trajectory includes obtaining lane line information according to the vehicle driving trajectory;

[0012] According to the distribution characteristics of the trajectory points on the road, add the following constraint conditions: Condition 1. On the same road, the lane width is usually equal; Condition 2. The driving methods of the trajectories on different lanes of the same road are mostly similar;

[0013] Extract the lane lines according to the above two conditions to determine the number of drivable lanes.

[0014] Preferably, in S3, the matching of the time information of the interference event with the vehicle includes matching the time-tagged information of the interference event with the time information of the vehicle; calculate through the time series similarity algorithm:

[0015] Set the time series when the interference event occurs as X, X = x1, x2...x n , set the time series of the vehicle driving as Y, Y = y1, y2...y m , where n and m are the lengths of the two time series respectively, and the purpose of this algorithm is to find an optimal alignment path.

[0016] Preferably, in S3, the slicing process based on the data information collected by multiple sensors includes slicing the image data information. The slicing process based on the collected image data information and the tagging process of the interference event with the time information include the following steps:

[0017] Save each frame of the collected image data by slicing. The number of slices in the vertical and horizontal directions is N h and N w , after obtaining the image slices, the time period when the interferer appears can be determined, and relevant slice data can be provided for the subsequent tagging process of the interference event.

[0018] Preferably, the tagging process of the interference event with the time information in S3 includes the following steps: perform slicing processing on the data information collected by multiple sensors; record the timestamps when the interferer appears during the vehicle driving and the timestamps at the end of each event to form time period information, and encapsulate this time period information containing interferer information into a tag M i , name the tags as M1, M2...M according to the time sequence L, where L is the number of generated tags. Based on the timeline information for the out-of-date matching, the tags on the time stamps are corresponded one by one to determine the occurrence time and time length of each event.

[0019] Preferably, the step of performing real-time processing based on information such as the image data of the interferer to reconstruct the virtual scene in S4 includes: based on the information such as the optimized interferer images and point clouds obtained by the key element selection method, threshold filtering, etc. from the data collected by multiple sensors, and attaching the predicted trajectory information of the interferer, the information of the interferer is correctly displayed under the unified global coordinates.

[0020] Preferably, the correct display of the information of the interferer is to convert the collected 2D image information into three-dimensional structure information;

[0021] It includes the following steps: obtaining multi-frame 2D picture information based on multiple sensors, generating environmental information and inferring viewpoint changes, using a convolutional neural network to extract effective features from the images for image reconstruction, extracting patterns in the images from local to global, and mapping these patterns into a three-dimensional structure.

[0022] Preferably, the real-time processing of the data information of the interferer includes the following steps:

[0023] Select and determine the number of sample points, use the threshold filtering method to obtain the point cloud data that can basically provide global geometric features, reduce the number of points to be processed, and improve the rate of real-time operation. Using the key element selection method, reduce the data information to be processed to meet the real-time requirements. After the selection process, a complete and small amount of point cloud data is obtained, which can still provide good global geometric characteristics, improve the speed of three-dimensional reconstruction, and meet the real-time requirements.

[0024] The present invention also proposes an image reconstruction device for the actual virtual scene for intelligent vehicle testing, which is applicable to the above-mentioned image reconstruction method for the actual virtual scene for intelligent vehicle testing, including:

[0025] The first acquisition module is used to acquire vehicle driving data with time information

[0026] The second acquisition module is used to acquire the driving information of the environmental vehicle, and determine the relative position relationship between the vehicles according to the driving information of the host vehicle and the environmental vehicle, and determine the relative positions of the vehicles in the environment with the test vehicle as the center;

[0027] The scene addition module is used to acquire the information of the test vehicle during driving, and determine the required scene information according to the road environment information and the label information of the interferer;

[0028] An optimization module, configured to optimize the information collected by the first and second acquisition modules with the scene information in the scene addition module, and obtain more accurate road event information based on the interference object tag information and the timeline information;

[0029] A real-time reconstruction module, configured to reconstruct a real-time virtual scene reflecting the real road scene based on the road event information and the scene information.

[0030] The beneficial effects of the present invention are:

[0031] 1. Compared with the traditional method of manually reconstructing images based on the collected information, the present application can make full use of the information collected by the vehicle and the road information, adopt multiple algorithms to improve the real-time performance of data processing, and perform information optimization processing, so that the information of the reconstructed real-time virtual scene is more comprehensive, and the authenticity of the intelligent vehicle test scene is improved;

[0032] 2. By setting corresponding constraint conditions, the accuracy of obtaining vehicle positioning information and driving data information can be improved, and the vehicle driving conditions can be fully corresponded to the interference events occurring during driving; at the same time, the time and time length of each event are also determined;

[0033] 3. The information of the interference object is fully and correctly displayed, and the image information is converted into three-dimensional structure image information, and the speed of three-dimensional reconstruction is increased to meet the real-time requirement. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a step diagram of the image reconstruction method for the real virtual scene for intelligent vehicle testing proposed by the present invention;

[0035] Figure 2 It is a schematic flow chart of the vehicle test scene composition in the image reconstruction method for the real virtual scene for intelligent vehicle testing proposed by the present invention;

[0036] Figure 3 It is a schematic diagram of the real-time reconstruction of the virtual scene in the image reconstruction method for the real virtual scene for intelligent vehicle testing proposed by the present invention;

[0037] Figure 4 It is a schematic diagram of coordinate system conversion in the image reconstruction method for the real virtual scene for intelligent vehicle testing proposed by the present invention;

[0038] Figure 5 It is a structural block diagram of the image reconstruction device for the real virtual scene for intelligent vehicle testing proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] In order to make the purpose, technical solutions and advantages of this application clearer, the following further describes this application in combination with the accompanying drawings and embodiments. It should be noted that the embodiments described here are only used to illustrate this application and are not limited to this application.

[0040] The present invention will be described below with reference to the accompanying drawings. As Figures 1-4 shown, an image reconstruction method for a virtual scenario for intelligent vehicle testing includes the following steps:

[0041] S1. Collect the driving data of the test vehicle with time information; the collected driving data with time information includes real-time data collected by various sensors under different road conditions, as well as the vehicle state information:

[0042] Among the collected real-time data, it includes perception information, such as the distance information between the host vehicle and surrounding vehicles, the position information with pedestrians, and the speed and acceleration information of moving objects in the horizontal and vertical directions. The vehicle state information includes heading angle, acceleration, steering wheel angle, and angular velocity, etc., which can reflect the vehicle's attitude during a certain period of time.

[0043] S2. Obtain the vehicle driving trajectory according to the vehicle positioning information and driving data information;

[0044] S3. Perform slicing processing on the data information collected by various sensors, label the interference events and time information, and match the timing information of the vehicle and the events;

[0045] S4. Perform real-time processing based on information such as the image data of the interfering object to realize the reconstruction of the virtual scenario.

[0046] In this embodiment, reconstructing the test scenario of the vehicle requires a vehicle with various sensors to drive under different road conditions and collect a large amount of real-time data, including interfering vehicles, pedestrians crossing the road, etc. The various sensors should include environmental perception devices such as cameras and radars, as well as a high-precision GPS system that can display vehicle positioning information.

[0047] Among them, the method for obtaining lane line information according to the vehicle driving trajectory is: using a Gaussian mixture model to extract lane information, and adding the following constraint conditions according to the distribution characteristics of the trajectory points on the road. Condition 1, on the same road, the lane width is usually equal. Condition 2, the driving modes of the trajectories on different lanes of the same road are mostly similar.

[0048] Specifically, the Gaussian mixture model is used to extract lane information, and according to the distribution characteristics of the trajectory points on the road, the following rules are added for constraint. On the same road, the lane widths are generally equal; the driving patterns of the vehicle trajectories on different lanes of the same road are similar. The constrained Gaussian model obtained after adding the above constraints can be expressed as:

[0049]

[0050] Where: ω j represents the weight corresponding to each Gaussian component; Δμ is the lane width; μ0 is the position of the center line of the leftmost lane; σ 2 is the variance; k is the number of lanes. The EM method is used to solve the model parameters. Through the EM method, the model parameters corresponding to the given k value can be calculated. Next, it is necessary to calculate the cost function values corresponding to different numbers of lanes, and the k value corresponding to the minimum cost function value is the optimal number of lanes. In one embodiment, the information slicing process is based on the image information collected by the camera, and the slicing method is as follows:

[0051] Set a suitable slicing time interval, and then determine the number N h and N w of the image slices according to the duration of the event. The number N h and N w of the slices are determined by the size H*W of the image, the slice size P h ×P w , the step size S h and S w .

[0052] The number of slices in the vertical direction is:

[0053]

[0054] The number of slices in the horizontal direction is:

[0055]

[0056] After obtaining the image slices, the time period when the interference appears can be determined, and relevant slice data can be provided for the subsequent labeling of interference events.

[0057] During the driving process of the vehicle, the environmental information, the time line with time stamps during vehicle driving, and various time lines with time stamps of interference events need to be collected. Based on this time line, label events are formed as the interference events that appear in the reconstructed real-time virtual test scenario.

[0058] The obtained image slice data is used to implement the construction of an image into a three-dimensional scene using a convolutional neural network, extracting patterns in the image from local to global and mapping these patterns to a three-dimensional structure. At the same time, a neural network can be used to estimate the depth of an object, more accurately representing the distance relationship information between the host vehicle and the interfering object.

[0059] In one embodiment, the method for matching the timestamp information of an interference event with the time information during vehicle driving is as follows: Calculate through a temporal similarity algorithm. Set the time series when the interference event occurs as X, X = x1, x2...x n , and set the time series of vehicle driving as Y, Y = y1, y2...y m , where n and m are the lengths of the two time series respectively. The purpose of this algorithm is to find an optimal alignment path to minimize the distance metric value between sequences X and Y.

[0060] Specifically, initialize the starting point and ending point of the alignment path as (1, 1) and (n, m), and on this basis, define a two-dimensional matrix D, where D(i, j) represents the metric value of the minimum distance between sequences x i and y i . Recursively calculate the value of each element in matrix D and select the minimum value on this path. The formula is as follows:

[0061] D(i, j) = d(i, j) + min(D(i - 1, j), D(i, j - 1), D(i - 1, j - 1))

[0062] where d(i, j) represents the distance metric value between sequences x i and y i , and the min() function represents taking the minimum value of the three parameters. Finally, traverse matrix D along the optimal path from (1, 1) to (n, m) to obtain the minimum distance metric value.

[0063] In one embodiment, the method for generating different label numbers and adding scenes according to interference object information and time information is as follows: Record the timestamps when interference objects appear and the timestamps at the end of each event during vehicle driving to form time period information. Package this time period information containing interference object information into a label M i , and name the labels as M1, M2...M L in chronological order, where L is the number of generated labels. Corresponding the labels on the timeline information after temporal matching to the timestamps one by one to determine the occurrence time and time length of each event, and use them as scene addition information to supplement the road scene.

[0064] In one embodiment, a tag event is generated. On a timeline with the timestamp of the interference event, a time threshold Δt is set. The occurrence time of interference event 1 is set as t1, and the end time is set as t′1. When Δt > t′1 - t1, it is determined that the occurrence time of this interference event is short, and it is identified as an invalid interference event. When Δt < t′1 - t1, it is determined that interference is formed on the test vehicle, and it is identified as a valid interference event. The valid interference event and the time period from t1 to t′1 are encapsulated to form a tag event, which contains time information and event information. A safety distance threshold L is set. When the distance of the event in the tag i > L, it is determined that the tag event is far from the host vehicle and is not sufficient to constitute an interference condition, and it is identified as an invalid interference event.

[0065] In one embodiment, when preprocessing the image, the optimized filtering algorithm is used to perform noise processing on various two-dimensional images collected by multiple vehicle-mounted cameras to enhance their quality. The loss function is as follows:

[0066]

[0067] Higher-quality image data information is obtained after denoising.

[0068] For the point cloud data collected by the radar, the threshold filtering method is used to remove noise points and abnormal points. For each point P n (x, y, z), the threshold conditions are set as:

[0069] x - ε ≤ x ≤ x + ε, y - ε ≤ y ≤ y + ε, z - ε ≤ z ≤ z + ε

[0070] Points that do not meet the conditions are removed accordingly. Then, point cloud normalization, segmentation, and feature extraction are performed to obtain the filtered point cloud data. The data of various sensors are unified into the global coordinate system:

[0071] P world = T s-to-w * P senior

[0072] T s-to-w is the transformation matrix from the sensor to the world coordinate, and P senior is the point collected by various sensors.

[0073] In one embodiment, it is necessary to perform dense depth estimation on the images collected by the vehicle to display the depth distribution in the scene, so as to determine information such as the position and distance from obstacles. Taking the depth estimation of a monocular camera but not limited to a monocular camera as an example, combined with the real-time reconstruction deep learning model:

[0074] D(x, y) = f(P(x, y); θ)

[0075]

[0076] P(x, y) is the input azimuth image, and D(x, y) is the depth map predicted in real time.

[0077] f(*; θ) is a real-time reconstruction deep learning model with parameters θ.

[0078] In one embodiment, through the algorithm of multi-sensor fusion, in order to meet the real-time requirement, it is necessary to quickly process and optimize the data. Using the key element selection method, the data information to be processed is reduced to improve the real-time requirement:

[0079]

[0080] Choice p is the key element point selected by the algorithm, and n v is all data points, and p i is the coordinate of each point. After the selection process, a complete and smaller amount of point cloud data is obtained, which can still provide good global geometric characteristics, improve the speed of 3D reconstruction, and meet the real-time requirement.

[0081] In one embodiment, referring to Figure 4 , when reconstructing the real-time scene, the position and perspective of the camera and sensors such as radar need to be updated as the scene changes. In order to generate the correct perspective, the view matrix and projection matrix are used to describe the position, orientation, and projection of the camera. In the virtual test scene, the scene information also changes with the position of the camera. It is necessary to perform coordinate system transformation on the interference object information, convert the object from the world coordinate system to the camera coordinate system, and finally project it onto the device.

[0082] The following formula is used for coordinate system transformation:

[0083] P camera = R * P world + T

[0084] where P camera = [X c , Y c , Z c T is the three-dimensional coordinate in the camera coordinate system,

[0085] P world = [X w , Y w , Z w T

[0086] Let \(R\) be the rotation matrix. Taking the camera images from multiple perspectives as an example, multi-view geometric reconstruction is carried out as follows:

[0087]

[0088] \(P\) z is the pixel coordinate at the perspective \(z\), and \(K\) z , \([R\) z |t z are the intrinsic and extrinsic parameters of the camera \(z\).

[0089] In one embodiment, referring to Figure 3 , the insertion of dynamic interference objects is carried out. By tracking the interference objects, predicting their trajectories to assist the rapid update of real-time modeling, and recording the historical trajectories of the obstacles at the \(t\) slice moment as \(\{(x\) t , \(y\) t , \(z\) t )\}, the predicted driving trajectories of the obstacles are obtained through a trained prediction neural network (but not limited to using a neural network). At the \(t + \epsilon\) slice moment \(\{(x\) t+ε , \(y\) t+ε , \(z\) t+ε )\}. According to the movement information of the obstacles collected by multiple sensors, combined with the predicted driving trajectories of the obstacles, the position information and rotation information of the obstacles are updated:

[0090]

[0091] \(\theta\) t+Δt =\(\theta\) t +\(\omega\)*\(\Delta t\)

[0092] The obstacle information is dynamically generated according to the position or speed of the test vehicle, including the appearance of new obstacles, the relative movement between the original obstacles and the host vehicle, and the self-movement of various obstacles, etc., to ensure that a more realistic and highly real-time virtual scene can be obtained in the test site. Among them, the movement characteristics of the interference objects include but are not limited to linear motion and curvilinear motion.

[0093] Referring to Figure 5 , an image reconstruction device for the factual virtual scene used in intelligent vehicle testing, which is applicable to the above-mentioned image reconstruction method for the factual virtual scene used in intelligent vehicle testing, includes a first acquisition module for obtaining vehicle driving data with time information to provide selection and use as a data set for subsequent various optimization processes.

[0094] Among them, a second acquisition module is used to obtain the driving information of the surrounding vehicles, and determine the relative position relationship between the vehicles according to the driving information of the host vehicle and the surrounding vehicles, and determine the relative positions of the surrounding vehicles with the test vehicle as the center.

[0095] Further, a scenario adding module is configured to obtain information of a test vehicle during driving, and determine the scenario information to be added according to road environment information and label information of interference objects. This module demonstrates the most basic vehicle driving environment and limits the route, ensuring the authenticity of the test scenario.

[0096] Furthermore, the above-mentioned optimization module is configured to optimize the information collected by the first and second acquisition modules and the scenario information in the scenario adding module, and obtain more accurate road event information based on the interference object label information and timeline information;

[0097] In this embodiment, the above-mentioned real-time reconstruction module is configured to use a real-time optimization algorithm to select appropriate data based on the data information collected by multiple sensors, and improve the real-time update rate according to the obtained prediction information, ensuring the real-time of virtual scene modeling.

[0098] Based on the fusion calculation of multiple sensors and combined with the driving position information of the test vehicle, the present invention provides an image reconstruction method for a real-time virtual test scenario centered on the host vehicle. Due to the different numbers and cooperation methods of cameras and other sensors of various test vehicles, different application solutions can be selected to match the reconstruction of a virtual scene with higher real-time performance and authenticity. Compared with the traditional image reconstruction method of real-time virtual scenes, it can reduce the processing and repeated operations of multiple groups of unnecessary data, and can more accurately display the relative position and motion state of interference objects according to the predicted trajectory and timeline information, improving the authenticity of the test scenario.

[0099] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. An image reconstruction method of a virtual scene for intelligent vehicle testing, characterized in that: The steps include: S1. Collecting driving data of the test vehicle with time information; S2. Obtaining the vehicle driving trajectory according to the vehicle positioning information and driving data information; S3: Slice the data collected by various sensors, label the interference events and time information, and match the timing information of vehicles and events; S4. Real-time processing is performed based on the image data and other information of the interference object to achieve reconstruction of the virtual scene.

2. The image reconstruction method of the actual virtual scene for intelligent vehicle testing according to claim 1 is characterized in that: In S2, obtaining the vehicle driving trajectory according to the vehicle positioning information and the driving data information includes obtaining lane line information according to the vehicle driving trajectory; According to the distribution characteristics of trajectory points on the road, the following constraints are added: Condition 1: On the same road, the lane widths are usually equal; Condition 2: The trajectory driving modes on different lanes of the same road are mostly similar; Lane lines are extracted based on the above two conditions to determine the number of drivable lanes.

3. The image reconstruction method of the actual virtual scene for intelligent vehicle testing according to claim 1, characterized in that: The matching of the time information of the interference event with the vehicle in S3 includes matching the time labeling information of the interference event with the time information of the vehicle; and calculating by a time series similarity algorithm: Set the time series of interference events to X, X = x1, x2...x n , the time series of vehicle travel is set as Y, Y = y1, y2...y m , n and m are the lengths of the two time series respectively. The algorithm is used to find an optimal alignment path.

4. The image reconstruction method of the actual virtual scene for intelligent vehicle testing according to claim 1, characterized in that: The slicing process based on the data information collected by multiple sensors in S3 includes slicing the image data information, and the slicing process based on the collected image data information and labeling the interference events and time information includes the following steps: Each frame of the acquired image data is sliced ​​and saved, and the number of slices in the vertical and horizontal directions is N respectively. h and N w After obtaining the image slices, the time period in which the interference object appears can be determined, and relevant slice data can be provided for the subsequent labeling and processing of the interference event.

5. The image reconstruction method of the actual virtual scene for intelligent vehicle testing according to claim 4 is characterized in that: The labeling of interference events and time information in S3 includes the following steps: slicing data information collected by multiple sensors; recording the timestamps of the interference objects appearing during the driving process of the vehicle and the timestamps of the end of each event to form time period information; and packaging the time period information containing the interference object information into a label M. i , and name the labels M1, M2...M according to the time sequence. L , L is the number of generated labels, and the time and time length of each event are determined by corresponding the labels on the timestamp one by one according to the timeline information that has been time-series matched.

6. The image reconstruction method of the actual virtual scene for intelligent vehicle testing according to claim 1, characterized in that: In the S4, the image data and other information of the interference object are processed in real time to realize the reconstruction step of the virtual scene, including: obtaining the optimized interference object image, point cloud and other information based on the information collected by multiple sensors and through the key element selection method, threshold filtering and other methods, and attaching the predicted interference object trajectory information, and correctly displaying the interference object information under the unified global coordinates.

7. The image reconstruction method of the actual virtual scene for intelligent vehicle testing according to claim 6 is characterized by: The correct display of the information of the interference object is to convert the collected 2D image information into three-dimensional structure information; It includes the following steps: acquiring multiple frames of 2D image information based on multiple sensors, generating environmental information and inferring viewpoint changes, extracting effective features from the image for image reconstruction with the help of convolutional neural networks, extracting patterns in the image from local to global, and mapping these patterns to three-dimensional structures.

8. The image reconstruction method of the actual virtual scene for intelligent vehicle testing according to claim 6 is characterized by: The real-time processing of the data information of the interferent comprises the following steps: The number of sample points is selected and determined, and the threshold filtering method is used to obtain the most basic point cloud data that can provide global geometric features, reduce the number of points required for processing, and improve the rate of real-time operations; the key element selection method is used to reduce the data information that needs to be processed to improve the real-time requirements. After the selection process, a complete and small number of point cloud data are obtained, which can still provide good global geometric characteristics, improve the speed of 3D reconstruction, and meet the real-time requirements.

9. An image reconstruction device for a virtual scene of a smart car test, adapted to the image reconstruction method for a virtual scene of a smart car test according to any one of claims 1 to 8, characterized in that: include: The first acquisition module: used to obtain vehicle driving data with time information; The second acquisition module is used to obtain the driving information of the environment vehicle, determine the relative position relationship between the vehicles according to the driving information of the host vehicle and the environment vehicle, and determine the relative position of each vehicle in the environment with the test vehicle as the center; A scene adding module is used to obtain the information of the test vehicle during driving, and determine the scene information to be added according to the road environment information and the label information of the interference object; Optimization module: used for optimizing the information collected by the first and second acquisition modules and the scene information in the scene adding module, and obtaining more accurate road event information based on the interference object label information and the timeline information; Real-time reconstruction module: used to reconstruct a real-time virtual scene reflecting a real road scene based on the road event information and the scene information.

Images