Three-dimensional modeling method, device, and storage medium

Abstract

The application relates to a three-dimensional modeling method, device and storage medium. The method comprises the following steps: acquiring a plurality of frames of original point cloud data collected for a driving area; the driving area comprises a plurality of objects; receiving a perception result of the objects, and determining a target object and a plurality of frames of point cloud data corresponding to the target object from the plurality of frames of original point cloud data according to the perception result; performing cumulative processing on the plurality of frames of point cloud data to obtain accumulated to-be-processed point cloud data; performing point cloud registration processing on the to-be-processed point cloud to obtain registered target point cloud data; and performing three-dimensional modeling on the target object according to the target point cloud data. The method can improve the reconstruction efficiency of the three-dimensional model and ensure the authenticity of the model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a three-dimensional modeling method, apparatus, and storage medium. Background Technology

[0002] With the development of computer technology, 3D model reconstruction technology plays an important role in fields such as autonomous driving. By performing 3D simulation modeling of target objects in road scenes, realistic autonomous driving test scenarios can be obtained.

[0003] Currently, existing modeling methods typically involve manually creating multiple CAD models first, and then reconstructing a 3D simulation model of the target object based on these multiple CAD models. Since manually creating models is time-consuming, and multiple different CAD models need to be created for different target objects, how to efficiently perform 3D modeling of target objects is the problem this application aims to solve. Summary of the Invention

[0004] Therefore, it is necessary to provide a 3D modeling method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve modeling efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a three-dimensional modeling method. The method includes:

[0006] Acquire multiple frames of raw point cloud data collected in the driving area; the driving area includes multiple objects.

[0007] Receive the perception result of the object, and determine the target object and the corresponding multi-frame point cloud data from the multi-frame raw point cloud data based on the perception result;

[0008] The multi-frame point cloud data is accumulated to obtain the accumulated point cloud data to be processed;

[0009] Perform point cloud registration processing on the point cloud to be processed to obtain the registered target point cloud data;

[0010] A 3D model of the target object is performed based on the target point cloud data.

[0011] In one embodiment, the perception result includes a detection box and a confidence score; determining the target object and the corresponding multi-frame point cloud data from the multi-frame raw point cloud data based on the perception result includes: filtering the target object from the objects included in the raw point cloud data based on the confidence score of the object's detection box; and segmenting the current frame of raw point cloud data according to the detection box of the target object for each frame of raw point cloud data in the multi-frame raw point cloud data to obtain the current frame point cloud data corresponding to the target object.

[0012] In one embodiment, before segmenting the current frame raw point cloud data according to the detection box of the target object to obtain the current frame point cloud data corresponding to the target object, the method further includes: establishing a lidar coordinate system based on the data acquisition location of the driving area; establishing a world coordinate system based on the detection box position of the target object; determining the parameter transformation matrix between the lidar coordinate system and the world coordinate system; and transforming the raw point cloud data and the detection box of the target object to the same coordinate system according to the parameter transformation matrix. The step of segmenting the current frame raw point cloud data according to the detection box of the target object to obtain the current frame point cloud data corresponding to the target object includes: for each detection box in the multiple detection boxes of the target object, segmenting the current frame raw point cloud data in the same coordinate system according to the current detection box of the target object to obtain the current frame point cloud data corresponding to the target object.

[0013] In one embodiment, the process of accumulating the multi-frame point cloud data to obtain accumulated point cloud data to be processed includes: determining a target accumulation position; for each frame of point cloud data in the multi-frame point cloud data, determining a coordinate transformation matrix corresponding to the current frame point cloud data based on the target accumulation position; and transforming the point cloud coordinates corresponding to the current frame point cloud data to the target region corresponding to the target accumulation position using the coordinate transformation matrix to obtain the current frame point cloud data to be processed.

[0014] In one embodiment, after accumulating the multi-frame point cloud data to obtain accumulated point cloud data to be processed, the method further includes: detecting and identifying the point cloud data to be processed to determine the structure type of the target object; when the structure type is axisymmetric, determining the axis of symmetry parameter of the target object; performing mirror processing on the point cloud data to be processed of the target object according to the axis of symmetry parameter to obtain mirror point cloud data corresponding to the point cloud data to be processed; and filling the missing blank point cloud data in the point cloud data to be processed with the mirror point cloud data to obtain filled point cloud data to be processed.

[0015] In one embodiment, the step of performing point cloud registration processing on the point cloud to be processed to obtain registered target point cloud data includes: determining the current frame of point cloud data to be processed in multiple frames of point cloud data to be processed; determining the adjacent point cloud data adjacent to the current frame of point cloud data to be processed; performing a similarity judgment on the current point cloud data to be processed and the adjacent point cloud data; when the similarity is greater than or equal to a similarity threshold, performing point cloud registration processing on the current point cloud data to be processed and the adjacent point cloud data to obtain target point cloud data; using the target point cloud data as the new current frame of point cloud data to be processed, and returning to the step of determining the adjacent point cloud data adjacent to the current frame of point cloud data to be processed to continue execution, until each frame of point cloud data to be processed in the multiple frames of point cloud data to be processed is registered with the adjacent point cloud data.

[0016] In one embodiment, after performing point cloud registration processing on the point cloud to be processed to obtain registered target point cloud data, the method further includes: determining abnormal point cloud data in the target point cloud data, and performing abnormal point cloud filtering processing on the target point cloud data to obtain target point cloud data after initial filtering; determining a first normal vector and a fitting plane of the target point cloud data after initial filtering; and performing smoothing filtering processing on the first normal vector through the fitting plane to obtain target point cloud data after further filtering.

[0017] In one embodiment, performing 3D modeling of the target object based on the target point cloud data includes: determining an indicator function corresponding to the target object and a second normal vector of the target point cloud data; performing surface estimation of the target object based on the indicator function, the target point cloud data, and the second normal vector of the target point cloud data to obtain a surface estimation result; extracting a 3D isosurface of the target object based on the surface estimation result, and performing 3D modeling of the target object based on the 3D isosurface.

[0018] Secondly, this application also provides a three-dimensional modeling apparatus. The apparatus includes:

[0019] The point cloud acquisition module is used to acquire multiple frames of raw point cloud data collected in the driving area; the driving area includes multiple objects.

[0020] The point cloud processing module is used to receive the perception result of the object, and determine the target object and the corresponding multi-frame point cloud data from the multi-frame raw point cloud data based on the perception result; perform cumulative processing on the multi-frame point cloud data to obtain the accumulated point cloud data to be processed; and perform point cloud registration processing on the point cloud to be processed to obtain the registered target point cloud data.

[0021] The point cloud modeling module is used to perform three-dimensional modeling of the target object based on the target point cloud data.

[0022] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0023] Acquire multiple frames of raw point cloud data collected in the driving area; the driving area includes multiple objects.

[0024] Receive the perception result of the object, and determine the target object and the corresponding multi-frame point cloud data from the multi-frame raw point cloud data based on the perception result;

[0025] The multi-frame point cloud data is accumulated to obtain the accumulated point cloud data to be processed;

[0026] Perform point cloud registration processing on the point cloud to be processed to obtain the registered target point cloud data;

[0027] A 3D model of the target object is performed based on the target point cloud data.

[0028] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, performs the following steps:

[0029] Acquire multiple frames of raw point cloud data collected in the driving area; the driving area includes multiple objects.

[0030] Receive the perception result of the object, and determine the target object and the corresponding multi-frame point cloud data from the multi-frame raw point cloud data based on the perception result;

[0031] The multi-frame point cloud data is accumulated to obtain the accumulated point cloud data to be processed;

[0032] Perform point cloud registration processing on the point cloud to be processed to obtain the registered target point cloud data;

[0033] A 3D model of the target object is performed based on the target point cloud data.

[0034] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0035] Acquire multiple frames of raw point cloud data collected in the driving area; the driving area includes multiple objects.

[0036] Receive the perception result of the object, and determine the target object and the corresponding multi-frame point cloud data from the multi-frame raw point cloud data based on the perception result;

[0037] The multi-frame point cloud data is accumulated to obtain the accumulated point cloud data to be processed;

[0038] Perform point cloud registration processing on the point cloud to be processed to obtain the registered target point cloud data;

[0039] A 3D model of the target object is performed based on the target point cloud data.

[0040] The aforementioned 3D modeling method, apparatus, computer equipment, storage medium, and computer program product acquire multiple frames of raw point cloud data collected from a driving area and receive the object's perception results. Based on the perception results, the target object and its corresponding multiple frames of point cloud data are determined from the multiple frames of raw point cloud data. By accumulating and processing the multiple frames of point cloud data, accumulated point cloud data to be processed is obtained. Then, point cloud registration processing is performed on the point cloud to be processed to obtain registered target point cloud data. Thus, 3D modeling of the target object can be performed based on the target point cloud data. Since this application automatically performs 3D modeling of the target object based on the target point cloud data after processing multiple frames of point cloud data to obtain the target point cloud data, compared to the traditional method of manually editing the model, this application can automatically complete the 3D modeling of the target object, thereby greatly improving the efficiency of 3D modeling. Simultaneously, since the target point cloud data formed after accumulating and registering the point cloud data is used as the modeling data, the accuracy of 3D model reconstruction using the target point cloud data is also improved.

[0041] Furthermore, the modeling data for the 3D model reconstruction comes from the recorded real point cloud data, which ensures the authenticity of the constructed model and reduces the difference between simulation data and real data. This, in turn, ensures the reliability of using the 3D model for the development and testing of other algorithm modules such as autonomous driving. Attached Figure Description

[0042] Figure 1 This is a diagram illustrating the application environment of a 3D modeling method in one embodiment.

[0043] Figure 2 This is a flowchart illustrating a 3D modeling method in one embodiment;

[0044] Figure 3 This is a scene rendering of the driving area in one embodiment;

[0045] Figure 4 This is a rendering of multiple frames of point cloud data in one embodiment;

[0046] Figure 5 This is a rendering of the target point cloud data in one embodiment;

[0047] Figure 6 This is a flowchart illustrating a 3D modeling method in another embodiment;

[0048] Figure 7 This is a structural block diagram of a three-dimensional modeling device in one embodiment;

[0049] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] The 3D modeling method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. Terminal 102 can acquire multiple frames of raw point cloud data collected in the driving area and perception results of multiple objects in the driving area, and send the multiple frames of raw point cloud data and perception results to server 104. Server 104 processes the received perception results and multiple frames of raw point cloud data to obtain target point cloud data. Server 104 performs 3D modeling of the target object based on the target point cloud data and sends the 3D modeling result to terminal 102 so that terminal 102 can display the 3D model of the target object. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, smart vehicle devices, tablets, IoT devices, and portable wearable devices. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0052] In one embodiment, such as Figure 2 As shown, a 3D modeling method is provided. Taking the application of this method to a computer device as an example, the computer device can provide... Figure 1 The terminal or server in the system. The 3D modeling method includes the following steps:

[0053] Step 202: Obtain multiple frames of raw point cloud data collected for the driving area; the driving area includes multiple objects.

[0054] Point cloud data refers to the collection of point data on the surface of an object obtained through sensors, such as lidar, millimeter-wave radar, or ultrasonic radar. The raw point cloud data contains point cloud data of various objects in the driving area, such as point cloud data of vehicles, pedestrians, buildings, lane lines, street fences, etc.

[0055] Specifically, when a user needs to perform 3D modeling of vehicles in a driving area, the user can trigger the LiDAR to scan the driving area at a preset scanning frequency, for example, 10Hz, obtaining one frame of raw point cloud data corresponding to each scanning frequency, and inputting each frame of raw point cloud data into a computer device. Figure 3 As shown, Figure 3 This is a rendering of the scene in the driving area created by scanning.

[0056] In one embodiment, point cloud data of other vehicles can be collected by the main vehicle in the driving area. The main vehicle can be a vehicle equipped with sensors and an autonomous driving system. When the main vehicle is driving on the road, it records the surrounding environment through sensors and identifies objects in the surrounding environment through a perception module. The raw point cloud data acquired by the sensors and the object perception results after identification are stored in a database, for example, stored in the on-board hard drive of the main vehicle.

[0057] Step 204: Receive the perception results of the object, and determine the target object and the corresponding multi-frame point cloud data from the multiple frames of raw point cloud data based on the perception results.

[0058] Specifically, the computer equipment uses a pre-trained perception algorithm to perceive and identify objects in each frame of raw point cloud data, obtaining the perception result for each object. Among these, reference... Figure 3 As shown, the perception result can be a detection box. Based on the perception result, the computer device can determine the target object that the user needs to model in 3D. For example, objects whose predicted probability in the detection box reaches a probability threshold can be considered as target objects, where the target object can be a moving vehicle. Based on the perception result, the computing device can segment multiple frames of raw point cloud data to obtain multiple frames of point cloud data corresponding to the target object, that is, extract the point cloud data corresponding to the moving vehicle from the raw point cloud data, as exemplified by... Figure 4 As shown, Figure 4 This is a rendering of the multi-frame point cloud data generated after segmentation.

[0059] In one embodiment, perception and recognition can be performed offline, meaning that each frame of raw point cloud data is transmitted to the control terminal before perception and recognition are performed; or it can be performed online, meaning that the perception algorithm module in the host vehicle is directly activated for online perception processing. It is easy to understand that when performing online perception processing via a moving host vehicle, the number of detection boxes corresponding to each object is related to the duration of each object's presence within the host vehicle's field of view.

[0060] In one embodiment, the computer device segments the original point cloud data based on the detection bounding box of the target object, or removes the point cloud data of the target object, leaving only the background point cloud data corresponding to the background environment of the driving area. The computer device then performs fusion processing on each frame of background point cloud data corresponding to the background environment to obtain fused background point cloud data, and uses the fused background point cloud data to build a map model of the background environment.

[0061] In one embodiment, the computer device responds to the user's selection operation, determines the target object to be tested during autonomous driving simulation testing, and combines the three-dimensional model of the target object with the map model of the background environment to establish a realistic driving simulation environment for the target object.

[0062] In this embodiment, after segmenting the original point cloud data according to the detection box of the target object, it is possible to build a 3D model of the target object in the driving area, as well as a map model of the background environment in the driving area. Furthermore, based on the scenario required by actual users in autonomous driving simulation testing, the 3D model of the target object and the map model of the background environment can be combined, which further reflects the realism of the simulation test.

[0063] Step 206: Accumulate the multi-frame point cloud data to obtain the accumulated point cloud data to be processed.

[0064] Since the target object in the driving area is in motion, when the lidar scans the target object multiple times, it will cause a motion distortion in each frame of point cloud data of the target object. That is, each frame of point cloud data is not under the same coordinate. Therefore, it is necessary to accumulate each frame of point cloud data, that is, to perform coordinate registration on each frame of point cloud data under different coordinates.

[0065] In one embodiment, accumulating multiple frames of point cloud data to obtain accumulated point cloud data to be processed includes: determining a target accumulation position; for each frame of point cloud data in the multiple frames of point cloud data, determining a coordinate transformation matrix corresponding to the current frame of point cloud data based on the target accumulation position; and transforming the point cloud coordinates corresponding to the current frame of point cloud data to the target region corresponding to the target accumulation position using the coordinate transformation matrix to obtain the current frame of point cloud data to be processed.

[0066] The target accumulation location can be the detection box area corresponding to any frame of point cloud data, or it can be other detection box areas in the driving area defined by the user.

[0067] Specifically, the computer equipment determines the target accumulation position corresponding to each frame of point cloud data, and determines the coordinate transformation matrix corresponding to each frame of point cloud data based on the target accumulation position. It is easy to understand that the motion distortion generated by each frame of point cloud data is different, and the corresponding coordinate transformation matrices are not the same. The computer equipment performs accumulation processing on each frame of point cloud data separately, so that each frame of point cloud data is transformed to the target region corresponding to the target accumulation position. That is, the point cloud coordinates of each frame of point cloud data are multiplied by their respective coordinate transformation matrices to obtain the point cloud coordinates of each frame of point cloud data to be processed located in the target region.

[0068] In this embodiment, by converting each frame of point cloud data to the target region corresponding to the target accumulation position, the initial accumulation of point cloud data of the target object is completed, so that point cloud registration processing can be performed on the point cloud data to be processed in the future.

[0069] Step 208: Perform point cloud registration processing on the point cloud to be processed to obtain the registered target point cloud data.

[0070] Point cloud registration is a process of accurately matching each frame of point cloud data to be processed and merging the matched multiple frames of point cloud data into a single frame of target point cloud data.

[0071] Specifically, the computer device determines the similarity between two adjacent frames of point cloud data based on their point cloud features. Then, based on this similarity, it performs point cloud registration. The more similar the point cloud features between two frames, the higher the similarity. When the similarity is greater than or equal to a similarity threshold, a point cloud registration algorithm is used to improve the accuracy of point cloud accumulation. When the similarity is less than the threshold, occlusion may have occurred between adjacent frames, resulting in significant differences. Therefore, the point cloud registration step for these adjacent frames can be skipped.

[0072] In one embodiment, taking 10 consecutive frames of point cloud data to be processed as an example, point cloud registration is performed by calculating the similarity between adjacent frames. Assuming that during vehicle movement, object occlusion occurred when capturing the 5th and 6th frames, causing the similarity between the 5th and 6th frames to be less than the similarity threshold, while the similarity between any two adjacent frames in the first 5 frames and the last 5 frames is greater than the similarity threshold, point cloud registration is first performed on the first 5 frames. After completing the point cloud registration between the 4th and 5th frames, the step of registering the point cloud between the 5th and 6th frames is skipped, and the point cloud registration of the last 5 frames continues with the 6th frame as the new starting frame. Finally, a point cloud registration judgment is performed on the point cloud registration results of the first 5 frames and the last 5 frames. The target point cloud data obtained after point cloud registration is as follows: Figure 5 As shown, Figure 5 This is a rendering of the target point cloud data after fusion processing.

[0073] In one embodiment, the method for accurately matching two adjacent frames of point cloud data to be processed can be a point cloud registration algorithm, such as the iterative nearest point algorithm.

[0074] In one embodiment, when the similarity between two adjacent frames of point cloud data to be processed is much less than the similarity threshold, it can be assumed that the target object is occluded. After point cloud accumulation and point cloud registration, the shape of the object point cloud is still incomplete. The shape can be completed by using a pre-trained object point cloud model.

[0075] Step 210: Perform 3D modeling of the target object based on the target point cloud data.

[0076] The methods for 3D modeling can include region expansion reconstruction, implicit reconstruction, and statistical reconstruction. Computer equipment uses the Poisson algorithm to perform 3D modeling on unstructured target point cloud data, obtaining a 3D structural model of the target object. This 3D structural model can be a triangular network model.

[0077] In the aforementioned 3D modeling method, multiple frames of raw point cloud data collected from the driving area are acquired, and the object's perception results are received. Based on the perception results, the target object and its corresponding multiple frames of point cloud data are determined from the multiple frames of raw point cloud data. By accumulating the multiple frames of point cloud data, accumulated point cloud data to be processed is obtained. Then, point cloud registration processing is performed on the point cloud to be processed to obtain registered target point cloud data. Thus, 3D modeling of the target object can be performed based on the target point cloud data. Since this application automatically performs 3D modeling of the target object based on the target point cloud data after processing multiple frames of point cloud data to obtain the target point cloud data, compared to the traditional method of manually editing the model, this application can automatically complete the 3D modeling of the target object, thereby greatly improving the efficiency of 3D modeling. Simultaneously, since the target point cloud data formed after accumulating and registering the point cloud data is used as the modeling data, the accuracy of 3D model reconstruction using the target point cloud data is also improved.

[0078] In one embodiment, the perception result includes a detection box and a confidence score. Based on the perception result, the target object is determined from multiple frames of raw point cloud data, and the corresponding multiple frames of point cloud data are obtained to acquire multiple frames of point cloud data collected for the target object in the driving area. This includes: filtering the target object from the objects included in the raw point cloud data based on the confidence score of the object's detection box; and for each frame of raw point cloud data in the multiple frames of raw point cloud data, the current frame of raw point cloud data is segmented based on the detection box of the target object to obtain the current frame of point cloud data corresponding to the target object.

[0079] The detection box represents a perception result for each object in the driving area, including the coordinates of the object's center point in the driving area, the side length of the detection box, and the rotation angle of the detection box around the coordinate axis. The confidence score represents a predicted probability that the detection box contains a certain type of object through the perception algorithm. The number of frames in the original point cloud data is typically the same as the number of detection boxes corresponding to each object.

[0080] Specifically, the computer device determines the confidence level of each object based on a preset confidence threshold. If the confidence level of the current object is greater than the confidence threshold, the current object is designated as the target object; that is, there can be multiple target objects. For each frame of raw point cloud data in multiple frames, that is, for each detection box corresponding to the target object, the computer device determines the point cloud selected by the current detection box in the current frame of raw point cloud data, and segments the selected point cloud from the current frame of raw point cloud data to obtain the current frame point cloud data corresponding to the target object.

[0081] In one embodiment, a computer device can determine the bounding box identifiers for multiple detection boxes corresponding to each frame of raw point cloud data, wherein the detection box identifiers for the same object are identical. That is, a trained perception algorithm labels the detection boxes corresponding to each object with the same detection box identifier.

[0082] In this embodiment, after filtering out target objects based on the confidence level of the object's detection bounding box, the original point cloud data is segmented according to the target object's detection bounding box to obtain the point cloud data corresponding to the target object. This allows for accurate separation of the target object from the driving area's scene environment. Furthermore, since multiple target objects can be filtered out, multiple 3D models can be built simultaneously in batches, improving the flexibility of autonomous driving environment simulation. Simultaneously, because the original point cloud data is collected from the real environment of the driving area, the processed point cloud data corresponding to the target object has good realism.

[0083] In one embodiment, before segmenting the original point cloud data of the current frame based on the detection bounding box of the target object to obtain the current frame point cloud data corresponding to the target object, the method further includes: establishing a lidar coordinate system based on the data acquisition location of the driving area; establishing a world coordinate system based on the detection bounding box position of the target object; determining the parameter transformation matrix between the lidar coordinate system and the world coordinate system; transforming the original point cloud data and the detection bounding box of the target object to the same coordinate system according to the parameter transformation matrix; and segmenting the original point cloud data of the current frame based on the detection bounding box of the target object to obtain the current frame point cloud data corresponding to the target object, including: for each detection bounding box of the multiple detection bounding boxes of the target object, segmenting the original point cloud data of the current frame in the same coordinate system based on the current detection bounding box of the target object to obtain the current frame point cloud data corresponding to the target object.

[0084] The original point cloud data is collected in the LiDAR coordinate system established by the LiDAR, while the target object detection box is identified in the world coordinate system. Therefore, it is necessary to convert the original point cloud data and the target object detection box to the same coordinate system.

[0085] Specifically, the computer equipment acquires the parameter transformation matrix between the lidar coordinate system and the world coordinate system. This transformation matrix can be obtained through a pre-completion calibration process. Based on the transformation matrix, the original point cloud data and the detection boxes of the target object can be transformed to the same coordinate system, typically the world coordinate system. For each detection box within the multiple detection boxes of the target object, the computer equipment segments the current frame's original point cloud data in the same coordinate system based on the target object's current detection box. This means determining the point cloud selected by the current detection box within the current original point cloud data and segmenting the selected point cloud from the current frame's original point cloud data to obtain the current frame's point cloud data corresponding to the target object.

[0086] In this embodiment, by converting the original point cloud data and the detection box of the target object to the same coordinate system, and then segmenting the original point cloud data according to the detection box of the target object, the point cloud data of the target object can be obtained more accurately.

[0087] In one embodiment, after accumulating multiple frames of point cloud data to obtain accumulated point cloud data to be processed, the method further includes: detecting and identifying the point cloud data to be processed to determine the structure type of the target object; when the structure type is axisymmetric, determining the axis of symmetry parameter of the target object; performing mirror processing on the point cloud data to be processed of the target object according to the axis of symmetry parameter to obtain mirror point cloud data corresponding to the point cloud data to be processed; and filling the missing blank point cloud data in the point cloud data to be processed with the mirror point cloud data to obtain the filled point cloud data to be processed.

[0088] Among them, detection and recognition is a way to perform semantic recognition on target objects. Through semantic recognition, it can be determined whether the target object is a vehicle, pedestrian or building, and whether the structure type is axisymmetric, etc.

[0089] Specifically, the computer equipment detects and identifies the point cloud data to be processed to determine the structural type of the target object. When the target object is a vehicle, it can be determined that the target object is axisymmetric. Based on the detection frame of the target object, the computer equipment determines the axis of symmetry parameters, that is, the center line of the point cloud extending from the front to the rear of the vehicle is taken as the axis of symmetry, and the axis of symmetry parameters of the target object are determined based on the position information of the axis of symmetry in the detection frame.

[0090] Furthermore, the computer device uses the symmetry axis parameter as a reference to divide the point cloud data of the target object into left-side and right-side point cloud data. Based on the symmetry axis parameter, the left-side point cloud data is mirrored to obtain a corresponding right-side mirrored point cloud data. The computer device then uses the right-side mirrored point cloud data to fill in any missing blank point cloud data in the right-side point cloud data, obtaining the filled right-side point cloud data. Similarly, the filled left-side point cloud data can be obtained.

[0091] For example, if the point cloud data to be processed on the left includes point cloud A, point cloud B, and point cloud D, and the point cloud data to be processed on the right includes point cloud A, point cloud C, and point cloud D, by mirroring the point cloud data to be processed on the left, the mirrored point cloud data on the right includes point cloud A1, point cloud B1, and point cloud D1; by using the mirrored point cloud data on the right, the missing blank point cloud data in the point cloud data to be processed on the right is filled, and the point cloud data to be processed on the right includes point cloud A, point cloud B1, point cloud C, and point cloud D.

[0092] In one embodiment, the detection and recognition method can be a point cloud semantic segmentation method, which determines the type of target object in the driving area by assigning specific semantic labels to the point cloud.

[0093] In this embodiment, by determining the symmetry axis parameter of the target object, the point cloud data of the target object to be processed is mirrored, and the point cloud data of the target object is stably and accurately restored. This avoids affecting the effect of subsequent 3D reconstruction when only one side of the target object and incomplete point cloud data can be obtained.

[0094] In one embodiment, point cloud registration processing is performed on the point cloud to be processed to obtain registered target point cloud data, including: determining the current frame of point cloud data to be processed in multiple frames of point cloud data to be processed; determining the adjacent point cloud data adjacent to the current frame of point cloud data to be processed; performing a similarity judgment on the current point cloud data to be processed and the adjacent point cloud data; when the similarity is greater than or equal to a similarity threshold, performing point cloud registration processing on the current point cloud data to be processed and the adjacent point cloud data to obtain target point cloud data; using the target point cloud data as the new current frame of point cloud data to be processed, and returning to the step of determining the adjacent point cloud data adjacent to the current frame of point cloud data to be processed to continue execution, until each frame of point cloud data to be processed in multiple frames of point cloud data to be processed is registered with the adjacent point cloud data.

[0095] The similarity is determined by the point cloud features between the two frames of point cloud data to be processed. For example, point cloud features include point cloud feature values, echo intensity, elevation difference, etc.

[0096] Specifically, the computer device determines the current frame of point cloud data and its adjacent neighboring point cloud data from multiple frames of point cloud data to be processed. These adjacent point cloud data can consist of at most two frames, typically obtained from the previous and next scanning frequencies of the LiDAR. The computer device performs a similarity assessment based on the point cloud features corresponding to the current and adjacent point cloud data; the more similar the point cloud features are, the higher the similarity. When the similarity is greater than or equal to a similarity threshold, the computer device performs point cloud registration processing on the current and adjacent point cloud data to obtain the target point cloud data, essentially fusing two adjacent frames into a single target point cloud data frame. The computer device then uses the target point cloud data as the new current frame of point cloud data to be processed and returns to the step of determining the adjacent point cloud data to be processed, continuing this process until each frame of point cloud data to be processed is fused with its adjacent point cloud data.

[0097] In this embodiment, by judging the similarity between two adjacent frames of point cloud data to be processed, and then performing point cloud matching processing on the two adjacent frames of point cloud data with a similarity greater than or equal to the similarity threshold, a precise target point cloud data is finally obtained, ensuring the accuracy of subsequent 3D modeling.

[0098] In one embodiment, after performing point cloud registration processing on the point cloud to be processed to obtain registered target point cloud data, the above method includes: identifying abnormal point cloud data in the target point cloud data and performing abnormal point cloud filtering processing on the target point cloud data to obtain target point cloud data after initial filtering; determining the first normal vector and fitting plane of the target point cloud data after initial filtering; and performing smoothing filtering processing on the first normal vector through the fitting plane to obtain target point cloud data after secondary filtering.

[0099] For ease of understanding, the fused point cloud data is considered as a set of multiple point clouds, and the abnormal point cloud data is considered as a set of multiple abnormal clouds. The fitting plane is a plane fitted by the least squares method after determining the K nearest neighbors of each point cloud.

[0100] Specifically, the computer device identifies anomalous point clouds based on the neighborhood characteristics between each point cloud in the target point cloud data. This involves determining the number of neighboring point clouds within a set range around each point cloud. If the number of neighboring point clouds is less than a set threshold, the current point cloud is considered anomalous. Multiple anomalous point clouds are then combined to obtain anomalous point cloud data. Based on this identified anomalous point cloud data, the computer device performs anomalous point cloud filtering on the target point cloud data, obtaining the first filtered target point cloud data. For each point cloud in the first filtered target point cloud data, the computer device determines its corresponding first normal vector and fitting plane. The first fitted normal vector of the current point cloud is determined using the fitting plane, which can be obtained using the least squares algorithm. The computer device then corrects the corresponding first normal vector using the first fitted normal vector of the current point cloud and projects the current point cloud along the corrected first normal vector direction onto the corresponding fitting plane. This process smooths the first normal vector using the fitting plane, resulting in the second filtered target point cloud data.

[0101] In one embodiment, the computer device performs statistical analysis on the neighborhood of each point cloud, that is, by calculating the average distance between each point cloud and its neighboring point clouds, to obtain a Gaussian distribution. For each point cloud, a mean and a standard deviation can be obtained. If the standard deviation of the current point cloud is greater than a preset range, the current point cloud can be regarded as an abnormal point cloud.

[0102] In this embodiment, by first performing abnormal point cloud filtering on the target point cloud data to obtain target point cloud data with abnormal point clouds removed, and then performing smoothing filtering on the target point cloud data to obtain target point cloud data after further filtering, the computational load of subsequent model reconstruction is reduced, the speed of 3D model reconstruction is improved, and the reconstruction of 3D model using target point cloud data is made more accurate.

[0103] In one embodiment, 3D modeling of a target object based on target point cloud data includes: determining an indicator function corresponding to the target object and a second normal vector of the target point cloud data; performing surface estimation on the target object based on the indicator function, the target point cloud data, and the second normal vector of the target point cloud data to obtain a surface estimation result; extracting a 3D isosurface of the target object based on the surface estimation result, and performing 3D modeling of the target object based on the 3D isosurface.

[0104] The indicator function is a function created using the Poisson algorithm to estimate the surface of the target object.

[0105] Specifically, the computer device transforms the 3D modeling of the target object into a spatial Poisson problem, determines the second normal vector of the target point cloud data, and determines the indicator function corresponding to the target object through implicit fitting. Based on the indicator function, the target point cloud data, and the second normal vector of the target point cloud data, the computer device performs surface estimation on the target object, obtaining a surface estimation result. Here, the target point cloud data represents the position of the target object's surface, and the second normal vector represents the inward and outward directions of the target object's surface. The computer device extracts the 3D isosurface of the target object based on the surface estimation result, and reconstructs a seamless triangular approximation based on the 3D isosurface, finally obtaining the reconstructed 3D model.

[0106] In this embodiment, the surface of the target object can be estimated by using the Poisson algorithm, and an accurate three-dimensional model can be quickly obtained based on the surface estimation results.

[0107] In one embodiment, such as Figure 6 As shown, Figure 6 This is a flowchart illustrating a 3D modeling method in another embodiment. S601: The computer device acquires each frame of raw point cloud data collected from the driving area and the perception result of the target object corresponding to each frame of raw point cloud data, i.e., the detection box of the target object; S602: The raw point cloud data of each frame and the corresponding detection box of the target object are transformed to the same coordinate system; S603: The raw point cloud data is segmented according to the detection box of the target object to obtain the point cloud data corresponding to the target object; S604: The point cloud data of each frame is accumulated to obtain the point cloud data to be processed; S605: It is determined whether the structure type of the target object is axisymmetric. If yes, S606 is executed; otherwise, S607 is executed; S606: The point cloud data to be processed of the target object is mirrored to complete the point cloud data to be processed. S607: Fill in the blank point cloud data in the cloud data to obtain the filled point cloud data to be processed; S608: Determine the similarity between two adjacent frames of point cloud data to be processed. If the similarity is greater than or equal to the similarity threshold, execute S608; if the similarity is less than the similarity threshold, execute S609; S608: Perform point cloud registration processing on the two adjacent frames of point cloud data to be processed to obtain the target point cloud data; S609: Perform point cloud filtering processing on the target point cloud data, including abnormal point cloud filtering processing and smooth fusion of point cloud data, to obtain the filtered target point cloud data; S610: Perform surface reconstruction of the target object's 3D model based on the target point cloud data; S611: Obtain the 3D model of the target object based on the reconstruction result of the 3D model's surface reconstruction.

[0108] In this embodiment, by processing multiple frames of point cloud data to obtain target point cloud data, 3D modeling of the target object can be completed based on the target point cloud data, greatly improving the efficiency of 3D modeling. Furthermore, since the target point cloud data, formed after accumulating and registering point cloud data, is used as modeling data, the accuracy and realism of 3D model reconstruction using the target point cloud data are also improved.

[0109] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0110] Based on the same inventive concept, this application also provides a three-dimensional modeling apparatus for implementing the three-dimensional modeling method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the three-dimensional modeling apparatus provided below can be found in the limitations of the three-dimensional modeling method described above, and will not be repeated here.

[0111] In one embodiment, such as Figure 7 As shown, a 3D modeling device 700 is provided, including: a point cloud acquisition module 702, a point cloud processing module 704, and a point cloud modeling module 706, wherein:

[0112] The point cloud acquisition module 702 is used to acquire multiple frames of raw point cloud data collected from the driving area; the driving area includes multiple objects.

[0113] The point cloud processing module 704 is used to receive the perception results of the object, and based on the perception results, determine the target object and the corresponding multi-frame point cloud data from multiple frames of raw point cloud data; perform cumulative processing on the multi-frame point cloud data to obtain the accumulated point cloud data to be processed; and perform point cloud registration processing on the point cloud to be processed to obtain the registered target point cloud data.

[0114] The point cloud modeling module 706 is used to perform 3D modeling of the target object based on the target point cloud data.

[0115] In one embodiment, the perception result includes a detection box and a confidence score. The point cloud processing module 704 further includes a segmentation processing module 7041, which is used to filter out the target object from the objects included in the original point cloud data based on the confidence score of the object's detection box. For each frame of original point cloud data in multiple frames, the current frame of original point cloud data is segmented based on the detection box of the target object to obtain the current frame point cloud data corresponding to the target object.

[0116] In one embodiment, the 3D modeling device 700 further includes a coordinate transformation module 708, used to establish a lidar coordinate system based on the data acquisition location of the driving area; establish a world coordinate system based on the detection box location of the target object; determine the parameter transformation matrix between the lidar coordinate system and the world coordinate system; transform the original point cloud data and the detection box of the target object to the same coordinate system according to the parameter transformation matrix; and perform segmentation processing on the original point cloud data of the current frame according to the detection box of the target object to obtain the current frame point cloud data corresponding to the target object, including: for each detection box in the multiple detection boxes of the target object, the original point cloud data of the current frame in the same coordinate system is segmented according to the current detection box of the target object to obtain the current frame point cloud data corresponding to the target object.

[0117] In one embodiment, the point cloud processing module 704 further includes an accumulation processing module 7042, which is used to determine the target accumulation position; for each frame of point cloud data in the multi-frame point cloud data, a coordinate transformation matrix corresponding to the current frame point cloud data is determined based on the target accumulation position; through the coordinate transformation matrix, the point cloud coordinates corresponding to the current frame point cloud data are transformed to the target area corresponding to the target accumulation position to obtain the current frame point cloud data to be processed.

[0118] In one embodiment, the point cloud processing module 704 further includes a mirror processing module 7043, used to detect and identify the point cloud data to be processed, determine the structure type of the target object; when the structure type is axisymmetric, determine the axis of symmetry parameter of the target object; perform mirror processing on the point cloud data to be processed of the target object according to the axis of symmetry parameter, to obtain mirror point cloud data corresponding to the point cloud data to be processed; and fill the missing blank point cloud data in the point cloud data to be processed using the mirror point cloud data, to obtain the filled point cloud data to be processed.

[0119] In one embodiment, the point cloud processing module 704 further includes a registration processing module 7044, configured to: determine the current frame of point cloud data to be processed in a multi-frame point cloud data set; determine adjacent point cloud data adjacent to the current frame of point cloud data to be processed; perform a similarity judgment on the current point cloud data to be processed and the adjacent point cloud data; when the similarity is greater than or equal to a similarity threshold, perform point cloud registration processing on the current point cloud data to be processed and the adjacent point cloud data to obtain target point cloud data; use the target point cloud data as the new current frame of point cloud data to be processed, and return to the step of determining the adjacent point cloud data adjacent to the current frame of point cloud data to continue execution until each frame of point cloud data to be processed in the multi-frame point cloud data set is registered with the adjacent point cloud data.

[0120] In one embodiment, the point cloud processing module 704 further includes a filtering processing module 7045, which is used to determine abnormal point cloud data in the target point cloud data, and perform abnormal point cloud filtering processing on the target point cloud data to obtain the target point cloud data after the first filtering; determine the first normal vector and the fitting plane of the target point cloud data after the first filtering; and perform smoothing filtering processing on the first normal vector through the fitting plane to obtain the target point cloud data after the second filtering.

[0121] In one embodiment, the point cloud modeling module 706 is further configured to determine the indicator function corresponding to the target object and the second normal vector of the target point cloud data; perform surface estimation on the target object based on the indicator function, the target point cloud data and the second normal vector of the target point cloud data to obtain the surface estimation result; extract the three-dimensional isosurface of the target object based on the surface estimation result, and perform three-dimensional modeling on the target object based on the three-dimensional isosurface.

[0122] Each module in the aforementioned 3D modeling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0123] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, this computer device includes a processor, memory, input / output interfaces (I / O), a communication interface, a display unit, and input devices. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface, display unit, and input devices are also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a three-dimensional modeling method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0124] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0125] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0126] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0127] In one embodiment, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. The camera's processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the camera to perform the steps described in the method embodiments above.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of three-dimensional modeling, characterized by, The method comprises: acquiring a plurality of frames of original point cloud data collected for a driving area; the driving area comprises a plurality of objects; receiving a perception result of the objects; the perception result comprises a detection box; determining a target object from the plurality of frames of original point cloud data according to the perception result; for each frame of original point cloud data in the plurality of frames of original point cloud data, performing segmentation processing on the original point cloud data according to the detection box of the target object to obtain point cloud data of the target object and background point cloud data corresponding to a background environment of the driving area; the target object is a driving vehicle; performing accumulation processing on the plurality of frames of point cloud data corresponding to the target object to obtain accumulated to-be-processed point cloud data; determining a current frame of to-be-processed point cloud data in the plurality of frames of to-be-processed point cloud data; determining adjacent point cloud data adjacent to the current frame of to-be-processed point cloud data; performing similarity judgment on the current frame of to-be-processed point cloud data and the adjacent point cloud data, wherein the similarity is determined by point cloud features between two frames of to-be-processed point cloud data; the point cloud features comprise a point cloud feature value, echo intensity and height difference; when the similarity is greater than or equal to a similarity threshold, performing point cloud registration processing on the current frame of to-be-processed point cloud data and the adjacent point cloud data to obtain target point cloud data; taking the target point cloud data as new current frame of to-be-processed point cloud data, and returning to the step of determining adjacent point cloud data adjacent to the current frame of to-be-processed point cloud data for continuous execution until each frame of to-be-processed point cloud data in the plurality of frames of to-be-processed point cloud data is registered with the adjacent point cloud data; when the number of adjacent point clouds in a set range around each point cloud in the target point cloud data is less than a set threshold, determining that the point cloud is an abnormal point cloud; performing abnormal point cloud filtering on the target point cloud data to obtain first filtered target point cloud data; for each point cloud in the first filtered target point cloud data, determining a first normal vector and a fitting plane corresponding to the point cloud, and determining a first fitting normal vector of the point cloud through the fitting plane; correcting the corresponding first normal vector through the first fitting normal vector, and projecting the point cloud along the direction of the corrected first normal vector to the corresponding fitting plane to perform smoothing filtering processing to obtain second filtered target point cloud data; performing three-dimensional modeling on the target object according to the second filtered target point cloud data; performing fusion processing on each frame of background point cloud data corresponding to the background environment to obtain fused background point cloud data, and establishing a map model of the background environment through the fused background point cloud data; combining the three-dimensional model of the target object and the map model of the background environment to establish a real driving simulation environment of the target object.

2. The method of claim 1, wherein, The perception result comprises a detection box and a confidence level; and the determining of the target object from the plurality of frames of original point cloud data according to the perception result comprises: screening the target object from the objects included in the original point cloud data according to the confidence level of the objects.

3. The method of claim 2, wherein, Before the method of segmenting the original point cloud data according to the detection box of the target object for each frame of original point cloud data in the multiple frames of original point cloud data, the method further comprises: establishing a laser radar coordinate system based on the data acquisition position of the driving area; establishing a world coordinate system based on the position of the detection box of the target object; determining a parameter conversion matrix between the laser radar coordinate system and the world coordinate system; converting the original point cloud data and the detection box of the target object to the same coordinate system according to the parameter conversion matrix; The method of segmenting the original point cloud data according to the detection box of the target object for each frame of original point cloud data in the multiple frames of original point cloud data comprises: For each detection box in the multiple detection boxes of the target object, the current frame of original point cloud data in the same coordinate system is segmented according to the current detection box of the target object to obtain the current frame of point cloud data corresponding to the target object.

4. The method of claim 1, wherein, The method of accumulating the multiple frames of point cloud data corresponding to the target object to obtain the accumulated to-be-processed point cloud data comprises: determining a target accumulation position; For each frame of point cloud data in the multiple frames of point cloud data, a coordinate conversion matrix corresponding to the current frame of point cloud data is determined based on the target accumulation position; The point cloud coordinates corresponding to the current frame of point cloud data are converted to the target region corresponding to the target accumulation position through the coordinate conversion matrix to obtain the current frame of to-be-processed point cloud data corresponding to the current frame of point cloud data.

5. The method of claim 1, wherein, After the multiple frames of point cloud data corresponding to the target object are accumulated to obtain the accumulated to-be-processed point cloud data, the method further comprises: detecting and identifying the to-be-processed point cloud data to determine the structure type of the target object; when the structure type is an axisymmetric type, determining the symmetry axis parameter of the target object; performing mirror processing on the to-be-processed point cloud data of the target object according to the symmetry axis parameter to obtain mirror point cloud data corresponding to the to-be-processed point cloud data; filling the missing blank point cloud data in the to-be-processed point cloud data through the mirror point cloud data to obtain the filled to-be-processed point cloud data.

6. The method of claim 1, wherein, The point cloud data refers to a point data set of the appearance surface of an object obtained by a sensor.

7. The method of claim 1, wherein, The method of performing three-dimensional modeling on the target object according to the target point cloud data comprises: determining an indicator function corresponding to the target object and a second normal vector of the target point cloud data; performing surface estimation on the target object based on the indicator function, the target point cloud data, and the second normal vector of the target point cloud data to obtain a surface estimation result; extracting a three-dimensional isosurface of the target object according to the surface estimation result, and performing three-dimensional modeling on the target object according to the three-dimensional isosurface.

8. A three-dimensional modeling apparatus, characterized by comprising: The device comprises: a point cloud acquisition module configured to acquire multiple frames of original point cloud data collected for a driving area; the driving area comprises multiple objects; The point cloud processing module is configured to receive a perception result of the object, wherein the perception result comprises a bounding box; determine a target object from the multiple frames of original point cloud data according to the perception result; for each frame of original point cloud data in the multiple frames of original point cloud data, perform segmentation processing on the original point cloud data according to the bounding box of the target object, to obtain point cloud data of the target object and background point cloud data corresponding to a background environment of a driving area; the target object is a driving vehicle; perform accumulation processing on the multiple frames of point cloud data corresponding to the target object, to obtain accumulated to-be-processed point cloud data; determine a current frame of to-be-processed point cloud data in the multiple frames of to-be-processed point cloud data; determine adjacent point cloud data adjacent to the current frame of to-be-processed point cloud data; perform similarity judgment on the current frame of to-be-processed point cloud data and the adjacent point cloud data, wherein the similarity is determined by point cloud features between two frames of to-be-processed point cloud data; the point cloud features comprise a point cloud feature value, echo intensity, and height difference; when the similarity is greater than or equal to a similarity threshold, perform point cloud registration processing on the current frame of to-be-processed point cloud data and the adjacent point cloud data, to obtain target point cloud data; take the target point cloud data as new current frame of to-be-processed point cloud data, and continue to perform the step of determining adjacent point cloud data adjacent to the current frame of to-be-processed point cloud data until registration of each frame of to-be-processed point cloud data in the multiple frames of to-be-processed point cloud data and the adjacent point cloud data is completed; for each point cloud in the target point cloud data, when a number of adjacent point clouds in a surrounding range of the point cloud is less than a set threshold, determine that the point cloud is an abnormal point cloud; perform abnormal point cloud filtering on the target point cloud data, to obtain first filtered target point cloud data; for each point cloud in the first filtered target point cloud data, determine a first normal vector corresponding to the point cloud and a fitting plane, and determine a first fitting normal vector of the point cloud through the fitting plane; correct the corresponding first normal vector through the first fitting normal vector, and project the point cloud to the corresponding fitting plane along a direction of the corrected first normal vector, to perform smoothing filtering processing, to obtain second filtered target point cloud data. The point cloud modeling module is configured to perform three-dimensional modeling on the target object according to the second filtered target point cloud data; perform fusion processing on each frame of background point cloud data corresponding to the background environment, to obtain fused background point cloud data, and establish a map model of the background environment through the fused background point cloud data; combine the three-dimensional model of the target object and the map model of the background environment, to establish a real driving simulation environment of the target object.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 7.

Images