A method, apparatus, device, and medium for transferring a perception model

Abstract

This application discloses a method, apparatus, device, and medium for transferring a perception model, which facilitates the transfer from the source domain to the target domain in point cloud perception tasks, enables the reuse of point cloud data from the original perception model, and improves the generalization ability of the perception model for different application scenarios. The method for transferring a perception model provided in this application includes: transferring the point cloud data of the original perception model to the target perception model according to the application scenario information of the target perception model obtained, thereby obtaining a truth database of the target perception model, wherein the truth database includes point cloud and label information of the perceived target; and training the model based on the truth database of the target perception model to obtain the target perception model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and medium for transferring a perception model. Background Technology

[0002] Point cloud data, with its superior ranging accuracy, can provide highly reliable 3D information within a certain spatial range, thus becoming an important source of benchmark information for tasks including image-related tasks. Point cloud detection can acquire information such as the spatial geometric position, size, and orientation of targets of interest, which is of great significance for tasks requiring high positioning accuracy. Summary of the Invention

[0003] This application provides a method, apparatus, device, and medium for migrating a perception model, which makes the migration from the source domain to the target domain in point cloud perception tasks more convenient, enables the reuse of point cloud data of the original perception model, and improves the generalization ability of the perception model for different application scenarios.

[0004] This application provides a method for transferring a perception model, comprising:

[0005] Based on the application scenario information of the target perception model obtained as needed, the point cloud data of the original perception model is migrated to the target perception model to obtain the truth database of the target perception model, wherein the truth database includes the point cloud and label information of the perceived target;

[0006] Based on the truth database of the target perception model, the model is trained to obtain the target perception model.

[0007] The method provided in this application embodiment, based on the application scenario information of the target perception model obtained as needed, migrates the point cloud data of the original perception model to the target perception model, obtaining a truth database of the target perception model, wherein the truth database includes the point cloud and label information of the perceived target; based on the truth database of the target perception model, model training is performed to obtain the target perception model. This makes the migration from the source domain to the target domain in the point cloud perception task more convenient, realizes the reuse of the point cloud label data of the original perception model, and enables target detection in the target scene, improving the generalization ability of the perception model to different application scenarios.

[0008] In some embodiments, the method further includes:

[0009] Based on the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model, the distribution density of the point cloud data in the truth database is adjusted to obtain the reconstructed dataset.

[0010] The process of training the model based on the truth database of the target perception model includes: training the model according to the reconstructed dataset.

[0011] Thus, by transferring point cloud density, the generalization ability of the perception model to different application scenarios is further improved. This can solve the problem of difficulty in generalizing to targets with small spatial volume, such as pedestrians and non-motorized vehicles, enabling the target perception model to achieve more accurate target perception for various target scenarios.

[0012] In some embodiments, based on the application scenario information of the target perception model obtained as needed, the point cloud data of the original perception model is migrated to the target perception model to obtain the truth database of the target perception model, including:

[0013] Based on the application scenario information of the target perception model obtained as needed, the ground truth database of the target perception model is obtained by using size scaling, upsampling, and / or downsampling methods on the point cloud data of the detected target of the original perception model.

[0014] In some embodiments, the distribution density of the point cloud data in the truth database is adjusted according to the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model to obtain a reconstructed dataset, including:

[0015] Determine the difference between the point cloud line bundle corresponding to the original perception model and the point cloud line bundle corresponding to the target perception model;

[0016] When the difference value is greater than a preset threshold, the point cloud data distribution density estimation map corresponding to the original perception model and the point cloud data distribution density estimation map corresponding to the target perception model are determined respectively. The point cloud data distribution density estimation map corresponding to the original perception model is compared with the point cloud data distribution density estimation map corresponding to the target perception model. Based on the comparison result, the point cloud data in the original perception model is upsampled and / or downsampled to obtain the reconstructed dataset.

[0017] In some embodiments, the distribution density of the point cloud data in the truth database is adjusted according to the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model to obtain a reconstructed dataset, including:

[0018] Determine the difference between the point cloud line bundle corresponding to the original perception model and the point cloud line bundle corresponding to the target perception model;

[0019] When the difference value is less than a preset threshold:

[0020] Determine the number of point cloud bundles corresponding to the original perception model and the pitch angle of each point cloud bundle; and determine the number of point cloud bundles corresponding to the target perception model and the pitch angle of each point cloud bundle.

[0021] Based on the number of point cloud bundles corresponding to the target perception model and the pitch angle of each point cloud bundle, as well as the number of point cloud bundles corresponding to the original perception model and the pitch angle of each point cloud bundle, the point cloud data on one or more point cloud bundles corresponding to the original perception model in the truth database are interpolated to obtain the reconstructed dataset.

[0022] In some embodiments, the difference value includes the difference between the density of the point cloud bundle corresponding to the original perception model and the density of the point cloud bundle corresponding to the target perception model.

[0023] In some embodiments, after obtaining the target perception model, the method further includes:

[0024] The obtained target perception model is used to perform target detection, and the detection results are then corrected to obtain the final detection result.

[0025] This application provides a perceptual model transfer device, comprising:

[0026] The first unit is used to migrate the point cloud data of the original perception model to the target perception model according to the application scenario information of the target perception model obtained as needed, so as to obtain the truth database of the target perception model, wherein the truth database includes the point cloud and label information of the perceived target.

[0027] The second unit is used to train the model based on the truth database of the target perception model to obtain the target perception model.

[0028] Another embodiment of this application provides an electronic device including a memory and a processor, wherein the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory and execute any of the methods described above according to the obtained program.

[0029] Another embodiment of this application provides a computer-readable storage medium storing computer-executable instructions for causing the computer to perform any of the methods described above. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A schematic diagram of the overall process of the transfer method of the perception model provided in the embodiments of this application;

[0032] Figure 2 This is a schematic flowchart illustrating the method for establishing a standard coordinate system provided in an embodiment of this application.

[0033] Figure 3 This is a schematic diagram illustrating the specific process of adjusting point cloud density provided in an embodiment of this application;

[0034] Figure 4 A schematic diagram illustrating the specific process of upsampling or downsampling point cloud data provided in the embodiments of this application;

[0035] Figure 5 A schematic diagram illustrating the process of determining the density estimation map of point cloud data distribution provided in an embodiment of this application;

[0036] Figure 6 A schematic diagram of linear velocity clustering of point clouds provided in an embodiment of this application;

[0037] Figure 7 This is a schematic diagram of point cloud data interpolation provided in an embodiment of this application;

[0038] Figure 8 A schematic diagram illustrating the specific process of model training provided in this application embodiment;

[0039] Figure 9 This is a schematic diagram illustrating the specific process for obtaining detection results provided in an embodiment of this application;

[0040] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0041] Figure 11 This is a schematic diagram of the structure of a perception model transfer device provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0043] This application provides a method, apparatus, device, and medium for migrating a perception model, which makes the migration from the source domain to the target domain in point cloud perception tasks more convenient, enables the reuse of point cloud data of the original perception model, and improves the generalization ability of the perception model for different application scenarios.

[0044] The methods, apparatus, equipment, and media are based on the same concept of the application. Since the methods, apparatus, equipment, and media solve problems in similar principles, the implementation of the apparatus, equipment, media, and methods can refer to each other, and repeated parts will not be described again.

[0045] The terms "first," "second," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0046] The following examples and embodiments are to be understood as illustrative only. While this specification may refer to "a," "an," or "some" examples or embodiments in several places, this does not mean that every such reference relates to the same example or embodiment, nor does it mean that the feature applies only to a single example or embodiment. Individual features of different embodiments may also be combined to provide other embodiments. Furthermore, terms such as "comprising" and "including" should be understood not to limit the described embodiments to consisting only of those features mentioned; such examples and embodiments may also include features, structures, units, modules, etc., not specifically mentioned.

[0047] The various embodiments of this application will now be described in detail with reference to the accompanying drawings. It should be noted that the order in which the embodiments are presented in this application represents only a chronological order and does not represent the superiority or inferiority of the technical solutions provided by the embodiments.

[0048] Due to factors such as the installation location, height, elevation angle, imaging principle, and wiring of the lidar, the point cloud perception model in different scenarios, namely the 3D perception model in lidar perception (referred to as the perception model or model), is highly dependent on the sensor characteristics and the scene. Most of them lack generalization ability and usually only achieve high accuracy in similar scenes in the training set, but cannot be effectively transferred to other scenes.

[0049] For example, the relevant datasets for driving scenarios are constantly increasing, and the amount of available data and the number of models are constantly increasing. However, the differences between vehicle-side and road-side scenarios, as well as the differences between sensors ranging from sixteen wire harnesses to hundreds of wire harnesses, mean that existing perception models are not universally applicable.

[0050] Therefore, when facing new scenarios or different models of LiDAR, it is often necessary to repeat the data collection, annotation, and training process in order to ensure that the trained perception model meets the actual needs.

[0051] This application provides a technical solution for transferring point cloud perception models in traffic scenarios, applicable to scenarios such as autonomous driving, vehicle-road cooperation, and holographic intersections. It solves the problem of difficult transfer from the source domain to the target domain in point cloud perception tasks, realizes label reuse of different datasets, and improves the generalization ability of perception models to different scenarios through point cloud data features and sensor imaging principles.

[0052] The following embodiments of this application focus on the transfer of perception models between different LiDARs (differences exist in beam type, installation location, resolution, etc.) and different scenarios (differences exist in target type, size, etc.), and propose a series of strategies to assist in the transfer.

[0053] See Figure 1 The perception model transfer method provided in this application includes:

[0054] Step S101: Based on the application scenario information of the target perception model obtained, migrate the point cloud data of the original perception model to the target perception model to obtain the truth database of the target perception model, wherein the truth database includes the point cloud and label information of the perceived target.

[0055] The application scenario information of the target perception model includes, for example, information such as the category, size, and distribution ratio of the detected targets in the application scenario of the target perception model.

[0056] The point cloud data of the original sensing model can be referred to as source domain data; the point cloud data of the target sensing model can be referred to as target domain data.

[0057] Source domain and target domain are concepts in transfer learning. The source domain is the existing knowledge domain, which in this case refers to the dataset that has been labeled; the target domain is the domain to be learned, which in this case refers to the dataset that has not been labeled.

[0058] The source domain and the target domain differ in point cloud morphology and target distribution. In this application, the embodiment uses source domain labels or models to assist in the target domain detection task.

[0059] The truth database refers to a database obtained by sampling from the source domain label data according to the distribution of the number of categories, and scaling the point cloud and labels according to the size distribution. The truth database stores the target point cloud and label information obtained from the application scenario segmentation corresponding to the original perception model.

[0060] This application addresses situations where there is a significant size difference between the detected targets in the target domain and the source domain. By constructing a ground truth database that conforms to the target domain scenario, the target perception model can detect zero-sample targets. For example, when the source domain only contains street scenes, while the target domain is located on an outer ring road and large trucks such as semi-trailers (i.e., zero-sample targets) frequently pass by, the constructed ground truth database can include large truck sample data obtained by scaling up and upsampling the smaller truck sizes in the source domain. Therefore, the target perception model also has a certain detection capability for large trucks.

[0061] Step S102: Based on the truth database of the target perception model, perform model training to obtain the target perception model.

[0062] In some embodiments, the method further includes:

[0063] Based on the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model, the distribution density of the point cloud data in the truth database is adjusted to obtain the reconstructed dataset.

[0064] The process of training the model based on the truth database of the target perception model includes: training the model according to the reconstructed dataset.

[0065] In other words, the embodiments of this application can map the point cloud distribution of the source domain to the target domain, making the point cloud distribution in the newly generated ground truth dataset more similar to the point cloud distribution of the target domain.

[0066] This application addresses the challenge of generalizing to targets with small spatial volumes, such as pedestrians and non-motorized vehicles, by modifying the distribution density (i.e., point cloud morphology) of the source domain point cloud data. This makes the source domain data as close to the target domain as possible, fully realizing the reuse of source domain labels. Data augmentation improves the original perception model's detection capabilities under different point cloud beams, LiDAR models, and weather conditions, making it compatible with different perception algorithms and sensor configurations. Furthermore, by adjusting the distribution density of the point cloud data in the original perception model, labeled point cloud data can be migrated across different scenarios, such as vehicle-side and road-side, and under different sensor configurations. This process is more flexible and controllable compared to domain-adaptive end-to-end deep network processing strategies.

[0067] In some embodiments, see Figure 2 Before step S101, the method also includes: establishing a unified standard coordinate system for the source and target domains. Specifically:

[0068] The system analyzes the category and size distribution of the source domain data to correct the original coordinate system. For example, the source domain label is denoted as I, and the label should include at least the center point location, size information, and yaw angle information of the detected targets. The system calculates the category distribution of the detected targets in the source domain, as well as the size distribution corresponding to different categories of detected targets. Furthermore, it calculates the ground elevation of the source domain dataset to correct the original coordinate system.

[0069] The detection target includes, for example, traffic participants that are of interest in road scenarios, such as motor vehicles, pedestrians, and non-motorized vehicles.

[0070] Regarding the label distribution: The labels contain the target's category and size information. Here, the size distribution is statistically analyzed according to the category based on the labels. For example, for the "car" category, all "car" targets in the labels are summarized, and their length, width, and height are calculated as statistical distribution histograms. This information is calculated primarily because point cloud perception models encode absolute coordinates, which are typically more sensitive to prior knowledge of target size. Subsequently, the source domain size distribution needs to be mapped to the target domain.

[0071] Regarding the ground height: Because the point cloud perception model encodes absolute coordinates, the point cloud is highly sensitive to ground height. Therefore, it is necessary to calibrate the point clouds in the source and target domains according to the ground. During model training and inference, the LiDAR coordinate system needs to be leveled, fixing the ground at a specific height, i.e., transforming it to the standard coordinate system. Calculating the ground height is to calculate the rotation and translation matrix from the original coordinate system to the standard coordinate system.

[0072] Since the point cloud of the input target perception model is in the standard coordinate system, it is necessary to transform the point clouds in the source domain and target domain in their respective lidar coordinate systems to the standard coordinate system.

[0073] Subsequent point cloud feature encoding is performed in the standard coordinate system, and the detection results of the perception model are restored to the original coordinate system as needed. Because detection needs to be performed in the standard coordinate system, the point cloud data should be transformed to the standard coordinate system before being input into the perception model, and then restored to the original LiDAR coordinate system.

[0074] In other words, in this embodiment of the application, before transferring the original perception model to the target perception model, it is necessary to define a unified standard coordinate system for the source domain and the target domain.

[0075] In some embodiments, see Figure 2 The process of establishing a standard coordinate system includes, for example:

[0076] Step S201: Determine the rotation matrix and translation vector between the current lidar coordinate system (original coordinate system) and the standard coordinate system.

[0077] For example: the radar orientation is defined as the forward x-axis, the z-axis as the direction perpendicular to the ground, and the coordinate system is a right-handed coordinate system. With the sensor fixed, the ground plane is fitted, the ground normal vector is calculated using the Rodrigues rotation formula, and ground leveling is performed. The point cloud is then translated to unify the ground height, thereby determining the rotation matrix and translation vector between the current lidar coordinate system and the standard coordinate system. Since ground leveling is required, the ground is fixed at a specific height.

[0078] Step S202: Determine whether the source field of view is smaller than the target field of view. If so, proceed to step S203; otherwise, proceed to step S204.

[0079] Step S203: When the field of view of the source domain is smaller than the field of view of the target domain, fill the point cloud region within the field of view of each frame of the point cloud in the source domain, segment the fan-shaped region of the point cloud from the edge of the field of view that does not contain the target, so that the field of view of the segmented point cloud is equal to the difference between the field of view of the target domain and the source domain, and fill the point cloud to the edge of the point cloud in the source domain, so that the field of view of the processed point cloud is consistent with the field of view of the target domain, and the spatial range of the source domain data is consistent with that of the target domain.

[0080] Step S204: When the source field of view is greater than or equal to the target field of view, the source field of view is clipped. For example, with the target field of view as a reference, the point cloud in the source field that exceeds the field of view cone of the target field of view is directly removed so that the clipped source field of view is consistent with the target field of view.

[0081] Specifically, step S101 can be based on the established standard coordinate system to construct a truth database.

[0082] In some embodiments, step S101 includes:

[0083] Based on the application scenario information of the target perception model obtained as needed, the ground truth database of the target perception model is obtained by using size scaling, upsampling, and / or downsampling methods on the point cloud data of the detected target of the original perception model.

[0084] The application scenario information of the target perception model includes the category, size, distribution ratio, etc. of the detected targets in the application scenario, such as whether there are a large number of pedestrians, traffic flow, and whether large trucks are allowed to pass.

[0085] Based on the application scenario information of the target perception model, the source domain data is sampled according to the category of the detected target to construct a ground truth database that conforms to the target domain category and size distribution (the length, width, and height distribution of targets according to category statistics). For example, large trucks often appear in actual deployment scenarios of the target domain, so it is necessary to add corresponding targets from the source domain data to the ground truth database. If the target of this type does not exist in the source domain data, the size of smaller trucks of the same type is stretched, or the source domain data is upsampled or downsampled to make the samples in the ground truth database more diverse. Among them, downsampling can use algorithms such as voxel downsampling and uniform downsampling; upsampling can use algorithms such as moving least squares and PU-Net.

[0086] In this embodiment, a ground truth database containing zero-sample motor vehicle targets is constructed based on the characteristics of the source and target domains. This database simulates special conditions such as rainy days, making full use of existing labels to simulate different weather and scene conditions. The resulting point cloud data is more closely aligned with real-world scenarios compared to scene and sensor simulations. Furthermore, the full utilization of scene information, including visual images and fixed structured roads, in this embodiment can compensate for the impact of insufficient resolution of the LiDAR sensor on the detection results.

[0087] In some embodiments, the distribution density of point cloud data in the truth database is adjusted according to the density of the point cloud bundles corresponding to the original sensing model and the density of the point cloud bundles corresponding to the target sensing model to obtain a reconstructed dataset. That is, under the condition that the field of view of the source domain and the target domain are consistent, that is, in the established standard coordinate system, the point cloud morphology is changed by sampling or interpolation according to the point cloud distribution of the source domain and the target domain. The point cloud morphology includes the emission angle of the laser bundles, so as to change the point cloud distribution density and ultimately improve the detection accuracy of the target sensing model.

[0088] In some embodiments, see Figure 3 Based on the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model, the distribution density of the point cloud data in the truth database is adjusted to obtain the reconstructed dataset, specifically including:

[0089] Step S301: Determine whether the difference between the point cloud bundles between the source domain and the target domain is greater than a preset threshold. If yes, proceed to step S302; otherwise, proceed to step S303.

[0090] For example, determine the difference between the density of the point cloud bundles corresponding to the original perception model (e.g., 4 bundles) and the density of the point cloud bundles corresponding to the target perception model (e.g., 6 bundles), and determine whether the difference is greater than a preset threshold. If it is, then execute step S302; otherwise, execute step S303.

[0091] In addition, the difference in point cloud bundles between the source and target domains can also be reflected by other parameters (such as the pitch angle of the point cloud bundles), such as the difference in pitch angle of the point cloud bundles, that is, it is not limited to the difference in density of the point cloud bundles.

[0092] The purpose of adjusting the distribution density of point cloud data in the truth database is to make the distribution pattern of the point cloud data after the original perception model is adjusted to better match the distribution pattern of the point cloud data of the target perception model that needs to be achieved.

[0093] Step S302: Based on the density of the point cloud bundles corresponding to the original sensing model and the density of the point cloud bundles corresponding to the target sensing model, the distribution density of the point cloud data in the truth database is upsampled or downsampled, so that the difference between the density of the point cloud bundles corresponding to the original sensing model and the target sensing model is less than a preset threshold, that is, the distribution pattern of the adjusted point cloud data in the original sensing model is more in line with the requirements of the target sensing model.

[0094] In some embodiments, see Figure 4 Step S302 in the figure includes:

[0095] S401. Determine the distribution density estimation map of the point cloud data of the original perception model and the distribution density estimation map of the point cloud data of the target perception model.

[0096] S402. Compare the distribution density estimation map of the point cloud data of the original perception model with the distribution density estimation map of the point cloud data of the target perception model.

[0097] S403. Based on the comparison results, the point cloud data in the original perception model is upsampled or downsampled so that the distribution pattern of the adjusted point cloud data in the original perception model meets the requirements of the target perception model.

[0098] The determination of the aforementioned density estimation map includes, for example:

[0099] The point cloud density distribution is calculated for both the source and target domains. A frame of point cloud data that does not contain moving targets is selected from the current scene (either the target domain or the dataset in the source domain), and the ground point cloud region is segmented using a plane fitting algorithm. The plane fitting algorithm can employ random sample consensus, raster sampling fitting, etc. Because the point cloud density distribution is uneven, a raster is constructed to estimate the point cloud distribution from a bird's-eye view perspective. See [link to relevant documentation] Figure 5 For both the source and target domains, the ground point cloud region is divided into 2D grids. The number of ground points in each grid is counted to construct a pseudo-image (i.e., an image composed of grids, where the pixel value corresponds to the number of points in the current grid. The pixel value of the pseudo-image differs from that of a regular image, which ranges from 0 to 255; the pixel value of the pseudo-image actually represents the number of points in the grid). Based on this, mean filtering is performed to obtain a density estimation map of dense point cloud data from a bird's-eye view (BEV) perspective. For example, if there are 8 points in a certain grid, the pseudo-image obtained by constructing the grid corresponds to position 8. To ensure that the density map is uniform overall, mean filtering is also performed on the pseudo-image to obtain a point cloud density estimation map. Finally, the point cloud data density estimation map corresponding to the original perception model (referred to as the first density estimation map) and the point cloud data density estimation map corresponding to the target perception model (referred to as the second density estimation map) are obtained respectively.

[0100] Comparing the first density estimation map and the second density estimation map, for each grid cell, if the target domain point cloud density value within that grid cell is lower than the source domain point cloud density value, the source domain point cloud data corresponding to that grid cell is downsampled; otherwise, the source domain point cloud data corresponding to that grid cell is upsampled. Since sampling is performed within a single grid cell, the downsampling algorithm may include uniform sampling, etc.

[0101] Step S303: When the source domain and target domain are relatively similar, i.e., when the difference between the point cloud line bundles corresponding to the original sensing model and the point cloud line bundles corresponding to the target sensing model is less than a preset threshold, the K-means clustering algorithm is first used to cluster the point cloud data of the source domain according to the elevation angle of the lidar corresponding to the source domain, and the point cloud line bundles of the source domain are encoded according to the clustering results. Similarly, the point cloud data of the target domain is clustered according to the elevation angle of the lidar corresponding to the target domain, and the point cloud line bundles of the target domain are encoded according to the clustering results.

[0102] For example, see Figure 6 The lidar emits beams of light in its surroundings, and the number of clusters is equal to the number of laser point cloud beams. Figure 6 Taking a 4-line lidar as an example, with 4 clusters, clustering is performed based on the elevation angle (the angle between the line connecting the radar and the point cloud points and the horizontal direction) to obtain... Figure 6The four clustering results shown are as follows: the first cluster corresponds to 3 points, the second cluster corresponds to 2 points, the third cluster corresponds to 4 points, and the fourth cluster corresponds to 1 point.

[0103] The difference in point cloud beamwidth between the source and target domains can include differences in point cloud beamwidth and elevation angle, which directly determine the point cloud density and distribution. For example, when the beamwidth difference is within 20% and the elevation angle difference is within 10°, k-means clustering and interpolation should be used to process the point cloud data of the original sensing model to better simulate the laser point cloud beamwidth and working principle of the target sensing model. However, because distant points in the lidar point cloud are extremely sparse, when the point cloud density and distribution differences are large, the point cloud morphology restored by k-means clustering and interpolation is unreliable. In this case, the density distribution statistics and sampling method in step S302 should be used to process the point cloud data of the original sensing model.

[0104] Step S304: After obtaining the above clustering results, based on the clustering results and the pitch angle corresponding to the point cloud bundles in the target domain, the point cloud data of the source domain is processed so that the processed source domain has the same total number of bundles and pitch angle as the target domain. Then, the point coordinates of the newly added bundles are determined by interpolation. For example, see... Figure 7 The original perception model's point cloud clustering result included 4 clusters, while the simulated target perception model's clustering result included 6 clusters. The coarse position of the interpolation point was calculated according to the beam emission direction (i.e., Figure 7 The position of the center point of a circle with radius r in the graph is determined, and further, based on the point cloud points near the coarse position (e.g., the average position of the 5 nearest points to the coarse position), the interpolation point is determined (i.e., the position of the center point of the circle with radius r in the graph). Figure 7 The refined position of the point (shown in the fine position) is the final determined interpolation point position, so that the adjusted point cloud distribution shape of the source domain meets the requirements of the point cloud distribution shape of the target domain.

[0105] The point cloud density migration scheme provided in this application migrates the density distribution of point cloud data from the source domain to the target domain, enabling the labels in the source domain to be applied to sensors with different viewpoints, different installation heights, or even different types. This migration is performed only at the data level and does not bring more time or computational costs to inference.

[0106] In some embodiments, see Figure 8 Step S102 specifically includes:

[0107] Step S801: Using a preset feature extraction strategy, extract features from the reconstructed dataset and aggregate features at different scales to obtain the feature extraction results.

[0108] For example, by constructing voxel networks, or deep neural networks (like PointNet++ networks) for classifying and segmenting point cloud data with irregular shapes, features can be extracted from the reconstructed dataset and features at different scales can be aggregated.

[0109] Step S802: Determine the detection head and loss function for model training according to the preset feature extraction strategy.

[0110] For example, taking voxel networks as an example, the loss functions used may include loss functions for separating positive and negative samples (such as Focal Loss) and loss functions for location regression (such as SmoothL1 Loss, L1 Loss, DIOULoss, etc.).

[0111] After feature extraction, the detection head is used to revert the position of the detected target to the original coordinate system and classify it.

[0112] Step S803: Based on the feature extraction results, using the detection head and loss function, train the model based on the reconstructed dataset to obtain the target perception model.

[0113] Data augmentation strategies during training include ground truth sampling, global rotation and translation, and random noise. The ground truth sampling data is provided by the aforementioned ground truth database.

[0114] Furthermore, by simulating point cloud occlusion through visibility simulation, ground points in the ground value database are interpolated or sampled to restore or simulate point cloud loss caused by ground water accumulation. Point cloud augmentation technology is then used to simulate real-world scenarios under different weather conditions.

[0115] In some embodiments, it also includes:

[0116] Step S804: Detect the target using the obtained target perception model, obtain the detection result, and use other information to correct the detection result to obtain a more accurate detection result.

[0117] Other information includes, for example, image data and road information at the road end. The specific correction method is not limited in this application embodiment.

[0118] In some embodiments, see Figure 9 Step S804 includes:

[0119] Step S901: When the target domain scene contains image or video data with reliable calibration relationships, the confidence of the corresponding point cloud detection box under the projection relationship is reassigned by detecting the single frame visual data.

[0120] The calibration relationship is reliable, meaning that the lidar and camera need to be jointly calibrated so that the projection relationship from the point cloud to the image can be calculated using intrinsic and extrinsic parameter matrices.

[0121] The projection relationship, that is, the correspondence between point cloud points and image pixel positions, transforms the lidar coordinate system to the camera coordinate system, and then to the image plane.

[0122] The confidence level is calculated as follows:

[0123] First, the 3D detection bounding boxes of the point cloud are projected onto the image plane. Then, based on the distance between the center of the different 2D bounding boxes and the center of the image detection bounding boxes, the association between the LiDAR and the image detection bounding boxes is determined by the Hungarian matching algorithm (i.e., which LiDAR detection bounding box corresponds to which image detection bounding box). Finally, a new confidence score is calculated based on the matching results.

[0124] For each 3D detection box, we have:

[0125] score new =λ*score lidar +(1-λ)*score camera

[0126] Among them, score new Indicates the new confidence level, score lidar The score represents the confidence level of the lidar. camera The confidence score of the image detection box matched with the LiDAR is represented by λ∈(0,1), which is a preset constant used to measure the reliability of the detection results.

[0127] Step S902: When the lidar is installed in a fixed scene at the roadside and the target domain contains a large amount of data from the same scene, specify the detection area for each scene and crop the detection area.

[0128] Step S903: Project the point cloud data of the cropped detection area onto the overhead view and draw a scatter plot of the corresponding area;

[0129] Step S904: Based on the scatter plot, mark the road area and non-road area by image labeling to obtain the labeling result;

[0130] Step S905: Map the marking results to the point cloud coordinate system (map the marking results on the pseudo image back to the point cloud data), and filter out the vehicle detection results in non-road areas to minimize the interference of the background on the results.

[0131] The cropping mentioned above is mainly to unify the field of view of the source domain and the field of view of the target domain, thereby mitigating the impact of field of view differences on the model.

[0132] The step of labeling road and non-road areas in the image and transforming scatter points onto the image aims to reduce the computational complexity of determining whether scatter points are within polygons. Since road shapes are often irregular, directly calculating whether scatter points are within polygons typically requires high complexity or consideration of special cases. Therefore, a strategy of converting the scatter plot into a pseudo-image is adopted, mapping scatter points to regular pixel positions, and using image annotation tools (such as LabelMe) to delineate road areas. Pixel values ​​within road areas are recorded as 1 (similar to a mask), while other areas are recorded as 0. When the pixel at the image position mapped to a scatter point is 1, the corresponding point cloud point is considered to be within the road area. Based on this, the calculation of whether all point cloud points are within the road area is performed.

[0133] The following describes the device or apparatus provided in the embodiments of this application, and the explanations or examples of the same or corresponding technical features as those described in the above methods will not be repeated hereafter.

[0134] This application provides an electronic device, which can be any device that needs to detect targets using a point cloud perception model. See also... Figure 10 For example, including:

[0135] Processor 600 is used to read the program from memory 620 and execute the following procedures:

[0136] Based on the application scenario information of the target perception model obtained as needed, the point cloud data of the original perception model is migrated to the target perception model to obtain the truth database of the target perception model, wherein the truth database includes the point cloud and label information of the perceived target;

[0137] Based on the truth database of the target perception model, the model is trained to obtain the target perception model.

[0138] In some embodiments, the processor 600 is further configured to read a program from the memory 620 and execute the following processes:

[0139] Based on the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model, the distribution density of the point cloud data in the truth database is adjusted to obtain the reconstructed dataset.

[0140] The process of training the model based on the truth database of the target perception model includes: training the model according to the reconstructed dataset.

[0141] In some embodiments, based on the application scenario information of the target perception model obtained as needed, the point cloud data of the original perception model is migrated to the target perception model to obtain the truth database of the target perception model, including:

[0142] Based on the application scenario information of the target perception model obtained as needed, the ground truth database of the target perception model is obtained by using size scaling, upsampling, and / or downsampling methods on the point cloud data of the detected target of the original perception model.

[0143] In some embodiments, the distribution density of the point cloud data in the truth database is adjusted according to the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model to obtain a reconstructed dataset, including:

[0144] Determine the difference between the point cloud line bundle corresponding to the original perception model and the point cloud line bundle corresponding to the target perception model;

[0145] When the difference value is greater than a preset threshold, the point cloud data distribution density estimation map corresponding to the original perception model and the point cloud data distribution density estimation map corresponding to the target perception model are determined respectively. The point cloud data distribution density estimation map corresponding to the original perception model is compared with the point cloud data distribution density estimation map corresponding to the target perception model. Based on the comparison result, the point cloud data in the original perception model is upsampled and / or downsampled to obtain the reconstructed dataset.

[0146] In some embodiments, the distribution density of the point cloud data in the truth database is adjusted according to the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model to obtain a reconstructed dataset, including:

[0147] Determine the difference between the point cloud line bundle corresponding to the original perception model and the point cloud line bundle corresponding to the target perception model;

[0148] When the difference value is less than a preset threshold:

[0149] Determine the number of point cloud bundles corresponding to the original perception model and the pitch angle of each point cloud bundle; and determine the number of point cloud bundles corresponding to the target perception model and the pitch angle of each point cloud bundle.

[0150] Based on the number of point cloud bundles corresponding to the target perception model and the pitch angle of each point cloud bundle, as well as the number of point cloud bundles corresponding to the original perception model and the pitch angle of each point cloud bundle, the point cloud data on one or more point cloud bundles corresponding to the original perception model in the truth database are interpolated to obtain the reconstructed dataset.

[0151] In some embodiments, the difference value includes the difference between the density of the point cloud bundle corresponding to the original perception model and the density of the point cloud bundle corresponding to the target perception model.

[0152] In some embodiments, after obtaining the target perception model, the processor 600 is further configured to read the program in the memory 620 and execute the following processes:

[0153] The obtained target perception model is used to perform target detection, and the detection results are then corrected to obtain the final detection result.

[0154] Transceiver 610 is used to receive and send data under the control of processor 600.

[0155] Among them, Figure 10 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 600 and memory represented by memory 620 together. The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 610 can be multiple components, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium, including wireless channels, wired channels, optical fibers, etc. For different user equipment, the user interface 630 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0156] The processor 600 is responsible for managing the bus architecture and general processing, while the memory 620 can store the data used by the processor 600 when performing operations.

[0157] In some implementations, the processor 600 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a CPLD (Complex Programmable Logic Device), and the processor may also employ a multi-core architecture.

[0158] The processor executes any of the methods described in the embodiments of this application according to the obtained executable instructions by calling a computer program stored in memory. The processor and memory may also be physically separated.

[0159] It should be noted that the apparatus provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0160] See Figure 11 Another video processing apparatus provided in this application embodiment includes:

[0161] The first unit 11 is used to migrate the point cloud data of the original perception model to the target perception model according to the application scenario information of the target perception model obtained as needed, so as to obtain the truth database of the target perception model, wherein the truth database includes the point cloud and label information of the perceived target.

[0162] The second unit 12 is used to train the model based on the truth database of the target perception model to obtain the target perception model.

[0163] In some embodiments, the first unit 11 is further configured to:

[0164] Based on the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model, the distribution density of the point cloud data in the truth database is adjusted to obtain the reconstructed dataset.

[0165] The second unit 12 is specifically used for: training the model based on the reconstructed dataset.

[0166] In some embodiments, based on the application scenario information of the target perception model obtained as needed, the point cloud data of the original perception model is migrated to the target perception model to obtain the truth database of the target perception model, including:

[0167] Based on the application scenario information of the target perception model obtained as needed, the ground truth database of the target perception model is obtained by using size scaling, upsampling, and / or downsampling methods on the point cloud data of the detected target of the original perception model.

[0168] In some embodiments, the distribution density of the point cloud data in the truth database is adjusted according to the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model to obtain a reconstructed dataset, including:

[0169] Determine the difference between the point cloud line bundle corresponding to the original perception model and the point cloud line bundle corresponding to the target perception model;

[0170] When the difference value is greater than a preset threshold, the point cloud data distribution density estimation map corresponding to the original perception model and the point cloud data distribution density estimation map corresponding to the target perception model are determined respectively. The point cloud data distribution density estimation map corresponding to the original perception model is compared with the point cloud data distribution density estimation map corresponding to the target perception model. Based on the comparison result, the point cloud data in the original perception model is upsampled and / or downsampled to obtain the reconstructed dataset.

[0171] In some embodiments, the distribution density of the point cloud data in the truth database is adjusted according to the density of the point cloud bundles corresponding to the original perception model and the density of the point cloud bundles corresponding to the target perception model to obtain a reconstructed dataset, including:

[0172] Determine the difference between the point cloud line bundle corresponding to the original perception model and the point cloud line bundle corresponding to the target perception model;

[0173] When the difference value is less than a preset threshold:

[0174] Determine the number of point cloud bundles corresponding to the original perception model and the pitch angle of each point cloud bundle; and determine the number of point cloud bundles corresponding to the target perception model and the pitch angle of each point cloud bundle.

[0175] Based on the number of point cloud bundles corresponding to the target perception model and the pitch angle of each point cloud bundle, as well as the number of point cloud bundles corresponding to the original perception model and the pitch angle of each point cloud bundle, the point cloud data on one or more point cloud bundles corresponding to the original perception model in the truth database are interpolated to obtain the reconstructed dataset.

[0176] In some embodiments, the difference value includes the difference between the density of the point cloud bundle corresponding to the original perception model and the density of the point cloud bundle corresponding to the target perception model.

[0177] In some embodiments, after obtaining the target perception model, the second unit 12 is further configured to:

[0178] The obtained target perception model is used to perform target detection, and the detection results are then corrected to obtain the final detection result.

[0179] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0181] Any of the devices or apparatuses provided in the embodiments of this application can specifically be desktop computers, portable computers, smartphones, tablet computers, personal digital assistants (PDAs), etc. They may include a central processing unit (CPU), memory, input / output devices, etc. Input devices may include keyboards, mice, touchscreens, etc., and output devices may include display devices such as liquid crystal displays (LCDs) and cathode ray tubes (CRTs).

[0182] The memory may include read-only memory (ROM) and random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In the embodiments of this application, the memory may be used to store the program of any of the methods provided in the embodiments of this application.

[0183] The processor executes any of the methods described in the embodiments of this application according to the program instructions stored in the memory.

[0184] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the methods described in the above embodiments. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0185] This application provides a computer-readable storage medium for storing computer program instructions used in the apparatus provided in the above-described embodiments, including a program for performing any of the methods provided in the above-described embodiments. The computer-readable storage medium may be a non-transitory computer-readable medium.

[0186] The computer-readable storage medium can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0187] It should be understood that:

[0188] The access technology used by entities in a communication network to transmit traffic can be any suitable current or future technology, such as WLAN (Wireless Local Access Network), WiMAX (Microwave Access Global Interoperability), LTE, LTE-A, 5G, Bluetooth, infrared, etc.; in addition, embodiments may also apply wired technologies, such as IP-based access technologies, such as wired networks or fixed lines.

[0189] An embodiment suitable for implementation as software code or as part thereof and for operation using a processor or processing function is independent of the software code and can be specified using any known or future-developed programming language, such as high-level programming languages ​​such as Objective-C, C, C++, C#, Java, Python, Javascript, other scripting languages, etc., or low-level programming languages ​​such as machine language or assembler.

[0190] The implementation of the embodiments is hardware-independent and can be implemented using any known or future-developed hardware technology or any combination thereof, such as microprocessors or CPUs (central processing units), MOS (metal-oxide-semiconductor), CMOS (complementary MOS), BiMOS (bipolar MOS), BiCMOS (bipolar CMOS), ECL (emitter-coupled logic), and / or TTL (transistor-transistor logic).

[0191] The embodiments may be implemented as individual devices, apparatuses, units, components or functions, or in a distributed manner. For example, one or more processors or processing functions may be used or shared in the process, or one or more processing segments or processing portions may be used and shared in the process, wherein one or more physical processors may be used to implement one or more processing portions dedicated to a particular process as described.

[0192] The device can be implemented by a semiconductor chip, a chipset, or a (hardware) module that includes such a chip or chipset.

[0193] The implementation can also be implemented as any combination of hardware and software, such as ASIC (Application-Specific IC (Integrated Circuit)) components, FPGA (Field Programmable Gate Array) or CPLD (Complex Programmable Logic Device) components or DSP (Digital Signal Processor) components.

[0194] The embodiments can also be implemented as computer program products, including a computer-usable medium in which computer-readable program code is embodied, the computer-usable program code being adapted to perform the processes described in the embodiments, wherein the computer-usable medium may be a non-transitory medium.

[0195] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0196] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0197] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0198] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0199] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for migrating a perception model, characterized in that The method includes: According to the application scenario information of the target perception model to be obtained, migrating the point cloud data of the original perception model to the target perception model to obtain the truth database of the target perception model, where the truth database includes the point cloud and label information of the perception target; Based on the truth database of the target perception model, perform model training to obtain the target perception model; The method further includes: According to the density of the point cloud beam corresponding to the original perception model and the density of the point cloud beam corresponding to the target perception model, adjust the distribution density of the point cloud data in the truth database to obtain a reconstructed data set; The performing model training based on the truth database of the target perception model includes: performing model training according to the reconstructed data set; Among them, according to the density of the point cloud beam corresponding to the original perception model and the density of the point cloud beam corresponding to the target perception model, adjusting the distribution density of the point cloud data in the truth database to obtain a reconstructed data set includes: Determine the difference value between the point cloud beam corresponding to the original perception model and the point cloud beam corresponding to the target perception model; When the difference value is greater than a preset threshold, respectively determine the point cloud data distribution density estimation map corresponding to the original perception model and the point cloud data distribution density estimation map corresponding to the target perception model, and compare the point cloud data distribution density estimation map corresponding to the original perception model with the point cloud data distribution density estimation map corresponding to the target perception model, and perform upsampling and / or downsampling on the point cloud data in the original perception model according to the comparison result to obtain a reconstructed data set.

2. The method according to claim 1, wherein According to the application scenario information of the target perception model to be obtained, migrating the point cloud data of the original perception model to the target perception model to obtain the truth database of the target perception model includes: According to the application scenario information of the target perception model to be obtained, for the point cloud data of the detection target of the original perception model, adopt methods such as size scaling, upsampling, and / or downsampling to obtain the truth database of the target perception model.

3. The method according to claim 1, wherein According to the density of the point cloud beam corresponding to the original perception model and the density of the point cloud beam corresponding to the target perception model, adjusting the distribution density of the point cloud data in the truth database to obtain a reconstructed data set further includes: When the difference value is less than the preset threshold: Determine the number of point cloud beams corresponding to the original perception model and the pitch angle of each point cloud beam; and determine the number of point cloud beams corresponding to the target perception model and the pitch angle of each point cloud beam; According to the number of point cloud beams corresponding to the target perception model and the pitch angle of each point cloud beam, and the number of point cloud beams corresponding to the original perception model and the pitch angle of each point cloud beam, interpolate the point cloud data on one or more point cloud beams corresponding to the original perception model in the truth database to obtain a reconstructed data set.

4. The method according to claim 1 or 3, characterized in that The difference value includes: the difference between the density of the point cloud beam corresponding to the original perception model and the density of the point cloud beam corresponding to the target perception model.

5. The method according to claim 1, wherein After obtaining the target perception model, the following steps are further included: Using the obtained target perception model for target detection to obtain a detection result, and correcting the detection result to obtain a final detection result.

6. A transfer device for a perception model, characterized in that, The device includes: A first unit, configured to migrate the point cloud data of the original perception model to the target perception model according to the application scenario information of the target perception model to be obtained, so as to obtain a truth database of the target perception model, where the truth database includes the point cloud and label information of the perception target; A second unit, configured to perform model training based on the truth database of the target perception model to obtain a target perception model; The first unit is further configured to: Adjust the distribution density of the point cloud data in the truth database according to the density of the point cloud wire harness corresponding to the original perception model and the density of the point cloud wire harness corresponding to the target perception model, so as to obtain a reconstructed data set; The performing model training based on the truth database of the target perception model includes: performing model training according to the reconstructed data set; Among them, adjusting the distribution density of the point cloud data in the truth database according to the density of the point cloud wire harness corresponding to the original perception model and the density of the point cloud wire harness corresponding to the target perception model to obtain a reconstructed data set includes: Determining the difference value between the point cloud wire harness corresponding to the original perception model and the point cloud wire harness corresponding to the target perception model; When the difference value is greater than a preset threshold, respectively determining the estimated distribution density map of the point cloud data corresponding to the original perception model and the estimated distribution density map of the point cloud data corresponding to the target perception model, and comparing the estimated distribution density map of the point cloud data corresponding to the original perception model with the estimated distribution density map of the point cloud data corresponding to the target perception model, and performing upsampling and / or downsampling on the point cloud data in the original perception model according to the comparison result to obtain a reconstructed data set.

7. An electronic device, characterized in that, It includes: A memory, configured to store program instructions; A processor, configured to call the program instructions stored in the memory and execute the method according to any one of claims 1 to 5 according to the obtained program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images