Training data set migration method and device and vehicle

Abstract

The invention relates to a training data set migration method and device and a vehicle, and the method comprises the steps: obtaining first vehicle data and second vehicle data, the first vehicle data represents a perception hardware parameter of a first vehicle model, the second vehicle data represents a perception hardware parameter of a second vehicle model, and the first vehicle model and the second vehicle model are different vehicle models; generating a perception transformation matrix based on difference information of the first vehicle data and the second vehicle data; obtaining original training data applied to a perception model of the first vehicle type, and performing transformation processing on the original training data based on the perception change matrix to obtain perception transformation data; and obtaining target training data based on the perceptual transformation data. According to the method and the device, the existing training data can be migrated to perception model training of different vehicle types, the training cost of the multi-vehicle-type perception model is reduced, the reuse rate of the training data is improved, data fusion is carried out based on vehicle type differences during training data migration, and the accuracy of the training model after training data migration can be ensured.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of vehicle perception, and in particular to a training dataset migration method and device and a vehicle. BACKGROUND

[0002] At present, the training of the perception model configured by the vehicle depends on a large-scale labeled accurate dataset. However, due to the differences in size, shape and sensor configuration, different vehicle models have different requirements for the perception model, and the training data required by the perception model is also different.

[0003] In the related art, the perception data of the corresponding vehicle model needs to be obtained and labeled separately for each vehicle model, which is a large amount of work and has low model training efficiency. SUMMARY

[0004] To solve at least one of the above technical problems, the present disclosure provides a training dataset migration method, device and vehicle.

[0005] According to some embodiments of the present disclosure, a training dataset migration method is provided, comprising: obtaining first vehicle data and second vehicle data, the first vehicle data representing perception hardware parameters of a first vehicle model, the second vehicle data representing perception hardware parameters of a second vehicle model, the first vehicle model and the second vehicle model being different vehicle models; generating a perception transformation matrix based on the difference information of the first vehicle data and the second vehicle data, the perception transformation matrix indicating a matrix for converting perception data adapted to the first vehicle model to perception data adapted to the second vehicle model; the perception data adapted to the first vehicle model is data obtained by the perception hardware in the first vehicle model, and the perception data adapted to the second vehicle model is data obtained by the perception hardware in the second vehicle model; obtaining original training data of a perception model applied to the first vehicle model, performing transformation processing on the original training data based on the perception transformation matrix to obtain perception transformation data; obtaining target training data based on the perception transformation data, the target training data indicating training data for training a perception model applied to the second vehicle model.

[0006] Based on the above scheme, the perception transformation matrix is generated based on the difference in perception hardware parameters of different vehicle models, and the existing training data is then transformed using the perception transformation matrix to obtain training data adapted to the perception model of the target vehicle model. This can realize the migration of the existing training data to the perception model training of different vehicle models, reduce the training cost of multi-vehicle perception models, improve the reuse rate of training data, and ensure the accuracy of the trained model after the migration of the training data based on the difference between the vehicle models.

[0007] In some possible implementations, obtaining target training data based on the perception transformation data includes: determining compensation parameters based on the perception transformation data, the compensation parameters indicating the difference information between the perception transformation data and the real-time perception data acquired by the perception hardware in the second vehicle model; and correcting the perception transformation data through the compensation parameters to obtain the target training data.

[0008] Based on the above scheme, compensation parameters are generated by the difference between the perception transformation data and the real-time perception data, and then the perception transformation data is corrected by the compensation parameters to ensure that the target training data can be accurately adapted to the perception model of the second vehicle.

[0009] In some possible implementations, generating a perception transformation matrix based on the difference information between the first vehicle data and the second vehicle data includes: establishing a first perception hardware model based on the first vehicle data, and establishing a second perception hardware model based on the second vehicle data; the first perception hardware model indicates the perception hardware information and vehicle model information of a first vehicle model, and the second perception hardware model indicates the perception hardware information and vehicle model information of a second vehicle model; determining a perception compensation feature and a perception hardware transformation matrix based on the difference between the first perception hardware model and the second perception hardware model, wherein the perception hardware transformation matrix indicates the matrix that converts the first perception hardware model into the second perception hardware model, and the perception compensation feature indicates the loss of perception content in the second perception hardware model compared to the first perception hardware model; and fusing the perception hardware transformation matrix with the perception compensation feature to obtain the perception transformation matrix.

[0010] Based on the above scheme, a perception hardware model is established using vehicle data. Then, based on the differences in the perception hardware model, the perception hardware transformation matrix and perception compensation features are determined respectively. This can achieve data space alignment and ensure that the training dataset can be adapted to the perception model of the target vehicle after migration.

[0011] In some possible implementations, determining the perception compensation features based on the differences between the first perception hardware model and the second perception hardware model includes: acquiring a simulated environment model; perceiving the simulated environment model through the first perception hardware model and the second perception hardware model respectively to obtain a first perception feature and a second perception feature; and determining the perception compensation features based on the difference information between the first perception feature and the second perception feature.

[0012] Based on the above scheme, the same simulated environment is perceived by the perception hardware model, and the perception compensation features are determined according to the differences in the obtained perception features. This can make up for the loss of perception features caused by changes in the perception hardware and improve the effectiveness of the target training data.

[0013] In some possible implementations, the method further includes: establishing a first associated feature based on the first perception model, the simulated environment model, and the first perception feature; establishing a second associated feature based on the second perception model, the simulated environment model, and the second perception feature; establishing a model information database, and storing the first associated feature and the second associated feature in the model information database.

[0014] Based on the above scheme, the perception features of the perception model, the simulation environment model, and the perception model perceiving the simulation environment model are associated and stored in the model information library. This can achieve model library standardization. The perception model of the new vehicle model can be transferred to a training dataset with low difference, which helps to improve the training efficiency of the perception model of multiple vehicle models.

[0015] In some possible implementations, before determining the compensation parameters based on the perception transformation data, the method further includes: generating mask data based on the second vehicle data, and using the mask data to mask the perception transformation data, wherein the mask data characterizes the occlusion of the perception hardware by the vehicle structure of the second vehicle model.

[0016] Based on the above scheme, using the second vehicle data to mask the perception transformation data can enhance generalization, making the perception transformation data closer to the real perception data, and also enhance the perception performance of the perception model.

[0017] In some possible implementations, after correcting the perception transformation data through the compensation parameters to obtain the target training data, the method further includes: acquiring a preset machine learning model; acquiring real-time perception data of the second vehicle model, and combining the target training data and the real-time perception data into mixed training data according to a preset ratio; training the preset machine learning model based on the mixed training data to obtain the perception model of the second vehicle model.

[0018] Based on the above scheme, the target training data and real-time perception data are mixed and processed, and the pre-set machine learning model is trained using the mixed training data. This can reduce the impact of the transformation error of the original training data, thereby improving the accuracy of the perception model.

[0019] In some possible implementations, the preset machine learning model includes a shared branch network and an adaptive branch network; training the preset machine learning model based on the mixed training data to obtain the perception model of the second vehicle model includes: in the shared branch network, extracting general environmental features based on the mixed training data, and determining a first branch loss coefficient based on the general environmental features, wherein the general environmental features indicate perception features not affected by vehicle model; in the adaptive branch network, extracting vehicle model adaptation features based on the mixed training data, and determining a second branch loss coefficient based on the vehicle model adaptation features, wherein the vehicle model adaptation features indicate perception features affected by vehicle model; determining the sum of the first branch loss coefficient and the second branch loss coefficient as the total loss coefficient; adjusting the network parameters of the shared branch network and the adaptive branch network based on the total loss coefficient to obtain the perception model of the second vehicle model.

[0020] Based on the above scheme, the perception model uses a dual-branch network to achieve feature decoupling and enhancement. The perception model can analyze vehicle-independent environmental features and vehicle-related geometric features.

[0021] According to some embodiments of the present invention, a training dataset transfer apparatus is also provided, comprising: The data acquisition module is used to acquire first vehicle data and second vehicle data. The first vehicle data represents the perception hardware parameters of a first vehicle model, and the second vehicle data represents the perception hardware parameters of a second vehicle model. The first vehicle model and the second vehicle model are different vehicle models. The vehicle data processing module is used to generate a perception transformation matrix based on the difference information between the first vehicle data and the second vehicle data. The perception transformation matrix indicates a matrix used to convert the perception data adapted to the first vehicle model into the perception data adapted to the second vehicle model. The perception data adapted to the first vehicle model is the data obtained by the perception hardware in the first vehicle model, and the perception data adapted to the second vehicle model is the data obtained by the perception hardware in the second vehicle model. The training data processing module is used to acquire the original training data of the perception model applied to the first vehicle model, transform the original training data based on the perception transformation matrix to obtain perception transformation data, and obtain target training data based on the perception transformation data, wherein the target training data indicates the training data used to train the perception model applied to the second vehicle model.

[0022] According to some embodiments of the present invention, a vehicle is also provided, including a controller, the controller being configured to execute the training dataset transfer method described in any of the above embodiments to obtain target training data, and to train a perception model mounted on the vehicle based on the target training data.

[0023] According to some embodiments of the present invention, an electronic device is also provided, the electronic device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement the training dataset transfer method as described above.

[0024] According to some embodiments of the present invention, a storage medium is also provided, which stores at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the training dataset transfer method as described above.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0026] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions and advantages in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a training dataset transfer method according to an embodiment of this disclosure is shown; Figure 2 A flowchart illustrating the establishment of a sensing hardware model according to an embodiment of this disclosure is shown; Figure 3 A flowchart illustrating the training process of a perception model for a second vehicle model according to an embodiment of the present disclosure is shown. Figure 4 A schematic diagram of a training dataset transfer apparatus according to an embodiment of the present disclosure is shown; Figure 5 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0029] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0031] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0032] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0033] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0034] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0035] Currently, training vehicle perception models relies on large-scale, accurately labeled datasets. However, different vehicle models have varying requirements for perception models due to differences in size, shape, and sensor configuration, resulting in different training data requirements. Traditional perception model training methods are often tailored to specific vehicle models, making it difficult to directly transfer these models to other models, which limits the widespread application of autonomous driving technology. Therefore, achieving rapid adaptation of perception models to different vehicle models has become a pressing issue in the field of autonomous driving.

[0036] To address the aforementioned technical problems, this disclosure provides a training dataset migration method. This method enables the reuse of training data for perception models across different vehicle models. It transforms the original training dataset of a perception model for one vehicle model, allowing the transformed training dataset to be directly used in the training of a perception model for another vehicle model. This solves the problem of difficulty in obtaining training data for perception models and reduces the training cost of multi-vehicle perception models.

[0037] Please refer to Figure 1 The training dataset transfer method of this disclosure includes: Step S101: Obtain first vehicle data and second vehicle data. The first vehicle data represents the perception hardware parameters of the first vehicle model, and the second vehicle data represents the perception hardware parameters of the second vehicle model. The first vehicle model and the second vehicle model are different vehicle models.

[0038] In this embodiment of the disclosure, the sensing hardware is a sensor device on the vehicle used to acquire sensing data, such as a lidar, a depth camera, and other types of sensor devices. Correspondingly, the sensing data is the point cloud image acquired by the lidar, the depth map captured by the depth camera, etc., and the sensing hardware parameters are the parameters of the aforementioned sensor devices. The sensing hardware parameters include the sensor's position parameters on the vehicle and the sensor's own parameters. In other words, the sensing hardware parameters include sensor parameters and vehicle model parameters. Among them, the sensor parameters include the sensor type, resolution, detection range, and data format, etc.

[0039] Step S102: Based on the difference information between the first vehicle data and the second vehicle data, a perception transformation matrix is ​​generated. The perception transformation matrix indicates the matrix used to convert the perception data adapted to the first vehicle model into the perception data adapted to the second vehicle model. The perception data adapted to the first vehicle model is the data acquired by the perception hardware in the first vehicle model, and the perception data adapted to the second vehicle model is the data acquired by the perception hardware in the second vehicle model.

[0040] In this embodiment of the disclosure, the perception transformation matrix is ​​generated based on the difference information of vehicle data. Specifically, the vehicle data includes the position information of the sensor on the vehicle and the information of the sensor itself. That is, the difference information of the vehicle data includes the difference in the position of the sensor on the vehicle and the difference in the sensor parameters (such as detection angle / horizontal angle, detection height / pitch angle, detection distance, etc.). Perception transformation vectors are generated based on the above two differences respectively, and then the perception transformation vectors are fused to obtain the perception transformation matrix.

[0041] In some specific implementations, vehicle sensors can be categorized according to their sensing direction / installation location. For example, sensors can be divided into forward sensing, lateral sensing, and rearward sensing. Alternatively, sensors can include roof sensors, front sensors, side sensors, and parking space sensors. Vehicle data can be split into multiple sets of data according to the above classification method. Each set of data can be converted into a feature vector, which stores information about the sensor parameters. Thus, vehicle data can be converted into a parameter matrix with multiple feature vectors. By analyzing the two parameter matrices, the sensing transformation matrix can be obtained.

[0042] It should be understood that the sensing hardware in the first vehicle model and the sensing hardware in the second vehicle model can be the same or different. For example, if the sensing hardware in the first vehicle model is a camera device, then the sensing data adapted to the first vehicle model is image data, while if the sensing hardware in the second vehicle model is a lidar, then the sensing data adapted to the second vehicle model is point cloud data. Correspondingly, the sensing transformation matrix can convert image data into point cloud data.

[0043] In some embodiments, the perception transformation matrix can be determined based on a calibration model. Specifically, generating the perception transformation matrix based on the difference information between the first vehicle data and the second vehicle data includes: Step S1021: Based on the first vehicle data, establish a first perception hardware model; based on the second vehicle data, establish a second perception hardware model.

[0044] Both the first and second sensing hardware models are calibration models, corresponding to specific vehicle models. These calibration models include the vehicle's structural dimensions and sensor configuration information. Specifically, the first sensing hardware model indicates the sensing hardware information and vehicle model information for the first vehicle model, while the second sensing hardware model indicates the sensing hardware information and vehicle model information for the second vehicle model. It should be understood that the data structures of the first and second sensing hardware models are identical.

[0045] In some cases, please refer to Figure 2 Vehicle data can be identified through a pre-set calibration file, thereby extracting parameter information of the perception hardware model. Finally, the parameter information is input into the model template to obtain the corresponding perception hardware model.

[0046] Step S1022: Based on the difference between the first and second sensing hardware models, determine the sensing compensation features and the sensing hardware transformation matrix. The sensing hardware transformation matrix indicates the matrix that converts the first sensing hardware model into the second sensing hardware model, and the sensing compensation features indicate the loss of sensing content in the second sensing hardware model compared to the first sensing hardware model.

[0047] In this embodiment, the sensing hardware transformation matrix includes positional difference information and sensor parameter difference information. The sensing hardware transformation matrix can be directly obtained based on the parameter differences between the first and second sensing hardware models. However, since the first and second sensing hardware models have geometric offsets such as sensing distance deviation, sensing angle deviation, and sensing height deviation, directly using the sensing hardware transformation matrix to convert the sensing data acquired by the first vehicle model's sensing hardware can easily lead to data loss. For example, if the first vehicle model is a sedan and the second vehicle model is a truck, their sensing heights and sensing field ranges differ. The sensing data obtained by converting the sedan's sensing data using the sensing hardware transformation matrix will have a loss of sensing content in the upper range. Therefore, corresponding compensation is determined to address this loss of sensing content.

[0048] In some embodiments, missing information about the perceived content can be determined by setting up a simulated environment. Specifically, based on the differences between the first and second sensing hardware models, sensing compensation features are determined, including: Step S10221: Obtain the simulation environment model.

[0049] In this embodiment, the simulated environment model can be set according to the actual vehicle driving scenario, such as highway scenario, underground parking garage scenario, urban road scenario, and rural road scenario. Different obstacle features can be set for different scenarios. In some possible cases, dynamic obstacles, such as pedestrians or vehicles, can also be set in the simulated environment model corresponding to some scenarios, thereby improving the model to be closer to the real scene.

[0050] Step S10222: Perceive the simulated environment model through the first sensing hardware model and the second sensing hardware model respectively to obtain the first sensing features and the second sensing features.

[0051] This embodiment does not limit the specific implementation method of obtaining perceptual features from the perceptual hardware model and the perceptual simulation environment model. For example, it can be achieved through simulated projection, that is, by determining each projection range through the perceptual hardware model, and then obtaining the perceptual features based on the interference between the projection range and the objects in the simulated environment model.

[0052] In addition, before sensing the simulation environment model through the first and second sensing hardware models respectively, the first and second sensing hardware models need to be aligned and calibrated. The purpose of alignment and calibration is to match the coordinates of the first and second sensing hardware models, and to ensure that the positions and orientations of the first and second sensing hardware models in the simulation environment model are consistent, thereby avoiding geometric deviations in the final sensing compensation features.

[0053] Step S10223: Determine the perception compensation features based on the difference information between the first perception feature and the second perception feature.

[0054] In this embodiment, the perception compensation feature represents the lack of perception content of the simulated environment model compared with the first perception feature. The role of the perception compensation feature is to fuse with the first perception feature to obtain the second perception feature.

[0055] In this embodiment, the perception compensation feature can be either a fixed compensation feature or a dynamic compensation feature. Specifically, when the perception compensation feature is a fixed compensation feature, the difference between the second perception feature and the first perception feature is directly calculated to obtain the missing feature. Then, the missing feature is mapped to the first perception feature to obtain the perception compensation feature. When the perception compensation feature is a dynamic compensation feature, it needs to be determined comprehensively based on the simulation environment models corresponding to multiple scenarios. It should be understood that as the simulation environment model changes and the pose of the perception hardware model changes within the simulation environment model, the difference between the first and second perception features changes. That is, the perception compensation features corresponding to different simulation environment models are different. In this case, corresponding fixed compensation features can be generated based on each model, and then multiple fixed compensation features can be fused again to obtain the dynamic compensation feature. Dynamic compensation features can adapt to data from multiple scenarios, but fixed compensation features have high accuracy for specific scenario data. In practical applications, the appropriate perception compensation feature can be flexibly selected.

[0056] Based on the above steps S10221 to S10223, the same simulated environment is perceived through the perception hardware model, and the perception compensation features are determined according to the differences in the obtained perception features. This can make up for the loss of perception features caused by changes in the perception hardware and improve the effectiveness of the target training data.

[0057] Step S1023: Fuse the sensing hardware transformation matrix with the sensing compensation features to obtain the sensing transformation matrix.

[0058] In this embodiment, the function of the perception hardware transformation matrix is ​​to transform the perception data of the first vehicle from the perception perspective of the second vehicle. The function of the perception compensation feature is to generate content missing from the perception data of the first vehicle based on the perception data of the second vehicle from the perception perspective. The two are then fused together, meaning the missing content is used to fill in the missing content in the transformed perception data. In practical applications, the perception transformation matrix may include two sub-matrices. The first sub-matrix implements the function of the perception hardware transformation matrix to generate a first transformed image, and the second sub-matrix implements the function of the perception compensation feature to generate a second transformed image. Finally, the first and second transformed images are stitched together to obtain the image corresponding to the perception transformation data.

[0059] Based on the above steps S1021 to S1023, a perception hardware model is established using vehicle data. Then, based on the differences in the perception hardware model, the perception hardware transformation matrix and perception compensation features are determined respectively, which can achieve data space alignment and ensure that the training dataset can be adapted to the perception model of the target vehicle after migration.

[0060] In some embodiments, a method for establishing a model library is also provided, the method specifically including: establishing a first associated feature based on a first perception model, a simulated environment model, and a first perception feature; establishing a second associated feature based on a second perception model, a simulated environment model, and a second perception feature; establishing a model information library, and storing the first associated feature and the second associated feature in the model information library.

[0061] In this embodiment, the associated features can parameterize and store the sensor layout, body size, and field of view of the vehicle model. Based on the above scheme, the perception features of the perception model, the simulation environment model, and the perception model perceiving the simulation environment model are associated and stored in the model information library, which can achieve model library standardization. The perception model of the new vehicle model can be transferred to a training dataset with low difference. Existing model data can be directly called, which can significantly shorten the data conversion time and help improve the training efficiency of multi-vehicle perception models.

[0062] Step S103: Obtain the original training data of the perception model applied to the first vehicle model, and transform the original training data based on the perception transformation matrix to obtain the perception transformation data. The original training data of the perception model for the first vehicle model includes the perception data acquired by the perception hardware in the first vehicle model. The perception data includes labeled and unlabeled data. Correspondingly, the perception transformation data includes labeled and unlabeled data.

[0063] Step S104: Obtain target training data based on the perceptual transformation data. The target training data indicates the training data used to train the perceptual model applied to the second vehicle model.

[0064] In some cases, the perceptual transformation data can be directly used as the target training data. In other cases, the perceptual transformation data needs to be processed to obtain the training data. For example, the perceptual transformation data needs to be transformed so that the resulting target training data is in a data format suitable for the perceptual model of the second vehicle model.

[0065] In some embodiments, the perceptual transformation data and the real-time perceptual data acquired by the perceptual hardware in the second vehicle have geometric differences, such as angle deviation and distance deviation, so the perceptual transformation data needs to be corrected and compensated. Specifically, the above-mentioned target training data obtained based on the perceptual transformation data includes: Step S1041: Determine compensation parameters based on the perception transformation data. The compensation parameters indicate the difference between the perception transformation data and the real-time perception data acquired by the perception hardware in the second vehicle model.

[0066] In some embodiments, the compensation parameters can be obtained based on a pre-trained feature training model. The perceptual transformation data and a small amount of real-time perceptual data acquired by the perceptual hardware in the second vehicle model are combined and input into the feature training model. The feature training model can separate the general environmental features (e.g., pedestrians, traffic signs) that are not affected by the vehicle model and the vehicle model adaptation features (e.g., vehicle occlusion) that are affected by the vehicle model in the perceptual transformation data, and extract and compare the parts affected by the vehicle model to obtain the compensation parameters.

[0067] Step S1042: Correct the perceptual transformation data by compensating the parameters to obtain the target training data.

[0068] Based on the above steps S1041 to S1042, compensation parameters are generated by the difference between the perception transformation data and the real-time perception data. The compensation parameters are then used to correct the perception transformation data, which can ensure that the target training data can be accurately adapted to the perception model of the second vehicle model.

[0069] In some embodiments, before determining compensation parameters based on the perceptual transformation data, the perceptual transformation data also needs to undergo application domain randomization. Application domain randomization is used to enhance generalization and can simulate the occlusion effects of different vehicle models. In some specific embodiments, the application domain randomization method includes: generating mask data based on the second vehicle data, and using the mask data to mask the perceptual transformation data. The mask data represents the occlusion of the perceptual hardware by the vehicle structure of the second vehicle model.

[0070] Based on the above scheme, using the second vehicle data to mask the perception transformation data can enhance generalization, making the perception transformation data closer to the real perception data, and also enhance the perception performance of the perception model.

[0071] The above embodiments have described in detail a training dataset transfer method of this disclosure. Based on the above steps S101 to S104, a perception transformation matrix is ​​generated by the differences in perception hardware parameters of different vehicle models. The existing training data is then transformed using the perception transformation matrix to obtain training data adapted to the perception model of the target vehicle model. This method can transfer existing training data to the training of perception models of different vehicle models, reduce the training cost of multi-vehicle perception models, improve the reusability of training data, and ensure the accuracy of the training model after the training data transfer by performing data fusion based on vehicle model differences.

[0072] Furthermore, this disclosure also provides a method for training a perception model. This method is used to train the perception model of a second vehicle model after the training dataset has been successfully transferred. Specifically, after obtaining the target training data by correcting the perception transformation data through compensation parameters, the method further includes: obtaining a preset machine learning model; obtaining real-time perception data of the second vehicle model, and combining the target training data and the real-time perception data into mixed training data according to a preset ratio; and training the preset machine learning model based on the mixed training data to obtain the perception model of the second vehicle model.

[0073] In this embodiment, the ratio of target training data to real-time perception data is greater than 1:1. In some preferred embodiments, the ratio of target training data to real-time perception data is 9:1. The real-time perception data can be obtained from driving records of the second vehicle over a preset time period (e.g., ten minutes). Based on the above scheme, mixing the target training data and real-time perception data and using the mixed training data to train a preset machine learning model can reduce the impact of transformation errors in the original training data, thereby improving the accuracy of the perception model.

[0074] In some embodiments, the perception model of the second vehicle model employs a dual-branch neural network design. For details, please refer to... Figure 3 The pre-defined machine learning model includes a shared branch network and an adaptive branch network. The shared branch network is used to extract general environmental features that are independent of the vehicle model, while the adaptive branch network is used to learn vehicle-specific geometric compensation parameters. The model's loss function integrates classification accuracy and geometric consistency constraints.

[0075] Based on the aforementioned pre-defined machine learning model, the above-mentioned training of the pre-defined machine learning model using mixed training data to obtain a perception model for the second vehicle model includes: in the shared branch network, extracting general environmental features based on the mixed training data, and determining the first branch loss coefficient based on the general environmental features, which indicate perception features unaffected by vehicle model interference; in the adaptation branch network, extracting vehicle model adaptation features based on the mixed training data, and determining the second branch loss coefficient based on the vehicle model adaptation features, which indicate perception features affected by vehicle model interference; determining the sum of the first branch loss coefficient and the second branch loss coefficient as the total loss coefficient; and adjusting the network parameters of the shared branch network and the adaptation branch network based on the total loss coefficient to obtain the perception model for the second vehicle model. Based on the above scheme, the perception model uses a dual-branch network to achieve feature decoupling and enhancement, and the perception model can analyze vehicle model-independent environmental features and vehicle model-related geometric features.

[0076] Please refer to Figure 4 This invention also provides a training dataset transfer apparatus, comprising: The data acquisition module 100 is used to acquire first vehicle data and second vehicle data. The first vehicle data represents the perception hardware parameters of the first vehicle model, and the second vehicle data represents the perception hardware parameters of the second vehicle model. The first vehicle model and the second vehicle model are different vehicle models. The vehicle data processing module 200 is used to generate a perception transformation matrix based on the difference information between the first vehicle data and the second vehicle data. The perception transformation matrix indicates a matrix used to convert the perception data adapted to the first vehicle model into the perception data adapted to the second vehicle model. The perception data adapted to the first vehicle model is the data obtained by the perception hardware in the first vehicle model, and the perception data adapted to the second vehicle model is the data obtained by the perception hardware in the second vehicle model. The training data processing module 300 is used to acquire the original training data of the perception model applied to the first vehicle model, transform the original training data based on the perception transformation matrix to obtain perception transformation data, and obtain target training data based on the perception transformation data. The target training data indicates the training data used to train the perception model applied to the second vehicle model.

[0077] In some embodiments, the training data processing module 300 further includes a compensation parameter determination module, which is used to determine compensation parameters based on the perception transformation data. The compensation parameters indicate the difference information between the perception transformation data and the real-time perception data acquired by the perception hardware in the second vehicle model. The perception transformation data is corrected by the compensation parameters to obtain the target training data.

[0078] In some embodiments, the vehicle data processing module 200 further includes a model processing module, configured to establish a first perception hardware model based on the first vehicle data and a second perception hardware model based on the second vehicle data; the first perception hardware model indicates the perception hardware information and vehicle model information of the first vehicle model, and the second perception hardware model indicates the perception hardware information and vehicle model information of the second vehicle model; based on the difference between the first perception hardware model and the second perception hardware model, a perception compensation feature and a perception hardware transformation matrix are determined, the perception hardware transformation matrix indicates the matrix that converts the first perception hardware model into the second perception hardware model, and the perception compensation feature indicates the loss of perception content in the second perception hardware model compared to the first perception hardware model; the perception hardware transformation matrix and the perception compensation feature are fused to obtain a perception transformation matrix.

[0079] In some embodiments, the model processing module further includes an environment simulation module for acquiring a simulated environment model; perceiving the simulated environment model through a first sensing hardware model and a second sensing hardware model respectively to obtain first sensing features and second sensing features; and determining sensing compensation features based on the difference information between the first sensing features and the second sensing features.

[0080] In some embodiments, the apparatus further includes a model library module for establishing a first associated feature based on a first perception model, a simulated environment model, and a first perception feature; establishing a second associated feature based on a second perception model, a simulated environment model, and a second perception feature; establishing a model information library; and storing the first associated feature and the second associated feature in the model information library.

[0081] In some embodiments, the vehicle data processing module 200 further includes a data correction module, which generates mask data based on the second vehicle data and performs masking processing on the perception transformation data through the mask data. The mask data represents the occlusion of the perception hardware by the vehicle structure of the second vehicle model.

[0082] In some embodiments, the apparatus further includes a model training module for acquiring a preset machine learning model; acquiring real-time perception data of the second vehicle model and combining the target training data and the real-time perception data into mixed training data according to a preset ratio; and training the preset machine learning model based on the mixed training data to obtain a perception model of the second vehicle model.

[0083] In some embodiments, the preset machine learning model includes a shared branch network and an adaptive branch network. Correspondingly, the model training module is further configured to: extract general environmental features based on mixed training data in the shared branch network, and determine a first branch loss coefficient based on the general environmental features, wherein the general environmental features indicate perception features not affected by vehicle model; extract vehicle model adaptation features based on mixed training data in the adaptive branch network, and determine a second branch loss coefficient based on the vehicle model adaptation features, wherein the vehicle model adaptation features indicate perception features affected by vehicle model; determine the sum of the first branch loss coefficient and the second branch loss coefficient as the total loss coefficient; and adjust the network parameters of the shared branch network and the adaptive branch network based on the total loss coefficient to obtain a perception model for the second vehicle model.

[0084] This disclosure also provides a vehicle, including a controller, which is configured to execute the training dataset transfer method described in any of the above embodiments to obtain target training data, and to train a perception model mounted on the vehicle based on the target training data.

[0085] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program. The processor loads and executes the at least one instruction or at least one program to implement the training dataset transfer method described above.

[0086] Memory is used to store software programs and modules. The processor executes these stored software programs and modules to perform various functional applications and data processing. Memory can primarily consist of a program storage area and a data storage area. The program storage area stores the operating system, application programs required for functionality, etc.; the data storage area stores data created based on device usage, etc. Furthermore, memory can include high-speed random access memory (RAM) and non-volatile memory, such as at least one hard disk drive, flash memory, or other volatile solid-state storage devices. Correspondingly, memory can also include a memory controller to provide the processor with access to the memory.

[0087] The methods and embodiments provided in this application can be executed in electronic devices such as mobile terminals, computer terminals, servers, or similar computing devices. Figure 5 This is the electronic device provided in the embodiments of this application. For example... Figure 5 As shown, the electronic device 900 can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) 910 (CPUs 910 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 930 for storing data, and one or more storage media 920 (e.g., one or more mass storage devices) for storing application programs 923 or data 922. The memory 930 and storage media 920 may be temporary or persistent storage. The program stored in the storage media 920 may include one or more modules, each module may include a series of instruction operations on the electronic device. Furthermore, the CPU 910 may be configured to communicate with the storage media 920 and execute the series of instruction operations in the storage media 920 on the electronic device 900. Electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0088] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 900. In one example, the input / output interface 940 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 940 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0089] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 900 may also include... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0090] Embodiments of this application also provide a storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the training dataset transfer method as described above.

[0091] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A training dataset transfer method, characterized in that, include: Acquire first vehicle data and second vehicle data, wherein the first vehicle data represents the perception hardware parameters of a first vehicle model and the second vehicle data represents the perception hardware parameters of a second vehicle model, and the first vehicle model and the second vehicle model are different vehicle models; Based on the difference information between the first vehicle data and the second vehicle data, a perception transformation matrix is ​​generated. The perception transformation matrix indicates a matrix used to convert the perception data adapted to the first vehicle model into the perception data adapted to the second vehicle model. The perception data adapted to the first vehicle model is the data obtained by the perception hardware in the first vehicle model, and the perception data adapted to the second vehicle model is the data obtained by the perception hardware in the second vehicle model. Obtain the original training data of the perception model applied to the first vehicle model, and transform the original training data based on the perception transformation matrix to obtain perception transformation data; Target training data is obtained based on the perceptual transformation data, and the target training data indicates the training data used to train the perceptual model applied to the second vehicle model.

2. The method according to claim 1, characterized in that, The process of obtaining target training data based on the perceptual transformation data includes: Compensation parameters are determined based on the perception transformation data, and the compensation parameters indicate the difference information between the perception transformation data and the real-time perception data acquired by the perception hardware in the second vehicle model; The target training data is obtained by correcting the perceptual transformation data using the compensation parameters.

3. The method according to claim 1, characterized in that, The step of generating a perception transformation matrix based on the difference information between the first vehicle data and the second vehicle data includes: Based on the first vehicle data, a first perception hardware model is established, and based on the second vehicle data, a second perception hardware model is established; the first perception hardware model indicates the perception hardware information and vehicle model information of the first vehicle model, and the second perception hardware model indicates the perception hardware information and vehicle model information of the second vehicle model. Based on the differences between the first and second perception hardware models, perception compensation features and perception hardware transformation matrices are determined. The perception hardware transformation matrix indicates the matrix that converts the first perception hardware model into the second perception hardware model. The perception compensation features indicate the loss of perception content in the second perception hardware model compared to the first perception hardware model. The sensing hardware transformation matrix is ​​fused with the sensing compensation feature to obtain the sensing transformation matrix.

4. The method according to claim 3, characterized in that, The step of determining the perception compensation features based on the differences between the first perception hardware model and the second perception hardware model includes: Obtain the simulation environment model; The simulated environment model is perceived through the first sensing hardware model and the second sensing hardware model respectively, and the first sensing feature and the second sensing feature are obtained. The perception compensation feature is determined based on the difference information between the first perception feature and the second perception feature.

5. The method according to claim 4, characterized in that, The method further includes: Based on the first perception model, the simulated environment model, and the first perception feature, a first association feature is established; Based on the second perception model, the simulated environment model, and the second perception features, a second association feature is established; Establish a model information database and store the first associated feature and the second associated feature in the model information database.

6. The method according to claim 2, characterized in that, Before determining the compensation parameters based on the perceptual transformation data, the method further includes: Based on the second vehicle data, mask data is generated, and the perception transformation data is masked using the mask data. The mask data represents the occlusion of the perception hardware by the vehicle structure of the second vehicle model.

7. The method according to claim 4, characterized in that, After obtaining the target training data by correcting the perceptual transformation data using the compensation parameters, the method further includes: Obtain a preset machine learning model; Acquire real-time perception data of the second vehicle model, and combine the target training data and the real-time perception data into mixed training data according to a preset ratio; The preset machine learning model is trained based on the mixed training data to obtain the perception model of the second vehicle model.

8. The method according to claim 7, characterized in that, The preset machine learning model includes a shared branch network and an adaptive branch network; the step of training the preset machine learning model based on the hybrid training data to obtain the perception model of the second vehicle model includes: In the shared branch network, general environmental features are extracted based on the mixed training data, and the first branch loss coefficient is determined based on the general environmental features. The general environmental features indicate perception features that are not affected by vehicle type. In the adaptation branch network, vehicle model adaptation features are extracted based on the mixed training data, and the second branch loss coefficient is determined based on the vehicle model adaptation features. The vehicle model adaptation features indicate the perceptual features affected by vehicle model interference. The sum of the first branch loss coefficient and the second branch loss coefficient is determined as the total loss coefficient; Based on the total loss coefficient, the network parameters of the shared branch network and the adapted branch network are adjusted to obtain the perception model of the second vehicle model.

9. A training dataset transfer device, characterized in that, include: The data acquisition module is used to acquire first vehicle data and second vehicle data. The first vehicle data represents the perception hardware parameters of a first vehicle model, and the second vehicle data represents the perception hardware parameters of a second vehicle model. The first vehicle model and the second vehicle model are different vehicle models. The vehicle data processing module is used to generate a perception transformation matrix based on the difference information between the first vehicle data and the second vehicle data. The perception transformation matrix indicates a matrix used to convert the perception data adapted to the first vehicle model into the perception data adapted to the second vehicle model. The perception data adapted to the first vehicle model is the data obtained by the perception hardware in the first vehicle model, and the perception data adapted to the second vehicle model is the data obtained by the perception hardware in the second vehicle model. The training data processing module is used to acquire the original training data of the perception model applied to the first vehicle model, transform the original training data based on the perception transformation matrix to obtain perception transformation data, and obtain target training data based on the perception transformation data, wherein the target training data indicates the training data used to train the perception model applied to the second vehicle model.

10. A vehicle, characterized in that, The system includes a controller configured to execute the training dataset transfer method according to any one of claims 1-8 to obtain target training data, and to train a perception model mounted on a vehicle based on the target training data.

Images