Vehicle model building method and system based on depth image fusion

Abstract

The application discloses a vehicle model building method and system based on depth image fusion, and belongs to the field of image processing, wherein the method comprises the following steps: obtaining a first vehicle; performing regional division based on the first vehicle to obtain N vehicle sub-regions; collecting basic information of the N vehicle sub-regions respectively to obtain N vehicle sub-region data sets; performing preprocessing on the N vehicle sub-region data sets to obtain N vehicle sub-data partitions; modeling the N vehicle sub-data partitions respectively to obtain N vehicle sub-models; collecting multi-angle images of the first vehicle to obtain vehicle multi-angle images, performing deep learning on the vehicle multi-angle images to generate multi-dimensional sub-region fusion features; and fusing the N vehicle sub-models to obtain a first vehicle twin model. The application solves the technical problem that a vehicle model cannot be accurately and efficiently built in the prior art, and achieves the technical effect of improving the accuracy and efficiency of vehicle model building.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a method and system for building vehicle models based on deep image fusion. Background Technology

[0002] With the development of computer and automation technologies, the manufacturing industry is undergoing a digital transformation. Among these transformations, the application of digital twin technology has enabled a deep integration of the virtual and physical worlds, providing new ideas and methods for product development, manufacturing, and operation and maintenance management. In the automotive industry, building accurate vehicle digital twin models is crucial for vehicle research and development, production, and operation and maintenance. However, existing vehicle digital twin modeling methods cannot accurately represent the overall attributes and behaviors of vehicles, and cannot efficiently build precise vehicle models. Summary of the Invention

[0003] This application provides a method and system for building vehicle models based on deep image fusion, aiming to solve the technical problem that existing technologies cannot accurately and efficiently build vehicle models.

[0004] In view of the above problems, this application provides a method and system for building vehicle models based on deep image fusion.

[0005] In a first aspect, this application provides a method for building a vehicle model based on deep image fusion. The method includes: obtaining a first vehicle; dividing the first vehicle into regions to obtain N vehicle sub-regions, where N is a positive integer greater than 1; collecting basic information of the N vehicle sub-regions to obtain N vehicle sub-region datasets; traversing the N vehicle sub-region datasets for preprocessing to obtain N vehicle sub-data partitions; connecting a digital twin module to model the N vehicle sub-data partitions to obtain N vehicle sub-models; collecting multi-angle images of the first vehicle to obtain multi-angle vehicle images, and performing deep learning on the multi-angle vehicle images based on the N vehicle sub-regions to generate multi-dimensional sub-region fusion features; and fusing the N vehicle sub-models based on the multi-dimensional sub-region fusion features to obtain a first vehicle twin model.

[0006] Secondly, this application provides a vehicle model building system based on deep image fusion. The system includes: a first vehicle acquisition module for acquiring a first vehicle; a vehicle region division module for dividing the first vehicle into N vehicle sub-regions, where N is a positive integer greater than 1; a region information acquisition module for acquiring basic information from each of the N vehicle sub-regions to obtain N vehicle sub-region datasets; a data preprocessing module for traversing the N vehicle sub-region datasets and preprocessing them to obtain N vehicle sub-data partitions; a data partition modeling module for connecting to a digital twin module to model each of the N vehicle sub-data partitions to obtain N vehicle sub-models; an image deep learning module for acquiring multi-angle images of the first vehicle to obtain multi-angle vehicle images, and performing deep learning on the multi-angle vehicle images based on the N vehicle sub-regions to generate multi-dimensional sub-region fusion features; and a sub-model fusion module for fusing the N vehicle sub-models based on the multi-dimensional sub-region fusion features to obtain a first vehicle twin model.

[0007] Thirdly, this application also provides an electronic device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the vehicle model building method based on deep image fusion provided in this application.

[0008] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle model building method based on deep image fusion provided in this application.

[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0010] By employing a technical solution that divides the vehicle into multiple sub-regions based on the needs of modeling, then collects information and multi-angle images from each sub-region, uses deep learning to extract multi-dimensional features from each sub-region, and finally fuses the features and models of each sub-region to obtain a digital twin model of the entire vehicle, this solution solves the technical problem of the inability to accurately and efficiently build vehicle models in existing technologies, and achieves the technical effect of improving the accuracy and efficiency of vehicle model building.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0012] Figure 1 This application provides a possible flowchart illustrating a vehicle model building method based on depth image fusion for embodiments of the present application;

[0013] Figure 2 This application provides a schematic diagram illustrating the possible process of obtaining N vehicle sub-data partitions in a vehicle model building method based on deep image fusion.

[0014] Figure 3 This application provides a possible structural diagram of a vehicle model building system based on deep image fusion for embodiments of the present application;

[0015] Figure 4 This is a schematic diagram of the structure of an exemplary electronic device of this application.

[0016] Explanation of reference numerals in the attached drawings: First vehicle acquisition module 11, vehicle area division module 12, area information acquisition module 13, data preprocessing module 14, data partitioning modeling module 15, image deep learning module 16, sub-model fusion module 17, processor 31, memory 32, input device 33, output device 34. Detailed Implementation

[0017] The overall concept of the technical solution provided in this application is as follows:

[0018] This application provides a method and system for building a vehicle model based on deep image fusion. First, the vehicle to be modeled is obtained, and then multiple sub-regions are defined based on the vehicle's structure and kinematic characteristics. Next, data and multi-angle images of each sub-region are collected, and the collected data and images of each sub-region are preprocessed. A digital twin module is connected, and deep learning methods are used to model the data and images of each sub-region, extracting multi-dimensional features of the sub-regions. Finally, based on the multi-dimensional features of the sub-regions, the models of each sub-region are fused using a feature fusion method to obtain a digital twin model of the entire vehicle.

[0019] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0020] Example 1

[0021] like Figure 1 As shown in the embodiment of this application, a method for building a vehicle model based on deep image fusion is provided, the method including:

[0022] Step S100: Obtain the first vehicle;

[0023] Specifically, the target vehicle to be modeled, i.e., the first vehicle, is obtained. The first vehicle can be of any type, such as a car or a truck. Obtaining the first vehicle provides the information basis for regional division of the first vehicle.

[0024] Step S200: Divide the region based on the first vehicle to obtain N vehicle sub-regions, where N is a positive integer greater than 1;

[0025] Specifically, after obtaining the first vehicle, in order to accurately model and analyze it, the first vehicle is meticulously divided into N sub-regions. These N sub-regions are defined based on the key structures and components of the first vehicle, such as the body area, engine compartment area, and driver's cab area. Each sub-region corresponds to a specific local area or key structure of the first vehicle. The number of sub-regions, N, is a positive integer greater than 1.

[0026] By dividing the first vehicle into regions, the entire vehicle system is gradually decomposed into subsystems and component levels, which is beneficial for subsequent modeling and analysis. Each sub-region can be regarded as a local region, which reduces the difficulty of modeling and facilitates the realization of digital twins.

[0027] Step S300: Collect the basic information of the N vehicle sub-regions respectively to obtain N vehicle sub-region datasets;

[0028] Specifically, after dividing the first vehicle into N sub-regions, data is collected for each sub-region to obtain basic information about the sub-region. Each region corresponds to a sub-region dataset, resulting in a total of N vehicle sub-region datasets.

[0029] Basic information for each vehicle sub-region was collected through methods such as photography, 3D scanning, material quality inspection, and assembly relationship acquisition. Photography involved taking multi-angle photos of each sub-region to obtain 2D images. 3D scanning involved scanning each sub-region with 3D scanning equipment to obtain 3D point cloud data, enabling geometric dimension measurement for model building. Material inspection involved inspecting each sub-region with material inspection equipment to obtain surface material properties such as material type, thickness, and surface treatment method, used for realistic rendering of the sub-region model. Assembly relationship acquisition involved obtaining information on the assembly method, assembly sequence, and positional relationships between each sub-region and adjacent sub-regions or the vehicle body, used for assembling the sub-region model. After collecting basic information for all N sub-regions using the above methods, N sub-region datasets were obtained. Each sub-region dataset includes basic information for the corresponding sub-region, providing data support for subsequent sub-region modeling and analysis.

[0030] Step S400: Traverse the N vehicle sub-region datasets and perform preprocessing to obtain N vehicle sub-data partitions;

[0031] Specifically, after obtaining N sub-region datasets, in order to achieve accurate modeling of the sub-regions, each sub-region dataset is preprocessed to obtain N vehicle sub-data partitions.

[0032] Preprocessing of N vehicle sub-region datasets is achieved through methods such as information supplementation, data cleaning, format standardization, data fusion, and correlation completion. Information supplementation involves detecting missing information in the sub-region datasets, such as missing images from a certain perspective or missing parameters for a certain material, and collecting the necessary supplementary information. Data cleaning involves detecting abnormal, erroneous, or redundant data in the sub-region datasets and deleting, correcting, or matching them. Format standardization involves standardizing the formats of various information in the sub-region datasets to ensure a uniform format, facilitating subsequent computer reading and processing for modeling. Data fusion involves combining various types of information in the sub-region datasets, such as images, 3D data, and material data, to generate a fused sub-region dataset. Correlation completion involves completing the correlation information between the sub-region datasets and other sub-region datasets, such as spatial location relationships, motion relationships, and operational relationships, providing data support for the assembly of the sub-region models.

[0033] By preprocessing each of the N sub-region datasets, N sub-data regions are obtained. The preprocessed sub-data regions have more complete and accurate information, making them ideal inputs for sub-region modeling and helping to improve the accuracy and precision of the sub-region model.

[0034] Step S500: Connect the digital twin module and model the N vehicle sub-data partitions respectively to obtain N vehicle sub-models;

[0035] Specifically, after obtaining the preprocessed N sub-data areas, digital twin technology is used to model each sub-data area, generating a digital twin model of the corresponding sub-region, and obtaining N vehicle sub-models.

[0036] First, a simulation modeling platform was selected as the digital twin module, and a modeling environment was built to realize sub-region modeling. Then, information from each sub-data region was imported into the modeling environment, i.e., images and 3D data were imported into the simulation software. Subsequently, in the modeling environment, the information in each sub-data region was modeled through human-computer interaction; that is, the modelers directly performed 3D surface reconstruction and assembly within the simulation software. Next, corresponding configuration parameters were set for the modeling of each sub-region, such as mesh density, number of iterations, and fitting threshold in the simulation software. Afterward, modeling was run in the modeling environment to achieve automatic or interactive modeling, ultimately obtaining a digital twin model that expresses the characteristics of the corresponding sub-region. Simultaneously, each generated sub-model was examined to determine if it met the accuracy and realism requirements. If not, the modeling configuration or interactive process needed to be optimized and adjusted until a satisfactory sub-model was obtained. Finally, each sub-model generated in the modeling environment was exported to obtain N vehicle sub-models.

[0037] By repeatedly modeling all N sub-data regions, N vehicle sub-models that correspond to and express the characteristics of each sub-region can be obtained, i.e., digital twin models of N sub-regions. These sub-models contain multiple numerical information and characteristics of the corresponding sub-regions, providing support for subsequent sub-model fusion and whole-vehicle modeling.

[0038] Step S600: Acquire multi-angle images of the first vehicle to obtain multi-angle images of the vehicle, and perform deep learning on the multi-angle images of the vehicle based on the N vehicle sub-regions to generate multi-dimensional sub-region fusion features;

[0039] Specifically, in order to achieve the fusion of sub-models, multi-angle images of the first vehicle are collected, and deep learning technology is used to analyze the images and extract sub-region fusion features.

[0040] First, the first vehicle is captured in 360° images, obtaining high-resolution images from multiple angles including left, right, front, rear, and overhead views, thus acquiring multi-angle images of the vehicle. Then, deep convolutional neural networks, such as ResNet, DenseNet, and U-Net, are selected for sub-region recognition and feature extraction in the images. Next, each sub-region is manually labeled in the multi-angle images, generating a sub-region dataset, which serves as the training dataset for the deep learning network. Then, the deep learning network is trained using the sub-region dataset, enabling it to recognize each sub-region. Finally, the trained deep learning network is used to analyze the multi-angle images, achieving the localization and recognition of each sub-region and extracting features from each sub-region, such as color, texture, and edges, forming a multi-dimensional sub-region fusion feature.

[0041] By extracting features from each sub-region from multi-angle images, multi-dimensional sub-region fusion features are formed. These features express the relationship between the sub-regions and are key information for achieving sub-model fusion, which helps improve the accuracy of the final vehicle digital twin model.

[0042] Step S700: Based on the multi-dimensional sub-region fusion features, fuse the N vehicle sub-models to obtain the first vehicle twin model.

[0043] Specifically, after obtaining the multi-dimensional sub-region fusion features and N vehicle sub-models, the two are fused to assemble the digital twin models of each sub-region into a whole, thereby generating a digital twin model that accurately expresses the overall features of the first vehicle, thus obtaining the first vehicle twin model.

[0044] First, the N sub-models and multi-dimensional sub-region fusion features are imported into a digital twin modeling environment, such as 3D design software. Second, based on the actual positions of the sub-regions within the first vehicle, each sub-model is initially positioned and laid out, establishing initial spatial relationships. Third, based on the intrinsic relationships within the sub-regions expressed in the multi-dimensional sub-region fusion features, such as color similarity and texture progression, the correlation between each sub-model is analyzed, and the sub-models are precisely assembled using feature matching. Next, based on the motion dependencies between sub-regions, such as synchronous motion and linked motion, motion associations are set for related sub-models, establishing motion relationships. Subsequently, based on the functional relationships between sub-regions reflected in the multi-dimensional sub-region fusion features, such as input-output relationships, transmission relationships, and control relationships, corresponding functional dependencies are added to related sub-models. Finally, a relationship network including each sub-region is constructed in the digital twin modeling environment to express the spatial, motion, and functional dependencies between sub-regions, enabling precise association between each sub-model through network relationships. Simultaneously, based on sub-regions and their relationships, a knowledge graph is established to express the inherent logical connections between sub-regions, providing knowledge support for the accurate aggregation of sub-models. Then, the fused first vehicle digital twin model is tested to determine if it meets the requirements. If not, feature matching, relationship networks, or knowledge graphs are optimized and adjusted until an accurate model is generated. Finally, the optimized first vehicle digital twin model is exported from the digital twin modeling environment.

[0045] By deeply fusing sub-models and multi-dimensional sub-region fusion features, various intrinsic relationships between sub-regions are expressed, ultimately generating an accurate first vehicle digital twin model. This model contains rich sub-region relationships, enabling in-depth model building of the entire vehicle and achieving the technical effect of improving the accuracy and efficiency of vehicle model building.

[0046] Furthermore, such as Figure 2 As shown, embodiments of this application also include:

[0047] Step S410: Traverse the N vehicle sub-region datasets to obtain the first vehicle sub-region dataset;

[0048] Step S420: Perform modeling association analysis based on the first vehicle sub-region dataset to obtain the modeling association degree;

[0049] Step S430: Obtain the preset modeling correlation degree;

[0050] Step S440: Based on the modeling correlation degree, filter the first vehicle sub-region dataset to obtain a first sub-region modeling correlation dataset that satisfies the preset modeling correlation degree;

[0051] Step S450: Based on the first sub-region modeling association dataset, perform data cleaning to obtain the first vehicle sub-data partition, and add the first vehicle sub-data partition to the N vehicle sub-data partitions.

[0052] Specifically, after obtaining N vehicle sub-region datasets, the datasets are preprocessed to further improve the sub-region modeling effect.

[0053] First, the first vehicle sub-region dataset is obtained by traversing N vehicle sub-region datasets. This first vehicle sub-region dataset is any dataset selected from the N vehicle sub-region datasets. Then, various types of information, such as image information, 3D geometric information, and material information, are extracted from the first vehicle sub-region dataset. Among the extracted information, different types of information expressing the same sub-region features are identified, such as images and 3D models of the sub-region. These are matched to determine their consistency in spatial location, geometric dimensions, and detail representation, thus obtaining the information matching degree. The spatial connection degree is obtained by analyzing whether there are close relationships between the information regarding the location, orientation, and scale of the sub-regions in the expressed space. The material property correlation degree is obtained by checking whether the descriptions of the materials used in the sub-regions by the information are consistent. Finally, the information matching degree, spatial connection degree, and material property correlation degree are combined to generate the modeling correlation degree. A higher modeling correlation degree indicates a stronger correlation between the information in the first vehicle sub-region dataset in expressing the features of the sub-region.

[0054] The preset modeling relevance is a threshold value pre-set based on the requirements of sub-region modeling. Based on this preset relevance, the first sub-region dataset is filtered to obtain a first sub-region modeling relevance dataset that meets the preset relevance. Irrelevant information with a modeling relevance less than the preset relevance is removed from the first vehicle sub-region dataset, resulting in a more accurate first sub-region modeling relevance dataset. Then, the first sub-region modeling relevance dataset undergoes preprocessing such as information supplementation, error correction, and format normalization to generate a more complete dataset, which serves as the first vehicle sub-data partition and is added to the N vehicle sub-data partitions.

[0055] By selecting preprocessed datasets that meet the modeling association requirements from N sub-region datasets, vehicle sub-data partitions are generated, ultimately resulting in N vehicle sub-data partitions, providing data support for subsequent sub-region modeling.

[0056] Furthermore, embodiments of this application also include:

[0057] Step S451: Traverse the first sub-region modeling association dataset to perform completeness evaluation and obtain the data completeness evaluation result;

[0058] Step S452: Obtain the preset completeness;

[0059] Step S453: Determine whether the data integrity evaluation result meets the preset integrity requirement;

[0060] Step S454: If the data integrity evaluation result does not meet the preset integrity, obtain a first data supplementation instruction, and perform data compensation on the first sub-region modeling associated dataset based on the first data supplementation instruction.

[0061] Specifically, after obtaining the first sub-region modeling association dataset that meets the modeling association degree, its integrity is evaluated and necessary data compensation is performed.

[0062] First, the dataset associated with the first sub-region modeling is traversed, and its completeness is evaluated based on content completeness, spatial information completeness, material information completeness, accuracy completeness, and correlation completeness. This yields the completeness evaluation results for the dataset associated with the first sub-region modeling. For example, the evaluation considers whether the viewpoints and quantity of images or 3D data describing the spatial location information of the sub-region in the dataset can comprehensively represent the spatial morphology of the sub-region. If not, the dataset has low spatial information completeness. The evaluation also considers whether the accuracy of each piece of information in the dataset meets the requirements for sub-region modeling. If the accuracy does not meet the requirements, even with rich information content, high-precision modeling is difficult to achieve, and the dataset's completeness is low. Then, a preset completeness is obtained, where the preset completeness is a threshold of the required level of completeness of the dataset information, determined according to the requirements of sub-region modeling.

[0063] Then, it is determined whether the data completeness evaluation result meets the preset completeness. If the data completeness evaluation result reaches or exceeds the preset completeness, it indicates that the first sub-region modeling associated dataset is relatively complete; otherwise, it indicates that the dataset contains some incomplete information and needs to be supplemented. If the data completeness evaluation result does not meet the preset completeness, a first data supplementation instruction is obtained, and data compensation is performed on the first sub-region modeling associated dataset based on the instruction. The data supplementation instruction includes the types and quantities of information that need to be supplemented to achieve dataset completeness. Based on the instruction, information supplementation is performed, such as supplementing other perspectives of the sub-region image, supplementing other detection results of material parameters, etc., until the dataset reaches the preset completeness.

[0064] By evaluating the completeness of the dataset associated with the modeling of the first sub-region and supplementing it with necessary information, the integrity of the dataset is ensured, providing more comprehensive and accurate information support for sub-region modeling, thereby improving modeling accuracy and effectiveness.

[0065] Furthermore, embodiments of this application also include:

[0066] Step S461: Traverse the first sub-region modeling association dataset to perform data anomaly evaluation and obtain the data anomaly evaluation results;

[0067] Step S462: Obtain data anomaly evaluation constraints;

[0068] Step S463: Determine whether the data anomaly evaluation result meets the data anomaly evaluation constraints;

[0069] Step S464: If the data anomaly evaluation result does not meet the data anomaly evaluation constraint, obtain a first data anomaly correction instruction, and perform data correction on the first sub-region modeling associated dataset based on the first data anomaly correction instruction.

[0070] Specifically, firstly, the first sub-region modeling associated dataset is traversed to perform data anomaly evaluation, detecting the presence of erroneous, redundant, and illegal data, and obtaining data anomaly evaluation results that reflect the types and quantities of anomalous data in the dataset. Then, data anomaly evaluation constraints are obtained, specifying the maximum allowed types and quantities of anomalous data in the dataset, determined based on the sub-region modeling requirements and dataset size. Next, it is determined whether the data anomaly evaluation results meet the constraints. If the results are below the constraints, the anomalous data in the dataset is within acceptable limits; otherwise, the anomalous data exceeds the limits and requires correction. If the results do not meet the constraints, a first data anomaly correction instruction is obtained, and data correction is performed on the first sub-region modeling associated dataset based on the instruction. The data anomaly correction instruction specifies the content and quantity of data to be deleted or corrected. Based on the instruction, the specified anomalous data in the dataset is deleted, replaced, or corrected, achieving anomalous data correction for the first sub-region modeling associated dataset.

[0071] By detecting anomalous data in the first sub-region modeling associated dataset and performing necessary data corrections when constraints are exceeded, the correctness and validity of the dataset are ensured, thereby maximizing the accuracy and effectiveness of sub-region modeling.

[0072] Furthermore, embodiments of this application also include:

[0073] Step S610: Based on big data, collect a record set of vehicle fusion feature analysis data;

[0074] Step S620: Based on the vehicle fusion feature analysis record set, perform random data partitioning to obtain a first training set, a first test set, and a first validation set;

[0075] Step S630: Based on a fully connected neural network, train, test, and validate the model according to the first training set, the first test set, and the first validation set to obtain a vehicle fusion feature analysis model;

[0076] Step S640: Based on the N vehicle sub-regions, analyze the multi-angle images of the vehicles according to the vehicle fusion feature analysis model to generate the multi-dimensional sub-region fusion features.

[0077] Specifically, firstly, a dataset containing multi-angle images of vehicles is obtained from public datasets and internal enterprise datasets through web crawling. The obtained image set is preprocessed, including format conversion, resolution adjustment, and image cropping, to make the images suitable for visual feature extraction. Each sub-region is manually labeled on the preprocessed images as reference information for feature extraction. The images are then analyzed to locate and identify the sub-regions, and features such as color, texture, and contour are extracted from each sub-region to form a corresponding feature sequence for each image. Next, the features extracted from each sub-region in each image are fused to obtain multi-dimensional features expressing the association between the sub-regions, and these features are associated with records as the image's fusion features. The records and fusion features of each image are compiled into a feature analysis record set, resulting in a vehicle fusion feature analysis record set, which is used for subsequent model training.

[0078] Then, the vehicle fusion feature analysis record set was randomly divided into a training set, a test set, and a validation set. The training set was used for model training, the test set for evaluating training results, and the validation set for parameter tuning. Next, a fully connected neural network was selected as the analysis model. The model was trained using the training set, evaluated using the test set, and its structure and parameters were tuned using the validation set, ultimately obtaining a vehicle fusion feature analysis model for multi-angle image analysis. Finally, the trained analysis model was applied to multi-angle images of the target vehicle. Based on the information of each sub-region in the image, sub-region identification and feature extraction were achieved, generating multi-dimensional sub-region fusion features to express the association between sub-regions.

[0079] By constructing and training deep learning models, we can achieve automatic identification and feature extraction of each sub-region in multi-angle vehicle images, and maximize the expression of the intrinsic relationship between sub-regions. This is the key information for achieving accurate fusion of sub-models.

[0080] Furthermore, embodiments of this application also include:

[0081] Step S810: Based on the N vehicle sub-regions, identify the multi-angle images of the vehicle to obtain N sub-region images;

[0082] Step S820: Perform feature consistency evaluation on the N sub-region images and the N vehicle sub-models respectively to obtain N feature consistency evaluation results;

[0083] Step S830: Determine whether the consistency evaluation results of the N features meet the preset consistency evaluation results;

[0084] Step S840: When any one of the N feature consistency evaluation results does not meet the preset consistency evaluation result, a sub-model correction instruction is generated.

[0085] Specifically, after obtaining N sub-models, feature consistency evaluation is performed between the sub-models and the sub-region images to determine the accuracy of the sub-models.

[0086] First, based on the position and contour of each sub-region in the multi-angle images, image patches corresponding to each sub-region are extracted, resulting in N sub-region images. Then, each sub-region image is compared with its corresponding sub-model to determine their similarity in color, texture, shape, etc., obtaining feature consistency evaluation results for the N sub-regions. Higher evaluation results indicate greater consistency between the sub-model and the sub-region image in terms of features. The preset consistency evaluation result is a feature consistency threshold set according to the sub-region modeling accuracy requirements. If all N evaluation results exceed the threshold, it indicates that each sub-model is highly consistent with the sub-region image; otherwise, some differences exist. If any of the N feature consistency evaluation results does not meet the preset consistency evaluation result, a sub-model correction instruction is generated, including the type of sub-model to be corrected and the specific content of the correction. Sub-models with low feature consistency are corrected to improve accuracy.

[0087] By evaluating the feature consistency between the sub-model and the corresponding sub-region image, we can verify whether the accuracy of the sub-model meets the requirements. If it does not, we can promptly generate a sub-model correction instruction to make necessary corrections to the sub-model and ensure that it accurately represents the features of the sub-region.

[0088] In summary, the vehicle model building method based on depth image fusion provided in this application has the following technical effects:

[0089] The process involves: obtaining the first vehicle to provide research data for subsequent steps; dividing the vehicle into N sub-regions (where N is a positive integer greater than 1) to provide a basis for sub-region modeling and feature extraction; collecting basic information from each of the N sub-regions to obtain N vehicle sub-region datasets, providing data support for sub-region model construction; preprocessing the N vehicle sub-region datasets to obtain N vehicle sub-data partitions, improving data quality and facilitating deep learning; connecting to the digital twin module to model each of the N vehicle sub-data partitions, obtaining N vehicle sub-models, and extracting multi-dimensional features from the sub-regions; acquiring multi-angle images of the first vehicle and performing deep learning on these images based on the N vehicle sub-regions to generate multi-dimensional sub-region fusion features; and fusing the N vehicle sub-models based on these multi-dimensional sub-region fusion features to obtain the first vehicle twin model, achieving accurate and efficient vehicle model construction.

[0090] Example 2

[0091] Based on the same inventive concept as the vehicle model building method based on depth image fusion in the foregoing embodiments, such as Figure 3 As shown in the figure, this application provides a vehicle model building system based on deep image fusion, the system including:

[0092] First vehicle acquisition module 11, used to acquire the first vehicle;

[0093] The vehicle region division module 12 is used to divide the region based on the first vehicle to obtain N vehicle sub-regions, where N is a positive integer greater than 1.

[0094] The regional information acquisition module 13 is used to collect basic information of the N vehicle sub-regions respectively, and obtain N vehicle sub-region datasets.

[0095] Data preprocessing module 14 is used to traverse the N vehicle sub-region datasets for preprocessing to obtain N vehicle sub-data partitions;

[0096] The data partitioning modeling module 15 is used to connect to the digital twin module to model the N vehicle sub-data partitions respectively, and obtain N vehicle sub-models.

[0097] The image deep learning module 16 is used to acquire multi-angle images of the first vehicle, obtain multi-angle images of the vehicle, and perform deep learning on the multi-angle images of the vehicle based on the N vehicle sub-regions to generate multi-dimensional sub-region fusion features.

[0098] The sub-model fusion module 17 fuses the N vehicle sub-models based on the multi-dimensional sub-region fusion features to obtain a first vehicle twin model.

[0099] Furthermore, the data preprocessing module 14 includes the following steps:

[0100] Traverse the N vehicle sub-region datasets to obtain the first vehicle sub-region dataset;

[0101] Based on the first vehicle sub-region dataset, a modeling correlation analysis is performed to obtain the modeling correlation degree;

[0102] Obtain the preset modeling correlation degree;

[0103] Based on the modeling correlation degree, the first vehicle sub-region dataset is filtered to obtain a first sub-region modeling correlation dataset that satisfies the preset modeling correlation degree.

[0104] Data cleaning is performed on the associated dataset modeled based on the first sub-region to obtain the first vehicle sub-data partition, and the first vehicle sub-data partition is added to the N vehicle sub-data partitions.

[0105] Furthermore, the data preprocessing module 14 also includes the following steps:

[0106] Traverse the first sub-region to model the associated dataset and evaluate its completeness to obtain the data completeness evaluation result;

[0107] Achieve preset completeness;

[0108] Determine whether the data integrity evaluation result meets the preset integrity level;

[0109] If the data integrity evaluation result does not meet the preset integrity, a first data supplementation instruction is obtained, and data compensation is performed on the modeling associated dataset of the first sub-region based on the first data supplementation instruction.

[0110] Furthermore, the data preprocessing module 14 also includes the following steps:

[0111] Traverse the first sub-region to model the associated dataset and perform data anomaly evaluation to obtain the data anomaly evaluation results;

[0112] Obtain data anomaly evaluation constraints;

[0113] Determine whether the data anomaly evaluation result meets the data anomaly evaluation constraints;

[0114] If the data anomaly evaluation result does not meet the data anomaly evaluation constraint, a first data anomaly correction instruction is obtained, and data correction is performed on the modeling associated dataset of the first sub-region based on the first data anomaly correction instruction.

[0115] Furthermore, the image deep learning module 16 includes the following execution steps:

[0116] Based on big data, collect and analyze vehicle fusion feature records;

[0117] Based on the vehicle fusion feature analysis record set, random data partitioning is performed to obtain a first training set, a first test set, and a first validation set;

[0118] Based on a fully connected neural network, a vehicle fusion feature analysis model is obtained by training, testing, and validating according to the first training set, the first test set, and the first validation set.

[0119] Based on the N vehicle sub-regions, the multi-angle images of the vehicles are analyzed according to the vehicle fusion feature analysis model to generate the multi-dimensional sub-region fusion features.

[0120] Furthermore, embodiments of this application also include a correction instruction generation module, which includes the following execution steps:

[0121] Based on the N vehicle sub-regions, the multi-angle images of the vehicle are identified to obtain N sub-region images;

[0122] Feature consistency evaluation is performed on the N sub-region images and the N vehicle sub-models respectively to obtain N feature consistency evaluation results;

[0123] Determine whether the consistency evaluation results of the N features meet the preset consistency evaluation results;

[0124] When any one of the N feature consistency evaluation results does not meet the preset consistency evaluation result, a sub-model correction instruction is generated.

[0125] Example 3

[0126] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention, showing a block diagram of an exemplary electronic device suitable for implementing the embodiments of the present invention. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the present invention. Figure 4 As shown, the electronic device includes a processor 31, a memory 32, an input device 33, and an output device 34; the number of processors 31 in the electronic device can be one or more. Figure 4 Taking a processor 31 as an example, the processor 31, memory 32, input device 33, and output device 34 in an electronic device can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0127] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the vehicle model building method based on deep image fusion in this embodiment of the invention. The processor 31 executes various functional applications and data processing of the computer device by running the software programs, instructions, and modules stored in the memory 32, thereby realizing the aforementioned vehicle model building method based on deep image fusion.

[0128] In summary, any step of the method described above can be stored as a computer instruction or program in an unrestricted computer memory, and can be called and identified by an unrestricted computer processor to implement any method in the embodiments of this application, without any additional restrictions.

[0129] Furthermore, the "first" or "second" mentioned above may not only represent a sequential relationship, but may also represent a specific concept, and / or refer to the individual or collective selection of multiple elements. Clearly, those skilled in the art can make various modifications and variations to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A method for building vehicle models based on deep image fusion, characterized in that, The method includes: Obtain the first vehicle; Based on the first vehicle, the region is divided into N vehicle sub-regions, where N is a positive integer greater than 1; Collect basic information for each of the N vehicle sub-regions to obtain a dataset of N vehicle sub-regions; The N vehicle sub-region datasets are traversed and preprocessed to obtain N vehicle sub-data partitions. Connect the digital twin module to model the N vehicle sub-data partitions respectively, and obtain N vehicle sub-models; Multi-angle images of the first vehicle are acquired to obtain multi-angle images of the vehicle, and deep learning is performed on the multi-angle images of the vehicle based on the N vehicle sub-regions to generate multi-dimensional sub-region fusion features. The N vehicle sub-models are fused based on the multi-dimensional sub-region fusion features to obtain a first vehicle twin model; Preprocessing the N vehicle sub-region datasets yields N vehicle sub-data partitions, including: Traverse the N vehicle sub-region datasets to obtain the first vehicle sub-region dataset; Based on the first vehicle sub-region dataset, a modeling correlation analysis is performed to obtain the modeling correlation degree; Obtain the preset modeling correlation degree; Based on the modeling correlation degree, the first vehicle sub-region dataset is filtered to obtain a first sub-region modeling correlation dataset that satisfies the preset modeling correlation degree. Based on the first sub-region modeling association dataset, data cleaning is performed to obtain the first vehicle sub-data partition, and the first vehicle sub-data partition is added to the N vehicle sub-data partitions; Data cleaning is performed on the dataset associated with the first sub-region model, including: Traverse the first sub-region to model the associated dataset and evaluate its completeness to obtain the data completeness evaluation result; Achieve preset completeness; Determine whether the data integrity evaluation result meets the preset integrity level; If the data integrity evaluation result does not meet the preset integrity, a first data supplementation instruction is obtained, and data compensation is performed on the modeling associated dataset of the first sub-region based on the first data supplementation instruction.

2. The method as described in claim 1, characterized in that, The method includes: Traverse the first sub-region to model the associated dataset and perform data anomaly evaluation to obtain the data anomaly evaluation results; Obtain data anomaly evaluation constraints; Determine whether the data anomaly evaluation result meets the data anomaly evaluation constraints; If the data anomaly evaluation result does not meet the data anomaly evaluation constraint, a first data anomaly correction instruction is obtained, and data correction is performed on the modeling associated dataset of the first sub-region based on the first data anomaly correction instruction.

3. The method as described in claim 1, characterized in that, Based on the N vehicle sub-regions, deep learning is performed on the multi-angle images of the vehicle to generate multi-dimensional sub-region fusion features, including: Based on big data, collect and analyze vehicle fusion feature records; Based on the vehicle fusion feature analysis record set, random data partitioning is performed to obtain a first training set, a first test set, and a first validation set; Based on a fully connected neural network, a vehicle fusion feature analysis model is obtained by training, testing, and validating according to the first training set, the first test set, and the first validation set. Based on the N vehicle sub-regions, the multi-angle images of the vehicles are analyzed according to the vehicle fusion feature analysis model to generate the multi-dimensional sub-region fusion features.

4. The method as described in claim 1, characterized in that, After obtaining N vehicle sub-models, including: Based on the N vehicle sub-regions, the multi-angle images of the vehicle are identified to obtain N sub-region images; Feature consistency evaluation is performed on the N sub-region images and the N vehicle sub-models respectively to obtain N feature consistency evaluation results; Determine whether the consistency evaluation results of the N features meet the preset consistency evaluation results; When any one of the N feature consistency evaluation results does not meet the preset consistency evaluation result, a sub-model correction instruction is generated.

5. A vehicle model building system based on deep image fusion, characterized in that, The system is used to implement the vehicle model building method based on depth image fusion as described in any one of claims 1-4, the system comprising: First vehicle acquisition module, the first vehicle acquisition module is used to acquire the first vehicle; The vehicle region division module is used to divide the region based on the first vehicle to obtain N vehicle sub-regions, where N is a positive integer greater than 1. The regional information collection module is used to collect basic information of the N vehicle sub-regions respectively, and obtain N vehicle sub-region datasets. The data preprocessing module is used to traverse the N vehicle sub-region datasets for preprocessing to obtain N vehicle sub-data partitions. A data partitioning modeling module is used to connect to the digital twin module to model the N vehicle sub-data partitions respectively, thereby obtaining N vehicle sub-models. The image deep learning module is used to acquire multi-angle images of the first vehicle, obtain multi-angle images of the vehicle, and perform deep learning on the multi-angle images of the vehicle based on the N vehicle sub-regions to generate multi-dimensional sub-region fusion features. The sub-model fusion module fuses the N vehicle sub-models based on the multi-dimensional sub-region fusion features to obtain a first vehicle twin model.

6. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the vehicle model building method based on depth image fusion as described in any one of claims 1 to 4.

7. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the vehicle model building method based on deep image fusion as described in any one of claims 1 to 5.

Images