A feature matching model weight update iteration method, system and storage medium

Abstract

The application discloses a feature matching model weight updating iteration method, comprising the following steps: collecting image data containing position and posture information of smart glasses under different operating environments; generating a training data set according to the collected image data and uploading the training data set to a cloud device; performing incremental training on a feature matching model according to the training set to obtain a latest feature matching model and corresponding model weights; when it is judged that the latest feature matching model result is better than the current feature matching model, issuing the model weights corresponding to the latest feature matching model; and updating the current feature matching model used in the smart glasses according to the issued model weights. The application also discloses a corresponding system and a storage medium. Through the interaction between the terminal device and the cloud device, the application can realize the automatic updating iteration of the weights of the feature matching model, improve the updating efficiency and has strong adaptability.

Description

Technical Field

[0001] This invention relates to the technical field of updating and iterating deep learning models, and in particular to a weight update and iteration method, system, and storage medium for a feature matching model for in-vehicle smart glasses. Background Technology

[0002] The use of smart glasses (such as VR glasses) in vehicles is becoming increasingly common, with passengers wearing them to play games or watch movies while the car is in motion. To ensure the correct display of smart content and a comfortable user experience, the system needs to acquire the pose coordinates of the smart glasses within the vehicle in real time. Feature matching plays a crucial role in acquiring these pose coordinates.

[0003] Generally, multiple cameras installed inside the vehicle capture images of the smart glasses in real time. These images contain not only information about the smart glasses themselves but also information about the vehicle's environment. The system extracts features from the captured images, identifying key points (such as corners and edges) and their descriptors (i.e., attribute information of the feature points). These feature points serve as the basis for subsequent matching. The system matches feature points in the current image with feature points in previous images or pre-built maps. Deep learning-based feature matching algorithms can find matching points more accurately, maintaining a high matching rate even under different lighting, weather, or camera orientations. Using the matched feature point pairs, the system can calculate the smart glasses' current position and orientation. This information is then used to adjust the display angle and depth of the smart content to ensure the user receives the best viewing or gaming experience.

[0004] Feature matching can be divided into traditional matching algorithms and deep learning-based feature matching algorithms. Deep learning-based feature matching algorithms generally have higher accuracy and generalization ability than traditional feature matching algorithms. To obtain a higher quality feature matching model, a large amount of in-vehicle cabin image data needs to be collected to train the model, and then deployed to the in-vehicle cabin for visual positioning. However, the constantly changing cabin environment, such as car model, different lighting conditions, weather, different camera poses inside the vehicle, and occlusion, can degrade the model's performance.

[0005] Therefore, in the existing technology, incremental learning has emerged as a solution for updating models. However, most existing incremental learning solutions only consider fine-tuning the model from the algorithm side using small sample data, without comprehensively considering the resources required for actual model deployment, and lacking automated update and iteration steps. The actual deployment of the model still relies on manual judgment of the new model and manual deployment to the device, which is inefficient and cannot adapt to constantly changing needs and environments. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that it proposes a method, system and storage medium for updating and iterating the weights of a feature matching model. Through the interaction between terminal devices and cloud devices, the weights of the feature matching model can be automatically updated and iterated, which improves the update efficiency and has strong adaptability.

[0007] As one aspect of the present invention, a weight update iteration method for a feature matching model is provided, which includes at least the following steps:

[0008] Monitor the operating environment information of smart glasses inside the vehicle, and collect image data containing the position and attitude information of smart glasses under different operating environments;

[0009] A training dataset is generated based on the collected image data, and the training dataset is uploaded to a cloud device.

[0010] The cloud device incrementally trains the feature matching model based on the training set to obtain the latest feature matching model and the corresponding model weights.

[0011] The latest feature matching model is compared with the current feature matching model. When the result of the latest feature matching model is better than that of the current feature matching model, the model weight corresponding to the latest feature matching model is distributed.

[0012] Update the current feature matching model used in the smart glasses based on the issued model weights.

[0013] This further includes:

[0014] The performance metrics of the current feature matching model of the smart glasses are uploaded to the cloud device along with the training dataset;

[0015] The performance metrics are correlated with the current feature matching model; the performance metrics include at least: inference time, CPU usage, and model memory / GPU memory usage.

[0016] The training dataset, generated based on the collected image data, further includes:

[0017] A self-supervised optimization method is used to obtain image data corresponding to the current operating environment information, which includes weather, lighting conditions, and vehicle type.

[0018] Based on the predetermined proportions of different weather conditions, lighting conditions, and vehicle types in the operating environment information, corresponding image data are selected, and after reaching a predetermined quantity, a training dataset is formed for incremental training of the feature matching model.

[0019] This further includes:

[0020] After the training dataset is uploaded to the cloud device, an incremental training request signal is sent to the cloud device.

[0021] The comparison between the latest feature matching model and the current feature matching model further includes:

[0022] The performance metrics of the latest feature matching model are compared with those of the current feature matching model. The performance metrics are a single metric or a combination of precision, recall, model complexity, model inference time, and model occupancy.

[0023] Accordingly, another aspect of the present invention also provides a weight update and iteration system for a feature matching model, which includes at least a cloud device and a terminal device connected to smart glasses in a vehicle, wherein:

[0024] The terminal device is used to monitor the operating environment information of the smart glasses in the vehicle, and to collect image data containing the position and posture information of the smart glasses under different operating environments; to generate a training dataset based on the collected image data, and to upload the training dataset to the cloud device; and to receive model weights from the cloud device and update the current feature matching model used in the smart glasses.

[0025] A cloud device is used to incrementally train the feature matching model based on the training set to obtain the latest feature matching model and its corresponding model weights; the latest feature matching model is compared with the current feature matching model, and when the result of the latest feature matching model is better than that of the current feature matching model, the model weights corresponding to the latest feature matching model are sent to the terminal device.

[0026] The terminal device further includes:

[0027] The data acquisition module is used to monitor the operating environment information of the smart glasses in the vehicle, and to collect image data containing the position and posture information of the smart glasses under different operating environments; it is used to upload the training dataset and the performance indicators of the current feature matching model to the cloud device; the operating environment information includes: weather, lighting conditions and vehicle type, and the performance indicators include at least: inference time, CPU usage, and model memory / GPU memory usage information;

[0028] The model inference module contains the current feature matching model, which is used to update the current feature matching model used in the smart glasses after receiving the model weights from the cloud device.

[0029] The visual mapping and localization module is used to form a training dataset based on the image data collected by the data acquisition module;

[0030] The terminal update module is used by the data acquisition module to send an incremental training request signal to the cloud device after the training dataset has been uploaded to the cloud device.

[0031] The cloud device includes:

[0032] The database module is used to store the training dataset from the terminal device and the performance metrics of the current feature matching model;

[0033] The model training module is used to receive an incremental training request signal from the terminal device, call the training dataset in the database module, incrementally train the feature matching model, obtain the latest feature matching model and corresponding model weights, and store them in the database module.

[0034] The cloud update module is used to compare the latest feature matching model with the current feature matching model. When the result of the latest feature matching model is better than that of the current feature matching model, the model weight corresponding to the latest feature matching model is sent to the terminal device.

[0035] In the visual mapping and localization module, based on the predetermined proportions of different weather conditions, different lighting conditions, and different vehicle types in the operating environment information, corresponding image data is selected. After reaching a predetermined number, a training dataset is formed for incremental training of the feature matching model.

[0036] In the cloud update module, the performance metrics of the latest feature matching model are compared with the performance metrics of the current feature matching model. The performance metrics are a single or combined metric among precision, recall, model complexity, model inference time, and model occupancy.

[0037] Accordingly, another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the method as described above.

[0038] Implementing the embodiments of the present invention has the following beneficial effects:

[0039] This invention proposes a weight update and iteration method, system, and storage medium for a feature matching model. Through the interaction between terminal devices and cloud devices, the weights of the feature matching model can be automatically updated and iterated, improving update efficiency. Furthermore, it can continuously adapt to the changing automotive environment, making the pose estimation of smart glasses in the vehicle more accurate.

[0040] In the method provided by this invention, the application on the device and in the cloud is managed in the form of a service, which makes it easier to achieve flexible deployment and expansion of the application; at the same time, the data collected by the terminal device is uniformly managed through the cloud-based automated update service database, which facilitates maintenance and upgrades and improves data security; and the deployment of model inference service on the device improves the real-time performance and fast response of the smart glasses in the vehicle cabin. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0042] Figure 1 A schematic diagram of the structure of an embodiment of a weight update and iteration system for a feature matching model provided by the present invention;

[0043] Figure 2 This is a schematic diagram of the main flow of an embodiment of a weight update and iteration method for a feature matching model provided by the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0045] like Figure 1 The diagram shown is a structural schematic of an embodiment of a weight update and iteration system for a feature matching model provided by the present invention. In this embodiment, the weight update and iteration system 1 for the feature matching model includes at least a cloud device 2 and a terminal device 1 connected to smart glasses in a vehicle, wherein:

[0046] The terminal device 1 is used to monitor the operating environment information of the smart glasses in the vehicle, and to collect image data containing the position and posture information of the smart glasses under different operating environments; to generate a training dataset based on the collected image data, and to upload the training dataset to the cloud device; and to receive model weights sent by the cloud device and update the current feature matching model used in the smart glasses.

[0047] Cloud device 2 is used to incrementally train the feature matching model based on the training set to obtain the latest feature matching model and the corresponding model weights; compare the latest feature matching model with the current feature matching model, and when the result of the latest feature matching model is better than the current feature matching model, send the model weights corresponding to the latest feature matching model to the terminal device.

[0048] More specifically, the terminal device 1 further includes:

[0049] The data acquisition module 10 is used to monitor the operating environment information of the smart glasses in the vehicle, and to collect image data containing the position and posture information of the smart glasses under different operating environments; it is used to upload the training dataset and the performance indicators of the current feature matching model to the cloud device; the operating environment information includes: weather, lighting conditions, vehicle type and the current data acquisition time, and the performance indicators include at least: inference time, CPU usage, and model memory / video memory usage information.

[0050] In a specific example, the data acquisition module 10 can use a camera to capture real-time images inside the vehicle, including the position of the smart glasses and the environment inside the vehicle; it can use an inertial measurement unit (IMU) to measure the device's acceleration, angular velocity, and other motion states, thereby helping to determine the pose of the smart glasses; it can collect environmental parameters inside the vehicle through environmental sensors (such as temperature sensors, humidity sensors, and light sensors); and the vehicle information system can also obtain information such as the vehicle's model, speed, and direction through the vehicle's bus system (such as the CAN bus) to more accurately understand the motion state of the smart glasses inside the vehicle.

[0051] The model inference module 11 contains the current feature matching model, which is used to update the current feature matching model used in the smart glasses after receiving the model weights sent by the cloud device.

[0052] The visual mapping and localization module 12 is used to form a training dataset based on the image data collected by the data acquisition module;

[0053] The terminal update module 13 is used to send an incremental training request signal to the cloud device after the data acquisition module has uploaded the training dataset to the cloud device.

[0054] Understandably, in specific examples, the model inference module on the device terminal can further deploy image retrieval models, image feature extraction models, and feature matching models. These models are used by smart glasses for visual mapping and positioning services. The feature matching model is used to match feature points of the image and obtain the performance metrics of the current feature matching model, such as inference time, CPU and GPU resource usage, and model memory / video memory usage information.

[0055] Furthermore, the terminal device is typically an embedded low-power ARM64-based device installed in a car. Its computing power is higher than that of smart glasses but lower than that of cloud devices. It can be installed in locations such as the roof or behind the windshield.

[0056] More specifically, the cloud device 2 includes:

[0057] Database module 20 is used to store the training dataset from the terminal device and the performance metrics of the current feature matching model;

[0058] The model training module 21 is used to receive an incremental training request signal from the terminal device, call the training dataset in the database module, incrementally train the feature matching model, obtain the latest feature matching model and the corresponding model weights, and store them in the database module.

[0059] The weights mentioned here are parameter weights in the deep learning model. These weights are learned by the model during training and are used to represent the degree of contribution of different features or inputs to the model output. In specific examples, the weights can be, for example, weights of neural unconnected layers, weights of convolutional layers, weights of fully connected layers, and weights of other types of layers (such as pooling layers, batch normalization layers, etc.). When adjusting the weights, optimization algorithms (such as gradient descent, Adam, etc.) are usually used to iteratively update the weight values ​​to minimize the loss function (i.e., the difference between the model's prediction and the actual result). By continuously adjusting the weights, the model can perform better on the training data and is expected to maintain good performance on unseen test data.

[0060] The cloud update module 22 is used to compare the latest feature matching model with the current feature matching model. When the result of the latest feature matching model is better than the current feature matching model, the model weight corresponding to the latest feature matching model is sent to the terminal device.

[0061] In a specific example, in the visual mapping and localization module 12, according to the predetermined proportional relationship of different weather, different lighting conditions and different vehicle types in the operating environment information, the corresponding image data is selected, and after reaching a predetermined number, a training dataset for incremental training of the feature matching model is formed.

[0062] In the cloud update module 22, the performance metrics of the latest feature matching model are compared with the performance metrics of the current feature matching model. The performance metrics are a single or combined metric among precision, recall, model complexity, model inference time, and model occupancy.

[0063] Furthermore, the cloud device is configured as a high-computing platform, typically including GPUs and other accelerated neural network computing units.

[0064] The multiple modules in the terminal device and cloud device involved in this invention can be implemented in the form of services. Specifically, the terminal device employs model inference service, data acquisition service, visual mapping and localization service, and automatic model update service; while the cloud employs training service and automatic model update service.

[0065] The following will be combined Figure 2 The principles involved in this invention will be explained in more detail.

[0066] like Figure 2 The diagram illustrates the main flow of an embodiment of a weight update iteration method for a feature matching model provided by the present invention. In this embodiment, it employs the following... Figure 2 The system shown is used to implement this. More specifically, the method provided by this invention includes at least the following steps:

[0067] Step S10: Monitor the operating environment information of the smart glasses in the vehicle, and collect image data containing the position and attitude information of the smart glasses under different operating environments;

[0068] Specifically, within a data acquisition cycle, the data acquisition service is activated and automatically detects environmental conditions inside the vehicle, such as weather (sunny or rainy), light intensity, and vehicle model. Sensors such as cameras are used to capture image data under these conditions. The data acquisition service stores the captured image data and corresponding environmental condition labels locally.

[0069] Step S11: Generate a training dataset based on the collected image data, and upload the training dataset to the cloud device;

[0070] In a specific example, this step generates a training dataset based on the collected image data, which further includes:

[0071] A self-supervised optimization method is used to obtain image data corresponding to the current operating environment information, which includes weather, lighting conditions, and vehicle type.

[0072] Based on the predetermined proportions of different weather conditions, lighting conditions, and vehicle types in the operating environment information, corresponding image data are selected, and after reaching a predetermined quantity, a training dataset is formed for incremental training of the feature matching model.

[0073] In a specific example, the training dataset was collected according to the conditions shown in Table 1. The collection ratio under weather conditions was sunny: cloudy / foggy / rainy = 2:1:1. The collection ratio under lighting conditions was daytime: early morning: evening / night = 2:1:1. The collection ratio of vehicle models was sedan: SUV = 1:1. A total of 10,000 matching pairs were generated by a self-supervised optimization method to construct the training dataset.

[0074] Table 1

[0075] sunny 5:00-7:00 AM sedan Cloudy / Smoggy Day Daytime 8:00-18:00 SUV rain Evening / Evening 18:00-21:00

[0076] In a specific example, the best-performing feature matching model from the model inference service is invoked to process the collected image data. A training dataset is generated using self-supervised optimization, leveraging the correlation and spatial consistency between images. This training dataset contains positive samples (well-matched image pairs) and negative samples (mismatched or low-similarity image pairs). For instance, in one example, image data of sunny days, SUV models, and daytime lighting conditions are automatically acquired from the terminal device's data acquisition service, and a training dataset containing 1000 matching pairs is generated through self-supervised tuning.

[0077] Further includes:

[0078] The performance metrics of the current feature matching model for the smart glasses are uploaded to the cloud device along with the training dataset. These performance metrics are correlated with the current feature matching model and include at least: inference time, CPU usage, and model memory / GPU memory usage. This performance metric information is crucial for evaluating model performance and optimizing resource allocation. Through this information, the cloud can understand the current operating status and performance of the model on the terminal device, thus making more informed decisions. For example, if a model consumes excessive resources on the terminal device without significant performance improvement, the cloud may choose not to update the model or optimize it to reduce resource consumption.

[0079] After the training dataset is uploaded to the cloud device, an incremental training request signal is sent to the cloud device.

[0080] Step S12: The cloud device performs incremental training on the feature matching model based on the training set to obtain the latest feature matching model and the corresponding model weights.

[0081] Understandably, after receiving an incremental training request signal from the terminal device, the cloud device starts the training service and initializes it to waiting mode. It then calls the training dataset in the database module to perform incremental training, obtain the latest feature matching model and corresponding model weights, and evaluate the performance metrics of the new model (such as accuracy and recall). Finally, it stores the results (such as metrics) and weights of the latest feature matching model in the database module.

[0082] Step S13: Compare the latest feature matching model with the current feature matching model. When the result of the latest feature matching model is better than that of the current feature matching model, distribute the model weights corresponding to the latest feature matching model.

[0083] In this step, comparing the latest feature matching model with the current feature matching model further includes:

[0084] The performance metrics of the latest feature matching model are compared with those of the current feature matching model. The performance metrics are a single metric or a combination of precision, recall, model complexity, model inference time, and model occupancy.

[0085] For example, in one instance, if the cloud update module finds that the accuracy and inference time of the new model are both better than the old model, then it can be determined that the result of the latest feature matching model is better than the current feature matching model.

[0086] Step S14: Update the current feature matching model used in the smart glasses according to the issued model weights.

[0087] After receiving the model weights from the cloud device, the terminal device's model inference module restarts the device-side inference service to load and use the new feature matching model, thereby completing the update and iteration of the feature matching model.

[0088] It is understandable that, in some embodiments, the update frequency of the latest model may also be adopted in a periodic manner.

[0089] For more details, please refer to and combine with the aforementioned points. Figure 2 The description of that will not be repeated here.

[0090] It is understood that the method provided in this invention manages the automated updates and iterations of the feature matching model by deploying multiple services in both the terminal device and the cloud device. The terminal device deploys a model inference service, a data acquisition service, a visual mapping and localization service, and a model automated update service. Within a single acquisition cycle, the data acquisition service collects images of the vehicle cabin environment under different conditions, such as weather, lighting, and vehicle model. The visual mapping and localization service then processes the collected images using a self-supervised optimization method to generate a training dataset. This visual mapping and localization service calls upon the feature matching model with the best recent performance metrics deployed in the model inference service. Once the training dataset is generated, it is uploaded to the model automated update service database in the cloud device via the acquisition service, along with the best recent performance metrics. When the data upload on the device is complete, the model automated update service on the device notifies the model automated update service in the cloud to initiate the cloud-based training service. After the cloud training service is completed, the new model metric results and weights are added to the cloud-based model automatic update service data and compared with the most recent metric results in that data. If the new model metric results are better than the most recent results, the cloud-based automatic update service sends the new model weights to the device-side inference service and restarts the device-side inference service to complete the update and iteration of the feature matching model.

[0091] As another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the aforementioned... Figure 2 The steps of the described method. For more details, please refer to and combine with the foregoing explanation. Figure 2 The description of that will not be repeated here.

[0092] Implementing the embodiments of the present invention has the following beneficial effects:

[0093] This invention proposes a weight update and iteration method, system, and storage medium for a feature matching model. Through the interaction between terminal devices and cloud devices, the weights of the feature matching model can be automatically updated and iterated, improving update efficiency. Furthermore, it can continuously adapt to the changing automotive environment, making the pose estimation of smart glasses in the vehicle more accurate.

[0094] In the method provided by this invention, the application on the device and in the cloud is managed in the form of a service, which makes it easier to achieve flexible deployment and expansion of the application; at the same time, the data collected by the terminal device is uniformly managed through the cloud-based automated update service database, which facilitates maintenance and upgrades and improves data security; and the deployment of model inference service on the device improves the real-time performance and fast response of the smart glasses in the vehicle cabin.

[0095] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0097] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A weight update and iteration method for a feature matching model, characterized in that, It should include at least the following steps: Monitor the operating environment information of smart glasses inside the vehicle, and collect image data containing the position and attitude information of smart glasses under different operating environments; Based on the collected image data, a training dataset is generated and uploaded to a cloud device; The cloud device incrementally trains the feature matching model based on the training dataset to obtain the latest feature matching model and the corresponding model weights. The performance metrics of the latest feature matching model are compared with those of the current feature matching model. When it is determined that the performance metrics of the latest feature matching model are better than those of the current feature matching model, the model weights corresponding to the latest feature matching model are distributed. Update the current feature matching model used in the smart glasses based on the issued model weights.

2. The method as described in claim 1, characterized in that, Further includes: The performance metrics of the current feature matching model of the smart glasses are uploaded to the cloud device along with the training dataset; The performance metrics are associated with the current feature matching model; the performance metrics include at least: model inference time, CPU usage information, or model memory / GPU memory usage information.

3. The method as described in claim 2, characterized in that, Based on the collected image data, a training dataset is generated, which further includes: A self-supervised optimization method is used to obtain image data corresponding to the current operating environment information, which includes weather, lighting conditions, and vehicle type. Based on the predetermined proportions of different weather conditions, lighting conditions, and vehicle types in the operating environment information, corresponding image data are selected, and after reaching a predetermined quantity, a training dataset is formed for incremental training of the feature matching model.

4. The method as described in claim 3, characterized in that, Further includes: After the training dataset is uploaded to the cloud device, an incremental training request signal is sent to the cloud device.

5. The method as described in claim 4, characterized in that, in, The performance metrics of the latest feature matching model are compared with those of the current feature matching model. The performance metrics include a single or combined metric among precision, recall, model complexity, model inference time, and model occupancy.

6. A weight update and iteration system for a feature matching model, characterized in that, It includes at least cloud devices and terminal devices that connect to smart glasses in the vehicle, including: The terminal device is used to monitor the operating environment information of the smart glasses in the vehicle, and to collect image data containing the position and posture information of the smart glasses under different operating environments; to generate a training dataset based on the collected image data and upload it to the cloud device; and to receive model weights from the cloud device and update the current feature matching model used in the smart glasses. A cloud device is used to incrementally train the feature matching model based on the training dataset to obtain the latest feature matching model and its corresponding model weights; the performance metrics of the latest feature matching model are compared with the performance metrics of the current feature matching model; when it is determined that the performance metrics of the latest feature matching model are better than those of the current feature matching model, the model weights corresponding to the latest feature matching model are sent to the terminal device.

7. The system as described in claim 6, characterized in that, The terminal device further includes: The data acquisition module is used to monitor the operating environment information of the smart glasses in the vehicle, and to collect image data containing the position and posture information of the smart glasses under different operating environments; it is used to upload the training dataset and the performance indicators of the current feature matching model to the cloud device; the operating environment information includes: weather, lighting conditions and vehicle type, and the performance indicators include at least: model inference time, CPU usage information or model memory / GPU memory usage information; The model inference module contains the current feature matching model, which is used to update the current feature matching model used in the smart glasses after receiving the model weights from the cloud device. The visual mapping and localization module is used to form a training dataset based on the image data collected by the data acquisition module; The terminal update module is used by the data acquisition module to send an incremental training request signal to the cloud device after the training dataset has been uploaded to the cloud device.

8. The system as described in claim 7, characterized in that, The cloud device includes: The database module is used to store the training dataset from the terminal device and the performance metrics of the current feature matching model; The model training module is used to receive an incremental training request signal from the terminal device, call the training dataset in the database module, incrementally train the feature matching model, obtain the latest feature matching model and corresponding model weights, and store them in the database module. The cloud update module is used to compare the latest feature matching model with the current feature matching model. When it is determined that the result of the latest feature matching model is better than the current feature matching model, the model weight corresponding to the latest feature matching model is sent to the terminal device.

9. The system as described in claim 8, characterized in that, in: In the visual mapping and localization module, based on the predetermined proportions of different weather conditions, different lighting conditions, and different vehicle types in the operating environment information, corresponding image data is selected. After reaching a predetermined number, a training dataset is formed for incremental training of the feature matching model. In the cloud update module, the performance metrics of the latest feature matching model are compared with the performance metrics of the current feature matching model. The performance metrics include a single or combined metric among accuracy, recall, model complexity, model inference time, and model occupancy.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images