Model updating method, device, equipment and storage medium based on incremental data

Abstract

The embodiment of the present application provides a model updating method, apparatus, device and storage medium based on incremental data. The method includes: obtaining a training sub-data set corresponding to each newly added category; for each newly added category, performing comparative training on the category gap of a preset cloud model based on multiple target training samples contained in the corresponding training sub-data set and multiple first category prototypes corresponding to the multiple newly added categories, thereby obtaining a target cloud model and multiple target first category prototypes; sending the multiple target first category prototypes to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes, thereby obtaining a target edge model; wherein the intermediate edge model is obtained by performing comparative training on the category gap of a preset edge model based on multiple target training samples and multiple second category prototypes corresponding to the multiple newly added categories; thereby, the model's perception of incremental data and the performance of the model can be improved.

Description

Technical Field

[0001] The present application relates to the field of model updating technology, and in particular to a model updating method, apparatus, device and storage medium based on incremental data. Background Art

[0002] Cloud models are large-scale machine learning or deep learning models that can run in a cloud computing environment. They can leverage the cloud platform's powerful computing power and storage resources to process and analyze large amounts of data. However, with the proliferation of IoT devices, sending all data processing tasks to the cloud can overload cloud servers and reduce data processing efficiency.

[0003] Related technologies can extract key knowledge from cloud models through techniques such as model compression. Based on this knowledge, edge models can be deployed at the edge to process data, improving the overall system's processing power and efficiency. However, when faced with new types of data, cloud and edge models may struggle to quickly adapt to these changes, making it impossible for the models to effectively extract and utilize the features of this data. This in turn affects the cloud and edge models' ability to perceive incremental data and reduces model performance. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a model updating method, apparatus, device and storage medium based on incremental data, which can improve the model's perception ability of incremental data and the performance of the model.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application proposes a model update method based on incremental data, which is applied to a cloud service end. The method includes:

[0006] Get the training sub-dataset corresponding to each newly added category;

[0007] For each newly added category, a preset cloud model is trained on the category gap comparison based on multiple target training samples contained in the corresponding training sub-dataset and multiple first category prototypes corresponding to the multiple newly added categories, thereby obtaining a target cloud model and corresponding multiple target first category prototypes; wherein the first category prototype of each newly added category is obtained by feature averaging of the preset cloud model based on the corresponding training sub-dataset;

[0008] The multiple target first category prototypes are sent to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain the target edge model; wherein, the intermediate edge model is obtained by the edge end performing comparative training on the category gap of the preset edge model based on the multiple target training samples and the multiple second category prototypes corresponding to the multiple newly added categories; the second category prototype of each newly added category is obtained by the preset edge model according to the corresponding training sub-dataset by averaging the features.

[0009] Accordingly, a second aspect of an embodiment of the present application proposes a model updating device based on incremental data, which is applied to a cloud service end. The device includes:

[0010] The acquisition module is used to obtain the training sub-dataset corresponding to each newly added category;

[0011] A training module is configured to perform, for each newly added category, comparative training on a preset cloud model based on a plurality of target training samples contained in a corresponding training sub-dataset and a plurality of first-category prototypes corresponding to the plurality of newly added categories, thereby obtaining a target cloud model and a plurality of corresponding target first-category prototypes; wherein the first-category prototype of each newly added category is obtained by feature averaging of the preset cloud model based on the corresponding training sub-dataset;

[0012] A first sending module is used to send the multiple target first category prototypes to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain a target edge model; wherein, the intermediate edge model is obtained by the edge end performing comparative training on the category gap of the preset edge model based on the multiple target training samples and the multiple second category prototypes corresponding to the multiple newly added categories; the second category prototype of each newly added category is obtained by the preset edge model according to the corresponding training sub-dataset by feature averaging.

[0013] In some embodiments, the incremental data-based model updating apparatus further includes a second sending module configured to:

[0014] For each newly added category, multiple classification fuzzy samples uploaded by the edge are obtained, and based on the corresponding multiple target training samples and the multiple classification fuzzy samples, comparative training is performed on the target first category prototype corresponding to each newly added category and multiple historical category prototypes obtained from any historical task to obtain an updated cloud model and the corresponding multiple updated first category prototypes;

[0015] The multiple classification fuzzy samples are subjected to classification tests on multiple test samples contained in the test sub-dataset corresponding to each newly added category through the target edge model, and multiple uncertainty indices of each test sample obtained under the multiple newly added categories are determined;

[0016] The multiple updated first category prototypes are sent to the edge end, so that the edge end adjusts the parameters of the target edge model based on the multiple updated first category prototypes to obtain an updated edge model.

[0017] In some embodiments, the training module is further configured to:

[0018] For each newly added category, the preset cloud model averages the features of multiple training samples contained in the corresponding training sub-dataset to obtain the corresponding first category prototype;

[0019] Obtaining a first feature representation corresponding to each training sample through the preset cloud model, and determining a first distance between each first feature representation and the corresponding first category prototype;

[0020] Obtain a second distance between each first feature representation and other first category prototypes of other newly added categories;

[0021] determining a first category gap loss based on a difference between the first distance and the second distance;

[0022] Parameters of a preset cloud model are adjusted based on the first category gap loss to obtain a target cloud model and a corresponding plurality of target first category prototypes.

[0023] To achieve the above objectives, a third aspect of an embodiment of the present application proposes a model updating method based on incremental data, which is applied to an edge end. The method includes:

[0024] Get the training sub-dataset corresponding to each newly added category;

[0025] For each newly added category, a preset edge model is trained for category gap comparison based on multiple target training samples contained in the corresponding training sub-dataset and multiple second category prototypes corresponding to the multiple newly added categories, thereby obtaining an intermediate edge model and corresponding multiple target second category prototypes; wherein the second category prototype of each newly added category is obtained by feature averaging of the preset edge model based on the corresponding training sub-dataset;

[0026] Receive multiple target first category prototypes sent by the cloud service end, and adjust parameters of the intermediate edge model based on the multiple target first category prototypes to obtain a target edge model.

[0027] In some embodiments, the incremental data-based model updating method applied to the edge corresponds to an incremental data-based model updating device applied to the edge, wherein the incremental data-based model updating device includes a testing module for:

[0028] For each of the newly added categories, a classification test is performed on a plurality of test samples included in a test sub-dataset corresponding to each of the newly added categories using the target edge model to obtain an uncertainty index of each test sample under the plurality of newly added categories;

[0029] Determine a plurality of classification fuzzy samples according to a plurality of uncertainty indices corresponding to the plurality of test samples, and send the plurality of classification fuzzy samples to a cloud server, so that the cloud server performs comparative training on the target cloud model based on the plurality of target training samples corresponding to each newly added category and the plurality of classification fuzzy samples, and the target first category prototype corresponding to each newly added category and a plurality of historical category prototypes obtained from any historical task to obtain a plurality of updated first category prototypes;

[0030] Acquire multiple updated first category prototypes sent by the cloud server, and adjust parameters of the target edge model based on the multiple updated first category prototypes to obtain an updated edge model.

[0031] In some embodiments, the incremental data-based model updating device applied to the edge terminal further includes a computing module for:

[0032] For each of the newly added categories, calculating, by using the preset edge model, the Euclidean distance between each test sample included in the test sub-dataset corresponding to each of the newly added categories and the multiple second category prototypes corresponding to the multiple newly added categories;

[0033] For each test sample, calculating a logarithmic value of each corresponding Euclidean distance, and obtaining an uncertainty measure of each test sample in the corresponding second category prototype based on a product between the Euclidean distance and the corresponding logarithmic value;

[0034] Based on the multiple uncertainty measures corresponding to the multiple second category prototypes, an uncertainty index of each test sample under the multiple newly added categories is obtained.

[0035] In some embodiments, the incremental data-based model updating apparatus applied to the edge terminal further includes a determination module configured to:

[0036] For each newly added category, obtaining a corresponding target second category prototype and a target first category prototype, and determining a third distance between the target second category prototype and the target first category prototype;

[0037] Obtaining first-category prototypes of other newly added targets, and determining a fourth distance between the second-category prototype of the target and the first-category prototype of the other target;

[0038] determining a distance distillation loss based on a difference between the third distance and the fourth distance;

[0039] Parameters of the intermediate edge model are adjusted based on the distance distillation loss to obtain a target edge model.

[0040] Correspondingly, the fourth aspect of the embodiments of the present application proposes a computer device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the model update method based on incremental data of the embodiment of the first aspect of the present application, or any one of the embodiments of the third aspect of the present application.

[0041] Correspondingly, the fifth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the model updating method based on incremental data of the embodiment of the first aspect of the present application or any one of the embodiments of the third aspect of the present application.

[0042] The embodiment of the present application obtains a training sub-dataset corresponding to each newly added category; for each newly added category, based on the multiple target training samples contained in the corresponding training sub-dataset, the preset cloud model is subjected to comparative training of the category gap with the multiple first category prototypes corresponding to the multiple newly added categories, thereby obtaining a target cloud model and the corresponding multiple target first category prototypes; wherein, the first category prototype of each newly added category is obtained by the preset cloud model by averaging the features according to the corresponding training sub-dataset; the multiple target first category prototypes are sent to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain the target edge model; wherein, the intermediate edge model is obtained by the edge end by performing comparative training of the category gap with the preset edge model based on the multiple target training samples and the multiple second category prototypes corresponding to the multiple newly added categories; the second category prototype of each newly added category is obtained by the preset edge model by averaging the features according to the corresponding training sub-dataset. In this way, the collaborative update of the cloud model and the edge model can be achieved through the combination of comparative learning and knowledge distillation. Specifically, the present application fully trains the cloud model and the edge model based on the newly added samples. During the training process, the contrastive learning technology is used to enhance the discrimination between the new and old categories in the feature space, and the category prototype is introduced to accurately characterize the central features of the newly added categories, providing a stable reference benchmark for model training. In the process of model training, the parameter optimization mechanism guided by the category prototype calibrates the representation vector of the category prototype in the parameter space, prompting the model weights to quickly converge to the optimized interval of the new category distribution, thereby effectively improving the model's adaptability and optimization efficiency for the newly added categories. Furthermore, the present application can make full use of the computing power advantage of the cloud model, and transfer the knowledge of the cloud model to the edge model through the distance distillation between category prototypes, so that the edge model can quickly and accurately identify the new categories while maintaining its original recognition ability. In summary, the present application can improve the model's perception of incremental data and improve the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram of the architecture of the incremental data-based model update system provided in an embodiment of the present application;

[0044] Figure 2 This is a flow chart of a model updating method based on incremental data on a cloud service side provided by an embodiment of the present application;

[0045] Figure 3 This is a flow chart of a model updating method based on incremental data at the edge provided by an embodiment of the present application;

[0046] Figure 4 This is a general flow chart of the incremental data-based model updating method provided in an embodiment of the present application;

[0047] Figure 5 This is another overall flow chart of the incremental data-based model updating method provided in an embodiment of the present application;

[0048] Figure 6 This is a schematic diagram of the functional modules of the incremental data-based model updating device provided in an embodiment of the present application;

[0049] Figure 7 This is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0051] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0053] Cloud models are large-scale machine learning or deep learning models that can run in a cloud computing environment. They can leverage the cloud platform's powerful computing power and storage resources to process and analyze large amounts of data. However, with the proliferation of IoT devices, sending all data processing tasks to the cloud can overload cloud servers and reduce data processing efficiency.

[0054] Related technologies can extract key knowledge from cloud models through techniques such as model compression. Based on this knowledge, edge models can be deployed at the edge to process data, improving the overall system's processing power and efficiency. However, when faced with new types of data, cloud and edge models may struggle to quickly adapt to these changes, making it impossible for the models to effectively extract and utilize the features of this data. This in turn affects the cloud and edge models' ability to perceive incremental data and reduces model performance.

[0055] Based on this, the embodiments of the present application provide a model updating method, apparatus, device and storage medium based on incremental data, which can improve the model's perception ability of incremental data and the performance of the model.

[0056] The incremental data-based model updating method, apparatus, device, and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the incremental data-based model updating system in the embodiments of the present application is described.

[0057] Please refer to Figure 1 In some implementations, the model updating system includes a terminal 11 , a cloud service terminal 12 , and an edge terminal 13 .

[0058] Exemplarily, the terminal 11 may be a smart camera, an industrial sensor, a mobile device, etc. The terminal 11 may be used for data collection and annotation, user interaction and configuration, data preprocessing and uploading, etc.

[0059] Furthermore, the cloud service end 12 can be a high-performance computing server, a distributed storage system, a network infrastructure, etc. The cloud service end 12 can be used for large-scale data storage and management, training and updating of cloud models, and unified management of the device status of the edge end 13, etc.

[0060] Furthermore, the edge end 13 may be an embedded computing device, an industrial-grade edge server, etc. The edge end 13 may be used to deploy a lightweight small model inference engine and perform lightweight inference, localized decision-making, and the like.

[0061] Exemplarily, after the cloud server 12 and the edge terminal 13 receive the training data uploaded by the terminal 11, they can train the models separately and optimize the category prototypes through comparative learning technology. Furthermore, the cloud server 12 can further perform comparative training on the cloud model based on the uncertainty samples uploaded by the edge terminal 13, optimize the category prototype, and transfer the knowledge of the cloud model to the edge model of the edge terminal 13 through knowledge distillation to achieve efficient data processing and model updating. After the models of the cloud server 12 and the edge terminal 13 are trained, the data collected by the terminal 11 can be classified in real time and accurately.

[0062] The model updating method based on incremental data in the embodiments of the present application can be illustrated by the following embodiments.

[0063] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first. Moreover, the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0064] In the embodiment of the present application, the model updating device based on incremental data will be described from the perspective of the model updating device based on incremental data, which can be integrated into a computer device. Figure 2 , Figure 2 This is a flowchart of the steps of the incremental data-based model update method applied to a cloud server provided in an embodiment of the present application. In this embodiment of the present application, the incremental data-based model update device is specifically integrated into a computer device as an example. When the processor on the computer device executes the program instructions corresponding to the incremental data-based model update method, the specific process is as follows:

[0065] Step 101: Obtain a training sub-dataset corresponding to each newly added category.

[0066] In some embodiments, in order to ensure that cloud models and edge models can continue to execute efficiently and accurately in the face of ever-changing application requirements, training sub-datasets corresponding to each newly added category can be obtained, and the incremental data that the model needs to learn can be integrated to facilitate model learning.

[0067] New categories can be categories that have never appeared in the model's training history. In other words, new categories are categories that the model does not have the ability to classify but will need to be classified later. For example, in an intelligent transportation system, the model is initially trained only to recognize pedestrians and vehicles, but over time, new categories such as bicycles and electric vehicles may need to be recognized. These two categories would be new categories.

[0068] The training sub-dataset may be a set of samples collected for each newly added category.

[0069] Specifically, the current training task may include training tasks for multiple newly added categories, each of which has corresponding training sub-data and test sub-data. The multiple training sub-data of the current training task constitute the training dataset, and the multiple test sub-data constitute the test dataset.

[0070] For example, for different scenarios, the newly added categories obtained are also different. Taking the field of intelligent transportation as an example, the newly added categories can be electric vehicles, new traffic signs, and so on. The corresponding training sub-datasets can be collected by smart cameras or uploaded by users. Each training sub-dataset can contain multiple samples corresponding to the newly added categories. For example, the newly added category electric vehicles can contain 15 training samples of electric vehicles, and so on. It should be noted that this application is applicable to any scenario for classifying data, such as smart security scenarios, industrial manufacturing scenarios, medical imaging scenarios, agricultural crop recognition scenarios, etc. Therefore, it is not limited to specific scenarios and data categories.

[0071] By obtaining the training sub-dataset corresponding to each newly added category, it is easier for subsequent models to train the training sub-dataset to improve the generalization ability of the model.

[0072] Step 102: For each newly added category, a preset cloud model is trained on the category gap comparison between the multiple target training samples contained in the corresponding training sub-dataset and the multiple first-category prototypes corresponding to the multiple newly added categories to obtain a target cloud model and the corresponding multiple target first-category prototypes; wherein, the first-category prototype of each newly added category is obtained by feature averaging of the preset cloud model based on the corresponding training sub-dataset.

[0073] In some embodiments, in order to enable the cloud model to more accurately distinguish between new and old categories, for each newly added category, the preset cloud model can be trained comparatively using the training sub-dataset of the newly added category, so that the training samples of the same category are more clustered in the feature space, and the training samples of different categories are more dispersed.

[0074] The target training samples may be specific samples selected from the training sub-dataset of each newly added category.

[0075] The first category prototype can be the average value of features calculated by the preset cloud model based on the corresponding training sub-dataset for each newly added category in the initial stage or current state. The first category prototype represents the center position of the newly added category in the feature space.

[0076] The preset cloud model can be a pre-trained large cloud model that already has the ability to classify existing categories. After receiving the training sub-dataset corresponding to the newly added category, the preset cloud model can adapt to the newly added category through comparative training.

[0077] The target first category prototype can be a new category prototype generated by the target cloud model after comparative training is completed. The target first category prototype can be the category center position recalculated based on the new target training samples by the cloud server during parameter adjustment of the preset cloud model for each newly added category, so as to more accurately represent the overall characteristics of the newly added category and improve the cloud model's recognition ability for the newly added category.

[0078] In some embodiments, the first category prototype of each newly added category is the geometric center obtained by extracting features from all target training samples of the category through a preset cloud model and averaging the features. Any new category in , the preset cloud model can be obtained by averaging the following values: Corresponding first category prototype :

[0079] ;

[0080] in, Indicates a newly added category The total number of target training samples in , ( )yes New categories added in target training samples.

[0081] Furthermore, in the process of training the preset cloud model based on each newly added category, the calculated first category prototype can be used to train the preset cloud model through contrastive learning. The goal of contrastive learning is to reduce the distance between each target training sample of the same newly added category and the corresponding first category prototype, while increasing the distance between each target training sample and the first category prototype of different newly added categories. Specifically, for each target training sample of the newly added category p, the similarity of its features with the first category prototype of the newly added category p (positive sample pair), as well as the similarity with other first category prototypes (negative sample pair) are calculated. By optimizing the contrast loss function, the features of samples of the same type are made closer to the first category prototype of the newly added category to which they belong, and away from the first category prototypes of other newly added categories, thereby improving the cloud model's ability to distinguish the newly added categories. Exemplarily, according to any target training sample Features Calculate the contrast loss as follows :

[0082] ;

[0083] in, for The first category prototype (positive sample pair) corresponding to the newly added category, For The first category prototypes (negative sample pairs) corresponding to different newly added categories.

[0084] Through the above method, not only the model's perception of the newly added categories is enhanced, but also the model's ability to distinguish between the newly added categories is improved, greatly improving the training efficiency.

[0085] In some embodiments, to enhance the model's ability to perceive newly added categories, comparative training can be performed on the pre-set cloud model for each newly added category, and model parameters can be gradually optimized to improve the overall robustness and generalization ability of the model, so that the model can remain efficient and accurate in the face of constantly changing new data. For example, step 102 may include:

[0086] (102.1) For each newly added category, the preset cloud model averages the features of multiple training samples contained in the corresponding training sub-dataset to obtain the corresponding first category prototype;

[0087] (102.2) Obtaining a first feature representation corresponding to each training sample using a preset cloud model, and determining a first distance between each first feature representation and the corresponding first category prototype;

[0088] (102.3) obtaining a second distance between each first feature representation and other first category prototypes of other newly added categories;

[0089] (102.4) determining a first category gap loss based on the difference between the first distance and the second distance;

[0090] (102.5) Adjust the parameters of the preset cloud model based on the first-category gap loss to obtain a target cloud model and corresponding multiple target first-category prototypes.

[0091] The first feature representation may be a vector representation obtained after a preset cloud model performs feature extraction on each training sample.

[0092] The first distance may be the distance between the first feature representation of each training sample and the first category prototype of the newly added category to which it belongs. The first distance may be measured using Euclidean distance or cosine similarity.

[0093] The other first category prototypes may be, for each newly added category, the first category prototypes of all other newly added categories except the first category prototype of the category to which it belongs. The other first category prototypes represent the center positions of different newly added categories in the feature space.

[0094] The second distance may be the distance between the first feature representation of each training sample and other first category prototypes of other newly added categories.

[0095] The first category gap loss may be a loss value calculated based on the gap between the first distance and the second distance.

[0096] Furthermore, in the process of training the preset cloud model based on each newly added category, the calculated first category prototype can be used to train the preset cloud model through contrastive learning. The goal of contrastive learning is to reduce the distance between each target training sample of the same newly added category and the corresponding first category prototype, while increasing the distance between each target training sample and the first category prototype of different newly added categories. Specifically, for each target training sample of the newly added category p, the similarity of its features with the first category prototype of the newly added category p (positive sample pair), as well as the similarity with other first category prototypes (negative sample pair) are calculated. By optimizing the contrast loss function, the features of samples of the same type are made closer to the first category prototype of the newly added category to which they belong, and away from the first category prototypes of other newly added categories, thereby improving the cloud model's ability to distinguish new categories.

[0097] For example, according to any target training sample The first feature representation of The first category gap loss is calculated as follows :

[0098] ;

[0099] in, for The first category prototype (positive sample pair) corresponding to the newly added category, For The first category prototypes (negative sample pairs) corresponding to different newly added categories, is a hyperparameter set according to actual conditions, such as 0.8.

[0100] In some embodiments, the first distance and the second distance can be calculated by Euclidean distance or cosine distance. The calculation formula is as follows:

[0101] ;

[0102] in, Represents training samples The first feature representation of Indicates the first category prototype corresponding to the newly added category.

[0103] Taking the vehicle recognition scenario as an example, if the system adds new categories such as ambulance, fire truck, and truck, these three new categories can be used as a training task, and the preset cloud model can be trained for each new category in turn. Taking the ambulance as an example, if the training sub-dataset corresponding to the ambulance has 100 ambulance pictures, these 100 pictures are input into the preset cloud model to extract features, and the 100 first feature representations extracted are averaged to obtain the first category prototype corresponding to the ambulance. Similarly, we can get the first category prototype corresponding to the fire truck , the first category prototype corresponding to the truck .

[0104] Furthermore, for each ambulance picture, for example, the first feature representation of the first ambulance picture , the first category prototype corresponding to the ambulance can be calculated The first distance between them, and calculate Respectively The second distance between The preset cloud model is parameterized based on the first distance and the second distance to the fire truck and the truck respectively, to obtain a target cloud model and corresponding multiple target first category prototypes.

[0105] By constructing the first-category gap loss to update the model parameters, when the sum of the first distance and the hyperparameter is greater than the second distance, the first-category gap loss can be made positive, triggering a gradient update, forcing the preset cloud model to adjust parameters, and recalculating the first-category prototypes of all newly added categories, so that similar training samples converge towards the first-category prototype (during this period, the first-category prototype is also one of the adjusted parameters), while keeping similar training samples away from the heterogeneous first-category prototype.

[0106] Through the above methods, the intra-class compactness of the same newly added category can be improved, so that the sample features of the same newly added category are highly clustered, and the prototype distances of different categories are expanded, which effectively improves the efficiency and accuracy of the cloud model in identifying new types.

[0107] In step 103, multiple target first category prototypes are sent to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain the target edge model; wherein, the intermediate edge model is obtained by the edge end performing comparative training on the category gap of the preset edge model based on multiple target training samples and multiple second category prototypes corresponding to multiple newly added categories; the second category prototype of each newly added category is obtained by the preset edge model by averaging the features according to the corresponding training sub-dataset.

[0108] In some embodiments, in order to ensure that the edge model can obtain the latest category information in a timely manner and improve its ability to recognize new categories, the knowledge of the target cloud model can be distilled to the edge model so that the edge model can quickly adapt to the new categories and maintain a high recognition accuracy even when the computing power and training effect are limited.

[0109] The edge can be a device close to the data source or application scenario, such as a smart camera or smartphone. Lightweight edge models are deployed on the edge, which have limited computing resources but need to process data in real time and make classification decisions.

[0110] Among them, the intermediate edge model can be a small model in an intermediate state running on the edge, which has undergone a round of adjustment for comparative training of new categories but has not yet been fully optimized.

[0111] The target edge model can be the final optimized model obtained by sending multiple target first-category prototypes updated from the cloud-based large model to the edge and further adjusting the parameters of the intermediate edge model. Compared to the intermediate edge model, the target edge model has higher accuracy and robustness and can better handle recognition tasks for both new and old categories.

[0112] The second category prototype can be the feature average calculated by the preset edge model based on the training sub-dataset of each newly added category at the edge. The second category prototype is the initial center point used by the edge model when learning the newly added category.

[0113] In some embodiments, the second category prototype of each newly added category is the geometric center obtained by extracting features from all target training samples of the category through a preset edge model and averaging the features. Any new category in , the preset edge model can be obtained by averaging the following values Corresponding second category prototype :

[0114] ;

[0115] in, Indicates a newly added category The total number of target training samples in , ( )yes New categories added in target training samples.

[0116] Furthermore, in the process of training the preset edge model based on each newly added category, the calculated second category prototype can be used to train the preset edge model through contrastive learning. The goal of contrastive learning is to reduce the distance between each target training sample of the same newly added category and the corresponding second category prototype, while increasing the distance between each target training sample and the second category prototype of different newly added categories. Specifically, for each target training sample of the newly added category p, the similarity of its features with the second category prototype of the newly added category p (positive sample pair), as well as the similarity with other second category prototypes (negative sample pair) are calculated. By optimizing the contrast loss function, the features of samples of the same type are made closer to the second category prototype of the newly added category to which they belong, and away from the second category prototypes of other newly added categories, thereby improving the ability of the edge model to distinguish the newly added categories.

[0117] For example, according to any target training sample The second feature representation of The second category gap loss is calculated as follows :

[0118] ;

[0119] in, for The second category prototype (positive sample pair) corresponding to the newly added category, For The second category prototypes (negative sample pairs) corresponding to different newly added categories, is a hyperparameter set according to actual conditions, such as 0.8.

[0120] In some embodiments, the distance between the second feature representation and the second category prototype corresponding to the newly added category, and the distance between the second category prototypes corresponding to other newly added categories can be calculated by Euclidean distance or cosine distance, and the specific calculation process is not repeated here.

[0121] By constructing the second-category gap loss to update the model parameters of the preset edge model, similar training samples can be clustered towards the second-category prototype (during this period, the second-category prototype is also one of the adjusted parameters), while keeping similar training samples away from the heterogeneous second-category prototype, effectively improving the efficiency and accuracy of the edge model in identifying new types.

[0122] Furthermore, multiple target first category prototypes can be distilled to the edge end, and the distance between the target first category prototype and the target second category prototype of the intermediate edge model can be reduced by back propagation, so that the target first category prototype and the target second category prototype are aligned, so that the distilled target edge model has a classification performance comparable to that of the target cloud model. In some embodiments, the distance distillation loss calculated during the distillation process is It can be calculated as follows:

[0123] ;

[0124] in, Indicates the current training task The total number of categories of newly added categories included in , is the target first category prototype corresponding to the newly added category of this training, is the target second category prototype corresponding to the newly added category of this training, The target first category prototypes of other newly added categories for this training are: is a hyperparameter set according to actual conditions.

[0125] The distance distillation loss calculated using the above formula encourages the intermediate edge model's target second-category prototype to approach the target first-category prototype of the target cloud model for the same newly added category (e.g., car), and to distance itself from the target first-category prototype of the target cloud model for different newly added categories (e.g., bicycle, tractor). After the edge receives multiple target first-category prototypes corresponding to multiple newly added categories, it can use these multiple target first-category prototypes to adjust the parameters of the intermediate edge model. Specifically, the intermediate edge model adjusts its parameters by minimizing the distance distillation loss function to continuously optimize its classification performance.

[0126] By sending the target first-category prototypes trained on the cloud server to the edge, the edge can effectively adjust the parameters of the intermediate edge model based on these category prototypes, thereby obtaining an optimized target edge model. This process not only achieves efficient transfer of cloud model knowledge, but also significantly improves the edge model's recognition accuracy and robustness for newly added categories in resource-constrained environments, thereby enhancing the real-time responsiveness and deployment flexibility of the entire system.

[0127] In some embodiments, to further improve the classification accuracy of the cloud model and the edge model, the edge model can be used to filter out samples that are difficult to classify, so as to more quickly identify samples that require further analysis. The cloud model can then be trained based on fuzzy samples and then distilled again to effectively improve the classification accuracy and data sensitivity of the cloud model and the edge model. For example, after step 103, the following steps may also be included:

[0128] (A.1) For each newly added category, obtain multiple classification fuzzy samples uploaded by the edge, and perform comparative training on the category gap between the target first category prototype corresponding to each newly added category and multiple historical category prototypes obtained from any historical task based on the corresponding multiple target training samples and multiple classification fuzzy samples. This results in an updated cloud model and the corresponding multiple updated first category prototypes.

[0129] Among them, multiple classification fuzzy samples are classified and tested on multiple test samples contained in the test sub-dataset corresponding to each newly added category through the target edge model, and multiple uncertainty indices of each test sample under multiple newly added categories are determined;

[0130] (A.2) Sending the multiple updated first category prototypes to the edge end, so that the edge end adjusts parameters of the target edge model based on the multiple updated first category prototypes to obtain an updated edge model.

[0131] Classification fuzzy samples can be test samples that are difficult to clearly classify after the target edge model is used to classify the test sub-dataset of the newly added categories. Classification fuzzy samples have high uncertainty indexes in multiple newly added categories and have significant feature overlap with multiple newly added categories, making them difficult to distinguish.

[0132] A historical task can be a completed training task. A historical task can include at least one previously added category that was trained before retraining the preset cloud model and the preset edge model. For example, during the training phase corresponding to the historical task, training for the historically added categories of trucks and tractors was completed. The current training phase requires training for the newly added categories of cars and bicycles.

[0133] The historical category prototype may be the category prototype corresponding to the historical newly added category included in the historical task. The calculation method may refer to the calculation process of the first category prototype and the second category prototype described above.

[0134] The updated first category prototype may be a new category prototype generated by a large cloud model after comparative training, and the updated first category prototype is used to better characterize the central category features of each newly added category.

[0135] The test sub-dataset can be a data set collected from actual application scenarios to evaluate the performance of the model. Each newly added category corresponds to a test sub-dataset.

[0136] The test sample can be a specific sample instance in the test sub-dataset, which is used to perform classification testing on the model.

[0137] The uncertainty index can be used to characterize the classification confidence or uncertainty of each test sample under multiple newly added categories. A higher uncertainty index means that the test sample is difficult to be clearly classified into a specific category by the target edge model.

[0138] The updated edge model may be a target edge model after parameter adjustment.

[0139] In some embodiments, in order to quickly screen out fuzzy samples, multiple classified fuzzy samples can also be classified and tested on multiple test samples contained in the test sub-dataset corresponding to each newly added category through a preset edge model or an intermediate edge model, and multiple uncertainty indexes of each test sample under multiple newly added categories are determined.

[0140] In some implementations, in order to effectively avoid catastrophic forgetting during training, an updated dataset can be formed based on multiple target training samples and multiple classification fuzzy samples for each newly added category, and compared with the cross-task prototype (i.e., the prototype stored in the prototype set after the historical task training is completed). For example, for each sample data in the updated dataset corresponding to each newly added category, after the sample data is input into the target cloud model, the corresponding feature representation can be obtained. , through contrast training, we can narrow Target first category prototype corresponding to the same newly added category The distance between Multiple history category prototypes stored with history tasks Specifically, the category gap loss corresponding to this comparative training is The calculation formula is as follows ( are preset hyperparameters):

[0141] ;

[0142] Furthermore, through the above category gap loss Backpropagation is performed to update the parameters of the target cloud model until the model converges or reaches the preset number of training rounds, and the final updated cloud model can be obtained. In this way, the newly added categories can be more effectively identified while maintaining the recognition ability of existing categories.

[0143] Furthermore, since the target edge model cannot accurately classify the classification fuzzy samples, the updated first category prototype obtained by updating the cloud model can be distilled into the target edge model, so that the target edge model can adjust the parameters based on the updated first category prototype to achieve rapid alignment of category prototypes. Specifically, the distillation loss of this process is The formula is as follows:

[0144] ;

[0145] in, Indicates the total number of categories of new samples in the current training task, Represents the target third category prototype corresponding to the target edge model, Indicates the updated first category prototype corresponding to each newly added category currently being trained (for example, the newly added category currently being trained is truck). Indicates the updated first prototype of new categories other than the currently trained new categories (such as new categories such as cars and bicycles).

[0146] The distillation loss calculated using the above formula encourages the updated third-category prototype of the edge model to be close to the updated first-category prototype of the cloud model for the same newly added category (e.g., car), and away from the updated first-category prototype of the cloud model for different newly added categories (e.g., bicycle, tractor). After the edge receives multiple updated first-category prototypes corresponding to multiple newly added categories, it can use these updated first-category prototypes to adjust the parameters of the target edge model. Specifically, the edge model adjusts its parameters by minimizing the distance distillation loss function to continuously optimize its classification performance.

[0147] In some embodiments, after obtaining the updated cloud model and the updated edge model, the updated first category prototype corresponding to the updated cloud model and the updated third category prototype corresponding to the updated edge model can be stored in their respective prototype sets to facilitate subsequent classification of the model and training of other newly added categories.

[0148] In some embodiments, the process of determining ambiguous samples is based on the results of a marginal model classification test on samples in the newly added category test subdataset. This process aims to identify samples that the model is less certain about when classifying. These samples may belong to new categories or may be samples from old categories that are difficult to distinguish.

[0149] First, the target edge model (or pre-set edge model, or target edge model) can use its current knowledge to classify each test sample in the test sub-dataset. During the classification process, for some test samples, the target edge model may predict multiple categories, and the confidence level of each prediction is low. Therefore, the model's classification of these test samples is uncertain. To quantify this uncertainty, the information entropy (also known as the uncertainty index) of the confidence distribution of each test sample across all possible newly added categories can be calculated. The higher the information entropy, the less certain the target edge model is about the classification of the test sample. Therefore, all test samples can be sorted from high to low according to their uncertainty index, and the top K% (the value of K can be determined based on actual conditions, such as 10%, 20%, etc.) of the test samples with the highest scores after sorting are selected as fuzzy classification samples. These fuzzy classification samples are used in subsequent training to improve the overall performance and adaptability of the model.

[0150] By classifying fuzzy samples, screening, and retraining the target cloud model, and then distilling the trained updated first-category prototypes to the target edge model, we obtain an updated edge model. This significantly improves the model's ability to recognize newly added categories of data while maintaining performance for data from existing categories. This approach allows the updated edge model to run efficiently on resource-constrained edge devices while leveraging the powerful computing power of the cloud server to update and optimize the model.

[0151] In some embodiments, after the training of the current newly added category (such as a bicycle) is completed and the trained cloud model and edge model are obtained, the next newly added category (such as a car) of the current training task can be trained, and then the parameters of the cloud model and the edge model can be adjusted until all the newly added categories in the current training task are trained. In this way, the model can be efficiently trained within a small range of the same task, so that the model has the ability to efficiently distinguish the newly added categories, and the model will not forget old knowledge, thereby maintaining good performance.

[0152] In some embodiments, after the cloud model and edge model are trained, prototypes corresponding to multiple newly added categories can be added to the prototype set of the corresponding model, and the model performance test can be performed. Taking the performance test of the cloud model as an example, for any test data X, the trained cloud model can be used to predict the category to which the data X belongs (category i = 1, 2, ..., ), the probability can be calculated as follows:

[0153] ;

[0154] in, , It is the feature representation extracted by the cloud model. is the total number of categories that have been learned up to the current training task, Represents the Euclidean distance function of two given vectors. The prediction result of the cloud model for X is The largest category i.

[0155] The embodiment of the present application obtains a training sub-dataset corresponding to each newly added category; for each newly added category, based on the multiple target training samples contained in the corresponding training sub-dataset, the preset cloud model is subjected to comparative training of the category gap with the multiple first category prototypes corresponding to the multiple newly added categories, thereby obtaining a target cloud model and the corresponding multiple target first category prototypes; wherein, the first category prototype of each newly added category is obtained by the preset cloud model by averaging the features according to the corresponding training sub-dataset; the multiple target first category prototypes are sent to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain the target edge model; wherein, the intermediate edge model is obtained by the edge end by performing comparative training of the category gap with the preset edge model based on the multiple target training samples and the multiple second category prototypes corresponding to the multiple newly added categories; the second category prototype of each newly added category is obtained by the preset edge model by averaging the features according to the corresponding training sub-dataset. In this way, the collaborative update of the cloud model and the edge model can be achieved through the combination of comparative learning and knowledge distillation. Specifically, the present application fully trains the cloud model and the edge model based on the newly added samples. During the training process, the contrastive learning technology is used to enhance the discrimination between the new and old categories in the feature space, and the category prototype is introduced to accurately characterize the central features of the newly added categories, providing a stable reference benchmark for model training. In the process of model training, the parameter optimization mechanism guided by the category prototype calibrates the representation vector of the category prototype in the parameter space, prompting the model weights to quickly converge to the optimized interval of the new category distribution, thereby effectively improving the model's adaptability and optimization efficiency for the newly added categories. Furthermore, the present application can make full use of the computing power advantage of the cloud model, and transfer the knowledge of the cloud model to the edge model through the distance distillation between category prototypes, so that the edge model can quickly and accurately identify the new categories while maintaining its original recognition ability. In summary, the present application can improve the model's perception of incremental data and improve the performance of the model.

[0156] In some embodiments, in order to ensure that the edge model can dynamically adapt to the ever-changing application requirements, so that it can still maintain a high recognition accuracy when facing new categories, the parameters can be adjusted by receiving multiple target first category prototypes sent by the cloud service end, so as to achieve the improvement of model performance without significantly increasing the burden on the edge end. Therefore, in the second aspect, the present application proposes a model update method based on incremental data, which is applied to the edge end, and the method includes:

[0157] Step 201: Obtain a training sub-dataset corresponding to each newly added category;

[0158] Step 202: For each newly added category, a preset edge model is trained for category gap comparison based on multiple target training samples contained in the corresponding training sub-dataset and multiple second category prototypes corresponding to the multiple newly added categories, thereby obtaining an intermediate edge model and corresponding multiple target second category prototypes; wherein the second category prototype of each newly added category is obtained by feature averaging of the preset edge model based on the corresponding training sub-dataset;

[0159] Step 203: Receive multiple target first category prototypes sent by the cloud service end, and adjust parameters of the intermediate edge model based on the multiple target first category prototypes to obtain a target edge model.

[0160] The target second category prototype can be the category center point obtained by averaging the features of the intermediate edge model based on the training sub-dataset of each newly added category. Specifically, during the comparative training of the preset edge model on the edge, the original category prototype will be continuously adjusted. After the model training is completed and the intermediate edge model is obtained, the second target prototype can be obtained. After receiving multiple target first category prototypes sent by the cloud service end, parameter adjustment can be performed to obtain the target edge model and the target third category prototype.

[0161] In some embodiments, the edge end performs feature averaging based on the training sub-data set corresponding to each newly added category to obtain an embodiment of the first category prototype corresponding to the newly added category, which has been introduced above. For details, please refer to the above embodiments of obtaining the first category prototype and obtaining the second category prototype. This embodiment of the present application will not go into details.

[0162] In some embodiments, the process of performing comparative training of category gaps on a preset edge model based on multiple target training samples and multiple second category prototypes corresponding to multiple newly added categories to obtain an intermediate edge model has been introduced above, and the above also introduces the process of performing comparative training of category gaps on a preset cloud model based on multiple target training samples and multiple first category prototypes corresponding to multiple newly added categories on a cloud server to obtain a target cloud model. For details, please refer to the implementation process above. The embodiment of the present application does not elaborate on the processing process of training the preset edge model to obtain the intermediate edge model.

[0163] In some embodiments, the process of receiving multiple target first category prototypes sent by the cloud server and adjusting the parameters of the intermediate edge model accordingly to obtain the target edge model has been introduced above and will not be repeated here.

[0164] By combining contrastive learning and knowledge distillation, the coordinated update of the cloud model and the edge model can be achieved. Specifically, the present application fully trains the cloud model and the edge model based on the newly added samples. During the training process, the contrastive learning technology is used to enhance the discrimination between the new and old categories in the feature space, and the category prototype is introduced to accurately characterize the central features of the newly added categories, providing a stable reference benchmark for model training, thereby achieving in the process of model training, the parameter optimization mechanism guided by the category prototype calibrates the representation vector of the category prototype in the parameter space, prompting the model weight to quickly converge to the optimized interval of the distribution of the newly added categories, thereby effectively improving the model's adaptability and optimization efficiency for the newly added categories. Furthermore, it can also make full use of the computing power advantage of the cloud model, and transfer the knowledge of the cloud model to the edge model through the distance distillation between category prototypes, so that the edge model can quickly and accurately identify the newly added categories while maintaining its original recognition ability. In summary, the present application can improve the model's perception of incremental data and improve the performance of the model.

[0165] In some embodiments, to ensure that the cloud model and the edge model have similar performance for newly added categories in the feature space, while ensuring that similar samples are clustered and heterogeneous samples are dispersed, the knowledge of the target cloud model can be distilled to the intermediate edge model to enhance the robustness and generalization ability of the edge model, so that it can better handle complex and changing application scenarios. For example, step 203 may include:

[0166] (203.1) For each newly added category, obtain the corresponding target second category prototype and target first category prototype, and determine a third distance between the target second category prototype and the target first category prototype;

[0167] (203.2) Obtaining other target first category prototypes of other newly added categories and determining a fourth distance between the target second category prototype and the other target first category prototypes;

[0168] (203.3) determining a distance distillation loss based on a difference between the third distance and the fourth distance;

[0169] (203.4) The parameters of the intermediate edge model are adjusted based on the distance distillation loss to obtain the target edge model.

[0170] The third distance may be the distance between the target second category prototype of each newly added category and the target first category prototype corresponding to the same newly added category.

[0171] The other target first category prototypes may be, for each newly added category, the target first category prototypes of all other newly added categories except the target first category prototype of the current newly added category.

[0172] The fourth distance can be the distance between the second-category prototype of each newly added category and the first-category prototypes of other newly added categories. The fourth distance can be used to distinguish different newly added categories, ensuring that similar samples are clustered and heterogeneous samples are dispersed, thereby improving the discriminative ability of the edge model.

[0173] The distance distillation loss may be a loss value calculated based on the difference between the third distance and the fourth distance. By minimizing the distance distillation loss, the target second category prototype can be made as close as possible to the target first category prototype while maintaining a sufficient distance from other target first category prototypes.

[0174] In some embodiments, multiple target first category prototypes can be received through the edge end, and the distance between the target first category prototype and the target second category prototype of the intermediate edge model can be reduced by back propagation, so that the target first category prototype and the target second category prototype are aligned, so that the target edge model obtained by distillation has a classification performance comparable to that of the target cloud model. In some embodiments, the distance distillation loss calculated during the distillation process is It can be calculated as follows:

[0175] ;

[0176] in, Indicates the current training task The total number of categories of newly added categories included in , is the target first category prototype corresponding to the newly added category of this training, is the target second category prototype corresponding to the newly added category of this training, The target first category prototypes of other newly added categories for this training are: is a hyperparameter set according to actual conditions.

[0177] The distance distillation loss calculated using the above formula encourages the intermediate edge model's target second-category prototype to approach the target first-category prototype of the target cloud model for the same newly added category (e.g., car), and to distance itself from the target first-category prototype of the target cloud model for different newly added categories (e.g., bicycle, tractor). After the edge receives multiple target first-category prototypes corresponding to multiple newly added categories, it can use these multiple target first-category prototypes to adjust the parameters of the intermediate edge model. Specifically, the intermediate edge model adjusts its parameters by minimizing the distance distillation loss function to continuously optimize its classification performance.

[0178] By receiving the target first-category prototypes trained by the cloud server at the edge, the edge can effectively adjust the parameters of the intermediate edge model based on these category prototypes, thereby obtaining an optimized target edge model. This process not only enables efficient transfer of cloud model knowledge, but also significantly improves the edge model's recognition accuracy and robustness for newly added categories in resource-constrained environments, thereby enhancing the real-time responsiveness and deployment flexibility of the entire system.

[0179] In some embodiments, to further improve the classification accuracy of the cloud model and the edge model, the edge model can be used to filter out samples that are difficult to classify, so as to more quickly identify samples that require further analysis. The cloud model can also be trained based on fuzzy samples and then distilled again to effectively improve the classification accuracy and data sensitivity of the cloud model and the edge model. For example, after step 203, the following steps may also be included:

[0180] (B.1) For each newly added category, perform classification tests on multiple test samples contained in the test sub-dataset corresponding to each newly added category using the target edge model, and obtain the uncertainty index of each test sample under multiple newly added categories;

[0181] (B.2) Determine multiple classification fuzzy samples based on multiple uncertainty indices corresponding to multiple test samples, and send the multiple classification fuzzy samples to the cloud server, so that the cloud server performs comparative training on the target cloud model based on the multiple target training samples and the multiple classification fuzzy samples corresponding to each newly added category, the target first category prototype corresponding to each newly added category, and multiple historical category prototypes obtained from any historical task, to obtain multiple updated first category prototypes;

[0182] (B.3) Obtain multiple updated first-category prototypes sent by the cloud server, and adjust parameters of the target edge model based on the multiple updated first-category prototypes to obtain an updated edge model.

[0183] The uncertainty index may be an indicator for measuring the degree of classification uncertainty of each test sample under multiple newly added categories, and may be used to reflect the overall classification uncertainty of the edge model for a test sample under multiple newly added categories.

[0184] The updated edge model may be a target edge model after parameter adjustment.

[0185] In some embodiments, in order to quickly screen out classification fuzzy samples, multiple classification fuzzy samples can also be classified and tested on multiple test samples contained in the test sub-dataset corresponding to each new category through a preset edge model or an intermediate edge model, and multiple uncertainty indexes of each test sample under multiple new categories are determined.

[0186] In some implementations, in order to effectively avoid catastrophic forgetting during training, an updated dataset can be formed on the cloud server based on multiple target training samples and multiple classification fuzzy samples for each newly added category, and compared with the cross-task prototype (i.e., the prototype stored in the prototype set after the historical task training is completed). For example, for each sample data in the updated dataset corresponding to each newly added category, after the sample data is input into the target cloud model, the corresponding feature representation can be obtained. , through contrast training, we can narrow Target first category prototype corresponding to the same newly added category The distance between Multiple history category prototypes stored with history tasks Specifically, the category gap loss corresponding to this comparative training is The calculation formula is as follows ( are preset hyperparameters):

[0187] ;

[0188] Furthermore, the cloud server can use the above-mentioned category gap loss Backpropagation is performed to update the parameters of the target cloud model until the model converges or reaches the preset number of training rounds, and the final updated cloud model can be obtained. In this way, the newly added categories can be more effectively identified while maintaining the recognition ability of existing categories.

[0189] Furthermore, since the target edge model cannot accurately classify the classification fuzzy samples, the updated first category prototype obtained by updating the cloud model can be distilled into the target edge model, so that the target edge model can adjust the parameters based on the updated first category prototype to achieve rapid alignment of category prototypes. Specifically, the distillation loss of this process is The formula is as follows:

[0190] ;

[0191] in, Indicates the current training task The total number of categories of newly added samples in , Represents the target third category prototype corresponding to the target edge model, Indicates the updated first category prototype corresponding to each newly added category currently being trained (for example, the newly added category currently being trained is truck). Indicates the updated first prototype of new categories other than the currently trained new categories (such as new categories such as cars and bicycles).

[0192] The distillation loss calculated using the above formula encourages the updated third-category prototype of the edge model to be close to the updated first-category prototype of the cloud model for the same newly added category (e.g., car), and away from the updated first-category prototype of the cloud model for different newly added categories (e.g., bicycle, tractor). After the edge receives multiple updated first-category prototypes corresponding to multiple newly added categories, it can use these updated first-category prototypes to adjust the parameters of the target edge model. Specifically, the edge model adjusts its parameters by minimizing the distance distillation loss function to continuously optimize its classification performance.

[0193] In some implementations, to improve the model's accuracy in classifying newly added categories, a detailed evaluation of the edge model's classification uncertainty for each test sample can be performed to identify key samples that the model has difficulty accurately classifying. This allows for continuous improvement and optimization of the entire system, ensuring that the model can still adjust and improve its classification capabilities in a timely manner when faced with newly added categories. For example, (B.1) may include:

[0194] (B.1.1) For each newly added category, using the target edge model, calculate the Euclidean distance between each test sample contained in the test sub-dataset corresponding to each newly added category and the multiple second category prototypes corresponding to the multiple newly added categories;

[0195] (B.1.2) For each test sample, calculate the logarithm of each corresponding Euclidean distance, and obtain the uncertainty measure of each test sample in the corresponding second category prototype based on the product of the Euclidean distance and the corresponding logarithm;

[0196] (B.1.3) Based on the multiple uncertainty measures corresponding to the multiple second-category prototypes, obtain the uncertainty index of each test sample under the multiple newly added categories.

[0197] The Euclidean distance may be the distance between each test sample and multiple second category prototypes corresponding to multiple newly added categories.

[0198] The logarithm of the Euclidean distance may be the natural logarithm or the common logarithm of the calculated Euclidean distance.

[0199] The uncertainty measure may be an indicator calculated based on the Euclidean distance and its logarithmic value between each test sample and multiple second category prototypes, and is used to characterize the degree of uncertainty of the target edge model in classifying the test sample.

[0200] In some embodiments, for any test sample X in the test set, the corresponding feature representation can be obtained by the target edge model: ,calculate Second category prototypes corresponding to each newly added category Euclidean distance between ,in, represents the number of second-category prototypes corresponding to all newly added categories. Furthermore, the uncertainty measure of each test sample in the corresponding second-category prototype is .

[0201] Furthermore, based on the multiple uncertainty measures corresponding to the multiple second category prototypes, the uncertainty index of each test sample under multiple new categories can be obtained: :

[0202] ;

[0203] in, Represents the Euclidean distance between each test sample and the corresponding second category prototype.

[0204] Furthermore, for each test sample in the test sub-dataset, the corresponding uncertainty index can be calculated using the above formula, and all test samples are sorted from large to small according to their corresponding uncertainty indexes, and a preset proportion of test samples are selected as classification fuzzy samples. The preset proportion can be 15%, 20%, etc., and can be set according to actual conditions. For example, if 100 test samples are sorted from large to small according to their uncertainty index, 20% of the test samples can be selected as classification fuzzy samples, that is, the first 20 test samples are selected as classification fuzzy samples.

[0205] By calculating the uncertainty index of each test sample to screen and classify fuzzy samples, the uncertainty of the model's classification of each test sample can be effectively quantified, ensuring that samples that are difficult for the model to classify are sent to the cloud for further analysis and learning, so as to facilitate the subsequent optimization of the cloud model and edge model's adaptability to newly added categories, thereby improving the accuracy and robustness of the model in practical applications.

[0206] Please refer to Figure 4 and Figure 5 , in some embodiments, Figure 4 and Figure 5 Two schematic diagrams of the overall embodiment of this application are now combined with Figure 4 and Figure 5, an overall embodiment of the present application is introduced. Exemplarily, for each newly added category in the current training phase, the model is trained in two main phases. Specifically, in the first phase, for each newly added category in the current training task, the corresponding training sub-dataset can be used to train the preset cloud model and the preset edge model separately. On the cloud server, the preset cloud model is optimized through comparative learning, and the first category prototype (the center point of the feature space) of each newly added category is calculated to generate a target cloud model and an optimized target first category prototype. On the edge, the preset edge model is comparatively trained based on the same training sub-dataset to generate an intermediate edge model and the corresponding second category prototype, thereby giving the model the ability to recognize newly added categories. Since the cloud server has strong computing power and can train the model more thoroughly, after the target cloud model is trained on the cloud server, the intermediate edge model can be distilled using the target first category prototype. The parameters of the intermediate edge model can be adjusted using the distance distillation loss to align the second category prototype on the edge with the target first category prototype on the cloud, while increasing the distance from the prototypes of other categories to obtain the target edge model, thereby achieving efficient migration of high-level knowledge from the cloud to the edge model.

[0207] Furthermore, the target edge model can classify the test samples in the test sub-dataset, calculate the uncertainty index based on the Euclidean distance between the test samples and the second category prototype, screen out the samples with the lowest classification confidence as classification fuzzy samples, and upload them to the cloud server for retraining. Specifically, on the cloud server, a new data set can be composed based on the uploaded classification fuzzy samples and the original training samples, and cross-task comparative training can be performed on the target cloud model. During the comparative training process, the newly added category features are decoupled from the historical category features by optimizing the category distance loss, and an updated cloud model and an optimized updated first category prototype are generated. The updated first category prototype of the updated cloud model is then transferred to the target edge model through knowledge distillation, and the final updated edge model is obtained through parameter adjustment, thereby further improving the model's ability to recognize new categories.

[0208] Furthermore, the updated cloud model and edge model can be tested to evaluate the recognition accuracy of new categories and the ability to retain old categories to ensure that the model performance meets the requirements.

[0209] It is understandable that the training process for each new category is an iterative cycle. Until all new categories of the current training task are trained, the model can accurately identify new and old categories. If there are new categories that have not been trained, the above two-stage process is repeated until all new categories are trained. In this way, the model can continuously learn and adapt to new categories in a constantly changing environment while maintaining the ability to recognize old categories. This model collaborative update technology can be widely used in fields such as smart transportation and intelligent security.

[0210] See also Figure 6 The embodiment of the present application further provides a model updating device based on incremental data, which can implement the above-mentioned model updating method based on incremental data. The model updating device based on incremental data is applied to a cloud service end, and the model updating device based on incremental data includes:

[0211] An acquisition module 61 is used to obtain a training sub-dataset corresponding to each newly added category;

[0212] Training module 62 is configured to perform comparative training on a preset cloud model for each newly added category based on a plurality of target training samples contained in a corresponding training sub-dataset and a plurality of first-category prototypes corresponding to the plurality of newly added categories, thereby obtaining a target cloud model and a plurality of corresponding target first-category prototypes; wherein the first-category prototype of each newly added category is obtained by feature averaging of the preset cloud model based on the corresponding training sub-dataset;

[0213] The first sending module 63 is used to send multiple target first category prototypes to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain the target edge model; wherein, the intermediate edge model is obtained by the edge end performing comparative training on the category gap of the preset edge model based on multiple target training samples and multiple second category prototypes corresponding to multiple newly added categories; the second category prototype of each newly added category is obtained by the preset edge model according to the corresponding training sub-dataset by averaging the features.

[0214] The specific implementation of the incremental data-based model updating device is substantially the same as the specific implementation of the incremental data-based model updating method described above, and will not be described in detail here. Subject to meeting the requirements of the embodiments of the present application, the incremental data-based model updating device may further include other functional modules to implement the incremental data-based model updating method described above.

[0215] The present application also provides a computer device comprising a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned incremental data-based model update method. The computer device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0216] See also Figure 7 , Figure 7 The hardware structure of a computer device according to another embodiment is shown. The computer device includes:

[0217] The processor 71 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0218] The memory 72 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 72 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 72 and is called by the processor 71 to execute the incremental data-based model update method of the embodiments of this application.

[0219] Input / output interface 73, used for information input and output;

[0220] Communication interface 74, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0221] bus 75 , which transmits information between the various components of the device (e.g., processor 71 , memory 72 , input / output interface 73 , and communication interface 74 );

[0222] The processor 71 , the memory 72 , the input / output interface 73 and the communication interface 74 are connected to each other in communication within the device via a bus 75 .

[0223] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned model updating method based on incremental data.

[0224] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0225] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0226] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0227] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0228] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0229] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0230] It should be understood that in this application, "at least one (item)" and "several" refer to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0231] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0232] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0233] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0234] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0235] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A model updating method based on incremental data, characterized in that: Applied to a cloud server, the method includes: Get the training sub-dataset corresponding to each newly added category; For each newly added category, the preset cloud model performs feature averaging based on multiple training samples included in the corresponding training sub-dataset to obtain the corresponding first category prototype; the preset cloud model obtains the first feature representation corresponding to each training sample, and determines the first distance between each first feature representation and the corresponding first category prototype; obtains the second distance between each first feature representation and other first category prototypes of other newly added categories; determines the first category gap loss based on the difference between the first distance and the second distance; adjusts the parameters of the preset cloud model based on the first category gap loss to obtain a target cloud model and corresponding multiple target first category prototypes; wherein, the first category prototype of each newly added category is obtained by the preset cloud model performing feature averaging based on the corresponding training sub-dataset; The multiple target first category prototypes are sent to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain the target edge model; wherein, the intermediate edge model is obtained by the edge end performing comparative training on the category gap of the preset edge model based on the multiple target training samples and the multiple second category prototypes corresponding to the multiple newly added categories; the comparative training on the category gap is performed by the edge end on the preset edge model to reduce the distance between each target training sample of the same newly added category and the corresponding second category prototype, and at the same time increase the distance between each target training sample and the second category prototype of different newly added categories for the purpose of comparative learning; the second category prototype of each newly added category is obtained by the preset edge model performing feature averaging based on the corresponding training sub-dataset.

2. The model updating method based on incremental data according to claim 1, characterized in that: After sending the plurality of target first category prototypes corresponding to the target cloud model to the edge end so that the edge end adjusts the parameters of the intermediate edge model based on the plurality of target first category prototypes to obtain the target edge model, the method further includes: For each newly added category, multiple classification fuzzy samples uploaded by the edge are obtained, and based on the corresponding multiple target training samples and the multiple classification fuzzy samples, comparative training is performed on the target first category prototype corresponding to each newly added category and multiple historical category prototypes obtained from any historical task to obtain an updated cloud model and the corresponding multiple updated first category prototypes; The multiple classification fuzzy samples are subjected to classification tests on multiple test samples contained in the test sub-dataset corresponding to each newly added category through the target edge model, and multiple uncertainty indices of each test sample obtained under the multiple newly added categories are determined; The multiple updated first category prototypes are sent to the edge end, so that the edge end adjusts the parameters of the target edge model based on the multiple updated first category prototypes to obtain an updated edge model.

3. A model updating method based on incremental data, characterized in that: Applied to the edge, the method includes: Get the training sub-dataset corresponding to each newly added category; For each newly added category, based on multiple target training samples contained in the corresponding training sub-dataset and multiple second category prototypes corresponding to the multiple newly added categories, a preset edge model is subjected to comparative training of category gaps to obtain an intermediate edge model and the corresponding multiple target second category prototypes; the comparative training of category gaps is to train the preset edge model to reduce the distance between each target training sample of the same newly added category and the corresponding second category prototype, while increasing the distance between each target training sample and the second category prototypes of different newly added categories for the purpose of comparative learning; wherein, the second category prototype of each newly added category is obtained by the preset edge model by performing feature averaging based on the corresponding training sub-dataset; Receive multiple target first category prototypes sent by the cloud server, and adjust the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain a target edge model, wherein the target first category prototype is obtained by the cloud server through parameter adjustment of the preset cloud model based on the first category gap loss; the first category gap loss is determined based on the difference between the first distance and the second distance; the first distance between each first feature representation and the corresponding first category prototype is determined after obtaining the first feature representation corresponding to each training sample through the preset cloud model; the second distance between each first feature representation and other first category prototypes of other newly added categories is obtained by the cloud server; the first category prototype of each newly added category is obtained by the preset cloud model through feature averaging based on multiple training samples contained in the corresponding training sub-dataset.

4. The model updating method based on incremental data according to claim 3, characterized in that: The method further includes: receiving a plurality of target first category prototypes sent by the cloud service end, and adjusting parameters of the intermediate edge model based on the plurality of target first category prototypes to obtain the target edge model; For each of the newly added categories, a classification test is performed on a plurality of test samples included in a test sub-dataset corresponding to each of the newly added categories using the target edge model to obtain an uncertainty index of each test sample under the plurality of newly added categories; Determine a plurality of classification fuzzy samples according to a plurality of uncertainty indices corresponding to the plurality of test samples, and send the plurality of classification fuzzy samples to a cloud server, so that the cloud server performs comparative training on a target cloud model based on the plurality of target training samples corresponding to each newly added category and the plurality of classification fuzzy samples, and on the target first category prototype corresponding to each newly added category and a plurality of historical category prototypes obtained from any historical task, to obtain a plurality of updated first category prototypes; Acquire multiple updated first category prototypes sent by the cloud server, and adjust parameters of the target edge model based on the multiple updated first category prototypes to obtain an updated edge model.

5. The model updating method based on incremental data according to claim 4, characterized in that: For each of the newly added categories, performing classification tests on multiple test samples included in the test sub-dataset corresponding to each of the newly added categories using the target edge model to obtain an uncertainty index for each test sample under the multiple newly added categories includes: For each of the newly added categories, calculating, by using the target edge model, the Euclidean distance between each test sample included in the test sub-dataset corresponding to each of the newly added categories and a plurality of second category prototypes corresponding to the plurality of newly added categories; For each test sample, calculating a logarithmic value of each corresponding Euclidean distance, and obtaining an uncertainty measure of each test sample in the corresponding second category prototype based on a product between the Euclidean distance and the corresponding logarithmic value; Based on the multiple uncertainty measures corresponding to the multiple second category prototypes, an uncertainty index of each test sample under the multiple newly added categories is obtained.

6. The model updating method based on incremental data according to claim 3, characterized in that: The receiving of multiple target first category prototypes sent by the cloud service end and adjusting parameters of the intermediate edge model based on the multiple target first category prototypes to obtain the target edge model includes: For each newly added category, obtaining a corresponding target second category prototype and a target first category prototype, and determining a third distance between the target second category prototype and the target first category prototype; Obtaining first-category prototypes of other newly added targets, and determining a fourth distance between the second-category prototype of the target and the first-category prototype of the other target; determining a distance distillation loss based on a difference between the third distance and the fourth distance; Parameters of the intermediate edge model are adjusted based on the distance distillation loss to obtain a target edge model.

7. A model updating device based on incremental data, characterized in that: Applied to a cloud server, the device includes: The acquisition module is used to obtain the training sub-dataset corresponding to each newly added category; A training module is configured to, for each newly added category, perform feature averaging on a plurality of training samples included in a corresponding training sub-dataset by a preset cloud model to obtain a corresponding first-category prototype; obtain a first feature representation corresponding to each training sample through the preset cloud model, and determine a first distance between each first feature representation and the corresponding first-category prototype; obtain a second distance between each first feature representation and other first-category prototypes of other newly added categories; determine a first-category gap loss based on a difference between the first distance and the second distance; and adjust parameters of the preset cloud model based on the first-category gap loss to obtain a target cloud model and a plurality of corresponding target first-category prototypes; wherein, the first-category prototype of each newly added category is obtained by performing feature averaging on the preset cloud model based on the corresponding training sub-dataset; A first sending module is used to send the multiple target first category prototypes to the edge end, so that the edge end adjusts the parameters of the intermediate edge model based on the multiple target first category prototypes to obtain a target edge model; wherein, the intermediate edge model is obtained by the edge end performing comparative training on the category gap of the preset edge model based on multiple target training samples and multiple second category prototypes corresponding to multiple newly added categories; the comparative training on the category gap is performed by the edge end on the preset edge model to reduce the distance between each target training sample of the same newly added category and the corresponding second category prototype, and at the same time increase the distance between each target training sample and the second category prototype of different newly added categories for the purpose of comparative learning; the second category prototype of each newly added category is obtained by the preset edge model performing feature averaging based on the corresponding training sub-dataset.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the incremental data-based model updating method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the model updating method based on incremental data according to any one of claims 1 to 6 is implemented.

Images