Edge heterogeneous model training method and device, equipment, storage medium and program product

Abstract

The invention discloses an edge heterogeneous model training method and device, equipment, a storage medium and a program product, edge equipment calculates the first feature similarity of any two local data categories in a local data set, and then sends the first feature similarity to other edge equipment for synchronization; second feature similarities sent by other edge devices are received, the global similarity is obtained after the two similarities are summarized, a distributed external sample generator is trained based on the global similarity, and then a virtual sample data set is constructed; and training the edge heterogeneous model by using the local data set and the virtual sample data set to obtain a trained edge heterogeneous model. According to the method, the personalized virtual sample generator can be configured for each edge device, so that the virtual distribution external sample adaptive to the dimension of the edge heterogeneous model is generated, the virtual distribution external sample is combined with the local data set to train the model, and the reasoning capability of the edge heterogeneous model on the original data is ensured while the distribution external detection precision is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method, apparatus, device, storage medium, and program product for training edge heterogeneous models. Background Technology

[0002] Out-of-distribution (OoD) samples are those whose distribution (pattern) is completely different from that of the model's training data; the model has neither learned from them nor can it deduce them from its existing knowledge. OoD identification and processing capabilities are of significant value in edge computing applications in commercial and industrial fields and have attracted widespread attention in recent years. For example, IoT cameras on highways are mainly used to collect vehicle images, cameras on sidewalks primarily capture pedestrian images, and cameras used for wildlife monitoring mainly record animal images. Due to significant differences in computing power and storage capacity among edge devices, the models deployed on these devices are often inconsistent in type and scale. These models are mostly designed for specific tasks and pre-trained or fine-tuned based on specific local datasets; therefore, their inference capabilities are usually limited to a closed set of samples predefined during the training phase. In open-world environments, the distribution of inference samples often differs from the distribution of local training data, making it difficult for local models to accurately handle out-of-distribution samples. For example, in highway scenarios, cameras typically only need to run a model dedicated to vehicle classification. However, in real-world applications, due to uncontrollable factors such as pedestrians or animals accidentally entering the frame, cameras may capture non-vehicle targets, causing the model to be unable to identify these abnormal categories. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, device, storage medium, and program product for training edge heterogeneous models. It can configure personalized virtual sample generators for each edge device to generate virtual out-of-distribution samples adapted to the local model dimensions. These samples are then combined with the local dataset to train the edge heterogeneous model, thereby improving the out-of-distribution detection accuracy while ensuring the reasoning ability of the local edge heterogeneous model on the original data.

[0004] To achieve the above objectives, embodiments of the present invention provide a method for training edge heterogeneous models, comprising: Calculate the first feature similarity between any two local data categories in the local dataset; The first feature similarity is sent to the remaining edge devices, and the second feature similarity sent by the remaining edge devices is received. The global similarity is obtained by summing the similarity of the first feature and the similarity of the second feature. An out-of-distribution sample generator is trained based on the global similarity, and a virtual sample dataset is constructed using the trained out-of-distribution sample generator. The edge heterogeneous model is trained using the local dataset and the virtual sample dataset to obtain the trained edge heterogeneous model.

[0005] As an improvement to the above scheme, the off-distribution sample generator trained based on the global similarity includes: Initialize the out-of-distribution sample generator; wherein the out-of-distribution sample generator takes the one-hot vector of the target data category in the specified dataset and Gaussian noise as input, the specified dataset includes all local data categories in the local dataset, and the identification information of the out-of-distribution data category to be generated; The loss function of the out-of-distribution sample generator is constructed based on the global similarity. The out-of-distribution sample generator is iteratively trained and its parameters are optimized until the loss value output by the loss function is less than a preset first loss threshold, at which point the trained out-of-distribution sample generator is obtained.

[0006] As an improvement to the above scheme, the loss function for constructing the out-of-distribution sample generator based on the global similarity includes: Calculate the second similarity between the virtual out-of-distribution samples output by the out-of-distribution sample generator and each local data category in the local dataset; Obtain the target similarity from the global similarity; wherein the target similarity is the similarity value of the category pair corresponding to the second similarity, and the category pair consists of the out-of-distribution data category and the local data category; The loss function of the out-of-distribution sample generator is constructed based on the second similarity and the target similarity.

[0007] As an improvement to the above scheme, the step of constructing a virtual sample dataset using a trained out-of-distribution sample generator includes: Use a trained out-of-distribution sample generator to generate virtual out-of-distribution samples; Calculate the sample mean and sample covariance of the samples outside the virtual distribution; A Gaussian distribution model is constructed using the sample mean and the sample covariance; Multiple out-of-target-distribution samples are sampled from the Gaussian distribution model according to preset sampling probability values, and these multiple out-of-target-distribution samples are integrated to obtain a virtual sample dataset.

[0008] As an improvement to the above scheme, the step of training the edge heterogeneous model using the local dataset and the virtual sample dataset includes: The local dataset and the virtual sample dataset are used as joint training data, and the joint training data is input into the edge heterogeneous model; Construct a joint loss function that includes both in-distribution sample loss and out-of-distribution sample loss; The edge heterogeneous model is iteratively trained and its parameters are optimized until the loss value output by the joint loss function is less than a preset second loss threshold, at which point the iterative training of the edge heterogeneous model is stopped.

[0009] As an improvement to the above scheme, after stopping the iterative training of the edge heterogeneous model, the method further includes: Obtain test set data; wherein, the test set data includes samples of the same type as the joint training data, samples from the same reference dataset as the local dataset but not belonging to the joint training data, and samples from other datasets other than the reference dataset; The test set data is input into the trained edge heterogeneous model to obtain the test results; The processing capability of the edge heterogeneous model is evaluated based on the test results.

[0010] To achieve the above objectives, embodiments of the present invention also provide an edge heterogeneous model training device, comprising: The first feature similarity calculation module is used to calculate the first feature similarity between any two local data categories in the local dataset. The first feature similarity sending module is used to send the first feature similarity to the other edge devices; The second feature similarity receiving module is used to receive the second feature similarity sent by other edge devices; A global similarity generation module is used to summarize the first feature similarity and the second feature similarity to obtain a global similarity. An out-of-distribution sample generator training module is used to train an out-of-distribution sample generator based on the global similarity. The virtual sample dataset construction module is used to construct a virtual sample dataset using a trained out-of-distribution sample generator. The edge heterogeneous model training module is used to train the edge heterogeneous model using the local dataset and the virtual sample dataset to obtain the trained edge heterogeneous model.

[0011] To achieve the above objectives, embodiments of the present invention also provide an edge heterogeneous model training device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the edge heterogeneous model training method as described in any of the above embodiments.

[0012] To achieve the above objectives, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the edge heterogeneous model training method as described in any of the above embodiments.

[0013] To achieve the above objectives, embodiments of the present invention also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the edge heterogeneous model training method as described in any of the above embodiments.

[0014] Compared to existing technologies, this invention discloses a method, apparatus, device, storage medium, and program product for training edge heterogeneous models. It can configure personalized virtual sample generators for each edge device, form a global supervision signal by sharing category feature similarity among edge devices, and generate virtual out-of-distribution samples adapted to the local model dimension. These samples are then combined with the local dataset to train the edge heterogeneous model. The generator and model parameters are alternately optimized. This not only breaks through the heterogeneity limitation of edge models and solves the problems of insufficient sample representativeness and suitability only for homogeneous scenarios in traditional methods, but also improves the out-of-distribution detection accuracy while ensuring the reasoning ability of the edge heterogeneous model on the original data and adapting to the resource constraints of edge devices. Attached Figure Description

[0015] Figure 1 This is a flowchart of a method for training an edge heterogeneous model provided in an embodiment of the present invention; Figure 2 This is another flowchart of a method for training an edge heterogeneous model provided in an embodiment of the present invention; Figure 3 This is a flowchart of the training distribution out-of-distribution sample generator provided in an embodiment of the present invention; Figure 4 This is a flowchart of iterative training of an edge heterogeneous model provided in an embodiment of the present invention; Figure 5 This is a flowchart of testing an edge heterogeneous model provided in an embodiment of the present invention; Figure 6 This is a structural block diagram of an edge heterogeneous model device provided in an embodiment of the present invention; Figure 7 This is a structural block diagram of an edge heterogeneous model device provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] With the rapid development of edge computing, edge devices are widely deployed in scenarios such as smart manufacturing, autonomous driving, smart cities, security monitoring, and telemedicine. These scenarios involve diverse device types and model structures, making traditional out-of-distribution detection schemes that rely on homogeneous models ill-suited to practical needs. This invention focuses on the more prevalent out-of-distribution detection problem under heterogeneous model conditions, equipping each edge device with a lightweight, personalized out-of-distribution sample generator, enabling its widespread application in the aforementioned various real-world scenarios.

[0018] Consider M reference datasets, denoted as Mi, Mj, ... The m-th reference dataset contains Each data category is represented as Each data category is a classification identifier for a data sample. For example, in a traffic scenario dataset, the data categories could be cars, trucks, SUVs, etc.; in an industrial quality inspection dataset, the data categories could be qualified products, defect type A, defect type B, etc. The set of data categories for all reference datasets can be represented as... The edge system contains N edge devices (N>M), indexed by n. Each edge device's local dataset is a reference dataset (i.e., ...). A subset of one of them, namely the local dataset of edge device n. From the reference dataset ,satisfy In addition, the local dataset of edge device n Only contains Samples of some data categories in the middle, denoted as This also reflects the locality of data from edge devices, meaning that each edge device can only access a portion of the data categories within its local dataset.

[0019] In existing technologies, the classification function corresponding to the local model of edge device n can be expressed as: , For the sample space, For output space, This indicates that an input sample has been received. and for each The class calculates and generates a score. Local model of edge device n. This can be further expressed as: (1); in, This means that sample x is processed through the feature embedding function. Mapped to 3D feature embedding space, Model parameters are used This is used to define how input samples are mapped to the feature embedding space; For model parameters, Its function is to use this feature to embed the space. Received Dimensional feature embedding, to provide Each class generates a score, i.e. It should be noted that, due to model heterogeneity, the feature embedding dimensions vary between different edge devices. There may be significant differences.

[0020] According to the above formula (1), given a sample , where x is the feature representation of the sample, and y is the label of the sample; let This represents a single loss function for a single sample. All edge devices use their local datasets. To update local model parameters and If the model is a neural network, It can represent bias. It can represent weights, etc., and the specifics depend on the model structure. The process of constructing the loss function and updating the model parameters is shown in the following formulas (2) to (4): (2); (3); (4); in, This indicates that edge device n is in the local dataset. The overall loss function on the local dataset; Only contains A small number of tags in, denoted as ; and These are the updated model parameters; Represents the loss function For parameters The gradient represents The trend of loss changes during the change; Represents the loss function For parameters The gradient, similarly expressed The trend of loss changes during the change; The learning rate, or hyperparameter, controls the step size for parameter updates. Equation (2) indicates that the generalization error of the model needs to be approximated by the training set loss. Therefore, the single-sample loss is averaged on the local dataset to obtain the overall loss. Both formulas (3) and (4) use the gradient descent algorithm to update the parameters.

[0021] During training, neural networks learn training labels. The characteristics and patterns of in-vivo samples are analyzed to classify these samples more accurately. However, due to the overconfidence of neural networks, when a local model encounters a label that does not belong to... When a new sample (i.e., an out-of-distribution sample, such as a car that was only trained on animals) is encountered, the model does not actually have the knowledge to classify such samples. Although it cannot classify correctly, the neural network mechanism will force it to output a classification result and give a high confidence score, even if the classification is wrong. For example, when faced with a "car" sample, the model may force it to be classified as an "elephant" and give a high confidence score. Therefore, it is crucial to give the local model the ability to perform OoD detection, which helps the local model learn to distinguish between OoD samples and in-distribution samples. To achieve this goal, centralized OoD methods usually train the local model by introducing an external dataset. For example, the standard cross-entropy loss function is replaced by a regularized loss based on the energy definition (as shown in Equation (5)) for model training, thereby replacing the original loss function (Equation (2)).

[0022] (5); in, It is an external dataset. For samples from external datasets, This represents the regularization loss defined by energy. This is the regularization coefficient.

[0023] However, in resource-constrained edge environments, acquiring external datasets often faces significant challenges. To address this, this invention provides a method for training heterogeneous edge models based on a personalized out-of-distribution sample generator. This method designs an out-of-distribution sample generator for each edge device and enhances the training signal of the generator by sharing similarity information between feature embeddings of different categories across multiple edge devices. This out-of-distribution sample generator is then used to generate external datasets. Simultaneously, by fully utilizing the computing power and collaborative mechanisms of multiple edge devices, the performance of out-of-distribution detection is improved. The generated virtual out-of-distribution samples are combined with the in-distribution training set (i.e., the local dataset) to jointly train the local model. This allows the local model to not only effectively identify out-of-distribution samples but also maintain its reasoning ability regarding the original data distribution. This successfully overcomes the problem of insufficient representativeness of virtual samples generated by centralized methods and breaks through the limitation of only being applicable to homogeneous model scenarios, effectively addressing the model heterogeneity problem in edge computing scenarios.

[0024] See Figure 1 , Figure 1 This is a flowchart of an edge heterogeneous model training method provided by an embodiment of the present invention. The edge heterogeneous model training method is applied to edge devices and includes steps S1 to S5.

[0025] S1. Calculate the first feature similarity between any two local data categories in the local dataset.

[0026] For example, suppose the edge device n is a smart camera on a certain section of highway, then its local dataset The data categories can include cars, trucks, SUVs, etc. It is necessary to calculate pairwise similarity among these three data categories. It should be noted that calculating pairwise similarity between data categories is primarily to obtain the feature associations between different categories, and to use this globally agreed-upon relationship to assist subsequent tasks (such as out-of-distribution sample generation, model training, etc.).

[0027] For example, each edge device computes its local dataset. Any two data categories and The cosine similarity between the average feature embeddings is used to measure the distance between the two categories, resulting in the first feature similarity. The calculation process for the first feature similarity is as follows: (6); (7); in, Indicates the similarity of the first feature; This represents the cosine similarity metric. and Representing data categories respectively and The average feature embedding is calculated using formula (7). This is the number of samples labeled as category c. The definition is the same as in formula (1). It should be noted that the average feature embedding... This refers to the local dataset for edge device n. For all samples with the label of category c, first use the feature embedding function. Extract the feature embeddings of each sample, and then take the average of these feature embedding vectors. The result is the average feature embedding of category c, which represents the "central" or "typical" feature representation of the features of category c samples. By calculating the cosine similarity between the average feature embeddings of different categories, we can measure the degree of similarity between different categories in the feature space.

[0028] S2. Send the first feature similarity to the remaining edge devices, and receive the second feature similarity sent by the remaining edge devices.

[0029] For example, see Figure 2 , Figure 2 This is another flowchart of an edge heterogeneous model training method provided in an embodiment of the present invention. Each edge device broadcasts its first feature similarity information from its local dataset to the other edge devices, and receives the second feature similarity sent by the other edge devices. The calculation process of the second feature similarity is the same as the calculation process of the first feature similarity.

[0030] For example, after each edge device completes the first feature similarity calculation of its local dataset, it encapsulates its first feature similarity information into structured data consisting of "device identifier + category pair + similarity value" through a preset communication protocol, such as MQTT (Message Queuing Telemetry Transport) or P2P (point to point) communication between edge nodes, and then broadcasts it to other devices in the network. For instance, edge device 1 calculates the first feature similarity of "car-truck (category pair)" to be 0.6 and the first feature similarity of "car-SUV" to be 0.8, and sends information such as "device 1_car-truck_0.6" and "device 1_car-SUV_0.8" to edge devices 2 and 3. Edge devices 2 and 3 listen to and receive the structured data sent by edge device 1 through the network interface, and classify and store it according to "sending device ID" and "category pair".

[0031] S3. Summarize the first feature similarity and the second feature similarity to obtain the global similarity.

[0032] For example, due to differences in the feature embedding dimensions of different edge devices, feature embeddings from other edge devices cannot be used directly. To compute and compare these feature embeddings with different dimensions, they need to be projected into the same dimensional space. Therefore, this invention utilizes cosine similarity between feature embeddings from different categories to guide the training of the out-of-distribution sample generator. Each edge device aggregates the first feature similarity and the second feature similarity to obtain global cosine similarity information, the calculation of which satisfies the following formula: (8); in, Indicates data category and Global similarity between them; This indicates that for all items satisfying the "category" and Both belong to edge devices Local category collection "edge devices" The categories calculated on these devices and Similarity between Perform summation; Indicates edge device The categories calculated above and The similarity between them; This indicates that for all items satisfying the "category" and Both belong to edge devices Local category collection "edge devices" The number is counted, that is, how many edge devices simultaneously contain the category. and .

[0033] It should be noted that the core constraint of formula (8) is that it only applies to category pairs that coexist in a local category of a certain device. That is, through the collaboration of multiple edge devices, the local similarities calculated for devices that simultaneously contain both categories are fused (averaged) to obtain a global consensus on category similarity. Assume that the local categories of edge device 1 are sedan, truck, and SUV (all of which are in-distribution categories) and do not include pedestrians; the local categories of edge device 2 include pedestrians (which are out-of-distribution categories for edge device 1) but do not include "sedan". That is, sedans and pedestrians do not exist simultaneously in the local category set of any device. In this case, the denominator of formula (8) (the number of devices that simultaneously contain both pedestrian and sedan categories) will be 0, and the global similarity between the two cannot be calculated.

[0034] S4. Train an out-of-distribution sample generator based on the global similarity, and use the trained out-of-distribution sample generator to construct a virtual sample dataset.

[0035] For example, if there is a similarity between pedestrians and cars in the global consensus, assuming the similarity is 0.2, then when training the out-of-distribution sample generator of edge device 1, the similarity between the generated "virtual pedestrian samples" and "cars" can be constrained to approach 0.2, ensuring that the virtual samples conform to the feature relationships between categories in the real scene. After completing the training of the out-of-distribution sample generator, a virtual sample dataset can be constructed.

[0036] Further, see Figure 3 , Figure 3 This is a flowchart of the training distribution out-of-distribution sample generator provided in the embodiment of the present invention, wherein step S4 specifically includes steps S41 to S43.

[0037] S41. Initialize the out-of-distribution sample generator; wherein the out-of-distribution sample generator takes the one-hot vector of the target data category in the specified dataset and Gaussian noise as input, and the specified dataset includes all local data categories in the local dataset and the identification information of the out-of-distribution data category to be generated.

[0038] For example, out-of-distribution category identification information is used to clarify "which out-of-distribution data category the out-of-distribution sample generator needs to generate virtual samples of." Its core function is to provide the generator with a generation target, such as telling the generator whether to generate pedestrians, bicycles, or other unseen categories, avoiding the generator generating chaotic features without clear category affiliation. In technical implementation, this identification information can be represented using a one-hot vector, or it could be a category index, category name encoding, etc. For example, if the specified dataset is a global category set... , If the data includes cars, trucks, SUVs, pedestrians, and bicycles, then these categories are the target data categories; in this case, the local categories of edge device n are... Includes cars, trucks, and SUVs, distributed outside of categories. To include pedestrians and bicycles, the out-of-distribution category identification information (one-hot vector) for pedestrians can be represented as [0,0,0,1,0] (only the position corresponding to pedestrians is 1, and the rest are 0), and the out-of-distribution category identification information for bicycles is [0,0,0,0,1].

[0039] For example, the off-distribution sample generator for edge device n can be represented as , The application process for generating feature embeddings for out-of-distribution data categories satisfies the following formula: (9); in, It is the feature embedding of the out-of-distribution data category, whose dimension is the same as the feature embedding of the local dataset of the edge device n. same; This is the identification information for out-of-distribution data categories, and it is a long... The one-hot vector is used to indicate the class c to which the out-of-distribution sample to be generated belongs. The generator receives this vector. Then, based on the category identifier in the vector, the generation logic will be adjusted. For example, when receiving the one-hot vector of "pedestrian", a virtual embedding that conforms to the feature rules of "pedestrian" will be generated. ; It is Gaussian noise, used to increase the diversity of feature embeddings for the generated out-of-distribution data categories; These are the trainable parameters of the off-distribution sample generator for edge device n. Through training, the generator's generation behavior can be adjusted to generate feature embeddings that better meet the requirements. This refers to a virtual distributed dataset not included in the local dataset of edge device n. , indicating that belongs to But it does not belong to the edge device n. The category.

[0040] S42. Construct the loss function of the out-of-distribution sample generator based on the global similarity.

[0041] For example, the loss function of the out-of-distribution sample generator is used to characterize the degree of deviation between the feature embedding of the virtual out-of-distribution (OoD) data category output by the generator and the "feature association rules that out-of-distribution data categories should have" in the global similarity consensus. The core is to constrain the generator through the loss function so that the generated virtual samples not only conform to the characteristics of out-of-distribution (have reasonable differences from the local distribution categories) but also follow the global category relationship consensus determined by multi-device collaboration.

[0042] Further, step S42 specifically includes: calculating the second similarity between the virtual out-of-distribution sample output by the out-of-distribution sample generator and each local data category in the local dataset; obtaining the target similarity from the global similarity; wherein the target similarity is the similarity value of the category pair corresponding to the second similarity, and the category pair consists of the out-of-distribution data category and the local data category; and constructing the loss function of the out-of-distribution sample generator based on the second similarity and the target similarity.

[0043] For example, the loss function of the out-of-distribution sample generator satisfies the following formula: (10); (11); in, The loss function represents the out-of-distribution sample generator for edge device n; ,express For edge device n, it is an off-distribution data category; ,express For edge device n, it is the data category within the distribution, i.e., the local data category; Indicate category and The target similarity between them is determined by the global similarity aggregated from multiple edge devices, through category pairing. Obtained by indexing; The second similarity; The feature embeddings, i.e., categories, of the virtual out-of-distribution samples output by the out-of-distribution sample generator are represented. The feature embedding is calculated using formula (11), which is generated based on formula (9). For out-of-distribution data categories Identification information; The feature embedding representing the local data category in the local dataset, i.e., the category The feature embedding and its calculation process can be found in formula (7).

[0044] In this embodiment of the invention, the consensus on category relationships among multiple devices implied by global similarity can be utilized to provide precise constraints for the generation of out-of-distribution samples. This ensures that the generated virtual samples not only conform to the characteristics of out-of-distribution but also maintain a reasonable feature association with the local data categories. Furthermore, the loss function constructed based on these precise constraints can effectively guide the training of the out-of-distribution sample generator, enabling the generator to output virtual out-of-distribution samples that better match the category relationships of real-world scenarios. This provides high-quality training data for subsequent edge heterogeneous model detection of out-of-distribution samples, thereby improving the model's recognition accuracy and generalization ability for out-of-distribution samples.

[0045] S43. Iteratively train and optimize the parameters of the out-of-distribution sample generator until the loss value output by the loss function is less than the preset first loss threshold, and then obtain the trained out-of-distribution sample generator.

[0046] For example, edge device n uses the loss function described above to update This process can be implemented using the gradient descent algorithm. If the loss value is greater than or equal to the first loss threshold (which can be preset according to the actual situation), the loss is updated using the gradient descent algorithm. If the loss value is less than the first loss threshold, the iteration stops. At this point, the parameters of the out-of-distribution sample generator have been optimized to stably output virtual samples that conform to the global class relationship consensus, and training is complete.

[0047] Furthermore, The update process satisfies the following formula: (12); in, This represents the updated model parameters of the out-of-distribution sample generator, i.e., the parameters. The value after one or more updates is an intermediate or final result in the parameter optimization iteration process; The learning rate (i.e., hyperparameter) is used to control the step size of parameter updates. It determines the magnitude by which the parameters move in the opposite direction of the gradient in each iteration, affecting the speed and stability of training. Represents the loss function For parameters The gradient represents the gradient of the current parameter. At this point, the loss function varies. The trend and rate of change are key to determining the direction of parameter updates in gradient descent.

[0048] In this embodiment of the invention, by training an out-of-distribution sample generator based on global similarity, the generator can fully utilize the global category relationship consensus obtained through collaboration among multiple edge devices. Furthermore, the initialization explicitly includes local data categories and out-of-distribution category identifiers, indicating the generation direction for the generator. The loss function constructed based on global similarity effectively constrains the generator to produce virtual out-of-distribution samples that conform to the category association rules of the real scene. The finally trained generator can produce high-quality virtual out-of-distribution samples that fit global feature relationships, providing reliable data support for subsequent edge heterogeneous model training, helping the model to detect out-of-distribution samples more accurately, and improving the entire edge computing system's ability to identify and handle unknown category samples.

[0049] Further, in step S4, the step of constructing a virtual sample dataset using a trained out-of-distribution sample generator includes: generating virtual out-of-distribution samples using the trained out-of-distribution sample generator; calculating the sample mean and sample covariance of the virtual out-of-distribution samples; constructing a Gaussian distribution model using the sample mean and the sample covariance; sampling multiple target out-of-distribution samples from the Gaussian distribution model according to a preset sampling probability value, and integrating the multiple target out-of-distribution samples to obtain a virtual sample dataset.

[0050] For example, after training the out-of-distribution sample generator is complete, samples can be obtained from the out-of-distribution sample generator. Out-of-distribution samples of class c are denoted as... To ensure greater diversity of out-of-distribution samples used for training local edge heterogeneous models, all out-of-distribution samples are modeled as a class-related multivariate Gaussian distribution, the sample mean and sample covariance of which are calculated using formulas (13) and (14), respectively. Then, the Gaussian distribution model of formula (15) is constructed using the sample mean and sample covariance. Finally, multiple target out-of-distribution samples are sampled from formula (15), and the multiple target out-of-distribution samples are integrated to obtain a virtual sample dataset. This dataset will be used for training subsequent local edge heterogeneous models.

[0051] (13); (14); (15); in, The mean of the samples is used to describe the central position of the virtual feature of category c in the feature space. The feature embedding of the j-th out-of-distribution sample of category c generated by edge device n is the basic unit for calculating the mean. Sample covariance is used to describe the degree of correlation between different feature dimensions and the dispersion of features; express The transpose of the matrix; , It is the total number of out-of-distribution samples generated by analogy with all out-of-distribution samples; This represents the feature embedding of out-of-target distribution samples obtained from a multivariate Gaussian distribution, and is the sample ultimately used to construct the virtual dataset; It is a preset sampling probability value used to ensure that the feature embeddings of samples outside the target distribution are not random, and only samples with a probability density greater than 1 are retained. The sample; d represents the dimension of the feature embedding, that is, the length of each feature embedding vector.

[0052] It should be noted that formula (15) defines a multivariate Gaussian distribution using the sample mean and sample covariance as parameters. For the out-of-distribution samples (virtual features) output by the generator, samples conforming to the distribution law are selected through probability comparison. The feature embeddings of all out-of-distribution samples sampled from formula (15) together constitute the virtual sample dataset. .

[0053] In this embodiment of the invention, the trained out-of-distribution sample generator can generate preliminary virtual out-of-distribution samples, providing a foundation for subsequent operations. By calculating the sample mean and covariance, it accurately captures the statistical characteristics of the samples. The Gaussian distribution model constructed based on this model can effectively characterize the distribution pattern of samples in the feature space. Sampling from this model according to a preset sampling probability can filter out target samples that not only meet the characteristics of out-of-distribution samples but also possess high diversity. In addition, the virtual sample dataset obtained after integration can enrich the training data of edge devices, solve the problem of scarce out-of-distribution samples, help the subsequent local model learn the features of out-of-distribution samples more accurately, improve the model's ability to identify and generalize samples of unknown categories, and enhance the performance of the edge computing system in dealing with out-of-distribution samples.

[0054] S5. Train the edge heterogeneous model using the local dataset and the virtual sample dataset to obtain the trained edge heterogeneous model.

[0055] Further, see Figure 4 , Figure 4 This is a flowchart of iterative training of an edge heterogeneous model provided in an embodiment of the present invention, wherein step S5 specifically includes steps S51 to S53.

[0056] S51. Use the local dataset and the virtual sample dataset as joint training data, and input the joint training data into the edge heterogeneous model.

[0057] For example, the distributed sample dataset locally contained in the edge device Compared to the virtual distribution outsampled dataset constructed by the generator The data is merged into a joint training dataset, where local data provides references for known category features, and virtual data supplements information on unknown category features. This joint dataset is then fed into the edge heterogeneous model, allowing the model to be exposed to both in-distribution and out-of-distribution samples during training. This lays the data foundation for the model's ability to distinguish between known and unknown categories in subsequent training.

[0058] S52. Construct a joint loss function that includes both in-distribution sample loss and out-of-distribution sample loss.

[0059] For example, the joint loss function can be constructed with reference to formula (5), satisfying the following formula: (16); in, This represents the joint loss function, which consists of two parts: one part is the local dataset. The loss on the in-distribution samples ensures that the model can fit the in-distribution samples; this part can be referred to formula (2); the other part is the virtual sample dataset. Out-of-distribution sample loss Denotes out-of-distribution samples, where It is a feature representation of out-of-distribution samples. It refers to the labels (i.e., identification information) of out-of-distribution samples; the loss for out-of-distribution samples needs to be multiplied by a regularization coefficient. (Hyperparameters) guide the model to learn the characteristics of out-of-distribution samples, thereby improving the ability to identify out-of-distribution samples.

[0060] S53. Iteratively train and optimize the parameters of the edge heterogeneous model until the loss value output by the joint loss function is less than the preset second loss threshold, then stop the iterative training of the edge heterogeneous model.

[0061] For example, following the same principle as formulas (3) and (4), iterative training and parameter optimization are required for the edge heterogeneous model. The parameter optimization process is referenced in the following formulas (17) and (18): (17); (18); The meanings of the parameters in formulas (17) and (18) can be found in formulas (3) and (4) above, and will not be repeated here. In each round of collaborative training, the training of the local edge heterogeneous model and the training of the out-of-distribution sample generator are carried out alternately, that is, formulas (17), (18) and (12) are executed alternately. If the loss value output by the joint loss function is greater than or equal to the second loss threshold (which can be preset according to the actual situation), the model parameters are updated by the gradient descent algorithm. and If the loss value output by the joint loss function is less than the second loss threshold, the iteration stops. At this point, the model parameters of the edge heterogeneous model have been optimized to the optimal level, and training is complete.

[0062] In this embodiment of the invention, training the edge heterogeneous model using both local and virtual sample datasets allows the model to simultaneously learn features from both in-distribution and out-of-distribution samples, compensating for the scarcity of out-of-distribution samples on edge devices. Furthermore, constructing a joint loss function incorporating both in-distribution and out-of-distribution sample losses simultaneously constrains the model's performance in classifying known categories and detecting unknown categories. This enables the model to accurately identify out-of-distribution samples while fully fitting local data, improving its generalization and detection capabilities for unknown categories and enhancing the adaptability and reliability of edge heterogeneous systems to various types of samples in complex scenarios.

[0063] Further, see Figure 5 , Figure 5 This is a flowchart of testing an edge heterogeneous model provided in an embodiment of the present invention. After step S53 is executed, step S5 further includes steps S54 to S56.

[0064] S54. Obtain test set data; wherein, the test set data includes samples of the same type as the joint training data, samples from the same reference dataset as the local dataset but not belonging to the joint training data, and samples from other datasets other than the reference dataset.

[0065] For example, the joint training data used when training edge heterogeneous models on the test set data are different, and they include: 1) Samples of the same type as the joint training data, i.e., those similar to... The same samples are used to verify the model's basic classification accuracy for known categories trained on, ensuring that the model remembers the learned knowledge. 2) Samples that come from the same reference dataset as the local dataset but do not belong to the joint training data, i.e., samples from the same reference dataset. But not belonging to samples , recorded as ; Verify the model's ability to generalize to new categories that are from the same source but have not been trained. Since the data is from the same source as the training data and the data distribution is somewhat related, if the model can adapt well, it means that it has the ability to adapt to semi-familiar data. 3) Samples from datasets other than the reference dataset, i.e., samples from datasets other than the reference dataset. Other Samples from the remaining datasets are used to verify the robustness and generalization ability of the model to completely unfamiliar, heterogeneous data. This type of data has no direct distributional correlation with the training data and can best test the model's ability to cope with unknown scenarios. It is a key indicator of the reliability of edge heterogeneous models in complex real-world scenarios.

[0066] S55. Input the test set data into the trained edge heterogeneous model to obtain the test results.

[0067] For example, the test set data is processed according to the input format and then fed into the trained edge heterogeneous model one by one. The model performs feature extraction and inference calculation on the input samples using the optimized parameters, and outputs corresponding prediction results, including category labels for samples of the same type, classification judgments for samples of different types in the same reference dataset, and out-of-distribution detection results for samples from other datasets, forming a complete test result set.

[0068] S56. Evaluate the processing capability of the edge heterogeneous model based on the test results.

[0069] For example, key metrics can be extracted from test results, such as classification accuracy for similar samples, recognition accuracy for different samples within the same reference dataset, and out-of-distribution detection recall for samples from other datasets. By comparing these metrics with preset evaluation criteria (such as accuracy thresholds and false positive rate caps), the model's performance in fitting known classes, adapting to new samples, and generalizing to unknown distributions can be quantified to comprehensively determine whether the model's processing capability meets the standards. If it does not meet the standards, the model training phase can be reviewed, and the weighting of in-distribution / out-of-distribution losses in the joint loss function can be further analyzed (e.g., adjusting the weighting ratio). Alternatively, one could check the quality of the virtual sample dataset and retrain the out-of-distribution sample generator (e.g., by lowering the first loss threshold) to generate virtual samples that better fit the real distribution.

[0070] In this embodiment of the invention, after the edge heterogeneous model stops iterative training, three types of test set data are constructed to comprehensively cover familiar, semi-familiar, and unfamiliar data scenarios that the model may encounter in actual applications, avoiding evaluation bias caused by a single test set. Furthermore, based on the result evaluation processing capability, model performance can be verified from multiple dimensions, including "classification accuracy of known categories, adaptability to new homologous samples, and generalization across datasets." This ensures that the model maintains accurate processing of similar data that has been trained on, and also tests its ability to handle untrained homologous and heterologous samples. Ultimately, this provides a reliable basis for whether the model meets the needs of actual applications, preventing substandard models from being used and ensuring the accuracy and stability of data processing in edge computing scenarios.

[0071] See Figure 6 , Figure 6 This is a structural block diagram of an edge heterogeneous model training device 100 provided in an embodiment of the present invention. The edge heterogeneous model training device 100 includes: The first feature similarity calculation module 11 is used to calculate the first feature similarity between any two local data categories in the local dataset. The first feature similarity sending module 12 is used to send the first feature similarity to the other edge devices; The second feature similarity receiving module 13 is used to receive the second feature similarity sent by the other edge devices; The global similarity generation module 14 is used to summarize the first feature similarity and the second feature similarity to obtain the global similarity; The out-of-distribution sample generator training module 15 is used to train the out-of-distribution sample generator based on the global similarity. Virtual sample dataset construction module 16 is used to construct a virtual sample dataset using a trained out-of-distribution sample generator. The edge heterogeneous model training module 17 is used to train the edge heterogeneous model using the local dataset and the virtual sample dataset to obtain the trained edge heterogeneous model.

[0072] It is worth noting that the working process of each module in the edge heterogeneous model training device 100 described in the embodiments of the present invention can refer to the working process of the edge heterogeneous model training method described in the above embodiments, and will not be repeated here.

[0073] See Figure 7 , Figure 7 This is a structural block diagram of an edge heterogeneous model training device 200 provided in an embodiment of the present invention. The edge heterogeneous model training device 200 includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the various edge heterogeneous model training method embodiments described above.

[0074] For example, the computer program can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the edge heterogeneous model training device 200.

[0075] The edge heterogeneous model training device 200 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the edge heterogeneous model training device 200 and does not constitute a limitation on the edge heterogeneous model training device 200. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the edge heterogeneous model training device 200 may also include input / output devices, network access devices, buses, etc.

[0076] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the edge heterogeneous model training device 200, connecting all parts of the edge heterogeneous model training device 200 via various interfaces and lines.

[0077] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the edge heterogeneous model training device 200 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0078] The modules / units integrated in the edge heterogeneous model training device 200, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0079] Furthermore, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the edge heterogeneous model training method as described in any of the above embodiments.

[0080] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for training edge heterogeneous models, characterized in that, include: Calculate the first feature similarity between any two local data categories in the local dataset; The first feature similarity is sent to the remaining edge devices, and the second feature similarity sent by the remaining edge devices is received. The global similarity is obtained by summing the similarity of the first feature and the similarity of the second feature. An out-of-distribution sample generator is trained based on the global similarity, and a virtual sample dataset is constructed using the trained out-of-distribution sample generator. The edge heterogeneous model is trained using the local dataset and the virtual sample dataset to obtain the trained edge heterogeneous model.

2. The edge heterogeneous model training method as described in claim 1, characterized in that, The out-of-distribution sample generator trained based on the global similarity includes: Initialize the out-of-distribution sample generator; wherein the out-of-distribution sample generator takes the one-hot vector of the target data category in the specified dataset and Gaussian noise as input, the specified dataset includes all local data categories in the local dataset, and the identification information of the out-of-distribution data category to be generated; The loss function of the out-of-distribution sample generator is constructed based on the global similarity. The out-of-distribution sample generator is iteratively trained and its parameters are optimized until the loss value output by the loss function is less than a preset first loss threshold, at which point the trained out-of-distribution sample generator is obtained.

3. The edge heterogeneous model training method as described in claim 2, characterized in that, The loss function for constructing the out-of-distribution sample generator based on the global similarity includes: Calculate the second similarity between the virtual out-of-distribution samples output by the out-of-distribution sample generator and each local data category in the local dataset; Obtain the target similarity from the global similarity; wherein the target similarity is the similarity value of the category pair corresponding to the second similarity, and the category pair consists of the out-of-distribution data category and the local data category; The loss function of the out-of-distribution sample generator is constructed based on the second similarity and the target similarity.

4. The edge heterogeneous model training method as described in claim 1, characterized in that, The construction of a virtual sample dataset using a trained out-of-distribution sample generator includes: Use a trained out-of-distribution sample generator to generate virtual out-of-distribution samples; Calculate the sample mean and sample covariance of the samples outside the virtual distribution; A Gaussian distribution model is constructed using the sample mean and the sample covariance; Multiple out-of-target-distribution samples are sampled from the Gaussian distribution model according to preset sampling probability values, and these multiple out-of-target-distribution samples are integrated to obtain a virtual sample dataset.

5. The edge heterogeneous model training method as described in claim 1, characterized in that, The step of training the edge heterogeneous model using the local dataset and the virtual sample dataset includes: The local dataset and the virtual sample dataset are used as joint training data, and the joint training data is input into the edge heterogeneous model; Construct a joint loss function that includes both in-distribution sample loss and out-of-distribution sample loss; The edge heterogeneous model is iteratively trained and its parameters are optimized until the loss value output by the joint loss function is less than a preset second loss threshold, at which point the iterative training of the edge heterogeneous model is stopped.

6. The edge heterogeneous model training method as described in claim 5, characterized in that, After stopping the iterative training of the edge heterogeneous model, the method further includes: Obtain test set data; wherein, the test set data includes samples of the same type as the joint training data, samples from the same reference dataset as the local dataset but not belonging to the joint training data, and samples from other datasets other than the reference dataset; The test set data is input into the trained edge heterogeneous model to obtain the test results; The processing capability of the edge heterogeneous model is evaluated based on the test results.

7. A training device for edge heterogeneous models, characterized in that, include: The first feature similarity calculation module is used to calculate the first feature similarity between any two local data categories in the local dataset. The first feature similarity sending module is used to send the first feature similarity to the other edge devices; The second feature similarity receiving module is used to receive the second feature similarity sent by other edge devices; A global similarity generation module is used to summarize the first feature similarity and the second feature similarity to obtain a global similarity. An out-of-distribution sample generator training module is used to train an out-of-distribution sample generator based on the global similarity. The virtual sample dataset construction module is used to construct a virtual sample dataset using a trained out-of-distribution sample generator. The edge heterogeneous model training module is used to train the edge heterogeneous model using the local dataset and the virtual sample dataset to obtain the trained edge heterogeneous model.

8. A training device for edge heterogeneous models, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the edge heterogeneous model training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the edge heterogeneous model training method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the edge heterogeneous model training method as described in any one of claims 1 to 6.

Images